An image processing method for changing endoscope direction based on pupil movement

Yang Cao,Quanquan Liu,Kazuya Kawamura,Shigeki Sugano,Masakatsu G Fujie,Satoshi Miura,Yo Kobayashi

doi:10.1186/s40648-016-0042-6

Yang Cao, Quanquan Liu + Show 5 more

Open Access

https://doi.org/10.1186/s40648-016-0042-6

Copy DOI

Journal: ROBOMECH Journal	Publication Date: Jan 22, 2016
Citations: 19	License type: CC BY 4.0

Affiliation: Waseda University, Chiba University

Abstract

Increased attention has been focused on laparoscopic surgery because of its minimal invasiveness and improved cosmetic properties. However, the procedure of laparoscopic surgery is considerably difficult for surgeons, thus paving the way for the introduction of robotic technology to reduce the surgeon’s burden. Thus, we have developed a single-port surgery assistive robot with a master–slave structure that has two surgical manipulators and a sheath manipulator for the alteration of endoscope direction. During the development of the surgical robotic system, achieving intuitive operation is very important. In this paper, we propose a new laparoscope manipulator control system based on the movement of the pupils to enhance intuitive operability. We achieve this using a webcam and an image processing method. After the pupil movement data are obtained, the master computer transforms these data into an output signal, and then the slave computer receives and uses that signal to drive the robot. The details of the system and the pupil detection procedure are explained. The aim of the present experiment is to verify the effectiveness of the image processing method applied to the alteration of endoscope direction control system. For this purpose, we need to determine an appropriate pupil motion activation threshold to begin the sheath manipulator’s movement. We used four kinds of activation threshold, measuring the time cost of a particular operation: to move the image of the endoscope to a specific target position. Moreover, we identified an appropriate activation threshold that can be used to determine whether the endoscope is moving.

Highlights

Laparoscopic surgery is a technique whereby a laparoscope and surgical instruments are inserted into the patient’s body through an artificial or natural body cavity, followed by the surgeon operating the instruments based on the monitor image captured by the laparoscope [1]
The purpose of this paper is to introduce a control method for the alteration of endoscope direction using pupil-tracking into the Prototype 2 [16] system, which is achieved via image processing and the use of a webcam
We propose a threshold for pupil movement distance: the sheath manipulator, which is used for the alteration of endoscope direction, activates when the user’s pupil movement distance exceeds a threshold value; the manipulator remains in a static state when the pupil movement distance is below the threshold

Summary

Introduction

Laparoscopic surgery is a technique whereby a laparoscope and surgical instruments are inserted into the patient’s body through an artificial or natural body cavity, followed by the surgeon operating the instruments based on the monitor image captured by the laparoscope [1]. Naviot [5] provides surgeons with the possibility of solo surgery, and requires the surgeon to use one hand to hold a controller when. Cao et al Robomech J (2016) 3:3 they need to alter the direction of the laparoscope. The Da Vinci Surgical System [6] can change the control mode between the surgical manipulator and the endoscopic manipulator for the operator using a foot pedal. Such methods do not allow the surgeon to alter the operative field while simultaneously manipulating tissue [7]

Objectives

Methods

Results

Conclusion