Robot-Assisted Retinal Surgery Research Articles

Delta Robot Kinematic Calibration for Precise Robot-Assisted Retinal Surgery.

High precision is required for ophthalmic robotic systems. This paper presents the kinematic calibration for the delta robot which is part of the next generation of Steady-Hand Eye Robot (SHER). A linear error model is derived based on geometric error parameters. Two experiments with different ranges of workspace are conducted with laser sensors measuring displacement. The error parameters are identified and applied in the kinematics to compensate for modeling error. To achieve better accuracy, Bernstein polynomials are adopted to fit the error residuals after compensation. After the kinematic calibration process, the error residuals of the delta robot are reduced to satisfy the clinical requirements.

Spotlight-based 3D Instrument Guidance for Autonomous Task in Robot-assisted Retinal Surgery.

Retinal surgery is known to be a complicated and challenging task for an ophthalmologist even for retina specialists. Image guided robot-assisted intervention is among the novel and promising solutions that may enhance human capabilities during microsurgery. In this paper, a novel method is proposed for 3D navigation of a microsurgical instrument based on the projection of a spotlight during robot-assisted retinal surgery. To test the feasibility and effectiveness of the proposed method, a vessel tracking task in a phantom with a Remote Center of Motion (RCM) constraint is performed by the Steady-Hand Eye Robot (SHER). The results are compared to manual tracking, cooperative control tracking with the SHER and spotlight-based automatic tracking with SHER. The reported results are that the spotlight-based automatic tracking with SHER can reach an average tracking error of 0.013 mm and keeping distance error of 0.1 mm from the desired range demonstrating a significant improvement compared to manual or cooperative control methods alone.

Force-based Safe Vein Cannulation in Robot-assisted Retinal Surgery: A Preliminary Study.

Retinal vein cannulation (RVC) is a potential treatment for retinal vein occlusion (RVO). Manual surgery has limitations in RVC due to extremely small vessels and instruments involved, as well as the presence of physiological hand tremor. Robot-assisted retinal surgery may be a better approach to smooth and accurate instrument manipulation during this procedure. Motion of the retina and cornea related to heartbeat may be associated with unexpected forces between the tool and eyeball. In this paper, we propose a force-based control strategy to automatically compensate for the movement of the retina maintaining the tip force and sclera force in a predetermined small range. A dual force-sensing tool is used to monitor the tip force, sclera force and tool insertion depth, which will be used to derive a desired joint velocity for the robot via a modified admittance controller. Then the tool is manipulated to compensate for the movement of the retina as well as reduce the tip force and sclera force. Quantitative experiments are conducted to verify the efficacy of the control strategy and a user study is also conducted by a retinal surgeon to demonstrate the advantages of our automatic compensation approach.

Auditory Feedback Effectiveness for Enabling Safe Sclera Force in Robot-Assisted Vitreoretinal Surgery: a Multi-User Study.

Robot-assisted retinal surgery has become increasingly prevalent in recent years in part due to the potential for robots to help surgeons improve the safety of an immensely delicate and difficult set of tasks. The integration of robots into retinal surgery has resulted in diminished surgeon perception of tool-to-tissue interaction forces due to robot's stiffness. The tactile perception of these interaction forces (sclera force) has long been a crucial source of feedback for surgeons who rely on them to guide surgical maneuvers and to prevent damaging forces from being applied to the eye. This problem is exacerbated when there are unfavorable sclera forces originating from patient movements (dynamic eyeball manipulation) during surgery which may cause the sclera forces to increase even drastically. In this study we aim at evaluating the efficacy of providing warning auditory feedback based on the level of sclera force measured by force sensing instruments. The intent is to enhance safety during dynamic eye manipulations in robot-assisted retinal surgery. The disturbances caused by lateral movement of patient's head are simulated using a piezo-actuated linear stage. The Johns Hopkins Steady-Hand Eye Robot (SHER), is then used in a multi-user experiment. Twelve participants are asked to perform a mock retinal surgery by following painted vessels inside an eye phantom using a force sensing instrument while auditory feedback is provided. The results indicate that the users are able to handle the eye motion disturbances while maintaining the sclera forces within safe boundaries when audio feedback is provided.

Towards Bimanual Vein Cannulation: Preliminary Study of a Bimanual Robotic System With a Dual Force Constraint Controller.

Retinal vein cannulation is a promising approach for treating retinal vein occlusion that involves injecting medicine into the occluded vessel to dissolve the clot. The approach remains largely unexploited clinically due to surgeon limitations in detecting interaction forces between surgical tools and retinal tissue. In this paper, a dual force constraint controller for robot-assisted retinal surgery was presented to keep the tool-to-vessel forces and tool-to-sclera forces below prescribed thresholds. A cannulation tool and forceps with dual force-sensing capability were developed and used to measure force information fed into the robot controller, which was implemented on existing Steady Hand Eye Robot platforms. The robotic system facilitates retinal vein cannulation by allowing a user to grasp the target vessel with the forceps and then enter the vessel with the cannula. The system was evaluated on an eye phantom. The results showed that, while the eyeball was subjected to rotational disturbances, the proposed controller actuates the robotic manipulators to maintain the average tool-to-vessel force at 10.9 mN and 13.1 mN and the average tool-to-sclera force at 38.1 mN and 41.2 mN for the cannula and the forcpes, respectively. Such small tool-to-tissue forces are acceptable to avoid retinal tissue injury. Additionally, two clinicians participated in a preliminary user study of the bimanual cannulation demonstrating that the operation time and tool-to-tissue forces are significantly decreased when using the bimanual robotic system as compared to freehand performance.

Real Time Prediction of Sclera Force with LSTM Neural Networks in Robot-Assisted Retinal Surgery.

Retinal microsurgery is one of the most technically demanding surgeries, during which the surgical tool needs to be inserted into the eyeball and is constantly constrained by the sclerotomy port. During the surgery, any unexpected manipulation could cause extreme tool-sclera contact force leading to sclera damage. Although, a robot assistant could reduce hand tremor and improve the tool positioning accuracy, it cannot prevent or alarm the surgeon about the upcoming danger caused by surgeon's misoperations, i.e., applying excessive force on the sclera. In this paper, we present a new method based on a Long Short Term Memory recurrent neural network for predicting the user behavior, i.e., the contact force between the tool and sclera (sclera force) and the insertion depth of the tool from sclera contact point (insertion depth) in real time (40Hz). The predicted force information is provided to the user through auditory feedback to alarm any unexpected sclera force. The user behavior data is collected in a mock retinal surgical operation on a dry eye phantom with Steady Hand Eye Robot and a novel multi-function sensing tool. The Long Short Term Memory recurrent neural network is trained on the collected time series of sclera force and insertion depth. The network can predict the sclera force and insertion depth 100 milliseconds in the future with 95.29% and 96.57% accuracy, respectively, and can help reduce the fraction of unsafe sclera forces from 40.19% to 15.43%.

Enabling Technology for Safe Robot-Assisted Retinal Surgery: Early Warning for Unsafe Scleral Force.

Retinal microsurgery is technically demanding and requires high surgical skill with very little room for manipulation error. During surgery the tool needs to be inserted into the eyeball while maintaining constant contact with the sclera. Any unexpected manipulation could cause extreme tool-sclera contact force (scleral force) thus damage the sclera. The introduction of robotic assistance could enhance and expand the surgeon's manipulation capabilities during surgery. However, the potential intra-operative danger from surgeon's misoperations remains difficult to detect and prevent by existing robotic systems. Therefore, we propose a method to predict imminent unsafe manipulation in robot-assisted retinal surgery and generate feedback to the surgeon via auditory substitution. The surgeon could then react to the possible unsafe events in advance. This work specifically focuses on minimizing sclera damage using a force-sensing tool calibrated to measure small scleral forces. A recurrent neural network is designed and trained to predict the force safety status up to 500 milliseconds in the future. The system is implemented using an existing "steady hand" eye robot. A vessel following manipulation task is designed and performed on a dry eye phantom to emulate the retinal surgery and to analyze the proposed method. Finally, preliminary validation experiments are performed by five users, the results of which indicate that the proposed early warning system could help to reduce the number of unsafe manipulation events.

Adaptive Control of Sclera Force and Insertion Depth for Safe Robot-Assisted Retinal Surgery.

One of the significant challenges of moving from manual to robot-assisted retinal surgery is the loss of perception of forces applied to the sclera (sclera forces) by the surgical tools. This damping of force feedback is primarily due to the stiffness and inertia of the robot. The diminished perception of tool-to-eye interactions might put the eye tissue at high risk of injury due to excessive sclera forces or extreme insertion of the tool into the eye. In the present study therefore a 1-dimensional adaptive control method is customized for 3-dimensional control of sclera force components and tool insertion depth and then implemented on the velocity-controlled Johns Hopkins Steady-Hand Eye Robot. The control method enables the robot to perform autonomous motions to make the sclera force and/or insertion depth of the tool tip to follow pre-defined desired and safe trajectories when they exceed safe bounds. A robotic light pipe holding application in retinal surgery is also investigated using the adaptive control method. The implementation results indicate that the adaptive control is able to achieve the imposed safety margins and prevent sclera forces and insertion depth from exceeding safe boundaries.

User Behavior Evaluation in Robot-Assisted Retinal Surgery.

Retinal microsurgery is technically demanding and requires high surgical skill with very little room for manipulation error. The introduction of robotic assistance has the potential to enhance and expand a surgeon's manipulation capabilities during retinal surgery, i.e., improve precision, cancel physiological hand tremor, and provide sensing information. However, surgeon performance may also be negatively impacted by robotic assistance due to robot structural stiffness and nonintuitive controls. In complying with robotic constraints, the surgeon loses the dexterity of the human hand. In this paper, we present a preliminary experimental study to evaluate user behavior when affected by robotic assistance during mock retinal surgery. In these experiments user behavior is characterized by measuring the forces applied by the user to the sclera, the tool insertion/retraction speed, the tool insertion depth relative to the scleral entry point, and the duration of surgery. The users' behavior data is collected during three mock retinal surgery tasks with four users. Each task is conducted using both freehand and robot-assisted techniques. The univariate user behavior and the correlations of multiple parameters of user behavior are analyzed. The results show that robot assistance prolongs the duration of the surgery and increases the manipulation forces applied to sclera, but refines the insertion velocity and eliminates hand tremor.

First-in-human study of the safety and viability of intraocular robotic surgery.

Microsurgery of the retina would be dramatically improved by instruments that offer supra-human precision. Here, we report the results of a first-in-human study of remotely controlled robot-assisted retinal surgery performed through a telemanipulation device. Specifically, 12 patients requiring dissection of the epiretinal or inner limiting membrane over the macula were randomly assigned to either undergo robot-assisted-surgery or manual surgery, under general anaesthesia. We evaluated surgical success, duration of surgery and amount of retinal microtrauma as a proxy for safety. Surgical outcomes were equally successful in the robotic-surgery and manual-surgery groups. Differences in the amount of retinal microtrauma between the two groups were statistically insignificant, yet dissection took longer with robotic surgery (median time, 4 min 5 s) than with manual surgery (1 min 20 s). We also show the feasibility of using the robot to inject recombinant tissue plasminogen activator under the retina to displace sight-threatening haemorrhage in three patients under local anaesthesia. A safe and viable robotic system for intraocular surgery would enable precise and minimally traumatic delivery of gene therapy or cell therapy to the retina.

Automatic Tool Landmark Detection for Stereo Vision in Robot-Assisted Retinal Surgery

Computer vision and robotics are being increasingly applied in medical interventions. Especially in interventions where extreme precision is required, they could make a difference. One such application is robot-assisted retinal microsurgery. In recent works, such interventions are conducted under a stereo-microscope, and with a robot-controlled surgical tool. The complementarity of computer vision and robotics has, however, not yet been fully exploited. In order to improve the robot control, we are interested in three-dimensional (3-D) reconstruction of the anatomy and in automatic tool localization using a stereo microscope. In this letter, we solve this problem for the first time using a single pipeline, starting from uncalibrated cameras to reach metric 3-D reconstruction and registration, in retinal microsurgery. The key ingredients of our method are 1) surgical tool landmark detection, and 2) 3-D reconstruction with the stereo microscope, using the detected landmarks. To address the former, we propose a novel deep learning method that detects and recognizes keypoints in high-definition images at higher than real-time speed. We use the detected two-dimensional keypoints along with their corresponding 3-D coordinates obtained from the robot sensors to calibrate the stereo microscope using an affine projection model. We design an online 3-D reconstruction pipeline that makes use of smoothness constraints and performs robot-to-camera registration. The entire pipeline is extensively validated on open-sky porcine eye sequences. Quantitative and qualitative results are presented for all steps.