Robotic Eye Research Articles

An Integrated High-dexterity Cooperative Robotic Assistant for Intraocular Micromanipulation.

Retinal surgeons are required to manipulate multiple surgical instruments in a confined intraocular space, while the instruments are constrained at the small incisions made on the sclera. Furthermore, physiological hand tremor can affect the precision of the instrument motion. The Steady-Hand Eye Robot (SHER), developed in our previous study, enables tremor-free tool manipulation by employing a cooperative control scheme whereby the surgeon and robot can co-manipulate the surgical instruments. Although SHER enables precise and tremor-free manipulation of surgical tools, its straight and rigid structure imposes certain limitations, as it can only approach a target on the retina from one direction. As a result, the instrument could potentially collide with the eye lens when attempting to access the anterior portion of the retina. In addition, it can be difficult to approach a target on the retina from a suitable direction when accessing its anterior portion for procedures such as vein cannulation or membrane peeling. Snake-like robots offer greater dexterity and allow access to a target on the retina from suitable directions, depending on the clinical task at hand. In this study, we present an integrated, high-dexterity, cooperative robotic assistant for intraocular micromanipulation. This robotic assistant comprises an improved integrated robotic intraocular snake (I2RIS) with a user interface (a tactile switch or joystick unit) for the manipulation of the snake-like distal end and the SHER, with a detachable end-effector to which the I2RIS can be attached. The integrated system was evaluated through a set of experiments wherein subjects were requested to touch or insert into randomly-assigned targets. The results indicate that the high-dexterity robotic assistant can touch or insert the tip into the same target from multiple directions, with no significant increase in task completion time for either user interface.

Preview the Products on Display at Display Week 2021

Preview the Products on Display at Display Week 2021

Adaptive Control Improves Sclera Force Safety in Robot-Assisted Eye Surgery: A Clinical Study.

The integration of robotics into retinal microsurgery leads to a reduction in surgeon perception of tool-to-tissue interaction forces. This blunting of human tactile sensory input, which is due to the inflexible mass and large inertia of the robotic arm as compared to the milli-Newton scale of the interaction forces and fragile tissues during ophthalmic surgery, identifies a potential iatrogenic risk during robotic eye surgery. In this paper, we aim to evaluate two variants of an adaptive force control scheme implemented on the Steady-Hand Eye Robot (SHER) that are intended to mitigate the risk of unsafe scleral forces. The present study enrolled ten retina fellows and ophthalmology residents into a simulated procedure, which simply asked the trainees to follow retinal vessels in a model retina surgery environment. For this purpose, we have developed a force-sensing (equipped with Fiber Bragg Grating (FBG)) instrument to attach to the robot. A piezo-actuated linear stage for creating random lateral motions to the eyeball phantom has been provided to simulate disturbances during surgery. The SHER and all of its dependencies were set up in an operating room in the Wilmer Eye Institute at the Johns Hopkins Hospital. The clinicians conducted robot-assisted experiments with the adaptive controls incorporated as well as freehand manipulations. The results indicate that the Adaptive Norm Control (ANC) method, is able to maintain scleral forces at predetermined safe levels better than even freehand manipulations. Novice clinicians in robot training however, subjectively preferred freehand maneuvers over robotic manipulations. Clinician preferences once highly skilled with the robot is not assessed in this study.

57084 Combining artificial intelligence and robotics: a novel fully automated optical coherence tomography-based approach for eye disease screening

ABSTRACT IMPACT: Despite its importance in systemic diseases such as diabetes, the eye is notably difficult to examine for non-specialists; this study introduces a fully automated approach for eye disease screening, coupling a deep learning algorithm with a robotically-aligned optical coherence tomography system to improve eye care in non-ophthalmology settings. OBJECTIVES/GOALS: This study aims to develop and test a deep learning (DL) method to classify images acquired from a robotically-aligned optical coherence tomography (OCT) system as normal vs. abnormal. The long-term goal of our study is to integrate artificial intelligence and robotic eye imaging to fully automate eye disease screening in diverse clinical settings. METHODS/STUDY POPULATION: Between August and October 2020, patients seen at the Duke Eye Center and healthy volunteers (age ≥18) were imaged with a custom, robotically-aligned OCT (RAOCT) system following routine eye exam. Using transfer learning, we adapted a preexisting convolutional neural network to train a DL algorithm to classify OCT images as normal vs. abnormal. The model was trained and validated on two publicly available OCT datasets and two of our own RAOCT volumes. For external testing, the top-performing model based on validation was applied to a representative averaged B-scan from each of the remaining RAOCT volumes. The model’s performance was evaluated against a reference standard of clinical diagnoses by retina specialists. Saliency maps were created to visualize the areas contributing most to the model predictions. RESULTS/ANTICIPATED RESULTS: The training and validation datasets included 87,697 OCT images, of which 59,743 were abnormal. The top-performing DL model had a training accuracy of 96% and a validation accuracy of 99%. For external testing, 43 eyes of 27 subjects were imaged with the robotically-aligned OCT system. Compared to clinical diagnoses, the model correctly labeled 18 out of 22 normal averaged B-scans and 18 out of 21 abnormal averaged B-scans. Overall, in the testing set, the model had an AUC for the detection of pathology of 0.92, an accuracy of 84%, a sensitivity of 86%, and a specificity of 82%. For the correctly predicted scans, saliency maps identified the areas contributing most to the DL algorithm’s predictions, which matched the regions of greatest clinical importance. DISCUSSION/SIGNIFICANCE OF FINDINGS: This is the first study to develop and apply a DL model to images acquired from a self-aligning OCT system, demonstrating the potential of integrating DL and robotic eye imaging to automate eye disease screening. We are working to translate this technology for use in emergency departments and primary care, where it will have the greatest impact.

Study of a Five-Phase AC Driven Spherical Motor with Oscillatory Movement

The conventional motor is driven with one degree of freedom, whereas the spherical motor is driven with three degrees of freedom. As a result, compared to conventional motors, control to the target posture is easier in a mechanism using spherical motors. In addition, the difficulty of achieving ultra-high accuracy due to assembly errors and difficulty of maintenance will be reduced. Because of these features, spherical motors are expected to be used for robot eyes, joints, reaction wheels, and stirrers. In the culture of cells in biotechnology, it is necessary to constantly stir the solution containing cells, but currently the stirring is conducted by one-dimensional movement. There is a need for a rotation motion with sinusoidally changing the tilting angle of rotation axis. This study proposes a spherical motor to realize the rotation motion and has developed a prototype spherical motor of this type. The proposed spherical motor is driven by the electromagnets excited by five-phase alternating currents. This paper describes the structure of proposed spherical motor, its driving principle and rotation experiments of the developed prototype of spherical motor.

Auditory Feedback Effectiveness for Enabling Safe Sclera Force in Robot-Assisted Vitreoretinal Surgery: a Multi-User Study.

Robot-assisted retinal surgery has become increasingly prevalent in recent years in part due to the potential for robots to help surgeons improve the safety of an immensely delicate and difficult set of tasks. The integration of robots into retinal surgery has resulted in diminished surgeon perception of tool-to-tissue interaction forces due to robot's stiffness. The tactile perception of these interaction forces (sclera force) has long been a crucial source of feedback for surgeons who rely on them to guide surgical maneuvers and to prevent damaging forces from being applied to the eye. This problem is exacerbated when there are unfavorable sclera forces originating from patient movements (dynamic eyeball manipulation) during surgery which may cause the sclera forces to increase even drastically. In this study we aim at evaluating the efficacy of providing warning auditory feedback based on the level of sclera force measured by force sensing instruments. The intent is to enhance safety during dynamic eye manipulations in robot-assisted retinal surgery. The disturbances caused by lateral movement of patient's head are simulated using a piezo-actuated linear stage. The Johns Hopkins Steady-Hand Eye Robot (SHER), is then used in a multi-user experiment. Twelve participants are asked to perform a mock retinal surgery by following painted vessels inside an eye phantom using a force sensing instrument while auditory feedback is provided. The results indicate that the users are able to handle the eye motion disturbances while maintaining the sclera forces within safe boundaries when audio feedback is provided.

An Optimized Tilt Mechanism for a New Steady-Hand Eye Robot.

Robot-assisted vitreoretinal surgery can filter surgeons' hand tremors and provide safe, accurate tool manipulation. In this paper, we report the design, optimization, and evaluation of a novel tilt mechanism for a new Steady-Hand Eye Robot (SHER). The new tilt mechanism features a four-bar linkage design and has a compact structure. Its kinematic configuration is optimized to minimize the required linear range of motion (LRM) for implementing a virtual remote center-of-motion (V-RCM) while tilting a surgical tool. Due to the different optimization constraints for the robots at the left and right sides of the human head, two configurations of this tilt mechanism are proposed. Experimental results show that the optimized tilt mechanism requires a significantly smaller LRM (e.g. 5.08 mm along Z direction and 8.77 mm along Y direction for left side robot) as compared to the slider-crank tilt mechanism used in the previous SHER (32.39 mm along Z direction and 21.10 mm along Y direction). The feasibility of the proposed tilt mechanism is verified in a mock bilateral robot-assisted vitreoretinal surgery. The ergonomically acceptable robot postures needed to access the surgical field is also determined.

Human-Inspired Robotic Eye-Hand Coordination Enables New Communication Channels Between Humans and Robots.

This paper concerns human-inspired robotic eye-hand coordination algorithms using custom built robotic eyes that were interfaced with a Baxter robot. Eye movement was programmed anthropomorphically based on previously reported research on human eye-hand coordination during grasped object transportation. Robotic eye tests were first performed on a component level where accurate position and temporal control were achieved. Next, 11 human subjects were recruited to observe the novel robotic system to quantify the ability of robotic eye-hand coordination algorithms to convey two kinds of information to people during object transportation tasks: first, the transported object's delivery location and second, the level of care exerted by the robot to transport the object. Most subjects correlated decreased frequency in gaze fixations on an object's target location with increased care of transporting an object, although these results were somewhat mixed among the 11 human subjects. Additionally, the human subjects were able to reliably infer the delivery location of the transported object purely by the robotic eye-hand coordination algorithm with an overall success rate of 91.4%. These results suggest that anthropomorphic eye-hand coordination of robotic entities could be useful in pedagogical or industrial settings.

Towards Bimanual Vein Cannulation: Preliminary Study of a Bimanual Robotic System With a Dual Force Constraint Controller.

Retinal vein cannulation is a promising approach for treating retinal vein occlusion that involves injecting medicine into the occluded vessel to dissolve the clot. The approach remains largely unexploited clinically due to surgeon limitations in detecting interaction forces between surgical tools and retinal tissue. In this paper, a dual force constraint controller for robot-assisted retinal surgery was presented to keep the tool-to-vessel forces and tool-to-sclera forces below prescribed thresholds. A cannulation tool and forceps with dual force-sensing capability were developed and used to measure force information fed into the robot controller, which was implemented on existing Steady Hand Eye Robot platforms. The robotic system facilitates retinal vein cannulation by allowing a user to grasp the target vessel with the forceps and then enter the vessel with the cannula. The system was evaluated on an eye phantom. The results showed that, while the eyeball was subjected to rotational disturbances, the proposed controller actuates the robotic manipulators to maintain the average tool-to-vessel force at 10.9 mN and 13.1 mN and the average tool-to-sclera force at 38.1 mN and 41.2 mN for the cannula and the forcpes, respectively. Such small tool-to-tissue forces are acceptable to avoid retinal tissue injury. Additionally, two clinicians participated in a preliminary user study of the bimanual cannulation demonstrating that the operation time and tool-to-tissue forces are significantly decreased when using the bimanual robotic system as compared to freehand performance.

COVID-19 (Coronavirus Disease 2019): Opportunities and Challenges for Digital Health and the Internet of Medical Things in China.

OMICS: A Journal of Integrative BiologyVol. 24, No. 5 EditorialsCOVID-19 (Coronavirus Disease 2019): Opportunities and Challenges for Digital Health and the Internet of Medical Things in ChinaBiaoyang Lin and ShengJun WuBiaoyang LinAddress correspondence to: Biaoyang Lin, PhD, Department of Urology, University of Washington School of Medicine, Seattle, WA 98195, USA E-mail Address: biaoylin@gmail.comZhejiang-California International Nanosystems Institute (ZCNI) Proprium Research Center, Zhejiang University, Hangzhou, China.Collaborative Innovation Center for Diagnosis and Treatment of Infectious Diseases, The First Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, China.Department of Urology, University of Washington School of Medicine, Seattle, Washington, USA.Search for more papers by this author and ShengJun WuDepartment of Clinical Laboratories, Sir Run Run Shaw Hospital, School of Medicine, Zhejiang University, Hangzhou, China.Search for more papers by this authorPublished Online:7 May 2020https://doi.org/10.1089/omi.2020.0047AboutSectionsView articleView Full TextPDF/EPUB Permissions & CitationsPermissionsDownload CitationsTrack CitationsAdd to favorites Back To Publication ShareShare onFacebookTwitterLinked InRedditEmail View article"COVID-19 (Coronavirus Disease 2019): Opportunities and Challenges for Digital Health and the Internet of Medical Things in China." OMICS: A Journal of Integrative Biology, 24(5), pp. 231–232FiguresReferencesRelatedDetailsCited byEmerging technologies for COVID (ET-CoV) detection and diagnosis: Recent advancements, applications, challenges, and future perspectivesBiomedical Signal Processing and Control, Vol. 83Integration of cloud computing with the Internet of things for the treatment and management of the COVID-19 pandemic15 November 2022 | Information Systems and e-Business Management, Vol. 26A novel e-healthcare diagnosing system for COVID-19 via whale optimization algorithm24 September 2022 | Journal of Experimental & Theoretical Artificial Intelligence, Vol. 6Call for Papers: COVID-19 Systems Biology, Multi-Omics Integration, and Digital Technologies in Ecology, Health, and Society Vural Özdemir7 September 2022 | OMICS: A Journal of Integrative Biology, Vol. 26, No. 9From Personalized Medicine to Population Health: A Survey of mHealth Sensing TechniquesIEEE Internet of Things Journal, Vol. 9, No. 17The role of contemporary digital tools and technologies in COVID‐19 crisis: An exploratory analysis6 October 2021 | Expert Systems, Vol. 39, No. 6Coronavirus Disease (COVID-19): Challenges and Opportunities10 September 2020 | Disaster Medicine and Public Health Preparedness, Vol. 16, No. 2Anthropomorphic Robotic Eyes: Structural Design and Non-Verbal Communication Effectiveness15 April 2022 | Sensors, Vol. 22, No. 8Digital Transformation in Personalized Medicine with Artificial Intelligence and the Internet of Medical Things Biaoyang Lin and Shengjun Wu7 February 2022 | OMICS: A Journal of Integrative Biology, Vol. 26, No. 2A Review About the Main Technologies to Fight COVID-1915 April 2022Identifying and addressing digital health risks associated with emergency pandemic response: Problem identification, scoping review, and directions toward evidence-based evaluationInternational Journal of Medical Informatics, Vol. 157Unlocking the Public Perception of COVID-19 Vaccination Process on Social MediaToward Combatting COVID-19: A Risk Assessment SystemIEEE Internet of Things Journal, Vol. 8, No. 21An online COVID-19 self-assessment framework supported by IoMT technology6 August 2021 | Journal of Intelligent Systems, Vol. 30, No. 1A Security Management Framework for Big Data in Smart HealthcareBig Data Research, Vol. 25Connected Medical Kiosks to Counter COVID-19: Needs, Architecture & Design GuidelinesThe perspectives of biomarker-based electrochemical immunosensors, artificial intelligence and the Internet of Medical Things toward COVID-19 diagnosis and managementMaterials Today Chemistry, Vol. 20Internet of Things (IoT) Enabled Architecture for Social Distancing During Pandemic15 April 2021 | Frontiers in Communications and Networks, Vol. 2A Comprehensive Review of Applications of Internet of Things for Covid-19 PandemicIOP Conference Series: Materials Science and Engineering, Vol. 1055, No. 1Robotics and Drone-Based Solution for the Impact of COVID-19 Worldwide Using AI and IoT16 February 2021Digital Transformation and Emerging Technologies for COVID-19 Pandemic: Social, Global, and Industry Perspectives20 February 2021Telemedicine maybe an effective solution for management of chronic disease during the COVID-19 epidemic29 September 2021 | Primary Health Care Research & Development, Vol. 22Uma análise sobre o desenvolvimento de tecnologias digitais em saúde para o enfrentamento da COVID-19 no Brasil e no mundo1 January 2021 | Cadernos de Saúde Pública, Vol. 37, No. 3Enabled IoT Applications for Covid-191 May 2021Swarm Intelligence based Hierarchical Routing Protocols Study in WSNsCOVID-19 Digital Health Innovation Policy: A Portal to Alternative Futures in the Making Mustafa Bayram, Simon Springer, Colin K. Garvey, and Vural Özdemir3 August 2020 | OMICS: A Journal of Integrative Biology, Vol. 24, No. 8Digital Technologies-Enabled Smart Manufacturing and Industry 4.0 in the Post-COVID-19 Era: Lessons Learnt from a Pandemic3 July 2020 | International Journal of Environmental Research and Public Health, Vol. 17, No. 13Adoption of Digital Technologies in Health Care During the COVID-19 Pandemic: Systematic Review of Early Scientific Literature6 November 2020 | Journal of Medical Internet Research, Vol. 22, No. 11 Volume 24Issue 5May 2020 InformationCopyright 2020, Mary Ann Liebert, Inc., publishersTo cite this article:Biaoyang Lin and ShengJun Wu.COVID-19 (Coronavirus Disease 2019): Opportunities and Challenges for Digital Health and the Internet of Medical Things in China.OMICS: A Journal of Integrative Biology.May 2020.231-232.http://doi.org/10.1089/omi.2020.0047Published in Volume: 24 Issue 5: May 7, 2020Online Ahead of Print:April 20, 2020PDF download

Synthesis and methodology for optimal design of a parallel remote center of motion mechanism: Application to robotic eye surgery

This work reports on the synthesis of a parallel Remote Center of Motion (RCM) mechanism, and an optimized design for the use-case of robot-assisted vitreoretinal surgery. A 2-DoF planar RCM mechanism is proposed and synthesised as part of a 4-DoF RCM mechanism. The proposed design substantially reduces the occupied volume at the end-effector. This solves a major problem present in related state-of-the-art, which poses limitations on sterile end-effector design and surgical instrument compatibility. Subsequently, an optimal design algorithm is proposed and implemented for the given use-case. The workspace is determined within mechanism constraints, after which performance parameters such as workspace coverage, potential energy, manipulability, reflected stiffness are determined for specific areas of interest within a desired workspace. Subsequently, these parameters are combined in a score function identifying an optimal kinematic design. When compared with the closely-related prior art, the resulting design shows improvements in relevant workspace coverage, reduced gravity compensation effort, and more isotropic manipulability. Overall reflected stiffness is reduced and should be taken into account in future design phases. Future work includes the integration of the kinematic design into a detailed conceptual design and a first prototype development.

An entirely soft varifocal lens based on an electro-hydraulic actuator

Varifocal lenses have been promptly developed but show many shortcomings in the area of artificial pupils, such as rigid frames, prestress and large structures. This paper presents an entirely soft varifocal lens to adapt to the increasingly ingenious soft robotic structures. Such a varifocal lens is mainly composed of a PDMS frame, a liquid dielectric and a pair of flexible electrodes. The stable focal length of the lens varies from 533.5 to 49.4 mm by applying DC voltages from 3 to 12 kV. Surprisingly, a novel ‘jump’ phenomenon is found in the varifocal lens at the moment of power-on. Then, the varifocal effects are studied under alternating voltages with a square wave, a triangle wave or a sine wave. The varifocal range remains basically stable after repeated cycles under alternating voltages. As the frequency increases, the range of the focal length change also gradually decreases and becomes more stable. As a result, the stable focal length of the lens is able to vary as wide as from 690.3 to 24.5 mm by applying square waveform voltages from 1 to 8 kV at a frequency of 10 Hz. The proposed lens is simple, thin and flexible, which is expected to develop the artificial eyes of soft robots.

At first sight: robots’ subtle eye movement parameters affect human attentional engagement, spontaneous attunement and perceived human-likeness

AbstractHuman-robot interaction research could benefit from knowing how various parameters of robotic eye movement control affect specific cognitive mechanisms of the user, such as attention or perception. In the present study, we systematically teased apart control parameters of Trajectory Time of robot eye movements (rTT) between two joint positions and Fixation Duration (rFD) on each of these positions of the iCub robot. We showed recordings of these behaviors to participants and asked them to rate each video on how human-like the robot’s behavior appeared. Additionally, we recorded participants’ eye movements to examine whether the different control parameters evoked different effects on cognition and attention. We found that slow but variable robot eye movements yielded relatively higher human-likeness ratings. On the other hand, the eye-tracking data suggest that the human range of rTT is most engaging and evoked spontaneous involvement in joint attention. The pattern observed in subjective ratings was paralleled only by one measure in the implicit objective metrics, namely the frequency of spontaneous attentional following. These findings provide significant clues for controller design to improve the interaction between humans and artificial agents.

Real Time Prediction of Sclera Force with LSTM Neural Networks in Robot-Assisted Retinal Surgery.

Retinal microsurgery is one of the most technically demanding surgeries, during which the surgical tool needs to be inserted into the eyeball and is constantly constrained by the sclerotomy port. During the surgery, any unexpected manipulation could cause extreme tool-sclera contact force leading to sclera damage. Although, a robot assistant could reduce hand tremor and improve the tool positioning accuracy, it cannot prevent or alarm the surgeon about the upcoming danger caused by surgeon's misoperations, i.e., applying excessive force on the sclera. In this paper, we present a new method based on a Long Short Term Memory recurrent neural network for predicting the user behavior, i.e., the contact force between the tool and sclera (sclera force) and the insertion depth of the tool from sclera contact point (insertion depth) in real time (40Hz). The predicted force information is provided to the user through auditory feedback to alarm any unexpected sclera force. The user behavior data is collected in a mock retinal surgical operation on a dry eye phantom with Steady Hand Eye Robot and a novel multi-function sensing tool. The Long Short Term Memory recurrent neural network is trained on the collected time series of sclera force and insertion depth. The network can predict the sclera force and insertion depth 100 milliseconds in the future with 95.29% and 96.57% accuracy, respectively, and can help reduce the fraction of unsafe sclera forces from 40.19% to 15.43%.

Optimization of Internal Limiting Membrane Peeling Task in Robotic Eye Surgery

Optimization of Internal Limiting Membrane Peeling Task in Robotic Eye Surgery

A Novel Semi-Autonomous Control Framework for Retina Confocal Endomicroscopy Scanning.

In this paper, a novel semi-autonomous control framework is presented for enabling probe-based confocal laser endomicroscopy (pCLE) scan of the retinal tissue. With pCLE, retinal layers such as nerve fiber layer (NFL) and retinal ganglion cell (RGC) can be scanned and characterized in real-time for an improved diagnosis and surgical outcome prediction. However, the limited field of view of the pCLE system and the micron-scale optimal focus distance of the probe, which are in the order of physiological hand tremor, act as barriers to successful manual scan of retinal tissue. Therefore, a novel sensorless framework is proposed for real-time semi-autonomous endomicroscopy scanning during retinal surgery. The framework consists of the Steady-Hand Eye Robot (SHER) integrated with a pCLE system, where the motion of the probe is controlled semi-autonomously. Through a hybrid motion control strategy, the system autonomously controls the confocal probe to optimize the sharpness and quality of the pCLE images, while providing the surgeon with the ability to scan the tissue in a tremor-free manner. Effectiveness of the proposed architecture is validated through experimental evaluations as well as a user study involving 9 participants. It is shown through statistical analyses that the proposed framework can reduce the work load experienced by the users in a statistically-significant manner, while also enhancing their performance in retaining pCLE images with optimized quality.

Prosthetic Soul Mates: Sex Robots as Media for Companionship

Prosthetic Soul Mates: Sex Robots as Media for Companionship

Towards securing the sclera against patient involuntary head movement in robotic retinal surgery.

Retinal surgery involves manipulating very delicate tissues within the confined area of eyeball. In such demanding practices, patient involuntary head movement might abruptly raise tool-to-eyeball interaction forces which would be detrimental to eye. This study is aimed at implementing different force control strategies and evaluating how they contribute to attaining sclera force safety while patient head drift is present. To simulate patient head movement, a piezoelectric-actuated linear stage is used to produce random motions in a single direction in random time intervals. Having an eye phantom attached to the linear stage then an experienced eye surgeon is asked to manipulate the eye and repeat a mock surgical task both with and without the assist of the Steady-Hand Eye Robot. For the freehand case, warning sounds were provided to the surgeon as auditory feedback to alert him about excessive slclra forces. For the robot-assisted experiments two variants of an adaptive sclera force control and a virtual fixture method were deployed to see how they can maintain eye safety under head drift circumstances. The results indicate that the developed robot control strategies are able to compensate for head drift and keep the sclera forces under safe levels as well as the free hand operation.

Toward Safe Retinal Microsurgery: Development and Evaluation of an RNN-Based Active Interventional Control Framework.

Robotics-assisted retinal microsurgery provides several benefits including improvement of manipulation precision. The assistance provided to the surgeons by current robotic frameworks is, however, a "passive" support, e.g., by damping hand tremor. Intelligent assistance and active guidance are, however, lacking in the existing robotic frameworks. In this paper, an active interventional control framework (AICF) has been presented to increase operation safety by actively intervening the operation to avoid exertion of excessive forces to the sclera. AICF consists of the following four components: first, the steady-hand eye robot as the robotic module; second, a sensorized tool to measure tool-to-sclera forces; third, a recurrent neural network to predict occurrence of undesired events based on a short history of time series of sensor measurements; and finally, a variable admittance controller to command the robot away from the undesired instances. A set of user studies were conducted involving 14 participants (with four surgeons). The users were asked to perform a vessel-following task on an eyeball phantom with the assistance of AICF as well as other two benchmark approaches, i.e., auditory feedback (AF) and real-time force feedback (RF). Statistical analysis shows that AICF results in a significant reduction of proportion of undesired instances to about 2.5%, compared with 38.4% and 26.2% using AF and RF, respectively. AICF can effectively predict excessive-force instances and augment performance of the user to avoid undesired events during robot-assisted microsurgical tasks. The proposed system may be extended to other fields of microsurgery and may potentially reduce tissue injury.

Enabling Technology for Safe Robot-Assisted Retinal Surgery: Early Warning for Unsafe Scleral Force.

Retinal microsurgery is technically demanding and requires high surgical skill with very little room for manipulation error. During surgery the tool needs to be inserted into the eyeball while maintaining constant contact with the sclera. Any unexpected manipulation could cause extreme tool-sclera contact force (scleral force) thus damage the sclera. The introduction of robotic assistance could enhance and expand the surgeon's manipulation capabilities during surgery. However, the potential intra-operative danger from surgeon's misoperations remains difficult to detect and prevent by existing robotic systems. Therefore, we propose a method to predict imminent unsafe manipulation in robot-assisted retinal surgery and generate feedback to the surgeon via auditory substitution. The surgeon could then react to the possible unsafe events in advance. This work specifically focuses on minimizing sclera damage using a force-sensing tool calibrated to measure small scleral forces. A recurrent neural network is designed and trained to predict the force safety status up to 500 milliseconds in the future. The system is implemented using an existing "steady hand" eye robot. A vessel following manipulation task is designed and performed on a dry eye phantom to emulate the retinal surgery and to analyze the proposed method. Finally, preliminary validation experiments are performed by five users, the results of which indicate that the proposed early warning system could help to reduce the number of unsafe manipulation events.