Robotic Eye Research Articles

Adaptive Control of Sclera Force and Insertion Depth for Safe Robot-Assisted Retinal Surgery.

One of the significant challenges of moving from manual to robot-assisted retinal surgery is the loss of perception of forces applied to the sclera (sclera forces) by the surgical tools. This damping of force feedback is primarily due to the stiffness and inertia of the robot. The diminished perception of tool-to-eye interactions might put the eye tissue at high risk of injury due to excessive sclera forces or extreme insertion of the tool into the eye. In the present study therefore a 1-dimensional adaptive control method is customized for 3-dimensional control of sclera force components and tool insertion depth and then implemented on the velocity-controlled Johns Hopkins Steady-Hand Eye Robot. The control method enables the robot to perform autonomous motions to make the sclera force and/or insertion depth of the tool tip to follow pre-defined desired and safe trajectories when they exceed safe bounds. A robotic light pipe holding application in retinal surgery is also investigated using the adaptive control method. The implementation results indicate that the adaptive control is able to achieve the imposed safety margins and prevent sclera forces and insertion depth from exceeding safe boundaries.

Sclera Force Control in Robot-assisted Eye Surgery: Adaptive Force Control vs. Auditory Feedback.

Surgeon hand tremor limits human capability during microsurgical procedures such as those that treat the eye. In contrast, elimination of hand tremor through the introduction of microsurgical robots diminishes the surgeons tactile perception of useful and familiar tool-to-sclera forces. While the large mass and inertia of eye surgical robot prevents surgeon microtremor, loss of perception of small scleral forces may put the sclera at risk of injury. In this paper, we have applied and compared two different methods to assure the safety of sclera tissue during robot-assisted eye surgery. In the active control method, an adaptive force control strategy is implemented on the Steady-Hand Eye Robot in order to control the magnitude of scleral forces when they exceed safe boundaries. This autonomous force compensation is then compared to a passive force control method in which the surgeon performs manual adjustments in response to the provided audio feedback proportional to the magnitude of sclera force. A pilot study with three users indicate that the active control method is potentially more efficient.

Preliminary study of an RNN-based active interventional robotic system (AIRS) in retinal microsurgery.

Retinal microsurgery requires highly dexterous and precise maneuvering of instruments inserted into the eyeball through the sclerotomy port. During such procedures, the sclera can potentially be injured from extreme tool-to-sclera contact force caused by surgeon's unintentional misoperations. We present an active interventional robotic system to prevent such iatrogenic accidents by enabling the robotic system to actively counteract the surgeon's possible unsafe operations in advance of their occurrence. Relying on a novel force sensing tool to measure and collect scleral forces, we construct a recurrent neural network with long short-term memory unit to oversee surgeon's operation and predict possible unsafe scleral forces up to the next 200ms. We then apply a linear admittance control to actuate the robot to reduce the undesired scleral force. The system is implemented using an existing "steady hand" eye robot platform. The proposed method is evaluated on an artificial eye phantom by performing a "vessel following" mock retinal surgery operation. Empirical validation over multiple trials indicates that the proposed active interventional robotic system could help to reduce the number of unsafe manipulation events. We develop an active interventional robotic system to actively prevent surgeon's unsafe operations in retinal surgery. The result of the evaluation experiments shows that the proposed system can improve the surgeon's performance.

Robotic eyes that express personality

ABSTRACTWe create robots to help people, and we believe that our robots can be more helpful if they look and behave as humans do. People usually prefer help from those whom they like. Some people can be likeable for their personality. A robot, especially an android, may become likeable too if it expresses likeable personality. A robot, or at least an android, needs the ability to express personality such that it can become more likeable, that its help can be more readily accepted. As the first step, we have investigated how the robotic eyes of an android may express personality. Humans have expressive eyes. We postulate that human eyes express personality. We experimented to see whether our test subjects could perceive distinct personalities in an android we programmed to imitate the eye movements of two people. The result showed that the android did express distinct personalities, but not without limitations.

Academic Emotions Affected by Robot Eye Color: An Investigation of Manipulability and Individual-Adaptability

We investigate whether academic emotions are affected by the color of a robot’s eyes in lecture behaviors. In conventional human-robot interaction research on robot lecturers, the emphasis has been on robots assisting or replacing human lecturers. We expanded these ideas and examined whether robots could lecture using one’s behaviors that are impossible for humans. Psychological research has shown that color affects emotions. Because emotion is strongly related to learning, and a framework of emotion control is required. Thus, we considered whether emotions related to the learner’s academic work, called “academic emotions,” can be controlled by the color of a robot’s illuminated eye light. In this paper, we found that the robot’s eye light color affects academic emotions and that the effect can be manipulated and adapted to individuals. Furthermore, the manipulability of academic emotions by color was confirmed in a situation mimicking a real lecture.

Robot Eyes Wide Shut: Understanding Dishonest Anthropomorphism

The goal of this paper is to advance design, policy, and ethics scholarship on how engineers and regulators can protect consumers from deceptive robots and artificial intelligences that exhibit the problem of dishonest anthropomorphism. The analysis expands upon ideas surrounding the principle of honest anthropomorphism originally formulated by Margot Kaminsky, Mathew Ruben, William D. Smart, and Cindy M. Grimm in their groundbreaking Maryland Law Review article, “Averting Robot Eyes.” Applying boundary management theory and philosophical insights into prediction and perception, we create a new taxonomy that identifies fundamental types of dishonest anthropomorphism and pinpoints harms that they can cause. To demonstrate how the taxonomy can be applied as well as clarify the scope of the problems that it can cover, we critically consider a representative series of ethical issues, proposals, and questions concerning whether the principle of honest anthropomorphism has been violated.

Biomimetic pupils for augmenting eye emulation in humanoid robots

Contemporary approaches in the development of humanoid robots continually neglect holistic nuances, particularly in ocular prosthetic design. The standard solid glass and acrylic eye construction techniques implemented in humanoid robot design present the observer with an inaccurate representation of the natural human eye by utilising hardened synthetic materials which prohibit pupillary dynamics. Precise eye emulation is an essential factor in the development of a greater realistic humanoid robot as misrepresentation in ocular form and function will appear distinctly prevalent during proximity face to face communication as eye contact is the primary form of interpersonal communicative processing. This paper explores a new material approach in the development of a more accurate humanoid robotic eye construction by employing natural compounds similar in structure to that found in the organic human eye to replace the traditional glass and acrylic modelling techniques. Furthermore, this paper identifies a gap in current ocular system design as no robotic eye model can accurately replicate all the natural operations of the human iris simultaneously in reaction to light and emotive responsivity. This paper offers a new system design approach to augment future humanoid robot eye construction towards achieving a greater accurate and naturalistic eye emulation.

The eye of sport, robot, and martial arts — visual cognition activities and body action

The eye of sport, robot, and martial arts — visual cognition activities and body action

An ocular biomechanic model for dynamic simulation of different eye movements

Simulating and analysing eye movement is useful for assessing visual system contribution to discomfort with respect to body movements, especially in virtual environments where simulation sickness might occur. It can also be used in the design of eye prosthesis or humanoid robot eye. In this paper, we present two biomechanic ocular models that are easily integrated into the available musculoskeletal models. The model was previously used to simulate eye-head coordination. The models are used to simulate and analyse eye movements. The proposed models are based on physiological and kinematic properties of the human eye. They incorporate an eye-globe, orbital suspension tissues and six muscles with their connective tissues (pulleys). Pulleys were incorporated in rectus and inferior oblique muscles. The two proposed models are the passive pulleys and the active pulleys models. Dynamic simulations of different eye movements, including fixation, saccade and smooth pursuit, are performed to validate both models. The resultant force-length curves of the models were similar to the experimental data. The simulation results show that the proposed models are suitable to generate eye movement simulations with results comparable to other musculoskeletal models. The maximum kinematic root mean square error (RMSE) is 5.68° and 4.35° for the passive and active pulley models, respectively. The analysis of the muscle forces showed realistic muscle activation with increased muscle synergy in the active pulley model.

Dynamics-Based Motion Deblurring Improves the Performance of Optical Character Recognition During Fast Scanning of a Robotic Eye

This paper presents a quantitative evaluation of the dynamics-based deblurring method using an optical character recognition (OCR) technology. Although various image deblurring algorithms have been studied, there has been no standard performance metric; deblurred images have often been evaluated in a qualitative manner. In this study, blurry images containing alphanumeric characters were obtained in the course of rapid motion using a robotic vision system. The obtained blurry images were recovered by the dynamics-based deblurring method. For a quantitative evaluation, OCR rates from the deblurred images by the dynamics-based method were calculated and compared with those by other well-known methods. Experiment results show that the dynamics-based method has the best quantitative results.

Support Vector Machine working on FPGA and the segmentation method of brain MR screening

Field Programmable Gate Arrays (FPGA Field Programmable Gate Array) are devices containing programmable digital blocks and connections and have the capabilities to work in a fast and flexible way. Complex designs can be easily developed with these programmable digital Gates. FPGA’s are small in size and can work as mobile, independent from the computer,at higher speeds.Support vector machine which are important machine learning algorithms is developed to solve the classification problems. In the literature, Support Vector Machines are one of the important machine learning algorithms which are one of the tasks of data mining developed for the solution of the classification problem . Tumor analysis, face recognition, topics such as creating a robotic eye are some of the current, important and difficult problems which researchers are working on intensively in the field of image processing. In tumor analysis performed on the computer, problems of being slow in processing of graphics and pictures and being not mobile are overcome with the FPGA hardware image processing. In this study, it is aimed to find the presence of the tumor region and demonstrating the results of the required analysis by performing more realistic tumor analysis using support vector machines running on the FPGA Hardware. Thus the design of a useful hardware that can be used in the field of health will be carried out. Generally, these processes are done by computers because it is more easier. Therefore, because the number of work is very few which performs the process as described with embedded systems, this project carries originality value. In addition, VHDL (Very High Speed Integrated Circuit Hardware Description Language) , which is one the hardware description languages specific to FPGA will be used. For evaluation of segmentation results Uniformity Measure (UM) it is used. Keywords: Field Programmable Gate Array, FPGA, very high speed ıntegrated circuit hardware description language, vhdl, segmentation, support vector machine

Study on Motion of Sight Line of Communication Robot in Standby State

The crucial role of nonverbal communication by gaze in social interactions has been highlighted. In this study, an eye robot having two degrees-of-freedom was developed as a communication robot, and the motion of the sight line of this robot in a standby state with an absence of communication with people was investigated. We compared and evaluated the impression provided by a robot and a person, who imitated the action of the robot, to observers and demonstrated that both the robot and person provided improved impression if the line of sight is stationary.

Special Issue on Human-Robot Interaction Systems II

Six years have passed since the publication of our Special Issue on Human-Robot Interaction Systems in 2011. Since then, artificial intelligence and robotics have developed rapidly, and the opportunities for human beings and robots to work together have increased. The objective of this special issue’s twelve articles is to activate and expand high-quality research.In the first article, Y. Tamura, T. Akashi, and H. Osumi propose a computational model of robot’s gaze. In the second article, S. Hoshino and K. Uchida propose an interactive motion planner for robot navigation in dynamic environments. In the third article, T. Iio, Y. Yoshikawa, and H. Ishiguro develop a conversational robotic system based on human response. In the fourth article, K. Sakai, F. Dalla Libera, Y. Yoshikawa, and H. Ishiguro propose a method for generating bystander robots’ actions that is based on an analysis of the relative probabilities of human responses to robot actions. In the fifth article, T. Matsumaru and M. Narita present a newly developed support system for learning calligraphy strokes. In the sixth article, E. Tamura, Y. Yamashita, T. Yamashita, E. Sato-Shimokawara, and T. Yamaguchi present a method of driving a car simply by gesturing. In the seventh article, A. Kurosu and T. Hashimoto develop an eye robot with two degrees of freedom. It is intended for use as a communication robot. In the eighth article, T. Hashimoto, Y. Munakata, R. Yamanaka, and A. Kurosu report on a method for retrieving episodic memories. In the ninth article, Y. Nishikawa, Y. Kagawa, and A. Okazaki develop a spiral movement robot for inpatients. In the tenth article, Y. Umesawa, K. Doi, and H. Fujimoto develop an interface device that creates kinaesthetic illusions by inducing vibrations in muscle tendons, vibrations that coordinate with dual-joint movements. In the eleventh article, R. Horio, N. Uchiyama, and S. Sano propose a human-operated biped robot for transporting objects over rough terrain or up steps. In the closing contribution, T. Sakuraba, N. Uchiyama, S. Sano, and T. Sakaguchi present the design of a spring-based regenerative brake, and they verify its effectiveness by driving a system that uses it.We thank the referees for their comprehensive reviews and the staff members of Fuji Technology Press, Ltd. for their encouragement and advice.

Intraocular Snake Integrated with the Steady-Hand Eye Robot for Assisted Retinal Microsurgery.

Due to the confined intraocular space and physical constraints in tool manipulation, snake-like robots have a significant potential for use in retinal microsurgery. By enhancing the dexterity at the tool tip, not only the operable space on the retina can be enlarged, but also the delicate target tissues can be reached at an optimal angle minimizing the damage and making the operation much easier. In this study, we present an improved version of our earlier integrated intraocular snake (IRIS) robot, and combine it with another robotic assistant: the cooperatively controlled Steady-Hand Eye Robot (SHER). SHER is used to drive IRIS close to the retina with precision, while IRIS makes omnidirectional bends by combining its yaw and pitch motions and provides a significantly enhanced intraocular dexterity while holding the sclerotomy port fixed. For precise control of IRIS, its snake-like tip actuation has been characterized through experiments considering both a free tool tip and external loading at the tool tip. The workspace analysis showed ±45° yaw and pitch with excellent repeatability (±1°) despite the highly miniaturized articulated segment length (3 mm) and very thin shaft (Ø 0.9 mm). Our preliminary experiments in an artificial eye model have shown feasibility in reaching targets requiring bends up to 55° accurately.

A Novel Deep Neural Network that Uses Space-Time Features for Tracking and Recognizing a Moving Object

Abstract This work proposes a deep neural net (DNN) that accomplishes the reliable visual recognition of a chosen object captured with a webcam and moving in a 3D space. Autoencoding and substitutional reality are used to train a shallow net until it achieves zero tracking error in a discrete ambient. This trained individual is set to work in a real world closed loop system where images coming from a webcam produce displacement information for a moving region of interest (ROI) inside the own image. This loop gives rise to an emergent tracking behavior which creates a self-maintain flow of compressed space-time data. Next, short term memory elements are set to play a key role by creating new representations in terms of a space-time matrix. The obtained representations are delivery as input to a second shallow network which acts as “recognizer”. A noise balanced learning method is used to fast train the recognizer with real-world images, giving rise to a simple and yet powerful robotic eye, with a slender neural processor that vigorously tracks and recognizes the chosen object. The system has been tested with real images in real time.

Robotic Vitreoretinal Surgery

To review the current literature on robotic assistance for ophthalmic surgery, especially vitreoretinal procedures. MEDLINE, Embase, and Web of Science databases were searched from inception to August, 2016, for articles relevant to the review topic. Queries included combinations of the terms: robotic eye surgery, ophthalmology, and vitreoretinal. In ophthalmology, proof-of-concept papers have shown the feasibility of performing many delicate anterior segment and vitreoretinal surgical procedures accurately with robotic assistance. Multiple surgical platforms have been designed and tested in animal eyes and phantom models. These platforms have the capability to measure forces generated and velocities of different surgical movements. "Smart" instruments have been designed to improve certain tasks such as membrane peeling and retinal vessel cannulations. Ophthalmic surgery, particularly vitreoretinal surgery, might have reached the limits of human physiologic performance. Robotic assistance can help overcome biologic limitations and improve our surgical performance. Clinical studies of robotic-assisted surgeries are needed to determine safety and feasibility of using this technology in patients.

Modeling Active Vision During Smooth Pursuit of a Robotic Eye

The computational framework, basedon the conformal camera, is developedfor processingvisual information duringsmooth pursuit movements of a robotic eye. During smooth pursuit, the image of the tracked object remains nearly stationary while the image of a stationary background sweeps across the image plane of the camera. The background's image transformation derived in the second-order approximation enable the anticipation of the perceptual outcome of the camera pursuit. This can be used to support the correct visual information of the moving object in front of the stationary background, These results complement the author's previous study on the predictive image processingfor visual stability duringthe conformalcameramovementsresembling primate's saccadiceye rotations. The visual information processing algorithms that can support visual stability during smooth pursuit and saccadic movements of an anthropomorphic robotic camera are neededfor an autonomousrobot efficient interactions with the real world, in real time.

Traffic Road Sign Detection and Recognition for Automotive Vehicles

Traffic road sign detection and recognition is important to transport system with a robotic eyes or camera while driving in the road. This paper presents and overview the traffic road sign detection and recognition, we developed and implemented the procedure to extract the road sign from a natural complex image. The main objective of this paper is to design and construct a computer based system which can automatically detect the direction of the road sign. This paper is based upon a major approach to detect the direction. In this paper, we will demonstrate the basic idea of how detect the area and extract it. This system will play an important role for the detection purpose of specific domains like island, schools, traffic sign, universities, hospitals, offices etc.

Dynamics-based motion de-blurring for a PZT-driven, compliant camera orientation mechanism

This paper proposes a method for removing motion blur from images captured by a fast-moving robot eye. Existing image techniques focused on recovering blurry images due to camera shake with long exposure time. In addition, previous studies relied solely on properties of the images or used external sensors to estimate a blur kernel, or point spread function (PSF). This paper focuses on estimating a latent image from the blur images taken by the robotic camera orientation system. A PZT-driven, compliant camera orientation system was employed to demonstrate the effectiveness of this approach. Discrete switching commands were given to the robotic system to create a rapid point-to-point motion while suppressing the vibration with a faster response. The blurry images were obtained when the robotic system created a rapid point-to-point motion, like human saccadic motion. This paper proposes a method for estimating the PSF in knowledge of system dynamics and input commands, resulting in a faster estimation. The proposed method was investigated under various motion conditions using the single-degree-of-freedom camera orientation system to verify the effectiveness and was compared with other approaches quantitatively and qualitatively. The experiment results show that overall the performance metric of the proposed method was 27.77% better than conventional methods. The computation time of the proposed method was 50 times faster than that of conventional methods.

Deep Level Emotion Understanding Using Customized Knowledge for Human-Robot Communication

In this study, a method for acquiring deep level emotion understanding is proposed to facilitate better human-robot communication, where customized learning knowledge of an observed agent (human or robot) is used with the observed input information from a Kinect sensor device. It aims to obtain agentdependent emotion understanding by utilizing special customized knowledge of the agent rather than ordinary surface level emotion understanding that uses visual/acoustic/distance information without any customized knowledge. In the experiment employing special demonstration scenarios where a company employee’s emotion is understood by a secretary eye robot equipped with a Kinect sensor device, it is confirmed that the proposed method provides deep level emotion understanding that is different from ordinary surface level emotion understanding. The proposal is being planned to be applied to a part of the emotion understanding module in the demonstration experiments of an ongoing robotics research project titled “Multi-Agent Fuzzy Atmosfield.”