Robotic Eye Research Articles

Master-Slave Eye Robot System Driven by Ultrasonic Motors for Camera Operation

Using ultrasonic motors, we propose a compact and highly responsible eye robot that is moved by transmitting a operator eye movement to the robot's eye, to obtain the visual information of the environment surrounding the robot. In addition, we propose a graphical user interface (GUI) that makes the robot operation intuitive.

1A1-A01 視線情報を入力とする高応答眼球ロボットシステム

In this paper, we propose an eye robot system with master slave control using operator-gaze tracking data, which can produce a high response level such as that of the human eyes. The operators can move a remote robot camera by only moving their eye (master) in front of an eye tracking apparatus and obtain visual information of the environment surrounding the robot's eye (slave). The use of ultrasonic motors has enabled the high responsiveness of the robot eye and the compactness of the robot body. In addition, we propose a graphical user interface (GUI) that makes the robot operation intuitive.

Toward robotically assisted membrane peeling with 3-DOF distal force sensing in retinal microsurgery.

Retinal microsurgery requires steady and precise manipulation of delicate eye tissues in a very small space. Physiological hand tremor and lack of force sensing are among the main technical challenges, limiting surgical performance. We present a system that consists of the cooperatively controlled Steady-Hand Eye Robot and a miniaturized 3-DOF force sensing instrument to address these limitations. While the robot can effectively suppress hand tremor, enable steady and precise tissue manipulation, the force sensing instrument can provide three dimensional force measurements at the tool tip with submillinewton resolution. Auditory sensory substitution is used to give the user real time force information. Evaluation experiments are conducted using artificial and biological membrane peeling phantoms. Experimental results show that the robotic assistance and force-to-audio sensory substitution can effectively control the magnitude of the tool-to-tissue force. The direction profiles of the membrane peeling forces reflect the different delaminating strategies for different membrane phantoms.

Design and Control of 3-DoF Spherical Parallel Mechanism Robot Eyes Inspired by the Binocular Vestibule-ocular Reflex

With the wide application of robot in unstructured environment, cognizing and understanding environmental information accurately is the key for robots to complete the task. Thus, visual stability problems in these robot control systems have become important design issues. In this paper, to solve the problem of vision instability caused by attitude variation of the robot working under bumpy environment, a bionic eyes system of active compensation for robot visual error is proposed inspired by the binocular VOR. The bionic vision system is developed based on features of oculomotor behaviors and bionic control algorithm of the binocular VOR. According to the behaviors features of eye movement, the bionic eye mechanism is designed by using a 3-DoF spherical parallel mechanism(SPM). Meanwhile, to improve the performance, the mechanism parameters of SPM are optimized by utilizing the worst-case performance index as the optimizing target in required workspace. An adaptive control model of the binocular VOR is established on neural control mechanisms of eye movement, and the model is used as the bionic control algorithm to drive and control the eye mechanism. The results indicate that the model can actively compensate the visual errors from postural changes of robot. Physical robot experiments also show that the system is robust even with bumpy environment.

Learning robotic eye–arm–hand coordination from human demonstration: a coupled dynamical systems approach

We investigate the role of obstacle avoidance in visually guided reaching and grasping movements. We report on a human study in which subjects performed prehensile motion with obstacle avoidance where the position of the obstacle was systematically varied across trials. These experiments suggest that reaching with obstacle avoidance is organized in a sequential manner, where the obstacle acts as an intermediary target. Furthermore, we demonstrate that the notion of workspace travelled by the hand is embedded explicitly in a forward planning scheme, which is actively involved in detecting obstacles on the way when performing reaching. We find that the gaze proactively coordinates the pattern of eye-arm motion during obstacle avoidance. This study provides also a quantitative assessment of the coupling between the eye-arm-hand motion. We show that the coupling follows regular phase dependencies and is unaltered during obstacle avoidance. These observations provide a basis for the design of a computational model. Our controller extends the coupled dynamical systems framework and provides fast and synchronous control of the eyes, the arm and the hand within a single and compact framework, mimicking similar control system found in humans. We validate our model for visuomotor control of a humanoid robot.

The Robot Eye Witness

Eyewitness reporting, particularly on warfare, environmental disasters, and other dramatic news events, might involve considerable journalistic risk taking. This study investigates in what ways the innovation of drones for journalistic purposes might extend and improve the use of eyewitness accounts, especially in areas or fields where human coverage would be impossible or too dangerous. It examines the interrelationships among new technologies, journalistic innovation, and the audience’s quest for constant visual enforcements, drawing on diffusion theory and social constructivist models for process innovation. It argues that the emergent genre of drone journalism might come to exemplify a “disruptive innovation,” an innovation that has emerged accidentally, but disrupts existing conceptions of visual journalism, and subsequently contributes to the creation of new markets and value networks.

Analysis of Electric Machine Charateristics for Robot Eyes Using Analytical Electromagnetic Field Computation Method

This paper proposes an electromagnetic field computation method using an interpolation method with sigmoid functions. This method is useful for approximating and developing a 3-D electromagnetic field model. Particularly, the authors have been researching a spherical motor for a robotic eye. The spherical motor has a spherical air-gap unlike the radial air-gap motor or the axial air-gap motor. The spherical motor proposed in this paper realizes three degrees-of-freedom movements with only one motor. As the spherical motor has a complicated coil structure, the 3-D FEM would be essential for calculating torque while the rotor is tilted. However, 3-D FEM requires much computation time and a huge memory capacity to store the results. The electromagnetic field is rapidly calculated using an interpolation torque function proposed in this paper. The improvement of the spherical motor due to our method is verified through experiment, the results of which are compared with simulation data.

Recognized Face Tracking for CONBE Robot

In our research, we develop the robot to combine with the consciousness and behavior which is the conscious and behavior robot (CONBE). In this paper we present the basis system in the CONBE robot that consist of the two main sections as the face recognition and robot gaze tracking. This system uses the face recognition using Fisherfaces that also refer to linear discriminant analysis (LDA). For the robot gaze tracking, we implement with the head of the CONBE robot that has a camera, two displays used for robot eyes and the actuators of 2 degrees of freedom. The actuators use to control the robot gaze for tracking the human face which the robot can recognize. In this paper, we experiment and present the results of the face recognition system and the tracking control system such as the tracking error and time.

Designing robot eyes for communicating gaze

Abstract—Human eyes not only serve the function of enabling us “to see” something, but also perform the vital role of allowing us “to show” our gaze for non-verbal communication, such as through establishing eye contact and joint attention. The eyes of service robots should therefore also perform both of these functions. Moreover, they should be friendly in appearance so that humans may feel comfortable with the robots. Therefore we maintain that it is important to consider gaze communication capability and friendliness in designing the appearance of robot eyes. In this paper, we propose a new robot face with rear-projected eyes for changing their appearance while simultaneously realizing the showing of gaze by incorporating stereo cameras. Additionally, we examine which shape of robot eyes is most suitable for gaze reading and gives the friendliest impression, through experiments where we altered the shape and iris size of robot eyes. Keywords: Gaze reading; facial design; projector camera system

Sanal Gerçeklik Ortamları ve Uygulamalar: Spor ve Sanal Ortam Göstergeleri

Günümüzde gerçeklik imgesi ile sanallık imgesi sık tartışılan kavramlar olmuştur. Gerçeğin yansıması ya da “gerçekmiş” gibi yapılarak çeşitli uygulamalar gerçekleştirilmesi çoğu alanda gözlemlenmektedir. Gerçeğin göstergeleri ile “gerçekmiş” gibi aktarılan “şeylerin” göstergeleri farklı mıdır? Bu göstergelerin algılanmaları ve “gerçekmiş” gibi aktarılan dünyaların gerçek dünya göstergeleri ile bütünleşmesi ise gözlenen bir olgudur. Bu bağlamda, yeni medya teknolojileri ile yaratılan sanal ortamlarda yaşanan deneyimler, oluşturulan sanal dünyalar gerçek-gerçek olmayan karşıtlığını gözler önüne serse de, bu tür sanal dünyalar kimi alanlardaki uygulamalarla yaşamı da kolaylaştırmaktadır. Çalışmada örneğin; “robot göz” ya da simülasyon uygulamaları aracılığıyla teknolojik gelişmelerle yaratılan ve gerçek-sanal sınırını çizen, kimi zaman da sınırı aşan ortamlar, “spor etkinlikleri” bağlamında ele alınacaktır. Sporcuların performansları ve teknik özellikleri ile ilgili göstergeler, sanal ortama nasıl aktarıldığı; bu göstergeler anlamlandırıldıktan sonra, “gerçekte” nasıl bir iyileştirme ortamı yaratıldığı sorusunun yanıtı sorgulanmaya çalışılacaktır.

Adapting Multi-Robot Behavior to Communication Atmosphere in Humans-Robots Interaction Using Fuzzy Production Rule Based Friend-Q Learning

A behavior adaptation mechanism in humans-robots interaction is proposed to adjust robots’ behavior to communication atmosphere, where fuzzy production rule based friend-Q learning (FPRFQ) is introduced. It aims to shorten the response time of robots and decrease the social distance between humans and robots to realize the smooth communication of robots and humans. Experiments on robots/humans interaction are performed in a virtual communication atmosphere environment. Results show that robots adapt well by saving 44 and 482 learning steps compared to that by friend-Q learning (FQ) and independent learning (IL), respectively; additionally, the distance between human-generated atmosphere and robot-generated atmosphere is 3 times and 10 times shorter than the FQ and the IL, respectively. The proposed behavior adaptation mechanism is also applied to robots’ eye movement in the developing humans-robots interaction system, calledmascot robot system, and basic experimental results are shown in home party scenario with five eye robots and four humans.

Concept of Fuzzy Atmosfield for Representing Communication Atmosphere and its Application to Humans-Robots Interaction

The concept of Fuzzy Atmosfield (FA) is proposed to represent a communication atmosphere in society consisting of multiple individuals such as humans and/or robots, where the FA is characterized by 3D fuzzy cubic space with “Friendly-Hostile” (FH), “Lively-Calm” (LC), and “Casual-Formal” (CF) axes, and each state of the FA is visualized using shape-colorlength graphics. It is targeted to be a tool for identifying an atmosphere using quantitative analysis and graphical representation. By a humans-robots interaction experiment in which the FA is used to represent the real-time atmosphere created by four humans and five eye robots in a home party scenario, it shows that Pearson’s correlation coefficient values of 0.92, 0.86, and 0.72 for the FH, LC, and CF axes, respectively, indicate the correspondence between the proposed FA and results of questionnaires, and that subjective estimation of graphical representation of the FA achieves 84% accuracy for shape, 76% for color, and 58% for length. The FA is being extended to the representation of complex atmosphere generated by humans, robots, and background music, and part of results is also shown.

2A1-L03 眼科手術支援ロボットにおける正確性・安全性向上を目的とした動作制限・動作自動補正手法の開発(手術支援ロボティクス・メカトロニクス(1))

2A1-L03 眼科手術支援ロボットにおける正確性・安全性向上を目的とした動作制限・動作自動補正手法の開発(手術支援ロボティクス・メカトロニクス(1))

Soft-computing-based emotional expression mechanism for game of computer Go

The game of Go is considered one of the most complicated games in the world. One Go game is divided into three stages: the opening, the middle, and the ending stages. Millions of people regularly play Go in countries around the world. The game is played by two players. One is White and another is Black. The players alternate placing one of their stones on an empty intersection of a square grid-patterned game board. The player with more territory wins the game. This paper proposes a soft-computing-based emotional expression mechanism and applies it to the game of computer Go to make Go beginners enjoy watching Go game and keep their tension on the game. First, the knowledge base and rule base of the proposed mechanism are defined by following the standards of the fuzzy markup language. The soft-computing mechanism for Go regional alarm level is responsible for showing the inferred regional alarm level to Go beginners. Based on the inferred board situation, the fuzzy inference mechanisms for emotional pleasure and arousal are responsible for inferring the pleasure degree and arousal degree, respectively. An emotional expression mapping mechanism maps the inferred degree of pleasure and degree of arousal into the emotional expression of the eye robot. The protocol transmission mechanism finally sends the pre-defined protocol to the eye robot via universal serial bus interface to make the eye robot express its emotional motion. From the experimental results, it shows that the eye robot can support Go beginners to have fun and retain their tension while watching or playing a game of Go.

Public events at Fall Meeting 2012

Three much‐anticipated events at Fall Meeting are AGU's family science programs, which take place the Sunday before Fall Meeting begins. Beginning at noon on 2 December, the public lecture will be given by Michael Meyer, John Grotzinger, and Rebecca Williams. These three NASA scientists are working with the rover Curiosity, which is currently exploring Mars. They will engage the public in a discussion of Mars exploration and the latest activities of the most sophisticated explorer ever sent to another planet. The panelists will discuss the hopes and excitement of exploring Mars through a robot's eyes, nose, taste, and touch.

Spine Bull’s-Eye Robot Guidewire Placement With Pedicle Standard Axis View for Thoracic and Lumbar Pedicle Screw Fixation

Prospective. To explore the feasibility and accuracy of thoracic and lumbar pedicle guidewire placement using a newly developed Spine Bull's-Eye Robot with the pedicle standard axis view (PSAV). Placement of pedicle screws in the thoracic spine is considered to be particularly risky surgery and is associated with a relatively high rate of screw misplacement. Use of robot-assisted surgery may improve the accuracy of pedicle screw placement. The study included 204 pedicles (T1-L5) from 6 human vertebrae and 5 pedicles from 3 patients. Computed tomographic reconstruction of target vertebrae was performed for surgical planning. Using the PSAV, a guidewire (1.2 mm diameter, 13.5 cm length) was inserted into the "bull's-eye" (the center of the pedicle circular projection) using the remotely operated Spine Bull's-Eye Robot. After surgery, vertebral body axial and lateral radiographs, and computed tomographic scans were used to assess the guidewire trajectory. Planned and actual (postoperative) measures for transverse section angle, sagittal section angle, and the superior/inferior and medial/lateral vertical distances from the entry point were compared. The PSAV was acquired clearly in 203 of 209 (97.1%) pedicles; the guidewire was successfully inserted in each of these pedicles. There were no significant differences between any of the planned and actual (postoperative) measures. All of the pedicles for which guidewire placement was abandoned (because of lack of bull's-eye resolution in the PSAV) were small upper thoracic specimens, exhibiting evidence of decalcification, presumably because of preservation procedures. None of the patients experienced complications associated with guidewire placement. Use of the Spine Bull's-Eye Robot with PSAV seems be an accurate and feasible approach for guidewire (and thus pedicle screw) placement in the thoracic and lumbar spine.

햅틱스 시스템용 3D 재구성을 위한 LoG 방법과 DoG 방법의 성능 분석

본 연구의 목적은 "로봇의 시각"과 "로봇의 촉각"을 대체할 수 있는 스테레오 비전 기반 햅틱스 시스템에서 가장 적합하고 효과적인 3D 재구성(3D reconstruction) 방법을 제안하는 것이다. 삼차원 영상에 대하여 정확하게 촉감을 전달하려면 스테레오 영상에서 사물의 깊이 정보와 사물의 경계면에 대한 정확한 정보가 필요하다. 본 연구에서는 스테레오 영상에서 사물의 깊이 정보를 정확하게 얻기 위하여 전통적인 스테레오 정합과정에 경계면 추출 방법인 LoG(Laplacian of Gaussian) 방법과 DoG(Difference of Gaussian) 방법을 혼합적용하여 3D 영상을 재구성한 결과를 제시한다. 또한 어떤 방법이 햅틱 렌더링을 적용하는데 유용한 지 검증하기 위하여 연산 시간 및 오차 분석 실험을 수행한 결과, 본 연구처럼 비주얼 렌더링에 햅틱 렌더링을 추가하여 사용하는 경우에는 잡음 감소와 경계면 추출 성능이 더 우수한 DoG 방법이 더 효율적인 것으로 판단되었다. 본 논문에서 제안하는 스테레오 비전 기반 햅틱스 시스템을 위한 3D 재구성 방법은 이동형 정찰 로봇의 성능을 높이는 연구 등 여러 산업 분야와 군사 분야에 응용이 가능할 것이다. The objective of this study is to propose an efficient 3D reconstruction method for developing a stereo-vision-based haptics system which can replace "robotic eyes" and "robotic touch." The haptic rendering for 3D images requires to capture depth information and edge information of stereo images. This paper proposes the 3D reconstruction methods using LoG(Laplacian of Gaussian) algorithm and DoG(Difference of Gaussian) algorithm for edge detection in addition to the basic 3D depth extraction method for better haptic rendering. Also, some experiments are performed for evaluating the CPU time and the error rates of those methods. The experimental results lead us to conclude that the DoG method is more efficient for haptic rendering. This paper may contribute to investigate the effective methods for 3D image reconstruction such as in improving the performance of mobile patrol robots.

Study on Target Tracking Technology in Control of Robot Visual Servo

Robot visual servo system is an important field of robotics research. The paper regards CCD camera as the eye of robot, with the location information, using Kalman filter, predicts the tracking of moving target in the field of vision, achieves the target tracking and proves the effectiveness of this method by experiment.

1A1-J01 Study of Telexistence LXVIII : Reaching a remote object with 6DOF point of view and arm endpoint accuracy(Robot Hand Mechanism and Grasping Strategy(1))

In Telexistence, point of vision through robot eyes and posture of the robot's hand should be in a close correlation in order to feel the extended body sensation and transfer realistic haptic sensation. However, present Telexistence robots mostly operate its torso in less than 4 DOF, thus it is difficult to map the vector between the point of vision to arm endpoint in operator to the same vector of slave robot. In order to address this issue we designed and developed TELESAR V, with a 5 DOF torso and a 7 DOF arm attached to it so that the slave robot can have 6 DOF point of vision and 6 DOF arm endpoint accuracy with operator's posture. This structure enables the operator to freely move in his space and experience his extended bodily border while operating the robot and perform manipulatory task remotely. In addition ability to transfer the fingertip haptic and temperature sensation, operator can feel his bodily consciousness is extended to the robot.

Computer-Simulated Display to Advance the Understanding of Perceptual Motor Skills

Purpose: To review past attempts, current innovations, and future goals of robotic eye surgery. Methods: A Medline literature search using the words “robot” and “ophthalmology” was performed to identify all relevant literature. Pertinent articles were reviewed and content summarized based on context. Results: Purported potential benefits of robotic-assisted eye surgery include improved precision, reduced tremor, amplified scale of motion, and the potential of automation and telesurgical operation. Several investigators have created devices capable of performing individual intraocular tasks, and efforts are underway to develop platforms designed to allow completion of entire ophthalmic procedures. Conclusion: Although obstacles such as cost and availability exist, prior successes and future benefits of robotic eye surgery are promising reasons for the continuation of research efforts.