Haptic Teleoperation Research Articles

A psychophysical evaluation of haptic controllers: viscosity perception of soft environments

SUMMARYIn this paper, human viscosity perception in haptic teleoperation systems is thoroughly analyzed. An accurate perception of viscoelastic environmental properties such as viscosity is a critical ability in several contexts, such as telesurgery, telerehabilitation, telemedicine, and soft-tissue interaction. We study and compare the ability to perceive viscosity from the standpoint of detection and discrimination using several relevant control methods for the teleoperator. The perception-based method, which was proposed by the authors to enhance the operator's kinesthetic perception, is compared with the conventional transparency-based control method for the teleoperation system. The fidelity-based method, which is a primary method among perception-centered control schemes in teleoperation, is also studied. We also examine the necessity and impact of the remote-site force information for each of the methods. The comparison is based on a series of psychophysical experiments measuring absolute threshold and just noticeable difference for all conditions. The results clearly show that the perception-based method enhances both detection and discrimination abilities compare with other control methods. The results further show that the fidelity-based method confers a better discrimination ability than the transparency-based method, although this is not true with respect to detection ability. In addition, we show that force information improves viscosity detection for all control methods, as predicted from previous theoretical analysis, but improves the discrimination threshold only for the perception-based method.

A Study on the FPGA Implementation of the Bilateral Control Algorithm Towards Haptic Teleoperation

This paper presents the FPGA implementation of sliding mode control algorithm for bilateral teleoperation, such that, the problem of haptic teleoperation is addressed. The presented study improves haptic fidelity by widening the control bandwidth. For wide control bandwidth, short control periods as well as short sampling periods are required that was achieved by the FPGA. The presented FPGA design methodology applies basic optimization methods in order to meet the required control period as well as the required hardware resource consumption. The circuit specification was performed by the high-level programing language LabVIEW using the fixed-point data type. Hence, short design times for producing the FPGA logic circuit can be achieved. The proposed FPGA-based bilateral teleoperation was validated by master-slave experimental device.

Vision-based Autonomous Control of a Quadrotor UAV using an Onboard RGB-D Camera and its Application to Haptic Teleoperation

In this paper we present the design of a platform for autonomous navigation of a quadrotor UAV based on RGB-D technology. The proposed platform can safely navigate in an unknown environment while self-stabilization is done relying only on its own sensor perception. We developed an estimation system based on the integration of IMU and RGB-D measurements in order to estimate the velocity of the quadrotor in its body frame. Experimental tests conducted as teleoperation experiments show the effectiveness of our approach in an unstructured environment.

Adaptive Haptic Control for Telerobotics Transitioning Between Free, Soft, and Hard Environments

This paper presents an adaptive haptic control for a one degree-of-freedom master-slave teleoperated device. The aim is to reduce excessive collision forces that occur when there are significant time delays in master-slave communication. The control design also allows the operator to move the slave in free space and in a soft medium. Previous approaches to haptic teleoperation typically design for either movement in a medium or constrained contact with a solid surface; then, it is up to the operator to avoid collisions or precisely anticipate collisions. The proposed control runs on the slave side inner loop, with no time delay, and tracks commanded forces from the outer loop. A Lyapunov-stable backstepping-with-tuning-functions design provides a way to ensure smooth forces are applied that guarantee stability in the presence of unmodeled environmental stiffness and viscosity. Experiments using a Phantom hand controller interacting with simulated environment show that collision forces are substantially reduced compared to two other control methods. In collision-free operation, the performance is comparable to other methods.

Human-Oriented Control for Haptic Teleoperation

Haptic teleoperation enables the human to perform manipulation tasks in distant, scaled, hazardous, or inaccessible environments. The human closes the control loop sending haptic command signals to and receiving haptic feedback signals from the remote teleoperator. The main research question is how to design the control such that human decision making and action is supported in the best possible way while ensuring robust operation of the system. The human in the loop induces two major challenges for control design: 1) the dynamics of the human operator and the teleoperation system are tightly coupled, i.e., stability of the overall system is affected by the human operator dynamics; and 2) the performance of the teleoperation system is subjectively evaluated by the human, which typically means that standard control performance metrics are not suitable. This paper discusses recent control design successes in the area of haptic teleoperation. In particular, the importance and need of dynamic human haptic closed-loop behavior models and human perception models for the further improvement of haptic teleoperation systems is highlighted and discussed for real-world problem domains.

Haptic Teleoperation for 3-D Microassembly of Spherical Objects

In this paper, teleoperated 3-D microassembly of spherical objects with haptic feedback is presented. A dual-tip gripper controlled through a haptic interface is used to pick-and-place microspheres (diameter: 4-6 μm). The proposed approach to align the gripper with the spheres is based on a user-driven exploration of the object to be manipulated. The haptic feedback is based on amplitude measurements from cantilevers in dynamic mode. That is, the operator perceives the contact while freely exploring the manipulation area. The data recorded during this exploration are processed online and generate a virtual guide to pull the user to the optimum contact point, allowing correct positioning of the dual tips. A preliminary scan is not necessary to compute the haptic feedback, which increases the intuitiveness of our system. For the pick-and-place operation, two haptic feedback schemes are proposed to either provide users with information about microscale interactions occurring during the operation, or to assist them while performing the task. As experimental validation, a two-layer pyramid composed of four microspheres is built in ambient conditions.

Robust Haptic Teleoperation by FPGA

Abstract This paper describes a robust algorithm for haptic-based teleoperation. The robust control algorithm was derived by sliding mode control design approach. Such control assures robustness to the disturbances and model uncertainties. Furthermore, the presented design approach guaranties chattering-free performance. Moreover, it is also easy to implement, since the detailed knowledge of the model is not required. The control has been implemented by FPGA in order to realize high control rate which is strongly required for haptic teleoperation. The control algorithm was validated by the 2-DoF laboratory haptic experimental system with linear motors with built-in Hall sensors; therefore no external sensor is required. The experiments validated the proposed control algorithm regarding the robustness and haptic performance.

Disturbance observer-based control of non-linear haptic teleoperation systems

Teleoperation systems are subject to different types of disturbances. Such disturbances, when unaccounted for, may cause poor performance and even instability of the teleoperation system. This study presents a novel non-linear bilateral control scheme using the concept of ‘disturbance observer-based control’ for non-linear teleoperation systems. Lumping the effects of dynamic uncertainties and external disturbances into a single disturbance term enables us to design a disturbance observer to suppress these disturbances and alleviate their adverse effects on the teleoperation system. A disturbance observer-based control law is proposed for non-linear teleoperation systems which will guarantee global asymptotic force tracking and global exponential position and disturbance tracking when the bilateral teleoperation system is experiencing slow-varying disturbances. In the case of fast-varying disturbances, the tracking errors are shown to be globally uniformly ultimately bounded, with an ultimate bound that can be made as small as desired using the design parameters. Simulations are presented to show the effectiveness of the proposed approach.

Self-Organizing Fuzzy Haptic Teleoperation of Mobile Robot Using Sparse Sonar Data

Mobile robot teleoperation has been used in many areas of industrial automation, such as explosives disposal, nuclear waste manipulation, freight handling, or transportation. Here, the commonly provided audio-visual feedback often resulted in an inadequate perception of the remote environment. Haptic augmentation was shown to improve and positively enhance the control of the mobile robot. This paper presents novel self-organizing fuzzy adaptive mapping algorithm (SOFAMap) for a haptic teleoperation of mobile robots. The SOFAMap algorithm was specifically developed for a mobile robot with a rotational sonar sensory system, constituting an alternative to a traditionally used multisonar array. The main contributions of this paper are the following: 1) development of a specific self-organizing environment mapping structure inspired by the growing neural gas algorithm; 2) incorporating a fuzzy controller into the algorithm to adapt to robot's motion; and 3) resolving typical issues such as sensor noise, communication time delay, and low sampling rate. The experimental testing was performed in both virtual environment and on a real robotic platform, consisting of a Lego NXT mobile robot and a Novint Falcon 3-DOF haptic interface. The results showed that a high-fidelity haptic feedback can be successfully generated using a simpler and more affordable rotational sonar sensory system, as opposed to the typical multisonar array. Furthermore, it was demonstrated that the SOFAMap algorithm improves the operator's awareness of unstructured environments, making it applicable to wide range of mobile robot teleoperation systems.

Performance Analysis of a Haptic Telemanipulation Task under Time Delay

There is ample research on the effect of haptic teleoperation under delayed communication channels in terms of stability and system performance. Little attention, however, has been paid to the effect of delayed force feedback on users' task performance and whether force feedback is beneficial under significant communication delays. This paper investigates whether force feedback improves user's task performance in delayed teleoperation. We study peg-in-the-hole insertion/retraction, dexterous manipulation tasks involving high degrees of freedom and high forces at certain points during task execution. A user study involving unilateral (without force feedback), bilateral (with force feedback) and graphical feedback teleoperation under various delays is presented. We observed that for all feedback modalities, task completion times increase as delay increases. Haptic feedback helps reduce contact forces and the occurrence of large robot/environment forces. Furthermore, graphical feedback helps users maintain the lowest range of forces at the cost of higher task completion times. With users mindful of minimizing contact forces, haptic/graphical feedback causes the task to take more time than unilateral control. Therefore, when short completion times are crucial given a tolerance for larger forces, force feedback only serves to increase the time required to perform the task; thus, unilateral control may be sufficient.

Haptic tele-operation system control design for the ultrasound task: A loop-shaping approach

This paper introduces a step-by-step frequency domain loop-shaping procedure for tele-operation system controllers, in particular for the three-channel Environment Force Compensation (EFC) control architecture. The framework is explained by designing a controller for a tele-operated 5-DOF probe for ultrasound echo-cardiography. Models of the tele-operation system components are created and performance requirements are specified. A control design procedure is proposed using a generic tele-operation block-scheme. In this procedure, the choice for the EFC control architecture is underpinned and guidelines for the loop-shaping of this tele-operation controller for performance and stability robustness are given. The designed controllers are tuned using the introduced procedure. Passivity, performance and stability robustness of the newly designed controller are evaluated using experiments.

A survey of environment-, operator-, and task-adapted controllers for teleoperation systems

Bilateral haptic teleoperation systems allow humans to perform complex tasks in a remote or inaccessible environment, while providing haptic feedback to the human operator. The incorporation of online gained environment-, operator-, or task-specific (EOT) information in the controller structure can lead to significant improvements in robustness, task performance, feeling of presence, or fidelity without compromising stability. This article provides a classification as well as a survey of approaches, called EOT-adapted controllers, which have been developed in this area. A discussion of improvements and requirements is provided for each method. The performed analysis indicates that several methods require the usage of additional sensors or are based on accurate model assumptions. The benefit of EOT-adapted controllers is mostly application-dependent, as each method focuses on the improvement of a specific aspect like coping with time delay or avoiding forbidden regions.

Haptic Teleoperation of Humanoid Robot Hand Using Three-Dimensional Force Feedback

Abstract This paper presents experimental results of the haptic teleoperation of humanoid robot hand using three-dimensional force feedback. The humanoid robot hand called KH Hand, which has a thumb and four-fingers with 15 DOF, is teleoperated through a multi-fingered haptic interface robot called HIRO III. This haptic teleoperation system provides the touch sensation of three-dimensional force to the user's fingertips. Experiments show high performance in three-dimensional manipulation.

大きさの異なるマスタ・スレーブのバイラテラル制御

Bilateral control is one of control methods used to achieve haptic teleoperation between remote places. If the bilateral system is utilized for small or large environments such as those in brain surgery and space development, it is necessary that the system takes into account scaling factors between the master and slave. In this paper, we propose the bilateral control of master-slave system with different scaling on the basis of reproducibility and operationality which are one of the evaluation indices of bilateral control system. The proposed control system contains two kinds of scaling factors. One of them is scaling factors of position and force responses that are used to determine the scaling ratio between the master and the slave. The other one is used to adjust the control gains at the master and the slave, respectively. In this paper, some scaling types depending on the nominal masses of the systems are proposed. In order to compare the scaling methods, the analysis results of the proposed method on reproducibility and operationality are shown. Moreover, experimental results confirm the validity of the proposed method.

High-Fidelity Bilateral Teleoperation Systems and the Effect of Multimodal Haptics

In master-slave teleoperation applications that deal with a delicate and sensitive environment, it is important to provide haptic feedback of slave/environment interactions to the user's hand as it improves task performance and teleoperation transparency (fidelity), which is the extent of telepresence of the remote environment available to the user through the master-slave system. For haptic teleoperation, in addition to a haptics-capable master interface, often one or more force sensors are also used, which warrant new bilateral control architectures while increasing the cost and the complexity of the teleoperation system. In this paper, we investigate the added benefits of using force sensors that measure hand/master and slave/environment interactions and of utilizing local feedback loops on the teleoperation transparency. We compare the two-channel and the four-channel bilateral control systems in terms of stability and transparency, and study the stability and performance robustness of the four-channel method against nonidealities that arise during bilateral control implementation, which include master-slave communication latency and changes in the environment dynamics. The next issue addressed in the paper deals with the case where the master interface is not haptics capable, but the slave is equipped with a force sensor. In the context of robotics-assisted soft-tissue surgical applications, we explore through human factors experiments whether slave/environment force measurements can be of any help with regard to improving task performance. The last problem we study is whether slave/environment force information, with and without haptic capability in the master interface, can help improve outcomes under degraded visual conditions.

PASSIVE HAPTIC CONTROL DURING TELEOPERATION

The basis of haptic teleoperation focuses on augmenting a user's ability to perform a task in a remote environment. Using an energetically passive haptic master assures that the energy can only be redirected or removed from the human-manipulator pair, guaranteeing the user's safety. However using a passive master complicates the display of haptic forces due to the device's limitations. The master device cannot be controlled arbitrarily, but instead can only resist the motion of the human user. This research focuses on the use of a planar passive haptic device actuated via three magneto-rheological (MR) brakes. This device is being used to teleoperate a linear motor over the internet via a UDP connection. Both the master and the slave are controlled using National Instruments (NI) hardware and LabVIEW software. The following will present and compare two actuation schemes applying traditional active teleoperation rules for to the control of the passive haptic system. The inadequacies of these schemes will be illustrated, showing the need for a more complex control scheme to produce haptic feedback in a passive system.

Cooperative control of virtual objects using haptic teleoperation over the internet

Article Cooperative control of virtual objects using haptic teleoperation over the internet was published on October 1, 2005 in the journal International Journal on Disability and Human Development (volume 4, issue 4).

Haptic tele-operation of a mobile robot

Teleoperation systems have been developed in order to allow a human operator to perform complex tasks in remote environments. Mobile robots can be considered as a particular example of telemanipulation systems, since they can be operated remotely to perform particular tasks. As an example, the inspection of underwater structures and the removal of mines are performed by mobile platforms controlled by a remote operator, which generally takes advantage only of the visual feedback provided by vision systems. In this paper, the use of a haptic interface is proposed in order to increase the user's perception of the workspace of the mobile robot. In particular, a virtual interaction force is computed on the basis of obstacles surrounding the mobile vehicle in order to prevent dangerous contacts. The passivity of the overall system is preserved, so that stability of the virtual interaction is guaranteed.

A finite memory model for haptic teleoperation

This paper is concerned with modeling the haptic exploration and recognition of objects using a teleoperator mechanism, mainly underwater. Three factors are identified which determine the success with which objects may be recognized: a) correct feature identification, b) interpretation of spatial data between features, and c) the ability to retain in memory the sequence of features identified. A haptic cognitive model is proposed which incorporates these factors, with emphasis on part c). The performance of the cognitive model is assessed with respect to previously obtained data, and further experiments are conducted to determine the number of features an operator can correctly recall when probing a remote object.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>