Haptic Teleoperation Research Articles

A Novel Force Sensorless Reflecting Control for Bilateral Haptic Teleoperation System

This paper presents a novel force sensorless reflecting controller for a haptic-enabled device driven by a bilateral pneumatic artificial muscle system, which proposed configuration for the first time the bilateral haptic teleoperation. For details, an adaptive force observer scheme considered to be an alternative to direct force measurement is proposed to estimate the interaction force with an unknown environment for the force reflecting control design. Meanwhile, the separately fast finite time nonsingular terminal sliding mode control schemes are developed based on the force estimation in both subsystems to achieve good tracking performance and fast response. Thus, the great transparency performance with both force feedback and position tracking can be achieved simultaneously by using our proposed method. The finite-time stability of the total controlled system is demonstrated by the Lyapunov approach. Moreover, the comparative experiments are carried out on the developed testbench to validate the effectiveness and advantages of our proposed control design in the different working conditions.

Peg-in-a-hole by haptic teleoperation

This paper presents the experimental study on the application of the peg-in-a-hole task in the proposed haptic teleoperation system. Motion references from the user towards the remote manipulator and force feedback signals from the remote environment towards the user are transmitted through a virtual reality layer where the remote operation is simulated in real-tme. Various physical interactions occuring in the bilatral loop are modelled and controlled through the introduction of virtual spring-damper elements. Modeling and controller design of the teleoperation system is made modular and straightforward with the use of identical virtual dynamics. The design approach combines the introduction of the middle layer of virtual reality and modeling of all interactons by identical virtual spring-dampers. The haptic teleoperation system is experimentally evaluated with experienced and unexperienced users in the bilateral loop. Experimental results show that stable peg-in-a-hole operation is achieved in presence of bounded time-delays in all communication channels between the user, virtual reality layer and remote environment. Improvements of the teleoperation performances in terms of position tracking and total operation time are also observed in experiments.

Series elastic actuation for improved transparency in time delayed haptic teleoperation

This paper demonstrates that incorporating passive compliance to the follower (slave) by means of a series elastic actuator (SEA) improves system transparency for haptic teleoperation applications with large time delays. Time delay induced force and position tracking errors limit the practical implementation of teleoperation systems. Traditional approaches have focused on passifying the communication channel and estimation methods to improve transparency, but performance degrades in large delays. In a force-position architecture using a Smith predictor, the ability to implement force control through position control with an SEA aids in increasing transparency of the system. The position drift common in most time delayed systems is eliminated with the proposed method while maintaining an accurate force reflection for a constant round trip time delay of two seconds. The combined actuation and sensing capabilities of the SEA also provide a means for detecting and correcting for variations in the environment contact location. This aids in estimating the follower dynamics for the predictor as both position and force information of the environment are captured. The proposed method is validated through simulation and experiment involving stiff and soft unilateral environment contact. The results of haptic teleoperation with an SEA demonstrate the effectiveness of compliance for accurate system transparency and improved performance in comparison to stiff actuation, while reducing controller complexity.

Semi-autonomous bilateral teleoperation of six-wheeled mobile robot on soft terrains

Abstract With the increasing use of multi-wheeled mobile robots in various fields, a few new challenges pertaining to the design of the teleoperation system of such robots have emerged. This paper proposes a new two-level architecture for haptic teleoperation of a six-wheeled mobile robot (SWMR) with a rocker-bogie chassis on soft terrains. In teleoperation, the linear and angular velocities of the SWMR base follow the master robot’s positions. At the slave SWMR level, local compensation controllers are designed for the driving motors and the steering motors to track the desired velocities of the base and eliminate the negative influence of wheel slippage. Command-tracking errors caused by wheel slippage at environment termination (ET) are conveyed to the human operator through haptic (force) feedback. By guaranteeing the passivity of ET, the stability conditions of the teleoperation system are constrained by the Llewellyn criterion. Experiments conducted using the proposed controllers demonstrated that the tracking errors in the commands sent to the slave robot are effectively compensated for, and the controllers achieve stable teleoperation with satisfactory tracking performance.

Parametric Sensitivity Analyses for Perceived Impedance in Haptic Teleoperation

In this paper, sensitivity analyses (SA) are performed in order to find the effect of variations of master/slave site dynamics parameters on the perceived impedance in haptic teleoperation in the absence/presence of communication time delays. These analyses are useful in finding the dominant parameter perceived by the human operator during the teleoperation. In this regard, the comprehensive sensitivity analyses are performed for the prominent force reflecting teleoperation architectures such as (i) two-channel (2C) teleoperation architectures mainly (a) position error-based architecture (PE) and (b) direct force reflection architecture (PF), (ii) four-channel architecture (4C) and, (iii) three-channel (3C) architecture (a) OFC architecture and (b) EFC architecture. Several important results and conclusions are presented for sensitivity of perceived impedance in frequency domain using MATLAB/Simulink.

Reset observers alleviating the peaking and the robustness tradeoffs: A case study on force estimation in teleoperation

The peaking phenomenon is an undesirable effect appearing in observers and destroying controller performance. Several solutions have been proposed to mitigate peaking in state estimation. The literature shows that reset or impulsive observers are superior to linear (Luenberger) observers. However, the comparisons are based on particular choices of linear observers. This paper investigates this issue. First, comparative frameworks are proposed based on two traded-off performance indices: ensemble maximum-peak versus ensemble settling time for nominal conditions, and ensemble settling time versus size of the error asymptotic invariant set for quadratically bounded uncertain plants. Next, performance limitations of linear observers are represented by Pareto-optimal boundaries. In this way, not previously considered in the literature as far as known, the superiority of the chosen reset observer is more rigorously assessed. The framework is finally applied to force estimation in haptic teleoperation.

Framework for Human Haptic Perception With Delayed Force Feedback

Time delays in haptic teleoperation affect the ability of human operators to assess mechanical properties (damping, mass, and stiffness) of the remote environment. To address this, we propose a unified framework for human haptic perception of the mechanical properties of environments with delayed force feedback. In a first experiment, we found that the delay in the force feedback led our subjects to underestimate all the three mechanical properties. Moreover, subjects perceived additional damping or stiffness properties that the environment did not possess. It was found that the extents of these changes in the perception depend on both time-delay magnitude and the frequency of the movement with which subjects interacted with the environment. This was due to the fact that subjects were not able to distinguish the delay-caused phase shift in the movement–force relation from changes in the three mechanical properties. Based on this, we proposed a framework that allowed for a prediction of the change associated with delayed force in perception of mass–spring–damper environments. The framework was corroborated by a second experiment, in which a combined mass–damper environment was tested. Our hypotheses that the delay would cause subjects to underestimate the mass but overestimate the damping and that the extents of the under- and overestimation would differ between individual subjects due to the difference in the interaction frequency were confirmed.

Virtual Reality-Based Time-Delayed Haptic Teleoperation Using Point Cloud Data

This paper presents a methodology for bilateral teleoperation of robots based on the use of point cloud data obtained with an RGB-D camera for providing haptic feedback, as well as a virtual robot workspace that allows the user to interact with the teleoperated robot and the environment, regardless of the time-delay between the master device and the slave robot. The robot movements are executed first in the Virtual Workspace and then in the real one. The Virtual Workspace is created using point cloud data, and the haptic feedback is calculated directly using this data. The proposed methodology was validated in 2D and 3D trajectory-following experiments with 20 volunteers each teleoperating a KUKA industrial robot. In addition, we studied two inverse kinematic teleoperation modes: “Proportional Workspace” and “Delta end-effector movements”, with the objective of determining which one is more suitable for the haptic teleoperation methodology proposed. The results obtained confirm that the proposed haptic teleoperation methodology enhances the precision of the operator by reducing the average error between the robot end-effector trajectory and the goal trajectory. Furthermore, the likelihood that the end-effector will collide with objects in the environment is reduced. The methodology also diminishes the time that it takes to complete the trajectory-following task in an unknown workspace. The inverse kinematic teleoperation mode with the better results is the “Proportional Workspace”, which is more intuitive for operators (users).

Kinematic Teleoperation of Wheeled Mobile Robot With Slippage Compensation on Soft Terrains

The wheeled mobile robot (WMR) is widely employed in many industrial fields, and more new control difficulties appear when the WMR works on soft terrains (e.g., Lunar exploration), one of which is induced by wheel slippage. In this paper, a new approach for the haptic teleoperation of a differential WMR coupling with wheel slippage on soft terrains is proposed. In the proposed teleoperator, the linear/angular velocities of the slave WMR are directly mapped with the master robot’s positions. The command-tracking errors and the non-passivity at the slave WMR site induced by the wheel slippage are compensated by the proposed feedforward controllers for each wheel based on the online estimated slippage. The stability of the teleoperation system is guaranteed by the passivity theory while the environment termination is proved to be passive with the proposed feedforward controllers. The experiments validate that the proposed teleoperator is stable while the command-tracking performance of the slave WMR is obviously improved on soft terrains.

A Lyapunov Stable Controller for Bilateral Haptic Teleoperation of Single-Rod Hydraulic Actuators Subjected to Base Disturbance

In this paper, a control scheme is developed and evaluated for stable bilateral haptic teleoperation of a single-rod hydraulic actuator subjected to base disturbance. The proposed controller, based on Lyapunov stability technique, is capable of reducing position errors at master and slave sides, and provides a feel of the contact force between the actuator and the task environment to the operator without a need for direct measurement. The controller requires only the measurements of the actuator line pressures and displacements of the master and slave. The system stability is insensitive to the uncertainties of the physical parameters and of the measurement of the base point motion. Stability of the proposed controller incorporating hydraulic nonlinearities and operator dynamics with an estimated upper value for the base disturbance is analytically proven. Simulation studies validate that the proposed control system is stable while interacting with a task environment. Experimental results demonstrate the effectiveness of control scheme in maintaining stability, while having good position tracking by the hydraulic actuator as well as providing a haptic feel to the operator without direct measurement of interaction force, while the hydraulic actuator is subjected to base disturbance.

Adaptive Control of a Robot-Assisted Tele-Surgery in Interaction With Hybrid Tissues

In a haptic teleoperation system, which interacts with unknown and hybrid environments, it is important to achieve stability and transparency. In medical usages, the utilization of knowledge on the tissues behavior in a controller design can improve the performance of the surgery in a robot-assisted telesurgery. Simultaneous interaction with hard and soft tissues makes it difficult to achieve stability and transparency. To deal with this difficulty, two controller schemes are designed. At first, a nonlinear mathematical model (inspired by the Hunt-Crossley (HC) model), which has the properties of soft and hard tissues, is combined with the slave side dynamic. In the second approach, the reaction force applied by hybrid tissues during the transition between tissues of different properties is modeled as an unknown force acting on the slave side. In a four-channel (4-CH) architecture, nonlinear adaptive controllers are designed without any knowledge about the parameters of the master, the slave robot, and the environment. For both control schemes, Lyapunov candidate functions provide a way to ensure the stability and transparency in the presence of uncertainties. The testbed comprises two Novint Falcon robots functioning as master and slave robots. Moreover, the experiments are performed on various objects, including a soft cube, a hard cube, and a phantom tissue. This paper rigorously evaluates the performances of the proposed methods, comparing them with each other and other previous schemes. Experimental and numerical results demonstrate the effectiveness of the proposed control schemes.

A novel bilateral haptic teleoperation approach for hexapod robot walking and manipulating with legs

Teleoperation control of the multiple degrees-of-freedom (DOF) robot requires a strategy for improving its controllability and adaptability. A novel bilateral haptic teleoperation approach (multi-master single-slave) is proposed for a hexapod robot in which the first master control the primary task of the slave, e.g. the linear/steering motion; meanwhile, the second master can perform a minor task, e.g. to interact with the environment by leg. The designed teleoperation system consists of two parts: body-level and leg-level. In body-level subsystem, the linear/angular velocity of the slave body follow the first master‘s position, and the velocity loss caused by the system disturbances is fed back to the operator in the form of haptic force. In leg-level subsystem, a modified four-channel (4CH) teleoperation control architecture is proposed for the manipulable leg, which can be regarded as a manipulator, so as to guarantee the performance of the position/force tracking in the subsystem subject to parametric uncertainties, environmental perturbation and unmeasurable interaction force. Additionally, the stability of the multi-DOF bilateral haptic teleoperation system are verified via absolute stability criterion and Lyapunov theorem, respectively. Experiments of the proposed approach demonstrate that it can result in stable and transparent bilateral teleoperation with haptic force feedback.

Design of A Novel Passive Binary-Controlled Variable Stiffness Joint (BpVSJ) Towards Passive Haptic Interface Application

In this paper we present the design, development and experimental validation of a novel Binary-Controlled Variable Stiffness Joint (BpVSJ) towards haptic teleoperation and human interaction manipulators applications. The proposed actuator is a proof of concept of a passive revolute joint, where the working principle is based on the recruitment of series-parallel elastic elements. The novelty of the system lies in its design topology, including the capability to involve an (n) number of series-parallel elastic elements to achieve ( $2^{n}$ ) levels of stiffness, as compared to current approaches. Accordingly, the level of stiffness can be altered at any position without the need to revert to the initial equilibrium position. The BpVSJ has low energy consumption and short switching time, and is able to rotate freely at zero stiffness without limitations. Further smart features include scalability and relative compactness. This paper details the mathematical stiffness modeling of the proposed actuator mechanism, as well as the experimentally measured performance characteristics. The experimental results matched well with the physical-based modeling in terms of stiffness variation levels. Moreover, Psychophysical experiments were also conducted using (20) healthy subjects in order to evaluate the capability of the BpVSJ to display three different levels of stiffness that are cognitively realized by the users. The participants performed two tasks: a relative cognitive task and an absolute cognitive task. The results show that the BpVSJ is capable of rendering stiffness with high average relative accuracy (Relative Cognitive Task relative accuracy is 97.3%, and Absolute Cognitive Task relative accuracy is 83%).

Increasing stability in model-mediated teleoperation approach by reducing model jump effect

Model-mediated teleoperation is a predictive control approach for controlling haptic teleoperation systems whereby the environment force is virtually located on master side in order to increase the stability and transparency of the system. This promising approach, however, results in new challenges. One pivotal challenge is the model jump effect, which stems from the delay in correct creation of the virtual environment. Previous works have endeavored to reduce this effect; however, they either led to transparency decrease or assumed simplified environment models. In this paper, we propose a control approach for this aim based on the idea of decoupling. This means that when a new environment has been identified, the operation is interrupted and no signal is transmitted between master and slave sides. During this time, both sides are controlled by their own sliding mode controllers until the system reaches stability. The main advantage of this method is its independence from environment type, which makes it usable for different kinds of applications. To verify the effectiveness of the proposed approach simulation tests are conducted. The results show the system is stable in interaction with hard and soft environments in presence of large time delays in communication channels.

A Mixed-Initiative Haptic Teleoperation Strategy for Mobile Robotic Systems Based on Bidirectional Computational Trust Analysis

The goal of this paper is to improve the performance of joint human–robot systems while balancing human experience through computational trust analysis in telerobotics applications. A mixed-initiative approach is enabled by scaling the manual and autonomous controls using a function of computational human-to-robot trust. A dynamic haptic force feedback scaling strategy is performed via a function of computational robot-to-human trust to adjust the assistive force and, hence, physical workload of the operator. Passivity-based methods are used to guarantee the stability of the overall framework. Moreover, guidelines are provided to adjust the transparency of force feedback and velocity signals while guaranteeing passivity. An experimental study with 30 human subjects demonstrates a 12.8% increase in task performance and a 10.7% decrease in operator workload when the proposed strategy is used as compared to a manual autonomy allocation approach. The results also indicate that the proposed method yields higher operator satisfaction compared to manual and optimal autonomy allocation methods and is 10.1% more trusted by the operators than the optimal allocation method.

A new impedance force control of a haptic teleoperation system for improved transparency

In this study, a new impedance force control model of a haptic device for teleoperation is developed and analyzed. A new contact force model for the haptic device and the human hand is provided. Movements of the human hand give additional force disturbances to the force control system. Disturbance force model by haptic dynamics and hand movements is fully analyzed and included in the control system design. An adaptive control scheme is proposed to improve system transparency by achieving good force tracking performance, while simultaneously compensating human hand disturbances and sensor noises. A separate reference model for every DOF is proposed to satisfy tracking performances. Adaptive control gains cover force tracking performance and compensates for human hand disturbances while providing robustness to sensor noises.

Stable Nonlinear Trilateral Impedance Control for Dual-User Haptic Teleoperation Systems With Communication Delays

A new nonlinear adaptive impedance-based trilateral controller is proposed to ensure the absolute stability of multi-degrees-of-freedom (DOFs) dual-user haptic teleoperation systems subjected to communication delays. Using this strategy, reference impedance models are realized for the trilateral teleoperation system represented by a three-port network to facilitate cooperation of two human operators in order to perform a remote physical task. For this purpose, an impedance model defines the desired haptic interaction between the two human operators, while another impedance model specifies the desired behavior of the slave robot in terms of tracking the mater robots' trajectories during interaction with the remote environment. It is shown that different performance goals such as position synchronization and force reflection can be achieved via different adjustments to the impedance parameters. The sufficient conditions for the trilateral haptic system's absolute stability are investigated in terms of the impedance models' parameters. Accordingly, guidelines for modification of the impedance parameters are obtained to guarantee the absolute stability of the trilateral haptic system in the presence of communication time delays. A trilateral nonlinear version of the model reference adaptive impedance control (MRAIC) scheme is developed for implementing the proposed reference impedance models on the masters and the slave. The convergence of robots' trajectories to desired responses and the robustness against modeling uncertainties are ensured using the proposed controller as proven by the Lyapunov stability theorem. The proposed impedance-based control strategy is evaluated experimentally by employing a nonlinear multi-DOFs teleoperated trilateral haptic system with and without communication delays.

A Lyapunov Stable Controller for Bilateral Haptic Teleoperation of Single-Rod Hydraulic Actuators

A Lyapunov stable control scheme is designed and implemented for bilateral haptic teleoperation of a single-rod hydraulic actuator. The proposed controller is capable of reducing position errors at master and slave sides, as well as perceiving the interaction force between the actuator and the task environment without a need for direct measurement of force. The controller only requires the actuator's line pressures and displacements of the master and slave. Stability of the proposed controller incorporating hydraulic nonlinearities and operator dynamics is analytically proven. Simulation studies demonstrate that the proposed system can reach an equilibrium point while interacting with an environment exhibiting stiffness. Experimental results confirm that the controller is able to effectively maintain stability, while having good position tracking by the hydraulic actuator as well as perceiving the contact force between the actuator and the task environment without direct measurement. This kind of haptic feedback force is a suitable choice for applications where mounting a force sensor at the end-effector is not feasible, such as excavators and backhoes. This work contributes to enhancing the operator's ability to perform stable haptic-enabled teleoperation of hydraulic manipulators.

A Study on the FPGA Implementation of the Bilateral Control Algorithm Towards Haptic Teleoperation

This paper presents the FPGA implementation of the sliding mode control algorithm for bilateral teleoperation, such that, the problem of haptic teleoperation is addressed. The presented study improves haptic fidelity by the widening the control bandwidth. For wide control bandwidth, short control periods as well as short sampling periods are required that can be achieved by FPGA. The presented FPGA design methodology applies basic optimization methods in order to meet the requirements in terms of the control period and the hardware resource consumption. The circuit specification was performed by the high-level programing language LabVIEW and using the fixed-point data type. Furthermore, the design methodology allows achieving short design times for producing the FPGA logic circuit. The proposed FPGA-based bilateral teleoperation was validated by 2-DoF master-slave experimental device.

Soft Resource Reservation for Low-Delayed Teleoperation Over Mobile Networks

The emerging Tactile Internet (TI) will enable control-oriented networks for remotely accessing or manipulating objects or devices. One major challenge in this context is how to achieve ultra-low-delay communication between the local operator and the remote object/device to guarantee the stability of the global control loop and to maximize the user’s quality-of-experience (QoE). Being one of the major human-in-the-loop applications of the TI, haptic teleoperation inherits its delay-sensitive nature and requires the orchestration of communication and control approaches. In this paper, we focus on the radio access protocol, and its impact on the latency of wireless communication. We propose a novel soft resource reservation mechanism for the uplink scheduling of mobile networks that can significantly reduce the latency compared with the current legacy scheme. By leveraging the characteristics of teleoperation data traffic, and reserving resources accordingly, the proposed soft reservation scheme maintains the spectral efficiency while the human operator’s QoE is improved. The simulation results confirm the efficiency of the proposed scheme.