Time-delayed Teleoperation Research Articles

Online Bilateral Predictive Control for Time-Delayed Teleoperation of Snake-like Robots

Controlling the teleoperation of snake-like robots is challenging due to complex nonlinear dynamics and communication delays. This research proposes an online bilateral predictive control architecture to address these issues. This control structure is established by predicting environment force and the user’s future motion. The former uses a model-mediated approach by creating a virtual environment on the master side and the latter adopts an artificial neural network (ANN) for online operator’s motion prediction. The slave controller utilizes transmitted data from ANN to generate required backbone lengths, which are then transformed into the slave's local bending and torsional degrees of freedom through the inverse kinematics of the robot. Motion prediction is examined in two scenarios: when the ANN predicts the trained motions, and when it predicts a different motion. Simulation studies demonstrate that the proposed online bilateral predictive teleoperation structure successfully achieves real-time position synchronization and force feedback, by effectively bypassing communication delays.

Intention-reflected predictive display for operability improvement of time-delayed teleoperation system

Robotic teleoperation is highly valued for its ability to remotely execute tasks that demand sophisticated human decision-making or that are intended to be carried out by human operators from a distance. However, when using the internet as a communication framework for teleoperation, high latency, and fluctuations make accurate positioning and time-dependent tasks difficult. To mitigate the negative effects of time delay, this paper proposes a teleoperation system that uses cross reality (XR) as a predictive display of the outcome of operators’ intended actions and develops a time-delay aware shared control to fulfill the intention. The system targets a liquid pouring task, wherein a white ring that indicates the intended height of the liquid surface is overlayed onto the beaker in a delayed camera image to close the visual feedback loop on the leader side. Simultaneously, the shared control automatically completes the pouring action to track the intended liquid height. The performance of the proposed system is validated based on liquid pouring experiments performed by human subjects. When compared with direct control, the absolute error rate decreased significantly for a constant round-trip time delay of 0.8 s and 1.2 s, similarly for a time-varying delay of 0.4 s and 0.8 s. Moreover, when the time-varying delay was 0.8 s, operators achieved significantly higher accuracy while maintaining comparable operation time. These results indicate that our proposed system improves operability even in the presence of time-varying delays in communication networks.

Adaptive control of time delay teleoperation system with uncertain dynamics.

A bilateral adaptive control method based on PEB control structure is designed for a class of time-delay force feedback teleoperation system without external interference and internal friction to study the uncertainty of dynamic parameters and time delay. The stability and tracking performances of the closed-loop constant time delay teleoperation system are analyzed by Lyapunov stability theory. Finally, the controller designed in this paper is successfully applied to the teleoperation system composed of a two-degree of freedom rotating manipulator as the master robot and the slave robot. The simulation is carried out in no operator and environment force or with operator and environment force. The adaptive bilateral control method's control performance is compared with that of the traditional time-delay teleoperation system. Finally, it is verified that the method has good control performance.

Control of Time Delay Force Feedback Teleoperation System With Finite Time Convergence.

In order to make the teleoperation system more practical, it is necessary to effectively control the tracking error convergence time of the teleoperation system. By combining the terminal sliding mode control method with the neural network adaptive control method, a bilateral continuous finite time adaptive terminal sliding mode control method is designed for the combined teleoperation system. The Lyapunov theory is used to analyze the stability of the closed-loop system, and the position tracking error is able to effectively converge in time. Finally, the effectiveness of the proposed control scheme is verified by MATLAB Simulink numerical simulation, and the numerical analysis of the results shows that the method has better system performance. Compared with the traditional two-sided control method (TPDC) of PD time-delay teleoperation system, the control method in this paper has good performance, improves stability, and makes steady-state errors smaller and better tracking.

Passivity-Based Control of Robots: Theory and Examples from the Literature

The use of passivity in control theory was motivated by the earlier use of passivity in circuit theory and passive network synthesis, beginning in the 1950s. With the well known analogy between electrical and mechanical systems, it is natural that passivity should play an important role in the control of mechanical systems and, in particular, in the control of robots. Beginning in the 1980s, the passivity property and passivity-based control began to play an increasingly important role in a host of robotics applications, including robust and adaptive control of manipulators, and time-delayed teleoperation. This article provides a historical overview of passivity-based robot control from the authors’ perspective and their contributions in control of Lagrangian and Hamiltonian systems, networked control, teleoperation, and vision-based control.

Environment Recognition Based Shared Control for Operability Improvement of Time-Delayed Teleoperation System

Environment Recognition Based Shared Control for Operability Improvement of Time-Delayed Teleoperation System

Adaptive Neural Network Control of Time Delay Teleoperation System Based on Model Approximation.

A bilateral neural network adaptive controller is designed for a class of teleoperation systems with constant time delay, external disturbance and internal friction. The stability of the teleoperation force feedback system with constant communication channel delay and nonlinear, complex, and uncertain constant time delay is guaranteed, and its tracking performance is improved. In the controller design process, the neural network method is used to approximate the system model, and the unknown internal friction and external disturbance of the system are estimated by the adaptive method, so as to avoid the influence of nonlinear uncertainties on the system.

A Novel Energy Compensation Scheme for Quality Enhancement in Time-Delayed Teleoperation With Multi-DoF Haptic Data Reduction and Communication.

Efficient and low-delay exchange of hapticinformation plays a key role for the design of high-fidelity teleoperation systems that operate over real-world communication networks. In the presence of communication unreliabilities such as delay and packet loss, a combination of stability-ensuring control schemes and haptic data reduction approaches is essential to address the challenges from both the control and communication perspective. In this paper, we extend our previous one degree-of-freedom (DoF) solution to 3DoF that combines the time-domain passivity approach (TDPA) and the perceptual deadband-based (DB) haptic data reduction approach. We also extend the 1DoF energy compensation (EC) scheme to 3DoF for mitigating the control artifacts introduced by the inter-play of the TDPA and DB approaches. The 3DoF solution is analyzed in detail and verified through a real-world teleoperation setup. The performance of the proposed method and the existing solutions is compared though objective and subjective assessments. Experimental results show that by jointly considering the evaluated objective/subjective quality and the required packet rate, the proposed 3DoF solution with the EC scheme improves system performance significantly compared to the state-of-the-art solutions.

Adaptive-observer-based robust control for a time-delayed teleoperation system with scaled four-channel architecture

AbstractThis article presents an innovative adaptive-observer-based scaled four-channel (4-CH) control approach applying damping injection for nonlinear teleoperation systems, which unify the study of robotic dynamic uncertainties, operator/environment force acquirements and asymmetric time-varying delays in the same framework. First, a scaled 4-CH scheme with damping injection is developed to handle time-varying delay while guaranteeing the passivity of communication channels. Then, the improved extended active observer (IEAOB) is deployed to derive the operator/environment force while addressing the issues of measurement noise and model uncertainties. Furthermore, the system stability is analyzed by choosing Lyapunov functional. Finally, the proposed method is validated through simulation.

Stabilization of User-Defined Feedback Controllers in Teleoperation With Passive Coupling Reference

Ensuring stability in time-delayed teleoperation can be difficult in some cases where the feedback forces need to be modified as defined by the user, to increase transparency. In the present letter, we propose an approach to ensure stability of any feedback controller selected by the user by considering a basic passive framework as reference. Stability properties of the proposed method are discussed and we apply it on different feedback controllers as examples. We present the results of hardware experiments, with performance comparison of three different control architectures stabilized with the proposed approach, with different delays and environment stiffnesses.

Enhancing the Transparency by Onomatopoeia for Passivity-Based Time-Delayed Teleoperation

Robotic teleoperation with force feedback has been studied extensively since it was first developed in the 1940 s. Time delay is a common problem of bilateral teleoperation systems. Although many efforts on optimizing the control architectures have been made, there is always a trade-off between transparency and stability for bilateral systems, and the perfect transparency and stability can only be achieved simultaneously in ideal situations. In this letter, we propose a novel approach to compensate for the degraded transparency while using the conventional passivity-based approach to maintain system stability under time-delay. The proposed approach is based on visual feedback and enhances the transparency by displaying different kinds of onomatopoeia according to contact force detected on the slave side. The basic performance is evaluated by conducting a stiffness classification task under constant round trip time delays (0 ms, 500 ms and 1000 ms). The preliminary results indicate that the subjects have higher accuracy for classifying the stiffness of a remote object by using onomatopoeia enhanced force feedback compared with the conventional passivity-based position-force feedback.

Input-to-State Stable Bilateral Teleoperation by Dynamic Interconnection and Damping Injection: Theory and Experiments

In bilateral teleoperation, the human operating the master and the environment interacting with the slave are part of the force feedback loop. Yet, both have time-varying and unpredictable dynamics and are challenging to model. Conventional sidestepping of the demand for their models in the stability analysis assumes passive user and environment, and controls the master-communications-slave system to be passive too. This paper circumvents the need for user and environment models in a novel way: it regards their forces as external excitations for a semiautonomous feedback loop, which it outfits with a dynamic interconnection and damping injection controller that renders time-delay teleoperation exponentially input-to-state stable. The controller uses the position and velocity of the local robot and the delayed position transmitted from the other side to robustly synchronize the master and slave under the user and environment perturbations. Lyapunov-Krasovskii stability analysis shows that the strategy, first, can confine the position error between the master and the slave to an invariant set, and, second, can drive it exponentially to a globally attractive set. The approach has practical relevance for telemanipulation tasks with given precision requirements. Experiments with a pair of Geomagic Touch robots validate the strategy compared to state-of-the-art robust position tracking designs.

Energy-Optimized Consensus Formation Control for the Time-Delayed Bilateral Teleoperation System of UAVs

This paper proposes an energy-optimized consensus formation scheme for the time-delayed bilateral teleoperation system of multiple unmanned aerial vehicles (UAVs) in the obstructed environment. To deal with the asymmetric time-varying delays in aerial teleoperation, the local damping is independently distributed on both sides to enforce consensus formation and force tracking of the master haptic device and the slave UAVs. The stability of the time-delayed aerial teleoperation system is analyzed by the Lyapunov function. In addition, a flux-conserved force field is incorporated into the aerial teleoperation system to guarantee a collision-free consensus formation in the obstructed environment. Moreover, to reduce the communication complexity and energy dissipation of the formation, a top-down strategy of 3D optimal persistent graph is first proposed to optimize the formation topology. Under the optimized topology with environmental constraints, communication complexity and energy dissipation can be minimized while the rigid formation can be maintained and transformed persistently in the obstructed environment. Finally, the human-in-the-loop simulations are performed to validate the effectiveness of the proposed scheme.

Adaptive control for time-delay teleoperation systems with uncertain dynamics

In most teleoperation systems, the dynamics are uncertain and the communications exhibit time delays. In order to confront these problems, this paper reports a position error-based bilateral adaptive controller, in which the unknown operator dynamical parameters and environmental dynamical parameters are included in the unknown vector of the system to be evaluated, adaptive estimate laws are compensated by estimate errors and dissipation by time delays are compensated. By using Lyapunov-Krasovskii stability theorem, it is proved that both position errors and velocities of the teleoperation system asymptotically convergent to zero. Simulations are performed to compare the performance of the proposed controller with the traditional adaptive controller and to demonstrate the efficiency of the developed teleoperation control system.

Energy Prediction for Teleoperation Systems That Combine the Time Domain Passivity Approach with Perceptual Deadband-Based Haptic Data Reduction.

We study the combination of the perceptual deadband (PD)-based haptic packet rate reduction scheme with the time domain passivity approach (TDPA) for time-delayed teleoperation and propose a novel energy prediction (EP) scheme that deals with the conservative behavior of the resulting controller. The PD approach leads to irregular packet transmission, resulting in degraded system transparency and reduced teleoperation quality when the PD approach is combined with the TDPA. The proposed method (PD+TDPA+EP) adaptively predicts the system energy during communication interruptions and allows for larger energy output. This achieves less conservative control and improves the teleoperation quality. Evaluation of the displayed impedance shows that the PD+TDPA+EP method achieves improved system transparency, both objectively and subjectively, when compared with related approaches from literature. According to a subjective user study, the PD+TDPA+EP method allows for a high packet rate reduction (up to 80 percent) without noticeably distorting the perceived interaction quality. We also show that the PD+TDPA+EP method is preferred over related approaches from literature in a direct comparison test. Thus, with the proposed PD+TDPA+EP method, a high data reduction and a high teleoperation quality are simultaneously achieved for time-delayed teleoperation.

A self-adjusting compliant bilateral control scheme for time-delay teleoperation in constrained environment

When teleoperations are implemented in the constrained environment, the lack of environment information would lead to contacts and undesired excessive contact forces, which are more evident with the existence of time delays. In this paper, a hybrid compliant bilateral controller is proposed to deal with this problem. The controller adopts a self-adjusting selecting scheme to divide the subspaces online. The master and slave manipulators are synchronized in the position subspace through an adaptive bilateral control scheme. At the same time, the slave manipulator is controlled by a local sliding mode impedance controller in order to achieve the desired compliant motion when contacting with the environment. Theoretical analysis proves the stability of the hybrid bilateral controller and concludes the transient performance of the teleoperators. Simulations are carried out to verify the effectiveness of the proposed approach. The results show that the control goals are all achieved.

Enhanced position-force tracking of time-delayed teleoperation for robotic-assisted surgery.

The advance of minimally invasive surgery has been empowered by new medical/surgical robotic systems towards achieving less invasiveness, smaller or even no scars. Wireless communication possesses great potential to be utilized in miniaturized surgical robotic system. However time delay is inevitably introduced in the control loop which causes stability issues for robotic-assisted surgeries. Wave variable based teleoperation structure provides stable force reflecting teleoperation performance but with both position and force tracking performance compromised due to conservative passivity condition. Recently, we proposed a new wave variable compensated structure to improve position and force tracking performance together with energy reservoir based regulators for stability purpose. In this paper, different energy reservoir based regulators are proposed with consideration of passivity of master and slave system to avoid uncertain compensated wave variables. Experiments are designed to evaluate the performance of proposed structure in comparison with traditional wave variable structure. Quantitative analyses of the obtained results justify the efficiency of proposed method.

Transparency Improvement by External Force Estimation in a Time-Delayed Nonlinear Bilateral Teleoperation System

Teleoperation systems have been developed in order to manipulate objects in environments where the presence of humans is impossible, dangerous or less effective. One of the most attractive applications is micro telemanipulation with micropositioning actuators. Due to the sensitivity of this operation, task performance should be accurately considered. The presence of force signals in the control scheme could effectively improve transparency. However, the main restriction is force measurement in micromanipulation scales. A new modified strategy for estimating the external forces acting on the master and slave robots is the major contribution of this paper. The main advantage of this strategy is that the necessity for force sensors is eliminated, leading to lower cost and further applicability. A novel control algorithm with estimated force signals is proposed for a general nonlinear macro–micro bilateral teleoperation system with time delay. The stability condition in the macro–micro teleoperation system with the new control algorithm is verified by means of Lyapunov stability analysis. The designed control algorithm guarantees stability of the macro–micro teleoperation system in the presence of an estimated operator and environmental force. Experimental results confirm the efficiency of the novel control algorithm in position tracking and force reflection.

Design of a New Impedance Controller for a Time Delay Teleoperation System

Bilateral teleoperation systems connected to computer networks such as the internet have to deal with varying time delays depending on several factors such as congestion, bandwidth, or distance. Such systems can easily become unstable due to irregular or varying time delays. A passivity concept has been used as the framework to solve the stability problem in bilateral control of teleoperation systems and we demonstrate in this paper how to recover both passivity and tracking performance using a novel control architecture that incorporates time varying gains into the transmission path, feedforward, and feedback position control. Simulation results for a single-degree of freedom master/ slave system are presented which demonstrates the performance of the resulting control architecture.

Key Technologies of Telepresence in Time-Delayed Teleoperation System: Review and Analysis

This paper reviews some key technologies and progress in the field of telepresence of teleoperation, meaning human sense and control of remote telerobot. Telepresence is the symbol of teleoperation, phenomenon of which enables the human to feel present at a remote location however a time-delay exits always in the teleoperation system. There are three means to conquer the effect: bilateral control method, virtual model based control and virtual reality technology. By optimizing control strategy user will get a perfect force or visual telepresence. Finally, opinions are given as to the further development of the field.