Transparent Teleoperation Research Articles

Practical Approaches Towards Transparent and Stable Bilateral Teleoperation Under Time-Varying Network Delay

AbstractIn real-life teleoperation scenarios, the presence of time-varying network delays, particularly in wireless networks, poses significant challenges in maintaining stability in a bilateral teleoperation system. Various approaches have been proposed in the past to address stability concerns; however, these often come at the expense of system transparency. Nevertheless, increasing transparency is crucial in a teleoperation system to enable precise and safe operations, as well as to provide real-time decision-making capabilities for the operator. This paper presents our comprehensive approaches to maximize teleoperation transparency by minimizing system impedance, enhance the wave variable method to handle time-varying network delays, and alleviate non-smooth effects caused by network jitters in bilateral teleoperation. The proposed methodologies take into account the real-world challenges and considerations to ensure the practical applicability and effectiveness of the teleoperation system. Throughout these implementations, passivity analysis is employed to ensure system stability, guaranteeing a reliable and safe teleoperation experience. The proposed approaches were successfully validated in Team Northeastern’s Avatar telepresence system, which achieved the 3rd place in ANA Avatar XPRIZE challenge.

Towards Critical Industrial Wireless Control: Prototype Implementation and Experimental Evaluation on URLLC

Ultra reliable low latency communication (URLLC) is a promising technology to enable critical industrial wireless control, such as tactile sensing and motion control. This article presents a novel software-defined URLLC prototype and establishes a human-in-loop robotic teleoperation platform for experimental evaluations. First, by studying the control and communication requirements of robotic teleoperation, the key challenges for URLLC-based industrial wireless control are analyzed. Then, the prototype development including protocol stack and hardware architecture design is introduced in detail, and extensive experiments are performed to evaluate the communication latency and control transparency for robotic teleoperation. Experimental results show that the prototype can realize millisecond-level latency with ultra-high reliability, and support wireless teleoperation with similar transparency as wired teleoperation. Finally, challenges and future research issues towards 6G are discussed.

Mutual Impedance-Based Force/Velocity Transmission Improvement for Bilateral Teleoperation

Tackling communication delays attracts interest in improving the stability and transparency of bilateral teleoperation. This study presents a novel concept; mutual impedance. Under the delays, this impedance is intrinsic in four-channel acceleration-based bilateral control (4ch ABC). In 4ch ABC, wave propagations can characterize the transmissions of force and velocity, which are described in distributed-parameter systems. The delays induce interference between applied force and velocity response and decrease transparency. The mutual relationship between force and position controllers can explain the interference. The proposed mutual impedance is the indicator of the impact of the relationship. This study notes that the impedance comprises a ratio of force and position controllers. Thus, a force proportional-integral (PI) controller with a gain constraint between the position controller is introduced. One-degree-of-freedom (1-DOF) motion experiments compare the proposed method with conventional methods in superiority. The applicability of the proposed approach to delay fluctuations and multi-DOF motion is verified through 3-DOF manipulators.

A unifying framework for transparency optimized controller design in multilateral teleoperation with time delays

Multilateral teleoperation is an umbrella term for various N-robot haptic teleoperation schemes and contains bilateral teleoperation as a special case with N=2. While transparency of bilateral teleoperation systems can be analyzed through hybrid matrices and transparent controllers can be designed with four channel/Lawrence architectures, such a general analysis and design framework has not yet been established for multilateral teleoperation systems. Here, a framework is proposed to achieve transparency in multi-user shared control multilateral teleoperation systems even in the presence of time delays. First, a novel control law unifying a wide range of dominance factor, control and communication architectures is proposed. This control law enables the adaptation of all bilateral linear Lawrence architectures to multilateral teleoperation. Second, a novel transparency analysis approach is proposed and it is shown that TOPF architecture is the only Lawrence architecture that can potentially achieve ideal transparency in multilateral teleoperation under time delays. Third, based on proposed L2 stability criteria, a trade-off between stability robustness and transparency under varying time delays is demonstrated. Effectiveness of the proposed framework is validated with numerical analyses and experiments.

The wizard and I: How transparent teleoperation and self-description (do not) affect children\u2019s robot perceptions and child-robot relationship formation

It has been well documented that children perceive robots as social, mental, and moral others. Studies on child-robot interaction may encourage this perception of robots, first, by using a Wizard of Oz (i.e., teleoperation) set-up and, second, by having robots engage in self-description. However, much remains unknown about the effects of transparent teleoperation and self-description on children’s perception of, and relationship formation with a robot. To address this research gap initially, we conducted an experimental study with a 2 × 2 (teleoperation: overt/covert; self-description: yes/no) between-subject design in which 168 children aged 7–10 interacted with a Nao robot once. Transparency about the teleoperation procedure decreased children’s perceptions of the robot’s autonomy and anthropomorphism. Self-description reduced the degree to which children perceived the robot as being similar to themselves. Transparent teleoperation and self-description affected neither children’s perceptions of the robot’s animacy and social presence nor their closeness to and trust in the robot.

A Wave Variable Approach With Multiple Channel Architecture for Teleoperated System

Performance of teleoperation can be greatly influenced by time delay in the process of tele-manipulation with respect to accuracy and transparency. Wave variable is an effective algorithm to achieve a good stable capability. However, some traditional wave variable methods may decrease the performance of transparency and suffer the impacts of wave reflection. To deal with the problem of stability and transparency in teleoperation, in this paper, a novel wave variable method with four channel is presented to achieve stable tracking in position and force. In addition, the proposed method can achieve the distortion compensation and reduce the impacts of wave reflection. The simulation experimental results verified the tracking performance of the proposed method.

Mathematical Model for UGV Transparency Using Laser Range Finder

ABSTRACTFor extending human capabilities to perform tasks remotely, unmanned vehicles and robots are used,this scenario is called Teleoperation. In teleoperation, human executes tasks in a remote environment. There are many applications of vehicle teleoperation in hazardous environment, military, underwater, and space exploration. Operating a vehicle or a system remotely requires supplyingthe operator in real-time with an accurate data about the operation environment. This operation is subjected to data transmission time-delay, sensors fusion and stability problems.This paper proposes a mathematical model for the time-delay compensation algorithm to achieve transparency in teleoperation using Laser scanner data. The proposed model for compensation depends on predicting the vehicle position in the future time and the environment of operation is available for the operator with respect to this predicted position. Using this algorithm in compensating time-delay in data transmission achieving transparency. Simulation and real experiments are shown to validate the proposed model.

Model-mediated teleoperation with improved stability

Model-mediated teleoperation has been developed to improve both transparency and stability in teleoperation. It uses local model of remote environment to provide non-delayed force feedback rather t...

Model-Mediated Teleoperation: Toward Stable and Transparent Teleoperation Systems

Bilateral teleoperation systems with haptic feedback allow human users to interact with objects or perform complex tasks in remote or inaccessible environments. Communication delays in teleoperation systems jeopardize system stability and transparency, leading to degraded system performance and poor user experience. In this paper, we provide a survey of the model-mediated teleoperation (MMT) approach, which has been developed to guarantee both system stability and transparency in the presence of arbitrary communication delays. This survey focuses on two major parts: 1) the historical development of the MMT approach from the late 1980s to the present and 2) the main challenges facing the design of a reliable MMT system. Along with the discussion of the MMT challenges and the proposed solutions, a series of experiments has been conducted to compare the performance between the existing techniques and to supply data that were missing in the previous studies on the MMT approach.

Data-Driven Motion Mappings Improve Transparency in Teleoperation

A teleoperation system with high transparency enables the operator to focus on completing the task at hand instead of on controlling the robot. We previously proposed that modifying the mapping from human movement to desired robot movement might improve the transparency of teleoperators in ways similar to adding sensory feedback. Specifically, we created non-Cartesian motion mappings that correct for systematic reaching errors made by humans, so that the robot motion resembles the operator's intent rather than his or her produced movement. This article presents a study that compares subjects' performance on a virtual teleoperated targeting task under three different motion mappings: a Cartesian-scaling motion mapping that is typically implemented in teleoperators, a corrective variable-similarity motion mapping that is fit to aggregate data from subjects in a previous study, and a corrective variable-similarity motion mapping that is fit to calibration data collected from each subject. Twelve participants reached toward 120 targets under each of the three motion mappings with balanced random presentation order and a washout task between conditions. Subjects were able to complete the targeting task with higher accuracy in the initial direction of robot motion, at higher speeds, and with more natural and efficient reaching movements under the variable-similarity motion mappings. Subjects also overwhelmingly preferred the variable-similarity motion mappings. These results indicate that subjects experienced a higher level of transparency when using the virtual teleoperator with the variable-similarity motion mappings than with the standard Cartesian mapping. Therefore, mappings that correct for systematic errors in human motion, such as the variable-similarity motion mappings tested here, should be considered in teleoperator design.

Digital versus analog control of bilateral teleoperation systems: A task performance comparison

Controller discretization has the potential to jeopardize the stability of a bilateral teleoperation system. As reported in the literature, stability conditions impose bounds on the gains of the discrete-time controller and the sampling period and also a trade-off between the two. This paper shows a choice of task for which large sampling periods, necessitating low control gains for maintaining stability, lead to low teleoperation transparency and unacceptable task performance. It continues to show that users can successfully perform the same task if the controller is implemented using analog components. This highlights the advantages of analog haptics in tasks involving the display of highly stiff environments. The paper also highlights the constraints in designing analog haptic teleoperation controllers and proposes design guidelines to address them.

Model-based passive bilateral teleoperation with time delay

Noisy behaviour of time domain passivity control (TDPC) at low velocity is a well known problem for delayed teleoperation systems. This paper presents a novel adaptive model-based passive controller to alleviate some of the noise problems associated with delayed teleoperation system using the TDPC approach. By composing an online estimation of phantom human tissue model and passivity observer on the master side, a high transparency of teleoperation can be achieved while the system passivity is maintained by modifying the damping of the master. The performance of the developed approach was validated using one-degree-of-freedom master–slave robot system with constant time delay. Results show that the phantom tissue parameters can be accurately estimated. In addition, the passivity of the system is observed by the passivity observer using the output of the identified tissue model. Results demonstrate that stable teleoperation with time delay can be achieved and the environment force can be accurately reflected to the operator without chattering.

Improving Transparency in Teleoperation by Means of Cutaneous Tactile Force Feedback

A study on the role of cutaneous and kinesthetic force feedback in teleoperation is presented. Cutaneous cues provide less transparency than kinesthetic force, but they do not affect the stability of the teleoperation system. On the other hand, kinesthesia provides a compelling illusion of telepresence but affects the stability of the haptic loop. However, when employing common grounded haptic interfaces, it is not possible to independently control the cutaneous and kinesthetic components of the interaction. For this reason, many control techniques ensure a stable interaction by scaling down both kinesthetic and cutaneous force feedback, even though acting on the cutaneous channel is not necessary.We discuss here the feasibility of a novel approach. It aims at improving the realism of the haptic rendering, while preserving its stability, by modulating cutaneous force to compensate for a lack of kinesthesia. We carried out two teleoperation experiments, evaluating (1) the role of cutaneous stimuli when reducing kinesthesia and (2) the extent to which an overactuation of the cutaneous channel can fully compensate for a lack of kinesthetic force feedback. Results showed that, to some extent, it is possible to compensate for a lack of kinesthesia with the aforementioned technique, without significant performance degradation. Moreover, users showed a high comfort level in using the proposed system.

Analytical Study of Perceptual and Motor Transparency in Bilateral Teleoperation

In bilateral teleoperation, a human operator manipulates a remote environment through a pair of master and slave robots. The transparency quantifies the fidelity of the teleoperation system, and is typically defined as the ability to accurately display remote environment properties to the operator. We propose a novel multidimensional measure of transparency which takes into account the human operator and consists of three components: 1) perceptual transparency, which quantifies human perception of the remote environment, 2) local motor transparency, which quantifies how far is the movement of the human operator from ideal, and 3) remote motor transparency, which describes how far is the movement of the remote device from ideal. We suggest that for many practical applications, the goal of the transparency optimization is to maintain perceptual and remote motor transparency while sacrificing local motor transparency, and that it is plausible to take advantage of the gap between perception and action in the operators sensorimotor system. We prove analytically that for a teleoperation channel with a position and force scaling and a constant transmission delay, in a palpation and perception of stiffness task, it is possible to find gains that ensure perfect perceptual and remote motor transparency while maintaining stability. We also show that stability depends on the operator that maintain sufficient arm impedance relative to environment impedance and delay.

Bilateral Control of Nonlinear Pneumatic Teleoperation System With Solenoid Valves

In past research on the control of pneumatic actuators, typically proportional servovalves have been used for achieving high-performance control of the mass flow rate. In this brief, we instead use fast-switching ON/OFF valves due to their distinct advantages in terms of low cost and small size. Accurate control of pneumatic actuators with ON/OFF solenoid valves is a challenge since the system dynamics is both discrete input and highly nonlinear. In this brief, we apply a hybrid control algorithm to a pneumatic actuator with ON/OFF valves. Such a control approach is developed for choosing the best control vector at each sample time to track the reference state (i.e., desired force) in the inner force control loop within a bilateral teleoperation system. Experimental results show that good teleoperation transparency is achieved despite all the obstacles such as discrete input and nonlinear behavior of the pneumatic-actuated teleoperation system.

A passivity criterion for sampled-data bilateral teleoperation systems.

A teleoperation system consists of a teleoperator, a human operator, and a remote environment. Conditions involving system and controller parameters that ensure the teleoperator passivity can serve as control design guidelines to attain maximum teleoperation transparency while maintaining system stability. In this paper, sufficient conditions for teleoperator passivity are derived for when position error-based controllers are implemented in discrete-time. This new analysis is necessary because discretization causes energy leaks and does not necessarily preserve the passivity of the system. The proposed criterion for sampled-data teleoperator passivity imposes lower bounds on the teleoperator's robots dampings, an upper bound on the sampling time, and bounds on the control gains. The criterion is verified through simulations and experiments.

Absolute stability analysis of sampled-data scaled bilateral teleoperation systems

Stability of a bilateral teleoperation system may be jeopardized by controller discretization, which has been shown to involve energy leaks. This paper proposes a novel approach to analyzing the absolute stability of sampled-data bilateral teleoperation systems consisting of discrete-time controllers and continuous-time master, slave, operator, and environment. The proposed stability analysis permits scaling and delay in the master and the slave positions and forces. The absolute stability conditions reported here impose bounds on the gains of the discrete-time controller, the damping terms of the master and the slave, and the sampling time. A design-related application of these results is in proper selection of various control parameters and the sampling rate for stable teleoperation under discrete-time control. To explore the trade-off between the control gains and the sampling time, it is studied that how large sampling times, which require low control gains for maintaining stability, can lead to unacceptable teleoperation transparency and human task performance in a teleoperated switching task. This shows that the effect of sampling time must be taken into account because neglecting it (as in the absolute stability literature) undermines both stability and transparency of teleoperation. The resulting absolute stability condition has been verified via experiments with two Phantom Omni robots.

Swarm-based structure-specified controller design for bilateral transparent teleoperation systems via synthesis

Swarm-based structure-specified controller design for bilateral transparent teleoperation systems via synthesis

A Robust LMI-Based Stabilizing Control Method for Bilateral Teleoperation Systems

This paper introduces a novel structure to design a robust PID controller based on a Linear Matrix Inequality (LMI) approach. The objectives of the controller establishment are to achieve complete transparency and robust stability despite the large time-delay of communication channels and uncertainty in various parts of a teleoperation system. In this structure two local controllers must be implemented by applying an LMI-based H∞ theory. One of these controllers is responsible for tracking the master position in the remote site which is called slave controller. The slave controller is a novel robust PID controller which is implemented by employing an LMI-based H∞ framework. Furthermore, the other one controller called master controller is in charge of force tracking as well as guaranteeing stability of the closed-loop system and it requires to be established by applying an LMI-based multiobjective H2/H∞ method. Finding optimal values for the parameters of the controllers can be ensured by applying the convex optimization method in both sites. As it can be seen from the simulation results, the proposed control method is highly effective in providing a stable transparent teleoperation system under uncertain, but bounded, time-delay in communication channel and task environment. Since the simulation results illustrates time responses of the system for nominal values of the parameters, μ-analysis has been used to investigate stability of the overall system in spite of changes in the parameters of the master and/or slave manipulators in modified range.

Delay-dependent stability for transparent bilateral teleoperation system: an LMI approach

There are two significant goals in teleoperation systems: Stability and performance. This paper introduces an LMI-based robust control method for bilateral transparent teleoperation systems in presence of model mismatch. The uncertainties in time delay in communication channel, task environment and model parameters of master-slave systems are called a model mismatch. The time delay in communication channel is assumed to be large, unknown and asymmetric, but the upper bound of the delay is assumed to be known. The proposed method consists of two local controllers. One local controller namely local slave controller is located on the remote site to control the motion tracking and the other one is located on the local site namely local master controller to preserve the complete transparency by ensuring force tracking and the robust stability of the closed-loop system. To reduce the peak amplitude of output signal respect to the peak amplitude of input signal in slave site, the local slave controller is designed based on a bounded peak-to-peak gain controller. In order to provide a realistic case, an external signal as a noise of force sensor is also considered. Simulation results show the effectiveness of proposed control structure.