Multilateral Teleoperation Research Articles

Sliding Mode Impedance Control for Dual Hand Master Single Slave Teleoperation Systems

For the purpose of avoiding injury and realizing precise operations, the multilateral teleoperation system is the most efficient way to transport trace toxic or radioactive substances, to perform minimally invasive surgery, etc. It is essential to enhance the transparency of a multilateral teleoperation system including multiple masters and the single slave manipulator. However, there are few researchers focus on the allocation of the contact force of the single slave manipulator to different master manipulators. In this paper, we firstly introduce the concept of force translation for teleoperation systems consisting of dual hand master (left and right hands) manipulators and a single slave manipulator. Force translation reflects how the impedance on the single slave side is translated or allocated to contact forces on different master sides. To maintain the stability of the system and to improve transparency, we elucidate the mechanism of the force translation and analyze the relation among masters and the slave. Furthermore, the force translation mechanism is analyzed through numerical simulations and CHAI 3D virtual physical simulations. It is used to propose a multilateral impedance control, and the Lyapunov function is used to analyze the system stability. The results of numerical simulations and real robot experiments verify the effectiveness of the proposed control methods based on the proposed force translation mechanism.

An Improved Composite State Convergence Scheme with Disturbance Compensation for Multilateral Teleoperation Systems

An Improved Composite State Convergence Scheme with Disturbance Compensation for Multilateral Teleoperation Systems

Underwater multilateral tele-operation control with constant time delays

In the complex underwater environment, it is difficult to complete teleoperation tasks due to dynamic environments. Through cooperation among manipulators, a multilateral teleoperation system has the potential to achieve better performance than conventional bilateral teleoperation systems. In this paper, we propose a multilateral teleoperation control method that can be easily applied for underwater grasping tasks. Specifically, a new structure involving two (or multiple) masters and a single slave is investigated, which is treated as a linear combination of multiple subsystems under time delays. A control method based on linear matrix inequality exponential convergence and nonlinear disturbance is presented to estimate the model uncertainties of the system and unknown external disturbances during grasping process. The asymptotic stability is analyzed using the established Lyapunov–Krasovskii functional. Numerical simulations and real experiments performed for the task of grasping a plastic foam block are reported to verify the proposed control method in real applications.

A unifying framework for transparency optimized controller design in multilateral teleoperation with time delays

Multilateral teleoperation is an umbrella term for various N-robot haptic teleoperation schemes and contains bilateral teleoperation as a special case with N=2. While transparency of bilateral teleoperation systems can be analyzed through hybrid matrices and transparent controllers can be designed with four channel/Lawrence architectures, such a general analysis and design framework has not yet been established for multilateral teleoperation systems. Here, a framework is proposed to achieve transparency in multi-user shared control multilateral teleoperation systems even in the presence of time delays. First, a novel control law unifying a wide range of dominance factor, control and communication architectures is proposed. This control law enables the adaptation of all bilateral linear Lawrence architectures to multilateral teleoperation. Second, a novel transparency analysis approach is proposed and it is shown that TOPF architecture is the only Lawrence architecture that can potentially achieve ideal transparency in multilateral teleoperation under time delays. Third, based on proposed L2 stability criteria, a trade-off between stability robustness and transparency under varying time delays is demonstrated. Effectiveness of the proposed framework is validated with numerical analyses and experiments.

Hidden Markov-Model-Based Control Design for Multilateral Teleoperation System With Asymmetric Time-Varying Delays

This article focuses on investigating the synchronization and position/force tracking performance of the multilateral teleoperation system with asymmetric time delays. First, the synchronization and tracking error signals are proposed to transform the nonlinear dynamic system into a closed-loop system governed by the Markovian jumping parameter. Because of the presence of time delays, the information regarding the system states might become unavailable. Owing to this, a feedback control based on the hidden Markov model is developed through which the information on the current state of the actual system can be accessed through a series of observations. Moreover, the forward and backward delays of the master and slave manipulators are assumed to be asymmetric and varying with time. For the stability analysis, the Lyapunov–Krasovskii technique has been adopted for which its derivatives are dealt by employing Jensens’ inequality and extended reciprocal convex matrix inequality. Finally, simulation results were provided to validate the proposed methodology guaranteeing the closed-loop system to be asymptotically stable.

Multi-Lateral Teleoperation Based on Multi-Agent Framework: Application to Simultaneous Training and Therapy in Telerehabilitation.

In this paper, a new scheme for multi-lateral remote rehabilitation is proposed. There exist one therapist, one patient, and several trainees, who are participating in the process of telerehabilitation (TR) in this scheme. This kind of strategy helps the therapist to facilitate the neurorehabilitation remotely. Thus, the patients can stay in their homes, resulting in safer and less expensive costs. Meanwhile, several trainees in medical education centers can be trained by participating partially in the rehabilitation process. The trainees participate in a “hands-on” manner; so, they feel like they are rehabilitating the patient directly. For implementing such a scheme, a novel theoretical method is proposed using the power of multi-agent systems (MAS) theory into the multi-lateral teleoperation, based on the self-intelligence in the MAS. In the previous related works, changing the number of participants in the multi-lateral teleoperation tasks required redesigning the controllers; while, in this paper using both of the decentralized control and the self-intelligence of the MAS, avoids the need for redesigning the controller in the proposed structure. Moreover, in this research, uncertainties in the operators' dynamics, as well as time-varying delays in the communication channels, are taken into account. It is shown that the proposed structure has two tuning matrices (L and D) that can be used for different scenarios of multi-lateral teleoperation. By choosing proper tuning matrices, many related works about the multi-lateral teleoperation/telerehabilitation process can be implemented. In the final section of the paper, several scenarios were introduced to achieve “Simultaneous Training and Therapy” in TR and are implemented with the proposed structure. The results confirmed the stability and performance of the proposed framework.

Robust control design for teleoperation of multiple mobile manipulators under time delays

SummaryThis article addresses a novel multilateral teleoperation control scheme for single‐master multiple slave systems, which can be extended to n masters and n slaves without the loss of generality, where the master is a m degrees of freedom (DOF) manipulator arm and the slaves are m DOF mobile manipulators. The human operator operates the master robot to remotely control the slaves handling a target object. The master position signal is transmitted to the slave side to generate a desired object trajectory as well as the reference mobile base velocity. An adaptive robust controller is designed for the slaves to follow the desired trajectory from the master, which not only provides the excellent trajectory tracking but also optimize the internal force distribution of the object. A null space controller is designed for the mobile platforms of the mobile manipulators to achieve the velocity consensus while achieving the main task of object transportation. The novel control design uses the transmission of the environmental force feedback over the communication channel by the estimated parameters of the environment, which helps retain the stability of the overall system. The environmental force is predicted on the master side based on the estimated environmental parameters. The proposed control design can simultaneously achieve the objectives of stability, synchronization, and optimal internal force distribution. The simulation results of a single‐master and three slaves teleoperation system validate the efficacy of the proposed control algorithm.

Multilateral Teleoperation Over Communication Time Delay Using the Time-Domain Passivity Approach

A general framework to stabilize multilateral teleoperation system over a communication time delay based on the well-known time-domain passivity approach (TDPA) is proposed. The uniqueness of this framework is that it is independent of the amount of communication time delay, the multilateral control architecture, and the number of masters and slaves. The multilateral system was first decoupled into subsystems with respect to each terminal by identifying the input signals’ contributions to the output terminals, which was not straightforward due to the coupled nature of the multilateral teleoperation system. The decoupled subsystems were then converted into an electrical circuit using a mechanical–electrical analogy. Time-delay power network (TDPN) was introduced to clarify active energy sources from the time delay and passivity observer/passivity controller (PO/PC) was utilized to dissipate those active energies. A less-conservative method compared with prior work was proposed to guarantee the stability. Experiments with a trilateral teleoperation system and with a multilateral teleoperation system with a dual master and dual slave were conducted to validate the proposed framework.

A cooperative paradigm for task-space control of multilateral nonlinear teleoperation with bounded inputs and time-varying delays

In this article, a novel multilateral teleoperation structure is presented for cooperative control of a redundant remote robot in the task-space. In the structure, each local robot is assigned a real number between zero and one, as its dominance factor. The sum of all the assigned factors is equal to one, and a greater dominance factor implies that its corresponding operator will have more power in controlling the remote robot. The time-varying delays in the communication channels, the actuators saturation, and the nonlinear dynamics are incorporated into the controller development. Using Barbalat’s lemma and a Lyapunov-Krasovskii functional, the asymptotic stability analysis of the closed-loop dynamics are shown, and the performance claims are delivered. Simulation and experimental results are provided to verify the theoretical findings.

Three-channel control architecture for multilateral teleoperation under time delay

Multilateral teleoperation is an extension of bilateral/haptic teleoperation framework to multiple operators/robots and finds applications in haptic training. As in bilateral teleoperation, time delay is an important problem, and stability and transparency, which quantifies the performance of the teleoperation system, are critical in the design of multilateral control systems. This paper proposes a novel three-channel-based multilateral control architecture with damping injection to guarantee delay-independent L 2 stability and high transparency in multilateral teleoperation systems. The theoretical and computational analyses are verified with experiment results.

Distributed Multilateral Teleoperation Framework Using Passivity-Shortage

In this paper, we propose a multilateral teleoperation framework, to cooperatively control a distributed multi-agent system, using passivity-short controllers. The output of the multi-agent system is designed to follow a normal distribution and desired performance is achieved by controlling the features of distribution. The concept of incremental passivity-shortage is introduced, and it is used to achieve convergence of input-output error. The results show that the proposed method has L2 stability and zero convergence error, even in the presence of communication delays. We also provide an example, that shows delay-independent performance, for both symmetric and asymmetric multilateral teleoperation. The proposed method does not have an upper-bound on the number of agents to be controlled.

Multilateral Teleoperation With New Cooperative Structure Based on Reconfigurable Robots and Type-2 Fuzzy Logic.

This paper develops an innovative multilateral teleoperation system with two haptic devices on the master side and a newly designed reconfigurable multi-fingered robot on the slave side. A novel nonsingular fast terminal sliding-mode algorithm, together with varying dominance factors for cooperation, is proposed to offer this system's fast position and force tracking, as well as an integrated perception for the operator on the reconfigurable slave robot (manipulator). The Type-2 fuzzy model is used to describe the overall system dynamics, and accordingly a new fuzzy-model-based state observer is proposed to compensate for system uncertainties. A sliding-mode adaptive controller is designed to deal with the varying zero drift of the force sensors and force observers. The stability of the closed-loop system under time-varying delays is proved using Lyapunov-Krasovskii functions. Finally, experiments to grasp different objects are performed to verify the effectiveness of this multilateral teleoperation system.

Data-Driven Impedance Matching in Multilateral Teleoperation Systems

<span lang="EN-US">This paper addresses a tuning approach to improve transparency between master and slave manipulators in a multilateral teleoperation system. The slave manipulators comprised of multi-agents of simple mass are converted into a bilateral system through the passive decomposition technique. There are two main operations being considered, namely; grasping and handling. The object grasping is achieved through consensus control. For handling, an equalizer constructed by a Laguerre function connected in-feedback-loop to the master manipulator is chosen and tuned to achieve impedance matching between both sides of teleoperation. Fictitious Reference Iterative Tuning (FRIT) is adopted to properly tune the selected equalizer. The result shows that by introducing an optimally tuned equalizer, the overall impedance matching and transparency between the single master and multi-slave manipulators are improved. Finally, the effectiveness of the proposed tuning algorithm is presented through a numerical example.</span>

A Systematic Review of Multilateral Teleoperation Systems.

While conventional bilateral Single-Master/Single-Slave (SM/SS) teleoperation systems have received considerable attention during the past several decades, multilateral teleoperation is only recently being studied. Unlike an SM/SS system, which consists of one master-slave set, multilateral teleoperation frameworks involve a minimum of three agents in order to remotely perform a task. This paper presents an overview of multilateral teleoperation systems and classifies the existing state-of-the-art architecture based on topologies, applications, and closed-loop stability analysis. For each category, the review discusses control strategies used for various architectures as well as control challenges (e.g., closed-loop instability as a result of a delay in the communication network) for each methodology.

A Novel Wave-Variable Based Time-Delay Compensated Four-Channel Control Design for Multilateral Teleoperation System

Multilateral teleoperation system, which is regarded as an extension of bilateral teleoperation system, is a significant issue and becomes a hotspot in recent years to improve the ability of multiple robots to coordinate and perform efficiently and accurately in the complicated, dangerous and remote working environment. In this paper, a four-channel multilateral teleoperation control problem is discussed, where $n$ masters control the $n$ slaves to handle dangerous, unknown, and complicated tasks remotely under time-varying delays in the communication channel. A novel multilateral teleoperation system combining the time-delay compensator by the novel wave transform and four-channel architecture is built to meet the demands of the communication among multiple masters and slaves, where the weighing coefficients are provided to perform different weighed commands. The two design parameters are provided separately in the time-delay compensator to optimize the signal transmission and guarantee the stability in the four-channel architecture, and the ideal transparency conditions are theoretically derived to simultaneously achieve the stability and good transparency performance for multilateral teleoperation system. The practical experiments are carried out on a 2-master-2-slave teleoperation system to vilify the effectiveness of the proposed control scheme.

A Time Domain Passivity Approach for Asymmetric Multilateral Teleoperation System

Time-domain passivity control (TDPC) is a widely used reliable approach to ensure the stability of the teleoperation systems. Although TDPC almost guarantees the stability of the system, there can be a possibility of instability as TDPC inherits the zero division problem in its conventional control laws. Low value force or velocity signals can lead to zero division in control laws and can be a reason of control failure. A multilateral teleoperation system is much more complex as compared to a bilateral teleoperation system. A multilateral teleoperation system comprises of multiple humans and multiple master and slave robots which results in increased transmission of signals over a communication network. So, it is substantially invaluable for multilateral teleoperation system to have a control scheme which ensures a safe and stable operation. This paper, an extension of our previous work, presents a novel design of TDPC laws for multilateral teleoperation, which not only maintains the passivity of the system for stability but also avoids zero division, thus guaranteeing a stable operation. A new architecture of communication channel is introduced to assign different weights to the masters and slaves depending upon the task requirements. The control strategy proposed in this paper is valid for different numbers of masters and slaves. Simulation and experimental results are presented to demonstrate the efficacy of the control design.

Asymmetric wave variable compensation method in dual-master-dual-slave multilateral teleoperation system

Time delay existed in the communication channels leads to the appearance of some most challenging problems, such as decrease of stability and tracking performance, for dual-master-dual-slave (DMDS) teleoperation system. Based on the DMDS architecture, wave-variable-based method is proposed to handle those problems with the advantage of delay-independent passivity. To guarantee system stability and enhance the trajectory and force tracking performance, an asymmetric wave variable compensation method is presented, which compensates the backward wave variable by forward wave variable. Simulation and experiment results show the effectiveness of the proposed method in reducing tracking error and ensuring system stability.

Multilateral teleoperation under asymmetric time delays: L2 stability and robustness

Multilateral teleoperation under asymmetric time delays: <i>L</i>₂ stability and robustness

An Extended State Convergence Architecture for Multilateral Teleoperation Systems

State convergence is a novel scheme to control a teleoperation system in a bilateral mode. Starting from modeling an $n^{\text {th}}$ order teleoperation system on state space, the scheme offers a simple and elegant procedure, which requires 3n+1 design conditions to be solved in order to synchronize the master and slave systems, and to achieve the desired dynamic behavior of the teleoperation system. However, in its current form, the scheme cannot be applied in situations where more than one master and/or slave systems are involved to perform a certain task. To overcome this limitation, we first present an alpha-modified version of the standard state convergence architecture for a single-master/single-slave teleoperation system. This alpha-modified architecture is then used to develop extended state convergence architecture for a multi-master/multi-slave teleoperation system. The resulting extended state convergence architecture requires solving a set of $n(k+l)+(n+1)kl$ design equations to determine the control gains for synchronizing $k$ -master and $l$ -slave systems in a desired dynamic way. MATLAB simulations considering a one-degree-of-freedom dual-master/tri-slave teleoperation system are presented to show the efficacy of the proposed extended state convergence architecture for multilateral teleoperation systems.

A novel multilateral teleoperation scheme with power-based time-domain passivity control

Multilateral teleoperation systems, which are extended from the traditional bilateral teleoperation, have become subject to increasing attention in current years, with increasing industrial requirements, such as the remote operation of larger objects and more complex tasks. In this paper, a general multilateral teleoperation control problem is discussed, in which n masters remotely control n slaves through delayed communication channels. A novel communication structure is proposed to satisfy the multiple master–slave communication requirement, in which weighting coefficients are chosen freely to perform the weighted effects of different masters or slaves. Power-based time-domain passivity control is subsequently developed for the complex multiple master–slave communication channel, to achieve the passivity of multilateral teleoperation systems under time delay. Experiments on a teleoperation system with two masters and two slaves are described; the results verify the effectiveness of the proposed control scheme.