Multilateral Teleoperation System Research Articles

Sliding Mode Impedance Control for Dual Hand Master Single Slave Teleoperation Systems

For the purpose of avoiding injury and realizing precise operations, the multilateral teleoperation system is the most efficient way to transport trace toxic or radioactive substances, to perform minimally invasive surgery, etc. It is essential to enhance the transparency of a multilateral teleoperation system including multiple masters and the single slave manipulator. However, there are few researchers focus on the allocation of the contact force of the single slave manipulator to different master manipulators. In this paper, we firstly introduce the concept of force translation for teleoperation systems consisting of dual hand master (left and right hands) manipulators and a single slave manipulator. Force translation reflects how the impedance on the single slave side is translated or allocated to contact forces on different master sides. To maintain the stability of the system and to improve transparency, we elucidate the mechanism of the force translation and analyze the relation among masters and the slave. Furthermore, the force translation mechanism is analyzed through numerical simulations and CHAI 3D virtual physical simulations. It is used to propose a multilateral impedance control, and the Lyapunov function is used to analyze the system stability. The results of numerical simulations and real robot experiments verify the effectiveness of the proposed control methods based on the proposed force translation mechanism.

An Improved Composite State Convergence Scheme with Disturbance Compensation for Multilateral Teleoperation Systems

An Improved Composite State Convergence Scheme with Disturbance Compensation for Multilateral Teleoperation Systems

Underwater multilateral tele-operation control with constant time delays

In the complex underwater environment, it is difficult to complete teleoperation tasks due to dynamic environments. Through cooperation among manipulators, a multilateral teleoperation system has the potential to achieve better performance than conventional bilateral teleoperation systems. In this paper, we propose a multilateral teleoperation control method that can be easily applied for underwater grasping tasks. Specifically, a new structure involving two (or multiple) masters and a single slave is investigated, which is treated as a linear combination of multiple subsystems under time delays. A control method based on linear matrix inequality exponential convergence and nonlinear disturbance is presented to estimate the model uncertainties of the system and unknown external disturbances during grasping process. The asymptotic stability is analyzed using the established Lyapunov–Krasovskii functional. Numerical simulations and real experiments performed for the task of grasping a plastic foam block are reported to verify the proposed control method in real applications.

A unifying framework for transparency optimized controller design in multilateral teleoperation with time delays

Multilateral teleoperation is an umbrella term for various N-robot haptic teleoperation schemes and contains bilateral teleoperation as a special case with N=2. While transparency of bilateral teleoperation systems can be analyzed through hybrid matrices and transparent controllers can be designed with four channel/Lawrence architectures, such a general analysis and design framework has not yet been established for multilateral teleoperation systems. Here, a framework is proposed to achieve transparency in multi-user shared control multilateral teleoperation systems even in the presence of time delays. First, a novel control law unifying a wide range of dominance factor, control and communication architectures is proposed. This control law enables the adaptation of all bilateral linear Lawrence architectures to multilateral teleoperation. Second, a novel transparency analysis approach is proposed and it is shown that TOPF architecture is the only Lawrence architecture that can potentially achieve ideal transparency in multilateral teleoperation under time delays. Third, based on proposed L2 stability criteria, a trade-off between stability robustness and transparency under varying time delays is demonstrated. Effectiveness of the proposed framework is validated with numerical analyses and experiments.

Hidden Markov-Model-Based Control Design for Multilateral Teleoperation System With Asymmetric Time-Varying Delays

This article focuses on investigating the synchronization and position/force tracking performance of the multilateral teleoperation system with asymmetric time delays. First, the synchronization and tracking error signals are proposed to transform the nonlinear dynamic system into a closed-loop system governed by the Markovian jumping parameter. Because of the presence of time delays, the information regarding the system states might become unavailable. Owing to this, a feedback control based on the hidden Markov model is developed through which the information on the current state of the actual system can be accessed through a series of observations. Moreover, the forward and backward delays of the master and slave manipulators are assumed to be asymmetric and varying with time. For the stability analysis, the Lyapunov–Krasovskii technique has been adopted for which its derivatives are dealt by employing Jensens’ inequality and extended reciprocal convex matrix inequality. Finally, simulation results were provided to validate the proposed methodology guaranteeing the closed-loop system to be asymptotically stable.

Multilateral Teleoperation Over Communication Time Delay Using the Time-Domain Passivity Approach

A general framework to stabilize multilateral teleoperation system over a communication time delay based on the well-known time-domain passivity approach (TDPA) is proposed. The uniqueness of this framework is that it is independent of the amount of communication time delay, the multilateral control architecture, and the number of masters and slaves. The multilateral system was first decoupled into subsystems with respect to each terminal by identifying the input signals’ contributions to the output terminals, which was not straightforward due to the coupled nature of the multilateral teleoperation system. The decoupled subsystems were then converted into an electrical circuit using a mechanical–electrical analogy. Time-delay power network (TDPN) was introduced to clarify active energy sources from the time delay and passivity observer/passivity controller (PO/PC) was utilized to dissipate those active energies. A less-conservative method compared with prior work was proposed to guarantee the stability. Experiments with a trilateral teleoperation system and with a multilateral teleoperation system with a dual master and dual slave were conducted to validate the proposed framework.

Three-channel control architecture for multilateral teleoperation under time delay

Multilateral teleoperation is an extension of bilateral/haptic teleoperation framework to multiple operators/robots and finds applications in haptic training. As in bilateral teleoperation, time delay is an important problem, and stability and transparency, which quantifies the performance of the teleoperation system, are critical in the design of multilateral control systems. This paper proposes a novel three-channel-based multilateral control architecture with damping injection to guarantee delay-independent L 2 stability and high transparency in multilateral teleoperation systems. The theoretical and computational analyses are verified with experiment results.

Multilateral Teleoperation With New Cooperative Structure Based on Reconfigurable Robots and Type-2 Fuzzy Logic.

This paper develops an innovative multilateral teleoperation system with two haptic devices on the master side and a newly designed reconfigurable multi-fingered robot on the slave side. A novel nonsingular fast terminal sliding-mode algorithm, together with varying dominance factors for cooperation, is proposed to offer this system's fast position and force tracking, as well as an integrated perception for the operator on the reconfigurable slave robot (manipulator). The Type-2 fuzzy model is used to describe the overall system dynamics, and accordingly a new fuzzy-model-based state observer is proposed to compensate for system uncertainties. A sliding-mode adaptive controller is designed to deal with the varying zero drift of the force sensors and force observers. The stability of the closed-loop system under time-varying delays is proved using Lyapunov-Krasovskii functions. Finally, experiments to grasp different objects are performed to verify the effectiveness of this multilateral teleoperation system.

Data-Driven Impedance Matching in Multilateral Teleoperation Systems

<span lang="EN-US">This paper addresses a tuning approach to improve transparency between master and slave manipulators in a multilateral teleoperation system. The slave manipulators comprised of multi-agents of simple mass are converted into a bilateral system through the passive decomposition technique. There are two main operations being considered, namely; grasping and handling. The object grasping is achieved through consensus control. For handling, an equalizer constructed by a Laguerre function connected in-feedback-loop to the master manipulator is chosen and tuned to achieve impedance matching between both sides of teleoperation. Fictitious Reference Iterative Tuning (FRIT) is adopted to properly tune the selected equalizer. The result shows that by introducing an optimally tuned equalizer, the overall impedance matching and transparency between the single master and multi-slave manipulators are improved. Finally, the effectiveness of the proposed tuning algorithm is presented through a numerical example.</span>

A Systematic Review of Multilateral Teleoperation Systems.

While conventional bilateral Single-Master/Single-Slave (SM/SS) teleoperation systems have received considerable attention during the past several decades, multilateral teleoperation is only recently being studied. Unlike an SM/SS system, which consists of one master-slave set, multilateral teleoperation frameworks involve a minimum of three agents in order to remotely perform a task. This paper presents an overview of multilateral teleoperation systems and classifies the existing state-of-the-art architecture based on topologies, applications, and closed-loop stability analysis. For each category, the review discusses control strategies used for various architectures as well as control challenges (e.g., closed-loop instability as a result of a delay in the communication network) for each methodology.

A Novel Wave-Variable Based Time-Delay Compensated Four-Channel Control Design for Multilateral Teleoperation System

Multilateral teleoperation system, which is regarded as an extension of bilateral teleoperation system, is a significant issue and becomes a hotspot in recent years to improve the ability of multiple robots to coordinate and perform efficiently and accurately in the complicated, dangerous and remote working environment. In this paper, a four-channel multilateral teleoperation control problem is discussed, where $n$ masters control the $n$ slaves to handle dangerous, unknown, and complicated tasks remotely under time-varying delays in the communication channel. A novel multilateral teleoperation system combining the time-delay compensator by the novel wave transform and four-channel architecture is built to meet the demands of the communication among multiple masters and slaves, where the weighing coefficients are provided to perform different weighed commands. The two design parameters are provided separately in the time-delay compensator to optimize the signal transmission and guarantee the stability in the four-channel architecture, and the ideal transparency conditions are theoretically derived to simultaneously achieve the stability and good transparency performance for multilateral teleoperation system. The practical experiments are carried out on a 2-master-2-slave teleoperation system to vilify the effectiveness of the proposed control scheme.

A Time Domain Passivity Approach for Asymmetric Multilateral Teleoperation System

Time-domain passivity control (TDPC) is a widely used reliable approach to ensure the stability of the teleoperation systems. Although TDPC almost guarantees the stability of the system, there can be a possibility of instability as TDPC inherits the zero division problem in its conventional control laws. Low value force or velocity signals can lead to zero division in control laws and can be a reason of control failure. A multilateral teleoperation system is much more complex as compared to a bilateral teleoperation system. A multilateral teleoperation system comprises of multiple humans and multiple master and slave robots which results in increased transmission of signals over a communication network. So, it is substantially invaluable for multilateral teleoperation system to have a control scheme which ensures a safe and stable operation. This paper, an extension of our previous work, presents a novel design of TDPC laws for multilateral teleoperation, which not only maintains the passivity of the system for stability but also avoids zero division, thus guaranteeing a stable operation. A new architecture of communication channel is introduced to assign different weights to the masters and slaves depending upon the task requirements. The control strategy proposed in this paper is valid for different numbers of masters and slaves. Simulation and experimental results are presented to demonstrate the efficacy of the control design.

Asymmetric wave variable compensation method in dual-master-dual-slave multilateral teleoperation system

Time delay existed in the communication channels leads to the appearance of some most challenging problems, such as decrease of stability and tracking performance, for dual-master-dual-slave (DMDS) teleoperation system. Based on the DMDS architecture, wave-variable-based method is proposed to handle those problems with the advantage of delay-independent passivity. To guarantee system stability and enhance the trajectory and force tracking performance, an asymmetric wave variable compensation method is presented, which compensates the backward wave variable by forward wave variable. Simulation and experiment results show the effectiveness of the proposed method in reducing tracking error and ensuring system stability.

An Extended State Convergence Architecture for Multilateral Teleoperation Systems

State convergence is a novel scheme to control a teleoperation system in a bilateral mode. Starting from modeling an $n^{\text {th}}$ order teleoperation system on state space, the scheme offers a simple and elegant procedure, which requires 3n+1 design conditions to be solved in order to synchronize the master and slave systems, and to achieve the desired dynamic behavior of the teleoperation system. However, in its current form, the scheme cannot be applied in situations where more than one master and/or slave systems are involved to perform a certain task. To overcome this limitation, we first present an alpha-modified version of the standard state convergence architecture for a single-master/single-slave teleoperation system. This alpha-modified architecture is then used to develop extended state convergence architecture for a multi-master/multi-slave teleoperation system. The resulting extended state convergence architecture requires solving a set of $n(k+l)+(n+1)kl$ design equations to determine the control gains for synchronizing $k$ -master and $l$ -slave systems in a desired dynamic way. MATLAB simulations considering a one-degree-of-freedom dual-master/tri-slave teleoperation system are presented to show the efficacy of the proposed extended state convergence architecture for multilateral teleoperation systems.

A novel multilateral teleoperation scheme with power-based time-domain passivity control

Multilateral teleoperation systems, which are extended from the traditional bilateral teleoperation, have become subject to increasing attention in current years, with increasing industrial requirements, such as the remote operation of larger objects and more complex tasks. In this paper, a general multilateral teleoperation control problem is discussed, in which n masters remotely control n slaves through delayed communication channels. A novel communication structure is proposed to satisfy the multiple master–slave communication requirement, in which weighting coefficients are chosen freely to perform the weighted effects of different masters or slaves. Power-based time-domain passivity control is subsequently developed for the complex multiple master–slave communication channel, to achieve the passivity of multilateral teleoperation systems under time delay. Experiments on a teleoperation system with two masters and two slaves are described; the results verify the effectiveness of the proposed control scheme.

Stability Control of Force-Reflected Nonlinear Multilateral Teleoperation System under Time-Varying Delays

A novel control algorithm based on the modified wave-variable controllers is proposed to achieve accurate position synchronization and reasonable force tracking of the nonlinear single-master-multiple-slave teleoperation system and simultaneously guarantee overall system’s stability in the presence of large time-varying delays. The system stability in different scenarios of human and environment situations has been analyzed. The proposed method is validated through experimental work based on the 3-DOF trilateral teleoperation system consisting of three different manipulators. The experimental results clearly demonstrate the feasibility of the proposed algorithm to achieve high transparency and robust stability in nonlinear single-master-multiple-slave teleoperation system in the presence of time-varying delays.

Integrated adaptive robust control for multilateral teleoperation systems under arbitrary time delays

SummaryWith the increasing industrial requirements such as bigger size object, stable operation, and complex task, multilateral teleoperation systems extended from traditional bilateral teleoperation are widely developed. In this paper, the integrated control design is developed for multilateral teleoperation systems, where n master manipulators are operated by human to remotely control n slave manipulators cooperatively handling a target object. For the first time, the control objectives of multilateral teleoperation including stability, synchronization, transparency, and internal force distribution are clarified systematically. A novel communication architecture is proposed to cope with communication delays, where the estimated environmental parameters are transmitted from the slave side to the master, to replace the traditional environmental force measurement in the communication channel. A kind of nonlinear adaptive robust control technique is used to deal with nonlinearities, unknown parameters, and modeling uncertainties existing in the master, slave, and environmental dynamics, so that the excellent tracking performance is achieved in both master and slave sides. The coordinated motion/force control is designed in the slave side by the optimal internal force distribution among n slave manipulators, and the impedance control is designed in the master side to realize the target transparency behavior. In summary, the proposed control algorithm can achieve the guaranteed robust stability, the excellent synchronization and transparency performance, and the optimal internal force distribution simultaneously for multilateral teleoperation systems under arbitrary time delays and various modeling uncertainties. The simulation is carried out on a 2‐master/2‐slave teleoperation system, and the results show the effectiveness of the proposed control design. Copyright © 2015 John Wiley & Sons, Ltd.

A method for passivity analysis of multilateral haptic systems

This paper presents a novel criterion to study the stability of multilateral teleoperation systems based on passivity. Such systems (modeled as n-port networks) have recently found interesting applications in cooperative haptic teleoperation and haptic-assisted training. The criterion provides researchers with an analytical, closed-form, necessary, and sufficient condition useful for both analysis and design of multilateral haptic teleoperation systems. The paper shows that when the proposed conditions reduce to the well-known Raisbeck’s passivity criterion for 2-port networks. The proposed conditions are used to study the passivity (and consequently the stability) of a dual-user haptic system for control of a single teleoperated robot. Simulations and experiments are performed to further test the validity of the proposed criterion.

2A2-P01 計算機ネットワークを介した力覚情報共有のためのマルチラテラル・テレオペレーションシステムの構築(ネットワークロボティクス)

This paper reports the construction of a multilateral teleoperation system over the time-delayed computer network. A wave-variables-based control method is introduced to ensure the passivity of the system against time delay of the network. A control software, which supports commercial haptic devices, is developed and an experimental system with three devices is constructed. Experiments are conducted to see the stability of the control method and usefulness of the multilateral teleoperation.