Trilateral Teleoperation Research Articles

A Two-Layer Trilateral Teleoperation Architecture for Mentoring in Surgical Training

Abstract In this paper we propose a novel trilateral dual-master-single-slave teleoperation control architecture that can be used for the training of novice surgeons in surgical procedures. Starting from the concept of energy tank, we propose a flexible and stable trilateral interconnection over a delayed communication channel between the masters and the slave. Exploiting the flexibility provided by the controller, we design a training strategy for novice surgeons. The proposed architecture is experimentally validated.

Adaptive Control for Dual-Master/Single-Slave Nonlinear Teleoperation Systems With Time-Varying Communication Delays

In this article, two types of new control strategies are proposed with respect to dual-master/single-slave trilateral teleoperation systems in the presence of symmetrical and unsymmetrical time-varying communication delay, respectively. First, the problems of stability and expected position coordination of trilateral teleoperation system are solved on the basis of controller design of bilateral teleoperation system. Second, some impedance features of interaction forces between human and dual-master as well as single-slave and remote environment are measured by adaptive control strategies. Adaptive control methods are also used to deal with the problem that is relevant to gravities loading of dual-master and single-slave manipulators. Furthermore, the stability of trilateral teleoperation systems under the symmetrical and unsymmetrical time-varying communication delay is analyzed through constructing suitable delay-dependent Lyapunov-Krasovskii functionals. Finally, simulation results are presented to verify the effectiveness of proposed control strategies.

Multilateral Teleoperation Over Communication Time Delay Using the Time-Domain Passivity Approach

A general framework to stabilize multilateral teleoperation system over a communication time delay based on the well-known time-domain passivity approach (TDPA) is proposed. The uniqueness of this framework is that it is independent of the amount of communication time delay, the multilateral control architecture, and the number of masters and slaves. The multilateral system was first decoupled into subsystems with respect to each terminal by identifying the input signals’ contributions to the output terminals, which was not straightforward due to the coupled nature of the multilateral teleoperation system. The decoupled subsystems were then converted into an electrical circuit using a mechanical–electrical analogy. Time-delay power network (TDPN) was introduced to clarify active energy sources from the time delay and passivity observer/passivity controller (PO/PC) was utilized to dissipate those active energies. A less-conservative method compared with prior work was proposed to guarantee the stability. Experiments with a trilateral teleoperation system and with a multilateral teleoperation system with a dual master and dual slave were conducted to validate the proposed framework.

Teleoperación Trilateral con Retardo de un Robot Móvil

Este artículo analiza la estabilidad de un sistema de teleoperación trilateral de un robot móvil. La dinámica de dos manipuladores maestros de estructura similar y un robot móvil, asimismo se tienen en cuenta retrasos variables en el tiempo y asimétricos. Un conjunto de controladores P + D se incluyen en el sistema de teleoperación con retardo. A partir del análisis realizado, se propone un procedimiento para establecer los parámetros de control con el fin de asegurar la estabilidad del sistema. Finalmente, el resultado teórico es verificado en la práctica por medio de pruebas sobre el sistema de teleoperación trilateral con retardo donde dos operadores humanos en colaboración y simultáneamente para conducen en un simulador 3D un robot móvil para alcanzar una tarea establecida en un entorno remoto.

Adaptive neural network based position tracking control for Dual-master/Single-slave teleoperation system under communication constant time delays

The novel trajectory tracking control strategies for trilateral teleoperation systems with Dual-master/Single-slave robot manipulators under communication constant time delays are proposed in this article. By incorporating this design technique into the neural network (NN) based adaptive control framework, two controllers are designed for the trilateral teleoperation systems in free motion. First, with acceleration measurements, an adaptive controller under the synchronization variables containing the position and velocity error is constructed to guarantee the position and velocity tracking errors between the trilateral teleoperation systems asymptotically converge to zero. Second, without acceleration measurements, an adaptive controller under the new synchronization variables is presented such that the trilateral teleoperation systems can obtain the same trajectory tracking performance as the first controller. Third, in term of establishing suitable Lyapunov–Krasovskii functionals, the asymptotic tracking performances of the trilateral teleoperation systems can be derived independent of the communication constant time delays. Moreover, these two controllers are obtained without the knowledge of upper bounds of the NN approximation errors, respectively. Finally, simulation results are presented to demonstrate the validity of the proposed methods.

Stable Nonlinear Trilateral Impedance Control for Dual-User Haptic Teleoperation Systems With Communication Delays

A new nonlinear adaptive impedance-based trilateral controller is proposed to ensure the absolute stability of multi-degrees-of-freedom (DOFs) dual-user haptic teleoperation systems subjected to communication delays. Using this strategy, reference impedance models are realized for the trilateral teleoperation system represented by a three-port network to facilitate cooperation of two human operators in order to perform a remote physical task. For this purpose, an impedance model defines the desired haptic interaction between the two human operators, while another impedance model specifies the desired behavior of the slave robot in terms of tracking the mater robots' trajectories during interaction with the remote environment. It is shown that different performance goals such as position synchronization and force reflection can be achieved via different adjustments to the impedance parameters. The sufficient conditions for the trilateral haptic system's absolute stability are investigated in terms of the impedance models' parameters. Accordingly, guidelines for modification of the impedance parameters are obtained to guarantee the absolute stability of the trilateral haptic system in the presence of communication time delays. A trilateral nonlinear version of the model reference adaptive impedance control (MRAIC) scheme is developed for implementing the proposed reference impedance models on the masters and the slave. The convergence of robots' trajectories to desired responses and the robustness against modeling uncertainties are ensured using the proposed controller as proven by the Lyapunov stability theorem. The proposed impedance-based control strategy is evaluated experimentally by employing a nonlinear multi-DOFs teleoperated trilateral haptic system with and without communication delays.

Dual-user nonlinear teleoperation subjected to varying time delay and bounded inputs

A novel trilateral control architecture for Dual-master/Single-slave teleoperation system with taking account of saturation in actuators, nonlinear dynamics for telemanipulators and bounded varying time delay which affects the transmitted signals in the communication channels, is proposed in this paper. In this research, we will address the stability and desired position coordination problem of trilateral teleoperation system by extension of (nP+D) controller that is used for Single-master/Single-slave teleoperation system. Our proposed controller is weighted summation of nonlinear Proportional plus Damping (nP+D) controller that incorporate gravity compensation and the weights are specified by the dominance factor, which determines the supremacy of each user over the slave robot and over the other user. The asymptotic stability of closed loop dynamics is studied using Lyapunov-Krasovskii functional under conditions on the controller parameters, the actuator saturation characteristics and the maximum values of varying time delays. It is shown that these controllers satisfy the desired position coordination problem in free motion condition. To show the effectiveness of the proposed method, a number of simulations have been conducted on a varying time delay Dual-master/Single-slave teleoperation system using 3-DOF planar robots for each telemanipulator subjected to actuator saturation.

Delayed Trilateral Teleoperation of a Mobile Robot

This paper analyzes the stability of a trilateral teleoperation system of a mobile robot. This type of system is nonlinear, time-varying, and delayed and includes a master-slave kinematic dissimilarity. To close the control loop, three P+d controllers are used under a position master/slave velocity strategy. The stability analysis is based on Lyapunov-Krasovskii theory where a functional is proposed and analyzed to get conditions for the control parameters that assure a stable behavior, keeping the synchronism errors bounded. Finally, the theoretical result is verified in practice by means of a simple test, where two human operators both collaboratively and simultaneously drive a 3D simulator of a mobile robot to achieve an established task on a remote shared environment.

Nonlinear trilateral teleoperation stability analysis subjected to time-varying delays

A trilateral teleoperation system facilitates the collaboration of two users to share control of a single robot in a remote environment. While various applications of shared-control trilateral haptic teleoperation systems have recently emerged, they have mostly been studied in the context of single-DOF, LTI robotic systems. On the other hand, robotic manipulators with multiple degrees of freedom (DOF) and therefore nonlinear dynamics have recently found many applications such as in robotic-assisted surgery and therapy, space exploration and navigation systems. In this paper, considering the full nonlinear dynamical models of multi-DOF robots, stability analysis of a dual-user haptic teleoperation system is considered over a communication network subjected to asymmetrical time varying delays and through a dominance factor suitable for trainer–trainee applications. Stability in free motion and contact motion and asymptotic position tracking of the trilateral haptic teleoperation system in free motion are proven via Lyapunov stability analysis and Barbalat's lemma where operators and the environment are assumed to be passive. Simulation and experimental results concerning robot position tracking and user-perceived forces for three 2-DOF robots and experimental analysis of user-perceived stiffnesses for three 3-DOF robots validate the theoretical findings pertaining to the system stability and demonstrate the efficiency of the proposed controller.

Trilateral Predictor-Mediated Teleoperation of a Wheeled Mobile Robot With Slippage

With the widespread use of wheeled mobile robots (WMR) in various applications, new challenges have emerged in terms of designing its teleoperation system. One of such challenges is caused by wheel slippage and another is due to the strict need for ensuring WMR safety. This letter proposes a new trilateral teleoperation scheme for haptic teleoperation control of a WMR (slave) with longitudinal slippage. In this teleoperation system, a virtual model (predictor) of the slave WMR is utilized at the master site to guide the human operator to issue more effective commands and, by mediating between the master and the slave WMR, ensure that unsafe maneuvers are not performed by the WMR. Besides compensation for the WMR/terrain’s nonpassivity caused by the slippage, a shared control is proposed for the system. Theoretically, the system’s stability is shown via its passivity and it is shown that the force felt by the human operator is approximately equal to the forces applied by the environment of the predictor plus that of the slave robot, which is a satisfactory performance outcome. Experiments of the proposed WMR teleoperation system demonstrate that it results in stable trilateral teleoperation with a satisfactory tracking performance. The predictor at the master site is shown to compensate for lack of precise information about the slave robot.

Stability Control of Force-Reflected Nonlinear Multilateral Teleoperation System under Time-Varying Delays

A novel control algorithm based on the modified wave-variable controllers is proposed to achieve accurate position synchronization and reasonable force tracking of the nonlinear single-master-multiple-slave teleoperation system and simultaneously guarantee overall system’s stability in the presence of large time-varying delays. The system stability in different scenarios of human and environment situations has been analyzed. The proposed method is validated through experimental work based on the 3-DOF trilateral teleoperation system consisting of three different manipulators. The experimental results clearly demonstrate the feasibility of the proposed algorithm to achieve high transparency and robust stability in nonlinear single-master-multiple-slave teleoperation system in the presence of time-varying delays.

Neural Network-Based Control of Networked Trilateral Teleoperation With Geometrically Unknown Constraints.

Most studies on bilateral teleoperation assume known system kinematics and only consider dynamical uncertainties. However, many practical applications involve tasks with both kinematics and dynamics uncertainties. In this paper, trilateral teleoperation systems with dual-master-single-slave framework are investigated, where a single robotic manipulator constrained by an unknown geometrical environment is controlled by dual masters. The network delay in the teleoperation system is modeled as Markov chain-based stochastic delay, then asymmetric stochastic time-varying delays, kinematics and dynamics uncertainties are all considered in the force-motion control design. First, a unified dynamical model is introduced by incorporating unknown environmental constraints. Then, by exact identification of constraint Jacobian matrix, adaptive neural network approximation method is employed, and the motion/force synchronization with time delays are achieved without persistency of excitation condition. The neural networks and parameter adaptive mechanism are combined to deal with the system uncertainties and unknown kinematics. It is shown that the system is stable with the strict linear matrix inequality-based controllers. Finally, the extensive simulation experiment studies are provided to demonstrate the performance of the proposed approach.

Trilateral Teleoperation of Adaptive Fuzzy Force/Motion Control for Nonlinear Teleoperators With Communication Random Delays

In this paper, an adaptive fuzzy control scheme is proposed for hybrid motion/force of trilateral teleoperation systems with a dual-master-single-slave configuration under stochastic time-varying delays in communication channels. Different from previous works on bilateral teleoperation systems, this paper addresses dual-master trilateral control of a single holonomic-constrained robotic manipulator, where the communication delays are modeled as multiple Markov chains, and the motion/force controls are investigated under consideration of unsymmetric stochastic time-varying delays and system dynamical uncertainties. Using partial feedback linearization, the whole trilateral teleoperation system, which consists of both master and slave manipulator dynamics, is transformed into three subsystems. By integrating Markov jump systems to handle random delays, adaptive fuzzy control strategies are developed for the nonlinear teleoperators with modeling uncertainties and external disturbances by using the approximation property of the fuzzy logic systems (FLSs). It is proven that the trilateral teleoperation system is stochastically stable in mean square under specific linear matrix inequality (LMI) conditions, and all the signals of the resulting closed-loop system are uniformly bounded. The proposed scheme is validated by extensive simulations.

3-DOF Trilateral Teleoperation Using A Pair of 1-DOF and 2-DOF Haptic Devices: Stability Analysis

Abstract Humans are usually better than autonomous robots in operating in complex environments. In bilateral teleoperation, to take full advantage of the human's intelligence, experience, and sensory inputs for performing a dexterous task, a possibility is to use the two hands of the user to manipulate two master haptic devices in order to control a slave robot with multiple degrees-of-freedom (DOF); the total DOFs of the two masters are equal to the DOFs of the slave. In this paper, two 1-DOF and 2-DOF haptic robots are considered as the two masters while a 3-DOF robot acts as the slave in a trilateral teleoperation system. It is discussed how such a system can result in better task performance by splitting the various DOFs of a dexterous task between two hands or two users, e.g., during peg-in-the-hole insertion. The stability analysis of such a system is not trivial due to dynamic coupling across the different DOFs of the robots, the human operators, and the physical/virtual environments. Also, the unknown dynamics of the users and the environments exacerbate the problem. We present a novel, straightforward and convenient method for stability analysis of this teleoperation system. Simulation results demonstrate the validity of the approach.

A novel control framework for nonlinear time-delayed Dual-master/Single-slave teleoperation

A novel trilateral control architecture for the Dual-master/Single-slave teleoperation is proposed in this paper. This framework has been used in surgical training and rehabilitation applications. In this structure, the slave motion has been controlled by weighted summation of signals transmitted by the operator referring to task control authority through the dominance factors.The nonlinear dynamics for telemanipulators are considered which were considered as disregarded issues in previous studies of this field. Bounded variable time-delay has been considered which affects the transmitted signals in the communication channels. Two types of controllers have been offered and an appropriate stability analysis for each controller has been demonstrated. The first controller includes Proportional with dissipative gains (P+d). The second one contains Proportional and Derivative with dissipative gains (PD+d). In both cases, the stability of the trilateral control framework is preserved by choosing appropriate controller's gains. It is shown that these controllers attempt to coordinate the positions of telemanipulators in the free motion condition. The stability of the Dual-master/Single-slave teleoperation has been proved by an appropriate Lyapunov like function and the stability conditions have been studied. In addition the proposed PD+d control architecture is modified for trilateral teleoperation with internet communication between telemanipulators that caused such communication complications as packet loss, data duplication and swapping. A number of experiments have been conducted with various levels of dominance factor to validate the effectiveness of the new control architecture.

A Kinematic Control Framework for Single-Slave Asymmetric Teleoperation Systems

This paper is concerned with asymmetrical teleoperation, where the master/slave subsystems have different degrees of mobility. In particular, dual-master trilateral control of a possibly kinematically redundant slave robot (KRSR) and single-master control of a kinematically deficient slave robot (KDSR) are considered. In the case of a KDSR, the motion of the master robot is restricted to the natural motion constraint of the slave robot. Trilateral teleoperation is achieved via two master devices, each controlling a dedicated frame that is assigned on the slave robot. A novel control framework is presented that accomplishes two objectives: 1) motion and force tracking of the master and slave robots within their nonconstrained task spaces and 2) constrained master robot(s) motion that reflects the slave natural constraint (KDSR) or the kinematic constraint on each of the slave taskspace control frames (trilateral teleoperation). The proposed adaptive controller utilizes projection and generalized pseudoinverse matrices to achieve the stated teleoperation objectives. Stability and transparency of the asymmetric teleoperation system are demonstrated analytically and experimentally.

Trilateral teleoperation control of kinematically redundant robotic manipulators

Teleoperation control of kinematically redundant robots requires a strategy for resolving their redundancy. A trilateral two-master/one-slave control approach is proposed for delay-free applications in which the first master controls a primary task control frame, e.g. the slave end-effector frame; meanwhile, another master device can manipulate a secondary task frame attached to the slave robot, e.g. to avoid collision with obstacles in the task environment. Any remaining degrees of motion are resolved autonomously. Teleoperation control is achieved in three steps employing joint-space Lyapunov-based adaptive motion/force controllers, a velocity-level redundancy resolution method, and task-space coordinating reference commands. Priority can be given to either the primary or secondary control frame so that the high-priority task can be transparently carried out without interference from the other task. Whenever applicable, the lower-priority task control frame would be restricted to the natural constraints imposed by prioritization or otherwise, decoupling between the tasks is achieved with the use of an arbitrarily weighted pseudo-inverse. Experiments with a planar teleoperation system consisting of two master devices controlling a closed-chain four degree-of-motion redundant slave robot show the feasibility of the approach.