Slave Robot Research Articles

Optimisation of synchronisation control parameters for enhanced bilateral teleoperation system

Abstract Bilateral teleoperation systems encounter challenges in achieving synchronisation between master and slave robots due to communication time delays. This paper addresses the instability caused by these delays and proposes a solution through advanced control algorithms. Nonlinear optimisation algorithms might only sometimes deliver solutions in the allotted time, particularly when handling complicated, high-dimensional issues or when optimisation iterations are extensive. The study first develops a comprehensive mathematical model encompassing the dynamics and communication intricacies of both master and slave sides in teleoperation. By recognising the limitations of existing proportional-derivative controllers in compensating for communication errors, a sequential quadratic programming-proportional-integral-derivative (SQP-PID) controller is introduced. This controller accumulates and rectifies synchronisation delay errors, ensuring precise control without steady-state deviations. The proposed SQP-PID controller stands out for its ability to handle steady-state errors effectively, offering swift response and maintaining stability. Leveraging the SQP optimisation algorithm, it intelligently tunes the parameters, minimising synchronisation errors. The approach capitalises on the simplicity, performance, and robustness of the SQP-PID controller, providing a promising avenue for enhancing bilateral teleoperation systems’ accuracy and stability, maintaining initial discrepancy with a best fitness value of 0.98 % in varied operating conditions.

Adaptive event-triggered security control for networked teleoperation systems.

An adaptive event-triggered security control method is proposed for networked robotic teleoperation systems subject to time-varying delays and false data injection (FDI) attacks. An event-triggered scheme is designed via the position and velocity signals of the master and slave robots, where the triggering thresholds can change adaptively with the system states. The position and velocity triggered signals and the feedback error between the operator and the environment forces are utilized to detect whether the transmitted signals suffered from FDI attacks. Then, a switching controller is designed, which is able to adopt the corresponding control strategy according to the corresponding detection result. The stability of the system and the convergence of the tracking errors are proved through Lyapunov functions, and the proposed method is validated through practical experiments. The proposed method is able to adaptively adjust the triggering frequency and effectively reduce the transmitted data, thus saving the network resources. Meanwhile, it can ensure the stability of networked teleoperation systems under time-varying delays and FDI attacks, as well as the position and force tracking performance.

A Dexterous Hand-Arm Teleoperation System Based on Hand Pose Estimation and Active Vision.

Markerless vision-based teleoperation that leverages innovations in computer vision offers the advantages of allowing natural and noninvasive finger motions for multifingered robot hands. However, current pose estimation methods still face inaccuracy issues due to the self-occlusion of the fingers. Herein, we develop a novel vision-based hand-arm teleoperation system that captures the human hands from the best viewpoint and at a suitable distance. This teleoperation system consists of an end-to-end hand pose regression network and a controlled active vision system. The end-to-end pose regression network (Transteleop), combined with an auxiliary reconstruction loss function, captures the human hand through a low-cost depth camera and predicts joint commands of the robot based on the image-to-image translation method. To obtain the optimal observation of the human hand, an active vision system is implemented by a robot arm at the local site that ensures the high accuracy of the proposed neural network. Human arm motions are simultaneously mapped to the slave robot arm under relative control. Quantitative network evaluation and a variety of complex manipulation tasks, for example, tower building, pouring, and multitable cup stacking, demonstrate the practicality and stability of the proposed teleoperation system.

Development of a master–slave 3D printed robotic surgical finger with haptic feedback

Robotic surgery started nearly 30 years ago. It has achieved telepresence and the performance of repetitive, precise, and accurate tasks. The “master–slave” robotic system allows control of manipulators by surgeon at distant site. Robotic surgical fingers were developed to allow surgeons to move them with accuracy through sensors fixed on surgeon’s hand. Also, haptic sensors were developed to allow transmission of sensation from robotic finger to surgeon’s finger. A complete system of a, 3D printed by a stereolithography (SLA) 3D printer, robotic surgical finger with haptic feedback system is proposed. The developed system includes a master glove that controls the motion of a 3DOF robotic slave finger while getting haptic feedback of force/pressure exerted on it. The precise control of the slave robotic finger was achieved by applying a Proportional Integral and Derivative (PID), fast and robust, control algorithm using an Arduino based hardware and software module. The individual joint angles, metacarpophalangeal joint (MCP) and proximal interphalangeal joint (PIP), and wrist were measured using rotatory and inertial sensors respectively. The degree of movement for MCP, PIP, and Wrist joints were measured to be 0–86°, 0–71°, and 0–89° respectively. Motion to the robotic finger is mimicked by a glove motion requiring minimal learning curve for the device. The collected data for the slave motion is in good agreement with the master-glove motion data. The vibro-tactile haptic feedback system was developed to distinguish between three different materials to mimic human flesh, tumor, and bone. The master–slave system using robotic surgical finger with good simultaneous movement to surgeon’s finger and good haptic sensation will provide the surgeon with the opportunity to perform finger dissection in laparoscopic and robotic surgery, as it used to be in open surgery. 3D bio printing will make this process even cheaper with the added advantage of making surgical tools locally according to the need of the surgery. An ongoing work is to develop silicone based 8 mm robotic surgical finger with multiple type haptic feedback.

Chatter prediction for parallel mirror milling of thin-walled parts by dual-robot collaborative machining system

Thin-walled parts with double-sided features are widely used in the aerospace industry, but the machining of such parts still faces great challenges due to their weak rigid structure and the positioning errors caused by turning/re-clamping of the workpiece. For this purpose, a novel machining approach for parallel mirror robotic milling of thin-walled parts is proposed. Different from conventional single-sided milling, it can simultaneously machine both sides of the workpiece while avoiding the above problems and improving machining efficiency. However, the introduction of multiple cutting excitations makes the vibration behavior of the workpiece more complicated, thus making it difficult for the dynamic modeling of the dual-robot machining system. This paper aims to uncover the dynamic mechanism of chatter phenomenon in parallel mirror robotic milling process in consideration of the interaction of tool-workpiece-tool system. A dual-robot collaborative machining system is first developed for the milling of thin-walled workpiece, and the method of synchronous motion planning for dual robots is described. The slave robot synchronously follows the master robot and the two milling cutters mounted on tool flange of the robot are kept in mirror position. Over a tooth passing period, the engagement state of the cutters and the workpiece is analyzed considering the effect of rotation phase lag (RPL) between the twin cutters. After that, the vibration state and the corresponding instantaneous displacement of the workpiece can be expressed explicitly, then the dynamic resultant cutting force acting on the workpiece is calculated according to the mechanistic force model. Comprehensively considering the dynamic interaction of the tool-workpiece-tool system, multi-mode and position-dependent modal characteristics of thin-walled part, a time-varying delay differential equation is constructed and solved to explore the dynamical behavior of parallel mirror milling process. The predicted stability lobe diagram is validated by a series of parallel mirror milling tests on thin-walled PMMA plates. Comparisons of numerical and experimental results show that the dynamic model successfully predicts the chatter stability for the parallel mirror milling of thin-walled parts.

Prescribed Performance Control for Teleoperation System of Nonholonomic Constrained Mobile Manipulator Without Any Approximation Function

The teleoperation system of mobile manipulator has been widely applied for space exploration, medical assistance, and other fields. To improve operational efficiency, this paper presents a novel prescribed performance synchronization control scheme without applying any approximation functions for a class of teleoperation system of the mobile manipulator with time-varying delays and nonholonomic constraints. Compared with the general manipulator system, the mobile manipulator has a larger workspace and stronger operability. Yet, the heterogeneity of master-slave robots and the nonholonomic constraint of slave robot bring many difficulties to the control of the system. Firstly, a prescribed performance function (PPF) is designed to guarantee that the position synchronization errors are remained within the predetermined boundary and converge to a small preset area within a preset time. Then, the direct force feedback information is introduced into the controller to improve the transparency of the teleoperation. In addition, there are no approximation functions applied in the proposed controller, which can significantly reduce the complexity and computation of the system. Finally, simulation and experimental results are made to illustrate the availability of this method for practical application. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —The motivation of this paper is to develop an algorithm to improve the steady-state and transient-state performance of mobile manipulator teleoperation system without increasing the complexity of the controller. Compared with the existing works, in this paper, the control algorithm without any approximators is proposed and a time-varying prescribed performance function is introduced, which promotes the steady-state and transient-state performance of the system essentially. At the same time, the introduction of force feedback information improves the transparency of the teleoperation system. The feasibility and effectiveness of the algorithm are verified on UR5e-Husky experimental platform. In the future, we will strive to further improve the transparency of the mobile manipulator teleoperation system in order to achieve the immersive effect of operator.

Hierarchical Prescribed-Performance Nonlinear Control for Teleoperation of Networked Slave Robots with Coupling Constraints

Hierarchical Prescribed-Performance Nonlinear Control for Teleoperation of Networked Slave Robots with Coupling Constraints

Stability of tracking wheel mobile robot with teleoperation fuzzy neural network control system

The stability of the Tracking Wheel Mobile Robot with Teleoperation System and Path Following Method is discussed in this study. The path is to be tracked by the host computer which is the master robot. The response from the robot is captured on camera. As the slave robot approaches the target position, the camera captures the response robot’s position and as well as moving trajectory. The host computer receives all of the images, enabling mobile robot deviation recoveries. The slave robot can use teleoperation to follow the sensor based on the decisions made by the master robot. The Lyapunov function in the Fuzzy Neural Network (FNN) control structure assures the system’s stability and satisfactory performance. It supports a mobile robot’s ability to adhere to a reference trajectory without deviating from it. Finally, the outcome of the simulation demonstrates that our controller is capable of tracking different environmental conditions and maintaining stability.

Teleoperation mode and control strategy for the machining of large casting parts

Residual features of large casting parts in small batch are random in distribution, size, and shape. The manual mode is inefficient, dangerous, and labor-intensive. The automatic mode spends a lot of time on trajectory planning and programming. To overcome the challenges, a novel teleoperation machining mode and a modified master-slave control structure are proposed. Moreover, a hierarchical control strategy based on variable motion mapping (VMM) and elastic compensation (EC) is presented to improve machining efficiency by realizing the constant cutting force. The control laws of VMM, EC and interactive force are designed to reduce operator fatigue and mental burden. The forbidden virtual fixture (RVF) is generated by employing a six-degree-of-freedom (6-DoF) articulated measuring arm to constraint the movement of slave robot and protect the workpiece from being overcut. The velocity and acceleration constraints are formulated to avoid the impact of the slave robot. A large casting part (train bolster) is considered to carry out comparative experiments. The effectiveness of the teleoperation mode and control strategy are confirmed by experiment results, which strongly promotes the progress of unstructured workpiece manufacturing and provides a reference for other machining.

Design and research of bilateral teleoperation for CFETR tile assembly using haptic feedback

Bilateral teleoperation is a control strategy used in remote handling (RH) to enable an operator to control a robot or system while perceiving the interactive force in the environment. To enhance the immersive operating experience by reducing time delays, a control system based on OROCOS and DDS has been presented to rapidly establish the bilateral teleoperation system with real-time performance. Additionally, a bilateral control algorithm combining closed-loop pose tracking and force feedback is proposed to improve the pose tracking accuracy of the slave robot and enhance the sense of telepresence. As a part of the pre-study on the tile assembly conducted by CMOR in Chinese Fusion Engineering Test Reactor (CFETR), a lightweight experimental setup has been constructed based on the maintenance scenario in EAST. Relative experiments have been conducted to validate the accuracy and effectiveness of the bilateral teleoperation system. The results demonstrate low latency and smooth control during the bilateral teleoperation process.

Teleoperated Surgical Robot with Adaptive Interactive Control Architecture for Tissue Identification.

The remote perception of teleoperated surgical robotics has been a critical issue for surgeons in fulfilling their remote manipulation tasks. In this article, an adaptive teleoperation control framework is proposed. It provides a physical human-robot interaction interface to enhance the ability of the operator to intuitively perceive the material properties of remote objects. The recursive least square (RLS) is adopted to estimate the required human hand stiffness that the operator can achieve to compensate for the contact force. Based on the estimated stiffness, a force feedback controller is designed to avoid the induced motion and to convey the haptic information of the slave side. The passivity of the proposed teleoperation system is ensured by the virtual energy tank. A stable contact test validated that the proposed method achieved stable contact between the slave robot and the hard environment while ensuring the transparency of the force feedback. A series of human subject experiments was conducted to empirically verify that the proposed teleoperation framework can provide a more smooth, dexterous, and intuitive user experience with a more accurate perception of the mechanical property of the interacted material on the slave side, compared to the baseline method. After the experiment, the design idea about the force feedback controller of the bilateral teleoperation is discussed.

Design of Adaptive Sensor Coupling-Based Upper Limb 7-DOF Exoskeleton for Smooth Human Motion Tracking: ASC-EXO

The conventional sensor coupling of the sensor in an exoskeleton robot suffers from inaccurate human motion tracking due to irregular inertial loading on the sensor axis. It may cause sensor axis misalignment, performance degradation and damage the sensor permanently, especially in the multiple degrees of freedom (DOF) exoskeleton robot. In this paper, we proposed a novel design of an adaptive sensor coupling-based 7-DOF exoskeleton (ASC-EXO) for motion tracking and tele-manipulation. The adaptive sensor coupling mechanism has been designed based on the flexure structure cantilever with a slew ring, which suppresses irregular inertial loading on the sensor and is stiff enough along the sensor rotation in the z-axis and compliant on the other axes. The slew ring allows smooth joint rotation. Further, the sensor values are improved by a Butterworth filter. As a result, the adaptive sensor coupling suppressed the sensor noise 20 times more than the conventionally coupled sensor with the robot joint. Finally, we developed a prototype of the 7-DOF exoskeleton (ASC-EXO) using adaptive sensor coupling for smooth human motion tracking. The performance evaluation of adaptive sensor coupling and human motion tracking is presented. In addition, we demonstrated smooth control of the slave robot (KUKA) using the exoskeleton (ASC-EXO) as a master device.

Toward Unified Adaptive Teleoperation Based on Damping ZNN for Robot Manipulators With Unknown Kinematics

Traditional teleoperation systems are usually designed for a single class of slave robots and require the specific kinematic model. This leads to the problems of high computational cost, poor robustness, and poor portability in traditional teleoperation systems. In this paper, a unified adaptive teleoperation system is proposed for different kinds of slave robots. To this end, a closed-loop control system based on damping zeroing neural network (DampZNN) is proposed to achieve the unified kinematic control of both the redundant robot and the continuum robot with unknown kinematics. The control system is constructed based on two DampZNNs, which are utilized to estimate the unknown kinematic model and to solve the inverse kinematics problem respectively. By combining the DampZNN control system with teleoperation technology, a unified adaptive teleoperation system is proposed for human operators to teleoperate both the redundant slave robot and the continuum slave robot easily. The efficacy of the proposed teleoperation system is verified by experiments based on different slave robots and the advantages of the proposed teleoperation system are revealed by comparative studies.

Accuracy of on-site teleoperated milling with haptic assistance

PurposeIn bone surgery specialties, like orthopedics, neurosurgery, and oral and maxillofacial surgery patient safety and treatment success depends on the accurate implementation of computer-based surgical plans. Unintentional plan deviations can result in long-term functional damage to the patient. With on-site teleoperation, the surgeon operates a slave robot with a physically-decoupled master device, while being directly present at the operation site. This allows the surgeon to perform surgical tasks with robotic accuracy, while always remaining in the control loop.MethodsIn this study the master- and slave-side accuracy of an on-site teleoperated miniature cooperative robot (minaroHD) is evaluated. Master-side accuracy is investigated in a user study regarding scale factor, target feed rate, movement direction and haptic guidance stiffness. Scale factors are chosen to correspond to primarily finger, hand, and arm movements. Slave-side accuracy is investigated in autonomous milling trials regarding stepover, feed rate, movement direction, and material density.ResultsMaster-side user input errors increase with increasing target feed rate and scale factor, and decrease with increasing haptic guidance stiffness. Resulting slave-side errors decrease with increasing scale factor and are < 0.07 mm for optimal guidance parameters. Slave-side robot position errors correlate with the feed rate but show little correlation with stepover distance. For optimal milling parameters, the 95th percentile of tracked slave-side position error is 0.086 mm with a maximal error of 0.16 mm.ConclusionFor optimal guidance and milling parameters, the combined error of 0.23 mm is in the range of the dura mater thickness (< 0.27 mm) or mandibular canal wall (~ 0.85 mm). This corresponds to safety margins in high-demand surgical procedures like craniotomies, laminectomies, or decortication of the jaw. However, for further clinical translation, the performance and usability of on-site teleoperated milling must be further evaluated for real-life clinical application examples with consideration of all error sources in a computer-assisted surgery workflow.

An Incremental Broad-Learning-System-Based Approach for Tremor Attenuation for Robot Tele-Operation.

The existence of the physiological tremor of the human hand significantly affects the application of tele-operation systems in performing high-precision tasks, such as tele-surgery, and currently, the process of effectively eliminating the physiological tremor has been an important yet challenging research topic in the tele-operation robot field. Some scholars propose using deep learning algorithms to solve this problem, but a large number of hyperparameters lead to a slow training speed. Later, the support-vector-machine-based methods have been applied to solve the problem, thereby effectively canceling tremors. However, these methods may lose the prediction accuracy, because learning energy cannot be accurately assigned. Therefore, in this paper, we propose a broad-learning-system-based tremor filter, which integrates a series of incremental learning algorithms to achieve fast remodeling and reach the desired performance. Note that the broad-learning-system-based filter has a fast learning rate while ensuring the accuracy due to its simple and novel network structure. Unlike other algorithms, it uses incremental learning algorithms to constantly update network parameters during training, and it stops learning when the error converges to zero. By focusing on the control performance of the slave robot, a sliding mode control approach has been used to improve the performance of closed-loop systems. In simulation experiments, the results demonstrated the feasibility of our proposed method.

Hierarchical finite-time cooperative control for teleoperation of networked disturbed mobile manipulators

This paper investigates the teleoperation problem of networked disturbed mobile manipulators (NDMMs), in which the human operator remotely controls multiple slave mobile manipulators through a master manipulator. Each individual of the slave ones consisted of a nonholonomic mobile platform and a holonomic constrained manipulator that is mounted on the nonholonomic mobile platform. The cooperative control objective of the considered teleoperation problem includes: (1) synchronizing the states of the slave manipulators to the human-controlled master one; (2) forcing the mobile platforms of the slave ones to form a user-defined formation; (3) controlling the geometric center of all the platforms to track a reference trajectory. We present a hierarchical finite-time cooperative control (HFTCC) framework to achieve the cooperative control goal in a finite time. The presented framework includes the distributed estimator, the weight regulator and the adaptive local controller, where the estimator generates the estimated states of the desired formation and trajectory, the regulator selects the slave robot that the master one needs to track, as well as the presented adaptive local controller guarantees the finite-time convergence of the controlled states with model uncertainties and disturbances. Additionally, for improving the telepresence, a novel super twisting observer is presented to reconstruct the interaction force between the salve mobile manipulators and the remote operating environment on the master (i.e., the human) side. Finally, the effectiveness of the proposed control framework is demonstrated by several simulation results.

Adaptive Control of a Master-Slave Based Robotic Surgical System With Haptic Feedback

A master-slave (M-S) based robotic surgical system with haptic feedback is presented in this work for natural orifice transluminal endoscopic surgery (NOTES) using an in-vivo robot. A novel dual-loop control, including closed-loop inverse kinematics (CLIK) and adaptive control with optimal control parameters, is developed for the slave robot to track a random trajectory upon updating the dynamic parameters of the slave robot. The interaction between the slave robot tip position and environment is controlled by an impedance controller that prevents damage to healthy tissues. Upon tracking a given desired 3D trajectory as input for the slave robot by the human operator handling the master robot, the slave robot-tissue interaction force is sensed as haptic feedback at the master side. Lyapunov direct method based stability analysis and performance evaluations involving both simulation and experimental studies prove the efficacy of our novelly developed M-S robotic system aiming NOTES regulated by the proposed adaptive control strategy with optimized control parameters. Note to Practitioners: The M-S based robotic surgical system is developed to perform NOTES using a cable-driven in-vivo robot attached to the tip of a flexible endoscope. It performs tissue manipulation as per the direction given by the operator. The inclusion of haptic imitates the NOTES performed using the in-vivo robot as conventional surgery. The developed system includes the following aspects: a) The adaptive control scheme with optimal control parameters; b) Interaction force between the slave robot and tissue environment is controlled using impedance control and given as haptic feedback to the surgeon; c) The proposed system can be potentially utilized for actual teleoperated surgery and surgical training for NOTES.

A Novel Master-Slave Interventional Surgery Robot with Force Feedback and Collaborative Operation.

In recent years, master-slave vascular robots have been developed to address the problem of radiation exposure during vascular interventions for surgeons. However, the single visual feedback reduces surgeon immersion and transparency of the system. In this work, we have developed a haptic interface based on the magnetorheological fluid (MRF) on the master side. The haptic interface can provide passive feedback force with high force fidelity and low inertia. Additionally, the manipulation of the master device does not change the operating posture of traditional surgery, which allows the surgeon to better adapt to the robotic system. For the slave robot, the catheter and guidewire can be navigated simultaneously which allows the two degrees of action on the catheter and axial action of a guidewire. The resistance force of the catheter navigation is measured and reflected to the user through the master haptic interface. To verify the proposed master-slave robotic system, the evaluation experiments are carried out in vitro, and the effectiveness of the system was demonstrated experimentally.

유압시스템을 사용한 고 하중 양팔 로봇의 모션 추종 성능 향상을 위한 슬라이딩 모드 제어

This paper presents the control implementation of a heavy-duty dual-arm robot, Accident Response Manipulator (ARMstrong), configured as teleoperated control by a master robot. The mater-slave configuration for remote operation in dangerous areas has been implemented. The master robot is an exoskeleton type such that an operator can wear it to control the slave. The slave robot is actuated by the hydraulic system, which generates high power but produces larger motion-tracking errors that suffer from nonlinearity. The dead zone of each joint is identified and compensated to improve motion-following performance after the master. Sliding mode control further improves the positional accuracy under unknown payloads. Experimental studies are conducted to demonstrate the position control performance of the ARMstrong slave robot.

Communication and Cooperation for Spherical Underwater Robots by Using Acoustic Transmission

The application of miniature underwater robots in the marine area is becoming more extensive. Occasionally, the ability of the single miniature underwater robot is limited by its size, configuration, function or propulsion method. At this moment, if a group of robots can work together in a division of labor, that can solve more problems underwater. For autonomous underwater vehicles, the support of underwater wireless communication equipment is required to achieve communication and collaboration. Acoustic communication equipment has generally been used in large underwater vehicles due to its large size and power. This article employs a small acoustic communication module to enable underwater communication in a spherical robot with a diameter of only 45 cm. The fourth-generation spherical underwater robot is exploited as the prototype. The article also verifies the accuracy, efficiency and timeliness of the proposed underwater communication systems. Finally, a cooperative mode is established based on the base station, master robot and slave robots, and some cooperation motion experiments are carried out to verify its effectiveness. Early work on this article was presented at the IEEE ICMA 2018 conference S. Gu et al., 2018. In this conference article, the authors only provided theoretical research and a small number of experiments. On this basis, this article has conducted in-depth research, proposed a base station-mater-slave robot communication mode, and established an instruction set for further experiments.