Master Side Research Articles

RBFNN-Based Adaptive Sliding Mode Control Design for Nonlinear Bilateral Teleoperation System Under Time-Varying Delays

The bilateral teleoperation technique has drawn much attention with its attractive superiority to implement the tasks in hazardous environments. Transmission delays and uncertainties are the two main challenges in the nonlinear bilateral teleoperation system to guarantee stability and achieve good transparency performance (including position tracking and force feedback) simultaneously. In this paper, a radial basis function neural network (RBFNN)-based adaptive sliding mode control design is developed for the nonlinear bilateral teleoperation system with transmission delays and uncertainties. For details, the reference trajectory producer is designed in both the master and slave sides to produce the passive reference trajectories for the tracking of master/slave manipulators. The RBFNN-based adaptive sliding mode controller is designed separately for the master and slave to achieve the good tracking performance under system uncertainties. To mitigate the negative effect of transmission delays on the system's stability, a projection mapping by saturation function is applied in the master side to guarantee the boundedness of the delayed environmental torque. Thus, the global stability and the good transparency performance with both position tracking and force feedback can be simultaneously achieved for our proposed method. The comparative experiment is carried out, and the results show the significant performance improvement with our proposed control design.

A Lyapunov Stable Controller for Bilateral Haptic Teleoperation of Single-Rod Hydraulic Actuators Subjected to Base Disturbance

In this paper, a control scheme is developed and evaluated for stable bilateral haptic teleoperation of a single-rod hydraulic actuator subjected to base disturbance. The proposed controller, based on Lyapunov stability technique, is capable of reducing position errors at master and slave sides, and provides a feel of the contact force between the actuator and the task environment to the operator without a need for direct measurement. The controller requires only the measurements of the actuator line pressures and displacements of the master and slave. The system stability is insensitive to the uncertainties of the physical parameters and of the measurement of the base point motion. Stability of the proposed controller incorporating hydraulic nonlinearities and operator dynamics with an estimated upper value for the base disturbance is analytically proven. Simulation studies validate that the proposed control system is stable while interacting with a task environment. Experimental results demonstrate the effectiveness of control scheme in maintaining stability, while having good position tracking by the hydraulic actuator as well as providing a haptic feel to the operator without direct measurement of interaction force, while the hydraulic actuator is subjected to base disturbance.

A New Template and Teleoperation System for Human-Guided Spine Surgery.

Fluoroscopy-guided placement of pedicle screws is usually used to position pedicle screws, although it is highly risky due to lack of accuracy during operation and surgeon's intensive labor requirement during surgery. In this study, a new method was introduced to consider the issues and we tried to reduce the risk of pedicle screw placement and drilling process. To put pedicle screws in the correct position and orientation, a specific drill guide is designed and fabricated by additive manufacturing technology. In addition, since the drilling process is remarkably important and it is usually dependent on surgeon skill, therefore a teleoperation system is proposed to perform this task. In order to let the surgeon to have better control on the patient, a control scheme including position and velocity signals along with surgeon force and reaction force of vertebra was proposed. This helped the surgeon for proper control on the patient during surgery. A force estimation algorithm was presented to eliminate measuring external force signal. Consequently, 10 vertebras were used to evaluate specific drill guide and teleoperation system simultaneously. Then, the computed tomography evaluation demonstrated that Kirschner wire trajectories followed the planned axes with an error between 1 and 2mm in 20% of cases and less than 1 mm in 80% of cases. The results obtained by the teleoperation system showed that the surgeon, in the master side, has proper control over the patient body in the slave side because the slave robot followed the master position. Furthermore, the surgeon in the master side sensed the reaction force of vertebra in the slave side appropriately.

Coupling a NURBS contact interface with a higher order finite element discretization for contact problems using the mortar method

In this paper, a contact problem between two bodies, discretized by finite elements, is solved by adding an auxiliary NURBS layer between the bodies. The advantages of a smooth contact formulation in a NURBS approach are combined with simple mesh generation procedures for the bodies discretized with finite elements. Mesh tying conditions are used to couple the NURBS layer with the finite element discretization. The NURBS layer is the master side for contact and mesh tying. Mesh tying is enforced either using pointwise or mortar type approaches. Frictionless 2D and 3D contact problems are considered using small deformations. The contact problem is discretized with the mortar method and a penalty approach is used to enforce the contact constraints. A robust element-based quadrature is applied for mortar tying and contact discretizations, thus avoiding computationally expensive segmentation.

Magnetorheological Fluids Actuated Haptic-Based Teleoperated Catheter Operating System.

During conventional catheter endovascular procedures, surgeons needs to adjust the catheter intervention moving direction and velocity according to the direct sensation. Moreover, in the conventional method, both the surgeon and the patient are inevitable exposed to a large amount of, and for a long period of time, X-ray radiation during the surgical procedure. The purpose of this paper is to ensure surgical safety and to protect the surgeon from X-ray radiation during the surgical procedure by adopting a novel haptic-based robot-assisted master-slave system mode. In this paper, a kind of magnetorheological fluids (MR fluids)-based haptic interface has been developed to generate a kind of controllable haptic sensation providing to the catheter operator, and the catheter intervention kinematics parameters measured the motion capture part to control the salve robotic catheter operating system following the master side kinematics. The slave catheter operating the mechanical system has also been designed and manufactured to manipulate the clinical catheter by mimicking the surgeon operating the catheter intervention surgical procedure, which has a 2-DOF (advance, retreat, and rotate) catheter motion characteristic; in addition, the interaction force between the catheter and inner wall of vasculature can be measured by its force sensing unit and the feedback to the master system. The catheter intervention synchronous evaluation experiments between the master and slave system are tested. Also, the advantages of integrating the controllable haptic sensation to the master-slave system experimental evaluations have been done in vitro. The experimental results demonstrated that the proposed haptic-based robot-assisted master-slave system mode can reduce the surgical time and protect the surgeon from X-ray radiation.

Haptic based teleoperation with master-slave motion mapping and haptic rendering for space exploration

This paper presents a new solution to haptic based teleoperation to control a large-sized slave robot for space exploration, which includes two specially designed haptic joysticks, a hybrid master-slave motion mapping method, and a haptic feedback model rendering the operating resistance and the interactive feedback on the slave side. Two devices using the 3R and DELTA mechanisms respectively are developed to be manipulated to control the position and orientation of a large-sized slave robot by using both of a user’s two hands respectively. The hybrid motion mapping method combines rate control and variable scaled position mapping to realize accurate and efficient master-slave control. Haptic feedback for these two mapping modes is designed with emphasis on ergonomics to improve the immersion of haptic based teleoperation. A stiffness estimation method is used to calculate the contact stiffness on the slave side and play the contact force rendered by using a traditional spring-damping model to a user on the master side stably. Experiments by using virtual environments to simulate the slave side are conducted to validate the effectiveness and efficiency of the proposed solution.

Multilateral Teleoperation With New Cooperative Structure Based on Reconfigurable Robots and Type-2 Fuzzy Logic.

This paper develops an innovative multilateral teleoperation system with two haptic devices on the master side and a newly designed reconfigurable multi-fingered robot on the slave side. A novel nonsingular fast terminal sliding-mode algorithm, together with varying dominance factors for cooperation, is proposed to offer this system's fast position and force tracking, as well as an integrated perception for the operator on the reconfigurable slave robot (manipulator). The Type-2 fuzzy model is used to describe the overall system dynamics, and accordingly a new fuzzy-model-based state observer is proposed to compensate for system uncertainties. A sliding-mode adaptive controller is designed to deal with the varying zero drift of the force sensors and force observers. The stability of the closed-loop system under time-varying delays is proved using Lyapunov-Krasovskii functions. Finally, experiments to grasp different objects are performed to verify the effectiveness of this multilateral teleoperation system.

An interpolation method for coupling non-conforming patches in isogeometric analysis of vibro-acoustic systems

This paper presents an interpolation method for coupling non-conforming Non-Uniform Rational B-Splines (NURBS) patches in isogeometric analysis of vibro-acoustic systems. This coupling method is based on a master–slave formulation, in which a smooth interpolating function is first constructed to represent the sound pressure field of the master side and then is sampled to determine the discrete sound pressure of the slave side. The coupling constraints are studied by the interpolation matrix built in this process and imposed on the system by Lagrange multiplier method. For interfaces with weak non-conformity, the interpolating function is constructed by a direct linear combination of radial basis functions (RBF); for interfaces with strong non-conformity or curved shape, this basic interpolation method is enhanced by a moving least squares approximation to improve the coupling accuracy. The proposed method starts with non-conforming acoustic–acoustic interfaces and is extended to non-conforming structural–acoustic interfaces. In both cases, the RBF-based coupling constraints depend only on Euclidian distances between control points, allowing them to be generated in a problem-independent and dimension-insensitive way. The accuracy and convergence behavior of the proposed method are benchmarked by an eigenvalue analysis of a two-dimensional acoustic cavity. Its efficiency is further investigated through frequency response analyses of a three-dimensional car-like model, with both strong and weak structural–acoustic interactions.

Performance evaluation of a robot-assisted catheter operating system with haptic feedback.

In this paper, a novel robot-assisted catheter operating system (RCOS) has been proposed as a method to reduce physical stress and X-ray exposure time to physicians during endovascular procedures. The unique design of this system allows the physician to apply conventional bedside catheterization skills (advance, retreat and rotate) to an input catheter, which is placed at the master side to control another patient catheter placed at the slave side. For this purpose, a magnetorheological (MR) fluids-based master haptic interface has been developed to measure the axial and radial motions of an input catheter, as well as to provide the haptic feedback to the physician during the operation. In order to achieve a quick response of the haptic force in the master haptic interface, a hall sensor-based closed-loop control strategy is employed. In slave side, a catheter manipulator is presented to deliver the patient catheter, according to position commands received from the master haptic interface. The contact forces between the patient catheter and blood vessel system can be measured by designed force sensor unit of catheter manipulator. Four levels of haptic force are provided to make the operator aware of the resistance encountered by the patient catheter during the insertion procedure. The catheter manipulator was evaluated for precision positioning. The time lag from the sensed motion to replicated motion is tested. To verify the efficacy of the proposed haptic feedback method, the evaluation experiments in vitro are carried out. The results demonstrate that the proposed system has the ability to enable decreasing the contact forces between the catheter and vasculature.

Translational objects dynamic modeling and correction for point cloud augmented virtual reality–based teleoperation

It is a challenging task for operators to interact with the remote environment without its geometric and dynamic knowledge during teleoperation. In this article, a novel system architecture for implementing the translational object modeling and correction during remote interaction is proposed to reconstruct the haptic interaction and predict the object motion at the local virtual reality–based teleoperation. First, a stress mutation analysis method is proposed for segmenting the translational object motion into static phase, critical phase, and sliding phase. And the static limiting friction is originally estimated in the teleoperation area. Meanwhile, mass-damper-spring model and adapted Karnopp friction model are adopted for dynamic modeling in each phase. Second, a novel adaptive forgetting factor recursive least square method is studied for high-accuracy parameter estimation. With the estimated model parameters, the motion of the translational object is predicted at the master side. Meanwhile, for model consistence between the real and virtual environments, a new correction strategy is used to adaptively update the environment model. According to the experimental results, the translational object can be accurately modeled in real time, and its motion at the master side can be predicted precisely and corrected promptly.

Increasing stability in model-mediated teleoperation approach by reducing model jump effect

Model-mediated teleoperation is a predictive control approach for controlling haptic teleoperation systems whereby the environment force is virtually located on master side in order to increase the stability and transparency of the system. This promising approach, however, results in new challenges. One pivotal challenge is the model jump effect, which stems from the delay in correct creation of the virtual environment. Previous works have endeavored to reduce this effect; however, they either led to transparency decrease or assumed simplified environment models. In this paper, we propose a control approach for this aim based on the idea of decoupling. This means that when a new environment has been identified, the operation is interrupted and no signal is transmitted between master and slave sides. During this time, both sides are controlled by their own sliding mode controllers until the system reaches stability. The main advantage of this method is its independence from environment type, which makes it usable for different kinds of applications. To verify the effectiveness of the proposed approach simulation tests are conducted. The results show the system is stable in interaction with hard and soft environments in presence of large time delays in communication channels.

Improved bilateral teleoperation with proactive haptic sensing and transmission

This article presents a novel master-slave control configuration that is experimentally demonstrated to improve teleoperation performance under random network delay. This is achieved via the design of a proactive haptic sensing system based on a laser range sensor on the slave side, which in turn allows for the proactive transmission of force control from the master to the slave, thus compensating for Internet-based network delays. The proposed configuration introduces three main contributions to the literature of bilateral control systems: (1) fully decoupled position and force control systems, which allow for the controller gains of each loop to be tuned independently, eliminating the trade-off common to most previous literature; (2) a novel approach that exploits the slow variation nature of the environment parameters, resulting in a lower bandwidth requirement in comparison to previous force control methods; (3) capability to measure the slave environment location and the prediction of the contact force as a result, which provides the human operator with the capability to generate the reaction force proactively on the master side. The conducted experiments demonstrated a significantly improved performance in terms of synchronized forces and positions despite the random network delay between the master and slave systems.

Real haptics and its applications

Real haptics has been an unsolved problem since its beginning in the 1940s. After much R&D, including the early works in the United States and other countries, bilateral control designed in the acceleration space is a solution for this issue. The quality of touch and tactile sensation is heavily dependent on the transparency between the master side and the slave side. Robust motion control, which is equivalent to acceleration control, gives high transparency. Real haptics is now a key concept in 21st century where the quality of life should be the basis of our society. Various examples are also given. © 2017 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

Enhanced transparency dual-user shared control teleoperation architecture with multiple adaptive dominance factors

In traditional dual-user shared control teleoperation, the users’ operational transparency is influenced by the dominance factorα. Especially when α = 1 and α = 0 the trainer or trainee cannot receive any information form the slave side. For enlarging the control flexibility and enhancing the system transparency, a novel method with multiple adaptive dominance factors is proposed in this paper. An ideal application of this method is on-line operation and supervision. The supervisor and operator can adjust the factors and switch the operation roles and states freely. Once the slave robot handled by the operator diverges from the planned path, the dominance factors will change adaptive to limit the operator’s movement. The adaptive principles are concluded from the dynamic performance measured by the measuring functions. The conclusions suggest that the varying range of the system dynamic performance is wider than the traditional method. In addition, considering the time delays between the slave and master sides, we proved the system stability conditions covering all the range of dominance factors. Finally, we make a discussion of the applying area of the novel shared control architecture.

Viscoelastic model based bilateral teleoperation for robotic-assisted tele-palpation

PurposeThe purpose of this paper is to design a model-based bilateral teleoperation method to improve the feedback force and velocity/position tracking for robotic-assisted tasks (such as palpation, etc.) under constant and/or varying time delay with environment dynamic property. Time delay existing in bilateral teleoperation easily destabilizes the system. Proper control strategies are able to make the system stable, but at the cost of compromised performance. Model-based bilateral teleoperation is designed to achieve enhanced performance of this time-delayed system, but an accurate model is required.Design/methodology/approachViscoelastic model has been used to describe the robot tool-soft tissue interaction behavior. Kevin-Boltzmann (K-B) model is selected to model the soft tissue behavior due to its good accuracy, transient and linearity properties among several viscoelastic models. In this work, the K-B model is designed at the master side to generate a virtual environment of remote robotic tool-soft tissue interaction. In order to obtain improved performance, a self perturbing recursive least square (SPRLS) algorithm is developed to on-line update the necessary parameters of the environment with varying dynamics.FindingsWith fast and optimal on-line estimation of primary parameters of the K-B model, the reflected force of the model-based bilateral teleoperation at the master side is improved as well as the position/velocity tracking performance. This model-based design in the bilateral teleoperation avoids the stability issue caused by time delay in the communication channel since the exchanged information become position/velocity and estimated parameters of the used model. Even facing with big and varying time delay, the system keeps stably and enhanced tracking performance. Besides, the fast convergence of the SPRLS algorithm helps to track the time-varying dynamic of the environment, which satisfies the surgical applications as the soft tissue properties usually are not static.Originality/valueThe originality of this work lies in that an enhanced perception of bilateral teleoperation structure under constant/varying time delay that benefits robotic assisted tele-palpation (time varying environment dynamic) tasks is developed. With SPRLS algorithm to on-line estimate the main parameters of environment, the feedback perception of system can be enhanced with stable velocity/position tracking. The superior velocity/position and force tracking performance of the developed method makes it possible for future robotic-assisted tasks with long-distance communication.

Augmented reality for enhancing tele-robotic system with force feedback

In the teleoperation, the force feedback is indispensable, which can enhance the sense of presence of the operator and help the operator accomplish tasks comfortably. The time delay is one of the main challenges that influence the stability of the teleoperation systems, which leads to the discontinuous operation. Thus building a local virtual model in the master side is an effective way to solve this problem. In this paper, a new method is presented to reconstruct the virtual model of the remote object. The virtual model can estimate the real-time force feedback to the operator and eliminate the effects of the time delay. Then the tele-robotic system based on augmented reality technology is set up in our laboratory. In the tele-robotic system, the dynamic parameters including damping and stiffness of the virtual model are constantly updated by utilizing the positions and forces information from sensors of the remote robot site. Finally, the effectiveness of the proposed method and the correctness of the visual model parameters are verified by two experiments.

Shadow and Mirror Mode Bilateral Control for a Tele-Operated Robot System

This paper proposes the development of haptic robot system for rehabilitation exercises. Recently, haptic communication is widely applied in a robot system for sensing the force between robot and human operator. The bilateral control which consists of master and slave robot is used to transfer and receive action/ reaction force and position information. In this paper, the controlling of robot has been designed by considering the specification requirements of user. This master-slave robot system can be controlled by two different approaches: 1) by using the shadow mode and 2) by using the mirror mode. In the force control designed, such force sensor can use to measure the external force. It is shown that the reaction force response can be controlled. However, the problem of sensor noise and bandwidth are deteriorated the performance not only at the master or slave side but also at the whole control system. Thus, disturbance observer (DOB) has been proposed and used instead of force sensor for improve the performance and robustness of the whole system. In this study, the shadow and mirror mode of bilateral control is used with the master-slave robot for rehabilitation exercise. On the shadow mode slave robot is controlled according to the master robot like a shadow while mirror mode, slave robot has moved in the opposite direction like a reflection image. The results of each mode are represented in an experimental result. By using the proposed robot system, it is possible to apply in the rehabilitation process.

Multicontact Bilateral Telemanipulation With Kinematic Asymmetries

We propose a novel bilateral telemanipulation framework to tame master and slave devices having different structures. This condition applies to multicontact teleoperation scenarios where the number of contact points on the slave side and the number of interaction points on the master side are different. An example is a master device interacting with the thumb and the index fingertips of the human operator, and as slave device, a robotic arm with a multifingered robotic hand. In case of a manipulation task, it is not straightforward to transmit motion commands and reflect forces from the interaction with the environment. A general telemanipulation framework, that does not consider the specific kinematics of the devices involved, is needed. The main idea of this study is to take advantage of a virtual object as a mediator between the master and slave side. The arising forward and backward mapping algorithms are able to relate the motions and the exerted forces of very dissimilar systems. The approach has been evaluated in a case study consisting of two haptic interfaces used both to track the index and thumb motions and to render forces on the master side and a robotic arm with a multifingered hand as end effector on the slave side. The results presented in this paper can be extended to cooperative grasping scenarios where multiple robots telemanipulate the same object.

Position and force tracking in nonlinear teleoperation systems with sandwich linearity in actuators and time-varying delay

In this paper, a new bounded force feedback control law is proposed to guarantee position and force tracking in nonlinear teleoperation systems in the presence of passive and nonpassive input interaction forces, time varying delay in their communication channels and sandwich linearity in their actuators. The proposed control is a nonlinear-proportional plus nonlinear damping (nP+nD) controller with the addition of a nonlinear function of the environment force on the slave side and nonlinear function of the human force and force error on the master side, the transparency of the proposed scheme will be improved. The controller prevents the inputs from reaching their usual actuator bounds. Using a novel Lyapunov–Krasovskii functional, the asymptotic stability and tracking performance of the teleoperation system are established under some conditions on the controller parameters, actuator saturation characteristics and maximum allowable time delays.

Bilateral teleoperation of a group of mobile robots for cooperative tasks

A visual and force feedback-based teleoperation scheme is proposed for cooperative tasks. The bilateral teleoperation system includes a haptic device, an overhead camera and a group of wheeled robots. The commands of formation and average velocities of the multiple robots are generated by the operator through the haptic device. The state of the multiple robots and the working environment is sent to the human operator. The received information contains the feedback force through the haptic device and visual information returned by a depth camera. The feedback force based on the difference between the desired and actual average velocities is presented. The wave variable method is employed in the bilateral teleoperation of multiple mobile robots with time delays. The effectiveness of the bilateral teleoperation system is demonstrated by experiments. The robots in the slave side are able to follow the commands from the master side to interact with the environments, including moving in different formations and pushing a box. The results show that the scheme enables the operator to manipulate a group of robots to complete cooperative tasks freely.