Remote Center Of Motion Constraint Research Articles

Cerebellum-Inspired Model-Free Tracking Control and Visual Servoing of a Rigid-Flexible Hybrid Robotic Endoscope With RCM Constraints

This article designs a model-free control scheme based on a cerebellum-inspired network model for tracking control and visual servoing of a surgical rigid-flexible hybrid robotic endoscope. The hybrid robot is composed of a rigid robot manipulator and a flexible endoscope as the end-effector. Inspired by the cerebellum associated with human motion control, we proposed a cerebellum-inspired model-free control scheme for the visual servoing and tracking control of the robotic endoscope with RCM (remote center of motion) constraints, which does not need to know the kinematic model of the robot. This scheme combines liquid state machines (LSM) and zeroing neural network (ZNN), where the LSM is able to generate control signals and the ZNN is used as the training signal for the LSM. Simulations and physical experiments demonstrate that the proposed cerebellum-inspired control scheme is effective to handle the tracking task under RCM constraints as well as the visual servoing task.

An Autonomous Surgical Instrument Tracking Framework With a Binocular Camera for a Robotic Flexible Laparoscope

In minimally invasive surgery (MIS), the field of view (FOV) plays a vital role. To enhance the stability of FOV and lighten the burden on surgeons, robot-assisted laparoscope systems have been developed and introduced into surgery. However, most of the existing automatic surgical tool tracking schemes with robotic laparoscopes either do not consider tracking of the depth direction or the remote center of motion (RCM) constraints. To tackle these problems, this paper presents an autonomous three-dimensional surgical instrument tracking framework for a robotic laparoscope based on a binocular camera. A 3-degree-of-freedom visual tracking scheme is derived from the kinematics of the binocular robotic laparoscope. A programmable RCM constraint is introduced due to the MIS requirement. To precisely locate the surgical instruments in FOV, a surgical instrument tip segmentation dataset with 5471 images is built and a modified real-time instance segmentation network is trained based on the dataset. Then, a semi-global matching (SGM) algorithm is utilized with the binocular camera to estimate the depth in FOV. To meet the joint limit constraint and maintain the dexterity of the robotic system, a task-priority-based control method is adopted. The simulation and experiment results demonstrate the feasibility of the proposed framework for instrument tracking with the robotic laparoscope.

A Concurrent Framework for Constrained Inverse Kinematics of Minimally Invasive Surgical Robots.

Minimally invasive surgery has undergone significant advancements in recent years, transforming various surgical procedures by minimizing patient trauma, postoperative pain, and recovery time. However, the use of robotic systems in minimally invasive surgery introduces significant challenges related to the control of the robot's motion and the accuracy of its movements. In particular, the inverse kinematics (IK) problem is critical for robot-assisted minimally invasive surgery (RMIS), where satisfying the remote center of motion (RCM) constraint is essential to prevent tissue damage at the incision point. Several IK strategies have been proposed for RMIS, including classical inverse Jacobian IK and optimization-based approaches. However, these methods have limitations and perform differently depending on the kinematic configuration. To address these challenges, we propose a novel concurrent IK framework that combines the strengths of both approaches and explicitly incorporates RCM constraints and joint limits into the optimization process. In this paper, we present the design and implementation of concurrent inverse kinematics solvers, as well as experimental validation in both simulation and real-world scenarios. Concurrent IK solvers outperform single-method solvers, achieving a 100% solve rate and reducing the IK solving time by up to 85% for an endoscope positioning task and 37% for a tool pose control task. In particular, the combination of an iterative inverse Jacobian method with a hierarchical quadratic programming method showed the highest average solve rate and lowest computation time in real-world experiments. Our results demonstrate that concurrent IK solving provides a novel and effective solution to the constrained IK problem in RMIS applications.

Neural-learning-enhanced Cartesian Admittance control of robot with moving RCM constraints

AbstractIn this manuscript, a scheme for neural-learning-enhanced Cartesian Admittance control is presented for a robotic manipulator to deal with dynamic environments with moving remote center of motion (RCM) constraints. Although some research has been implemented to address fixed constrained motion, the dynamic moving movement constraint is still challenging. Indeed, the moving active RCM constraints generate uncertain disturbance on the robot tool shaft with unknown dynamics. The neural-learning-enhanced decoupled controller with disturbance optimisation is employed and implemented to maintain the performance under the kinematic uncertain and dynamic uncertain generated. In addition, the admittance Cartesian control method is introduced to control the robot, providing compliant behaviour to an external force in its operational space. In this proposed framework, a neural-learning-enhanced disturbance observer is investigated to calculate the external factor operating on the end effector premised on generalised momentum in order to ensure accuracy. Finally, the experiments are implemented using a redundant robot to validate the efficacy of the suggested approach with moving RCM constraints.

A Unified Monocular Camera-Based and Pattern-Free Hand-to-Eye Calibration Algorithm for Surgical Robots With RCM Constraints

With progressive advancements in modern technologies, image-guided automation, which plans and manipulates particular tasks by utilizing visual cues, is becoming an emerging sector in medical robotics. Its prevalence nowadays requires a reliable hand-to-eye calibration owing to various configurations of surgical robots, especially for those with remote center of motion (RCM) constraints, e.g., da Vinci Research Kit (dVRK). In this article, we proposed a novel unified calibration method, which leverages the structure characteristics and establishes an explicit geometric model to solve the hand-to-eye relationship only with a monocular camera’s feedback. By freely moving the patient side manipulator in dVRK, our method can first recover the 3-D poses of its shaft at various configurations using Plücker coordinates. To obtain the hand-to-eye translation relationship, these recovered poses are further converged by softly intersecting them within a minimal spatial sphere while preserving their coordinates using an aggregating sphere loss. Meanwhile, the orientation mapping can be recovered by leveraging the corresponding longitudinal-axis information. To improve the calibration accuracy, a local optimization strategy that utilizes the bundle adjustment to refine the transformation using the previous outcomes is also introduced. With such a hierarchical calibration scheme, the hand-to-eye relationship of any RCM constrained robot can be successfully achieved. Extensive simulations and experiments are conducted based on dVRK platform. By comparing with the traditional method, the validation results prove the feasibility and superiority of our unified algorithm in calibration accuracy and robustness.

A Kinematic Modeling and Control Scheme for Different Robotic Endoscopes: A Rudimentary Research Prototype

In image-guided robotic surgery, there exist different endoscopes coupled either with specialized surgical robots (SSRs) or general industrial robots (GIRs). In general, SSRs mechanically respect the remote-center-of-motion (RCM) constraints with directly and explicitly controllable degrees-of-freedom (DoFs), whereas GIRs meet the constraints at the algorithmic level in an implicit manner, with a loss of direct control of some RCM DoFs. Conventionally, different robotic endoscopes are treated as different monolithic systems. Then, kinematic models and control schemes are separately established to automate different robotic endoscopes. This means that a similar analysis must be followed for each system, which is tedious and time-consuming. This letter introduces a modular method to analyze the individual kinematics of the robotic holder, the endoscope shaft and the surgical endoscope with RCM constraints explicitly handled. For achieving automation, the integrated kinematics of a generic robotic endoscope is determined by combining the kinematics of the modular elements. Considering that cameras are intrinsically embedded sensors, a visual servo control scheme applicable to different robotic endoscopes is formulated to incorporate four explicit (virtual) DoFs characterizing the RCM constraints in the control law. Simulations and experiments performed using three different robotic endoscopes validate the effectiveness and practicality of the kinematic modeling and control scheme for automatic field-of-view adjustment.

Accelerated Dual Neural Network Controller for Visual Servoing of Flexible Endoscopic Robot With Tracking Error, Joint Motion, and RCM Constraints

Aiming at the requirements for endoscope in minimally invasive surgery, a dual neural network (DNN) controller is designed for a flexible endoscopic robot (FER) with 10 DOF. First, the FER's kinematics model with remote center of motion (RCM) constraints is established. Then, a quadratic programming control scheme involving the tracking error, joint motion, and RCM constraints of the FER is proposed. A DNN solver of the control scheme, which is activated by the sum of linear and sign-bi-power activation function with adjustable parameters (LSB-AF-AP), is designed, and its convergence rate (CR) and antinoise ability can be improved via adjusting the parameters of LSB-AF-AP. The DNN can converge rapidly in finite time and it is proved by utilizing the Lyapunov theory. Compared with the previous linear AFs applied to the DNN, theoretical analysis indicates that the DNN activated by the LSB-AF-AP has an accelerated CR. Meanwhile, it is proved that the solution obtained by using the designed DNN is the optimal solution of the control scheme. Finally, simulations using the ROS and Gazebo, and an experiment performed on a real FER are conducted. The simulative and experimental results verify that the control scheme can complete the target tracking task well, and the CR and antinoise ability of the algorithm can be further improved via adjusting the parameters of LSB-AF-AP.

Robot-Enabled Uterus Manipulator for Laparoscopic Hysterectomy With Soft RCM Constraints: Design, Control, and Evaluation

Laparoscopic hysterectomy is a common minimally invasive gynecologic surgery in which an assistant is required to manipulate the uterus during the procedure according to the verbal commands of the primary surgeon. However, uterus manipulation is a lengthy and laborious task, where human fatigue may lower operating safety. This paper presents a novel robot-enabled uterus manipulation system that provides accurate, tireless, and direct uterus manipulation. The system consists primarily of a 7-DoF robotic arm tailored for uterus manipulation in laparoscopic hysterectomy. The primary surgeon can directly control the robot using a footswitch pedal to move within the constraint of the Remote Center of Motion (RCM) via software and achieve the desired range of pitch and yaw motion. An additional rotational motion around the instrument axis is provided for twisting the uterus to enhance the exposure of the ligaments and peritoneum for resection. In addition, the robot arm consists of 7 compact modular actuators that provide the target payload for holding the uterus while maintaining a small footprint in a constrained operating environment. To facilitate operation, a passive mobile base is provided for supporting and positioning the robot. A 6-D force sensor is equipped that allows the assistant to quickly guide the robot into the work area and move the robot under RCM constraint using an admittance control approach. The uterus manipulation rod can be easily detached from the robot body by a quick pluggable mechanism for preoperative sterilization. A quick pluggable mechanism was proposed for easy separation and mounting of the uterus manipulation rod from the robot body for preoperative sterilization. Experiments were performed to measure the performance of the modular joints, RCM repeatability, and evaluate the feasibility of the robot in the simulated laparoscopic hysterectomy.

Intuitive Remote Robotic Nasal Sampling by Orientation Control With Variable RCM in Limited Space

When performing polymerase chain reaction testing, collecting nasopharyngeal swabs from patients is necessary. Hence, the development of a noncontact specimen collection system is desired to avoid the risk of secondary infection from direct contact between the medical personnel and patients. This study aims to develop a master-follower-type remote specimen collection system to achieve infection protection through adequate isolation. A teleoperated robot is used for specimen collection work instead of a human; the teleoperated robot moves remotely, and a cotton swab is grasped and inserted into the nostril by the robot. A remote center of motion (RCM) mechanism was implemented to prevent the cotton swabs from making excessive contact with the nostril during specimen collection. Further, a variable RCM constraint that combines safety and operability is proposed as the conventional RCM control modification. The evaluation results indicated that the variable RCM restraint allowed a quicker and safer collection of specimens than the conventional RCM constraint.

A Robotic System with Robust Remote Center of Motion Constraint for Endometrial Regeneration Surgery

Surgical robots have been widely used in different procedures to improve and facilitate the surgery. However, there is no robot designed for endometrial regeneration surgery, which is a new therapy for restoring fertility in women using stem cells. Endometrial regeneration surgery requires processing of the endometrium and transplantation of stem cells with minimal trauma to the uterus. In this paper, we introduce a surgical robotic system that consists of a dexterous hysteroscope, supporting arm, and additional novel instruments to facilitate the operation and decrease trauma to the uterus. Remote center of motion (RCM) constraint is required to protect the cervix of the uterus. First, the supporting arm and hysteroscope are controlled separately in kinematics to ensure that the RCM constraint and hysteroscope’s shape and posture are predictable. Then, a task-decoupled method is used to improve the robustness of the RCM constraint. Experiments confirm that the proposed method is more robust and achieves higher RCM accuracy. In addition, the master-slave control of a robot with RCM constraint is also verified. This study proposes the realization of a robot with robust RCM control for endometrial regeneration surgery.

Fuzzy Approximation-Based Task-Space Control of Robot Manipulators With Remote Center of Motion Constraint

The presence of unknown physical interaction between the patients’ body and surgical tool in laparoscopic surgery requires a secure end-effector positioning while assuring a reliable constraint motion. In this work, a task-space control approach based on fuzzy approximation is proposed for a teleoperated surgery scenario utilizing a serial redundant robot manipulator (7 degrees of freedom), the motions of which are constrained with respect to a point known as remote center of motion (RCM). The dynamical uncertainties due to the physical interaction are considered and estimated to maintain operational accuracy by introducing the decoupled adaptive fuzzy approximation technique. Experiments verify the presented approach’s effectiveness. Results indicate that with the proposed control method, the accuracy of the surgical operation is improved while the safety can be ensured for teleoperated surgery with RCM constraint.

Robotic system with programmable motion constraint for transurethral resection

We propose a tele-operated transurethral robotic system with programmable motion constraint for tissue resection. The system consists of a surgeon console with an interaction device, a 7-degree-of-freedom manipulator, and a surgical end-effector. The surgical end-effector holds a resectoscope with a motor driven mechanism to perform electrocautery. Instrumental motion with remote center-of-motion (RCM) constraint is important for transurethral procedures since the damage to the healthy structures can be minimized. A screw theory-based programmable RCM generator is proposed to map the inputs of the interaction device to the RCM manifold in the manipulator space. To achieve smooth real-time manipulator following control with RCM constraint, an online trajectory planner is presented. Experiments were performed to evaluate the motion precision and accuracy. The results show that both the RCM precision and accuracy were less than 1 mm. Ex-vivo experiments of robotic resection of soft tissue were also carried out. The results show that our robotic system could achieve instrumental manipulation and delicate cutting under the real-time control of a surgeon. Our robotic system could assist surgeons in performing transurethral procedures in a remote manner with potential advantages of being accurate, less laborious and clean.

Kinematic Control of Manipulator with Remote Center of Motion Constraints Synthesised by a Simplified Recurrent Neural Network

Redundancy manipulators need favorable redundancy resolution to obtain suitable control actions to guarantee accurate kinematic control. Among numerous kinematic control applications, some specific tasks such as minimally invasive manipulation/surgery require the distal link of a manipulator to translate along such fixed point. Such a point is known as remote center of motion (RCM) to constrain motion planning and kinematic control of manipulators. Recurrent neural network (RNN) which possesses parallel processing ability, is a powerful alternative and has achieved success in conventional redundancy resolution and kinematic control with physical constraints of joint limits. However, up to now, there still is few related works on the RNNs for redundancy resolution and kinematic control of manipulators with RCM constraints considered yet. In this paper, for the first time, an RNN-based approach with a simplified neural network architecture is proposed to solve the redundancy resolution issue with RCM constraints, with a new and general dynamic optimization formulation containing the RCM constraints investigated. Theoretical results analyze and convergence properties of the proposed simplified RNN for redundancy resolution of manipulators with RCM constraints. Simulation results further demonstrate the efficiency of the proposed method in end-effector path tracking control under RCM constraints based on a redundant manipulator.

Smart surgical control under RCM constraint using bio-inspired network

In this paper, we propose a control framework for intelligent surgical robots under the Remote Center of Motion (RCM). The goal of a surgical robot is to assist surgeons in performing complex surgeries. RCM constraint implies that the surgical tip attached to the end-effector of the surgical robot does not slide away from the point of the incision while performing surgery. Implementation of a control algorithm to comply with RCM constraints is a complicated task because of the nonlinear model of the surgical robots and stringent conditions of accuracy imposed by the patient’s safety. This paper proposes an optimization-driven approach to perform the surgical maneuver under RCM constraints. We then applied a bio-inspired optimization algorithm to solve the problem efficiently. For testing the performance of ZNNBAS, we used MATLAB to simulate a surgical procedure. A 7-DOF surgical robot (KUKA LBR IIWA 7) was used as a test bench for running the simulations. The simulation results show that the ZNNBAS is comparable with BAS, PSO, and GA and efficiently and robustly performed the task commanded maneuvers while enforcing the RCM constraints.

Incorporating model predictive control with fuzzy approximation for robot manipulation under remote center of motion constraint

Remote center of motion (RCM) constraint has attracted many research interests as one of the key challenges for robot-assisted minimally invasive surgery (RAMIS). Although it has been addressed by many studies, few of them treated the motion constraint with an independent workspace solution, which means they rely on the kinematics of the robot manipulator. This makes it difficult to replicate the solutions on other manipulators, which limits their population. In this paper, we propose a novel control framework by incorporating model predictive control (MPC) with the fuzzy approximation to improve the accuracy under the motion constraint. The fuzzy approximation is introduced to manage the kinematic uncertainties existing in the MPC control. Finally, simulations were performed and analyzed to validate the proposed algorithm. By comparison, the results prove that the proposed algorithm achieved success and satisfying performance in the presence of external disturbances.

A Controller to Impose a RCM for Hands-on Robotic-Assisted Minimally Invasive Surgery

In Robotic-Assisted Minimally Invasive Surgery a long and thin instrument attached to the robotic arm enters the human body through a tiny incision. To ensure that no injury occurs when the surgeon is manipulating the instrument, the incision point must be a remote center of motion (RCM) for the instrument. For this purpose, a novel target admittance model is designed in the joint space for hands-on procedures that can be applied in all commercially available general-purpose manipulators with six or more degrees of freedom. It is proved that the joint reference trajectories generated by the proposed target admittance model under the exertion of a human force are stable and satisfy the RCM constraint. The measurements of the human force and the robot's forward kinematic model are only required. Its use spans all hands-on surgical procedures. The proposed model can be easily extended to achieve additional objectives. Simulation results validate the theoretical findings and experimental results utilizing a KUKA LWR4+ demonstrate that trocar displacements are less than 1mm.

Toward Teaching by Demonstration for Robot-Assisted Minimally Invasive Surgery

Learning manipulation skills from open surgery provides more flexible access to the organ targets in the abdomen cavity and this could make the surgical robot working in a highly intelligent and friendly manner. Teaching by demonstration (TbD) is capable of transferring the manipulation skills from human to humanoid robots by employing active learning of multiple demonstrated tasks. This work aims to transfer motion skills from multiple human demonstrations in open surgery to robot manipulators in robot-assisted minimally invasive surgery (RA-MIS) by using TbD. However, the kinematic constraint should be respected during the performing of the learned skills by using a robot for minimally invasive surgery. In this article, we propose a novel methodology by integrating the cognitive learning techniques and the developed control techniques, allowing the robot to be highly intelligent to learn senior surgeons' skills and to perform the learned surgical operations in semiautonomous surgery in the future. Finally, experiments are performed to verify the efficiency of the proposed strategy, and the results demonstrate the ability of the system to transfer human manipulation skills to a robot in RA-MIS and also shows that the remote center of motion (RCM) constraint can be guaranteed simultaneously. Note to Practitioners-This article is inspired by limited access to the manipulation of laparoscopic surgery under a kinematic constraint at the point of incision. Current commercial surgical robots are mostly operated by teleoperation, which is representing less autonomy on surgery. Assisting and enhancing the surgeon's performance by increasing the autonomy of surgical robots has fundamental importance. The technique of teaching by demonstration (TbD) is capable of transferring the manipulation skills from human to humanoid robots by employing active learning of multiple demonstrated tasks. With the improved ability to interact with humans, such as flexibility and compliance, the new generation of serial robots becomes more and more popular in nonclinical research. Thus, advanced control strategies are required by integrating cognitive functions and learning techniques into the processes of surgical operation between robots, surgeon, and minimally invasive surgery (MIS). In this article, we propose a novel methodology to model the manipulation skill from multiple demonstrations and execute the learned operations in robot-assisted minimally invasive surgery (RA-MIS) by using a decoupled controller to respect the remote center of motion (RCM) constraint exploiting the redundancy of the robot. The developed control scheme has the following functionalities: 1) it enables the 3-D manipulation skill modeling after multiple demonstrations of the surgical tasks in open surgery by integrating dynamic time warping (DTW) and Gaussian mixture model (GMM)-based dynamic movement primitive (DMP) and 2) it maintains the RCM constraint in a smaller safe area while performing the learned operation in RA-MIS. The developed control strategy can also be potentially used in other industrial applications with a similar scenario.

An Accelerated Finite-Time Convergent Neural Network for Visual Servoing of a Flexible Surgical Endoscope With Physical and RCM Constraints.

This article designs and analyzes a recurrent neural network (RNN) for the visual servoing of a flexible surgical endoscope. The flexible surgical endoscope is based on a commercially available UR5 robot with a flexible endoscope attached as an end-effector. Most of the existing visual servo control frameworks of the robotic endoscopes or robot arms have not considered either the physical limits of the robot or the remote center of motion (RCM) constraints (i.e., the fulcrum effect). To tackle this issue, this article first conducts the kinematic modeling of the flexible robotic endoscope to achieve automation by visual servo control. The kinematic modeling results in a quadratic programming (QP) framework with physical limits and RCM constraints involved, making the UR5 robot applicable to surgical field. To solve the QP problem and accomplish the visual task, an RNN activated by a sign-bi-power activation function (AF) is proposed. The motivation of using the sign-bi-power AF is to enable the RNN to exhibit an accelerated finite-time convergence, which is more preferred in time-critical applications. Theoretically, the finite-time convergence of the RNN is rigorously proved using the Lyapunov theory. Compared with the previous AFs applied to the RNN, theoretical analysis shows that the RNN activated by the sign-bi-power AF delivers an accelerated convergence speed. Comparative validations are performed, showing that the proposed finite-time convergent neural network is effective to achieve visual servoing of the flexible endoscope with physical limits and RCM constraints handled simultaneously.

Improved recurrent neural network-based manipulator control with remote center of motion constraints: Experimental results

In this paper, an improved recurrent neural network (RNN) scheme is proposed to perform the trajectory control of redundant robot manipulators using remote center of motion (RCM) constraints. Firstly, learning by demonstration is implemented to model the surgical operation skills in the Cartesian space. After that, considering the kinematic constraints associated with the optimization control of redundant manipulators, we propose a novel RNN-based approach to facilitate accurate task tracking based on the general quadratic performance index, which includes managing the constraints on RCM joint angle, and joint velocity, simultaneously. The results of the conducted theoretical analysis confirm that the RCM constraint has been established successfully, and accordingly. The corresponding end-effector tracking errors asymptotically converge to zero. Finally, demonstration experiments are conducted in a laboratory setup environment using KUKA LWR4+ to validate the effectiveness of the proposed control strategy.

A Novel Flexible Robotic Endoscope With Constrained Tendon-Driven Continuum Mechanism

This letter presents a novel flexible robotic endoscope with a constrained tendon-driven continuum mechanism (CTCM) which is targeted for bariatric surgery. The robotic endoscope is composed of a UR5 robot and a CTCM-based flexible endoscope. By introducing a constraint tube, both the angulation and the length of the flexible segment can be controlled, ensuring improved flexibility and dexterity. The kinematic modeling of the flexible robotic endoscope is conducted with the remote center of motion (RCM) constraints and physical limits of the robotic system considered. To automate the endoscope robot, image-based visual servoing technique is employed. The kinematics modeling results in a quadratic programming (QP) based control framework with the physical limits and RCM constraints of the robotic endoscope involved. Experimental validation is performed to verify the effectiveness and superiority of the CTCM-based flexible endoscope for bariatric surgery as compared with conventional rigid and flexible endoscopes.