Robotic Endoscope Research Articles

Using a Guidance Virtual Fixture on a Soft Robot to Improve Ureteroscopy Procedures in a Phantom

Manipulating a flexible ureteroscope is difficult, due to its bendable body and hand–eye coordination problems, especially when exploring the lower pole of the kidney. Though robotic interventions have been adopted in various clinical scenarios, they are rarely used in ureteroscopy. This study proposes a teleoperation system consists of a soft robotic endoscope together with a Guidance Virtual Fixture (GVF) to help users explore the kidney’s lower pole. The soft robotic arm was a cable-driven, 3D-printed design with a helicoid structure. GVF was dynamically constructed using video streams from an endoscopic camera. With a haptic controller, GVF can provide haptic feedback to guide the users in following a trajectory. In the user study, participants were asked to follow trajectories when the soft robotic arm was in a retroflex posture. The results suggest that the GVF can reduce errors in the trajectory tracking tasks when the users receive the proper training and gain more experience. Based on the NASA Task Load Index questionnaires, most participants preferred having the GVF when manipulating the robotic arm. In conclusion, the results demonstrate the benefits and potential of using a robotic arm with a GVF. More research is needed to investigate the effectiveness of the GVFs and the robotic endoscope in ureteroscopic procedures.

Force feedback reduces test time and interaction forces in telemanipulated palpation using a robotic endoscope with series elastic actuated joints.

Abstract Force feedback is currently missing in most surgical robot systems, even though it would be essential for many surgical tasks, such as palpating tissue. In this study, we used a previously developed robotic endoscope based on series elastic actuation and tested the feasibility of using it for force feedback with a pilot study. The participants had a lower test time (−27%) and lower applied forces (−23%) with force feedback enabled. While further studies are needed for validation in realistic surgical scenarios, these results suggest improved safety and efficiency in telemanipulated surgeries.

Stealth Autoguide as Robotic Holder for Cranial Endoscopic Surgery: Technical Note and Proof of Concept

The Stealth Autoguide (Medtronic, Minneapolis, USA) is a robotic auto-targeting device for stereotactic brain biopsy, placement of stereoelectroencephalography electrodes, and laser ablation therapy. This paper evaluates its off-label application as robotic endoscope holder in transnasal and intraventricular neurosurgery. The potential to enhance stability, reduce manual adjustments, and improve surgical precision is discussed, alongside cost-effectiveness and feasibility. Patients who underwent endoscopic endonasal, transventricular, and microsurgical endoscopic-assisted surgeries using the Stealth Autoguide as robotic holder from August 2023 to March 2024 were included. Surgical data and surgeons' remarks on advantages, disadvantages and limitations of the technique were collected. In this preliminary experience, Stealth Autoguide has proven safe and effective for several approaches. It was successfully used for patients with sellar, parasellar and suprasellar lesions. Moreover, the holder was employed in a third ventriculocisternostomy, a septostomy, and a removal of intraventricular lesion. Finally, the Autoguide has been a useful holder for the endoscope after gross total microsurgical excision of a tumor: the robot aligned the endoscope to the chosen trajectory and allowed the surgeon to "look around the corner" and confirm the extent of resection. No complications occurred in the treated cases, and the system provided great stability and usefulness throughout the surgeries. This experience proves that the Stealth Autoguide can be safely and effectively used as a robotic support for endoscopic procedures. This device ensures precise positioning and manipulation of neuroendoscopes, facilitating visualization and targeting of the desired area. Further researches are needed to verify applications and limitations.

A Noniterative Neural Algorithm Involving Dini Derivatives for Visual Servoing of Joint-Constrained Robotic Endoscope

A Noniterative Neural Algorithm Involving Dini Derivatives for Visual Servoing of Joint-Constrained Robotic Endoscope

Modeling and Compensation of Stiffness-Dependent Hysteresis for Stiffness-Tunable Tendon-Sheath Mechanism in Flexible Endoscopic Robots

Modeling and Compensation of Stiffness-Dependent Hysteresis for Stiffness-Tunable Tendon-Sheath Mechanism in Flexible Endoscopic Robots

MonoLoT: Self-Supervised Monocular Depth Estimation in Low-Texture Scenes for Automatic Robotic Endoscopy.

The self-supervised monocular depth estimation framework is well-suited for medical images that lack ground-truth depth, such as those from digestive endoscopes, facilitating navigation and 3D reconstruction in the gastrointestinal tract. However, this framework faces several limitations, including poor performance in low-texture environments, limited generalisation to real-world datasets, and unclear applicability in downstream tasks like visual servoing. To tackle these challenges, we propose MonoLoT, a self-supervised monocular depth estimation framework featuring two key innovations: point matching loss and batch image shuffle. Extensive ablation studies on two publicly available datasets, namely C3VD and SimCol, have shown that methods enabled by MonoLoT achieve substantial improvements, with accuracies of 0.944 on C3VD and 0.959 on SimCol, surpassing both depth-supervised and self-supervised baselines on C3VD. Qualitative evaluations on real-world endoscopic data underscore the generalisation capabilities of our methods, outperforming both depth-supervised and self-supervised baselines. To demonstrate the feasibility of using monocular depth estimation for visual servoing, we have successfully integrated our method into a proof-of-concept robotic platform, enabling real-time automatic intervention and control in digestive endoscopy. In summary, our method represents a significant advancement in monocular depth estimation for digestive endoscopy, overcoming key challenges and opening promising avenues for medical applications.

Fast Convergent Antinoise Dual Neural Network Controller With Adaptive Gain for Flexible Endoscope Robots.

Manual rigid endoscopes have defects such as a low efficiency, difficult operation, and safety risks, and the antinoise interference ability, convergence speed, and control accuracy of the neural network control technology for the existing autonomous endoscopes are often ignored. Solving these problems is important for the stable operation of endoscopes. Therefore, a new adaptive fast convergent antinoise dual neural network (AFA-DNN) controller for the visual servo control of ten-degree of freedom flexible endoscope robots (FERs) with physical constraints is proposed in this work. First, the control scheme of the FERs is formulated as a quadratic programming problem, and then, an AFA-DNN visual servo controller is designed for the FERs. The adaptive gains of the controller can accelerate the convergence, improve the antinoise ability, and increase the convergence accuracy of the controller. Then, according to the Lyapunov theory, the fast convergence of the AFA-DNN in finite time is proven for both noise-free and noisy conditions. The experimental results indicate that the FER controlled by the proposed AFA-DNN can accurately track various trajectories and that the AFA-DNN has a better antinoise interference ability, higher convergence accuracy, and faster convergence speed than conventional methods. The convergence speed of the AFA-DNN is increased by a factor of 4.22 by using the adaptive gains. Experiments also indicate that the AFA-DNN remains well functioning under various noise disturbances (such as constant, periodic, linear, and Gaussian noise).

Robotic platforms in gynaecological surgery: past, present, and future.

More than two decades ago, the advent of robotic laparoscopic surgery marked a significant milestone, featuring the introduction of the AESOP robotic endoscope control system and the ZEUS robotic surgery system. The latter, equipped with distinct arms for the laparoscope and surgical instruments, was designed to accommodate remote connections, enabling the practice of remote telesurgery as early as 2001. Subsequent technological progress has given rise to a range of options in today's market, encompassing multi-port and single-port systems, both rigid and flexible, across various price points, with further growth anticipated. This article serves as an indispensable guide for gynaecological surgeons with an interest in embracing robotic surgery. Drawing insights from the experience of the Strasbourg training centre for minimally invasive surgery (IRCAD), this article offers a comprehensive overview of existing robotic platforms in the market, as well as those in development. Robotic surgical systems not only streamline established operative methods but also broaden the scope of procedures, including intra- and transluminal surgeries. As integral components of the digital surgery ecosystem, these robotic systems actively contribute to the increasing integration and adoption of advanced technologies, such as artificial intelligence-based data analysis and support systems. Robotic surgery is increasingly being adopted in clinical practice. With the growing number of systems available on the marketplace, the primary challenge lies in identifying the optimal platform for each specific procedure and patient. The seamless integration of robotic systems with artificial intelligence, image-guided surgery, and telesurgery presents undeniable advantages, enhancing the precision and effectiveness of surgical interventions. This article provides a guide to the robotic platforms available on the market and those in development for gynaecologists interested in robotic surgery.

Robotic-Assisted Inguinal Hernia Repair with The Dexter System™: A Prospective Multicentre Clinical Investigation

Abstract Background Robot-assisted transabdominal preperitoneal inguinal hernia repair (rTAPP) is established as safe and feasible with various robotic platforms. The Dexter robotic system is an open platform with a sterile surgeon’s console, two robotic instrument arms, and one robotic endoscope arm. Aims We conducted a prospective multicentre study (ClinicalTrials.gov NCT05873582) to assess the safety and performance of this new robotic platform. Methods Three centres in France, Germany, and Switzerland participated. The primary objectives of the study were to document the successful completion of the rTAPP procedures and to collect data on the occurrence of adverse events and device-related events perioperatively and up to 30 days post-surgery. The procedures were performed by three surgeons with varying levels of experience in robotic systems, including novice, intermediate and expert. Results Fifty patients were operated for inguinal hernia repair (33 unilateral, 17 bilateral). The mean age and BMI of the patients were 60 years (±15) and 25.7 kg/cm2 (±3.3), respectively. All surgeries were successfully completed as planned using three standard laparoscopy trocars sized 10/12mm. Each procedure was performed with robotic assistance, and there were no conversions to open surgery. No intra-operative complications, device deficiency, or adverse events of Clavien-Dindo Grade III to V occurred. The mean skin-to-skin operative time was 55 min (±14) for unilateral hernia repair and 95 min (±22) for bilateral hernia repair. The mean robotic console time was 34 min (±12) for unilateral hernia and 68 min (±21) for bilateral hernia repairs. The patients were discharged within 1day (± 1 day) after surgery. Conclusion Our rTAPP experience with the Dexter system confirms feasibility, safety, and comparable operative times to other robotic platforms. This study demonstrates the practicality and safety of the robotic techniques, positioning the Dexter system as a valuable tool in hernia repair.

An MRI-Conditional Flexible Endoscopic Robot with a Hydraulic Tendon-Driven Actuation System

This paper presents the conceptualization, mathematical modeling, and experimental validation of a magnetic resonance imaging (MRI)-conditional, flexible endoscopic robot for MRI-guided neurosurgery. The robotic assembly encompasses a straight rigid shaft, a tendon-driven steerable tip with two degrees of freedom, and a compact hybrid actuation system made of nonmagnetic modules. The robot’s small diameter, approximately 3 mm, allows for robot navigation through narrow cavities, such as nasal passages and the densely packed anatomy of the skull base. By filling the robot with deionized water, the MRI contrast of the robot can be improved, therefore enabling intra-operative tracking of the robot’s movement. In this paper, we present the design and fabrication process of the robotic system, kinematic analysis of the robot, and experimental studies to validate our model, characterize the robot, and test the feedback robot control. Furthermore, we demonstrate the MRI compatibility of the robot, along with the intra-operative tracking capability.

A miniature triaxial force sensor based on fiber Bragg gratings for flexible endoscopic robot

Contact force sensing is an important means to ensure the safe operation of surgical robots. This paper presents a miniature triaxial force sensor based on fiber Bragg grating (FBG) for detecting the interaction forces between the tip of flexible endoscopic surgical robots and tissue. The proposed sensor mainly consists of a force-sensitive flexure structure, a support body, and four FBG sensors. A composite hollow structure with a parallel helical slot and elliptical hinge has been utilized to design the force-sensitive flexure with an excellent linear measurement range. The four optical fibers arranged along the circumference of the flexure are configured as a tightly stretched status with their two points fixed. This configuration can improve sensitivity and avoid the drawbacks of FBG chirping and low repeatability. Finite element method-based simulation has been implemented to optimize the flexure’s structural parameters for balancing the force sensor’s axial and radial resolution. The prototyped sensor has been calibrated and achieves a high resolution of 0.77 mN and 0.79 mN for both lateral forces within −1 to 1 N and 1.25 mN for axial force within 0 to 2 N. Loading experiments have been conducted to validate the effectiveness of the proposed designs. The results show that the sensor can provide a high-accuracy measurement of three force components with an average RMS error of less than 3% in the total operating ranges. Such a result validates the feasibility of the designed sensor to measure the robot-tissue interaction forces.

Development of a flexible endoscopic robot with autonomous tracking control ability using machine vision and deep learning

Abstract. A flexible endoscopic robot is designed to solve the problem that it is difficult for auxiliary doctors to maintain a stable visual field in traditional endoscopic surgery. Based on geometric derivation, a motion control method under the constraint of the remote center motion (RCM) of the robot system is established, and a set of circular trajectories are planned for it. The RCM error of the robot during operation and the actual trajectory of the robot end in three-dimensional space are obtained through the motion capture system. The end of the robot is controlled by the heterogeneous primary–secondary teleoperation control algorithm based on position increments. Finally, the RTMDet deep learning object detection algorithm was selected to identify and locate surgical instruments through comparative experiments, and the autonomous tracking control was completed based on visual guidance. In the process of autonomous tracking, the RCM error was less than 1 mm, which met the actual surgical requirements.

MorphGI: A Self-Propelling Soft Robotic Endoscope Through Morphing Shape.

Colonoscopy is currently the best method for detecting bowel cancer, but fundamental design and construction have not changed significantly in decades. Conventional colonoscope (CC) is difficult to maneuver and can lead to pain with a risk of damaging the bowel due to its rigidity. We present the MorphGI, a robotic endoscope system that is self-propelling and made of soft material, thus easy to operate and inherently safe to patient. After verifying kinematic control of the distal bending segment, the system was evaluated in: a benchtop colon simulator, using multiple colon configurations; a colon simulator with force sensors; and surgically removed pig colon tissue. In the colon simulator, the MorphGI completed a colonoscopy in an average of 10.84 min. The MorphGI showed an average of 77% and 62% reduction in peak forces compared to a CC in high- and low-stiffness modes, respectively. Self-propulsion was demonstrated in the excised tissue test but not in the live pig test, due to anatomical differences between pig and human colons. This work demonstrates the core features of MorphGI.

Noise-Resistant Adaptive Gain Recurrent Neural Network for Visual Tracking of Redundant Flexible Endoscope Robot With Time-Varying State Variable Constraints

Noise-Resistant Adaptive Gain Recurrent Neural Network for Visual Tracking of Redundant Flexible Endoscope Robot With Time-Varying State Variable Constraints

Stereo Visual Servoing Control of a Soft Endoscope for Upper Gastrointestinal Endoscopic Submucosal Dissection.

Quickly and accurately completing endoscopic submucosal dissection (ESD) operations within narrow lumens is currently challenging because of the environment's high flexibility, invisible collision, and natural tissue motion. This paper proposes a novel stereo visual servoing control for a dual-segment robotic endoscope (DSRE) for ESD surgery. Departing from conventional monocular-based methods, our DSRE leverages stereoscopic imaging to rapidly extract precise depth data, enabling quicker controller convergence and enhanced surgical accuracy. The system's dual-segment configuration enables agile maneuverability around lesions, while its compliant structure ensures adaptability within the surgical environment. The implemented stereo visual servo controller uses image features for real-time feedback and dynamically updates gain coefficients, facilitating rapid convergence to the target. In visual servoing experiments, the controller demonstrated strong performance across various tasks. Even when subjected to unknown external forces, the controller maintained robust performance in target tracking. The feasibility and effectiveness of the DSRE were further verified through ex vivo experiments. We posit that this novel system holds significant potential for clinical application in ESD surgeries.

Research on the linear driving characteristics of endoscopic continuous robot

Due to the way line drive, the continuous endoscope robot will have a certain lag in the process of movement, which will affect the accuracy and flexibility of the operation. In addition, the hysteresis and return difference caused by wire rope transmission will also increase the hysteresis effect of continuous robots. In this paper, the motion characteristics of the continuous endoscopic robot are analyzed, the hysteresis of the robot is modeled theoretically based on the Preisach model, and the effectiveness of the model is verified. Preisach hysteretic nonlinear hyperbola model predicts the bending changes under different positive and negative drives. The experimental results show that the maximum error between the model and the theoretical prediction is 3.18 degrees. The hysteresis model can predict the hysteresis characteristics of the continuous endoscope robot well.

Development and Evaluation of a Markerless 6 DOF Pose Tracking Method for a Suture Needle from a Robotic Endoscope

Remarkable progress has been made in the field of robot-assisted surgery in recent years, particularly in the area of surgical task automation, though many challenges and opportunities still exist. Among these topics, the detection and tracking of surgical tools play a pivotal role in enabling autonomous systems to plan and execute procedures effectively. For instance, accurate estimation of a needle’s position and posture is essential for surgical systems to grasp the needle and perform suturing tasks autonomously. In this paper, we developed image-based methods for markerless 6 degrees of freedom (DOF) suture needle pose estimation using keypoint detection technique based on Deep Learning and Point-to-point Registration, we also leveraged multi-viewpoint from a robotic endoscope to enhance the accuracy. The data collection and annotation process was automated by utilizing a simulated environment, enabling us to create a dataset with 3446 evenly distributed needle samples across a suturing phantom space for training and to demonstrate more convincing and unbiased performance results. We also investigated the impact of training set size on the keypoint detection accuracy. Our implemented pipeline that takes a single RGB image achieved a median position error of 1.4 mm and a median orientation error of 2.9∘, while our multi-viewpoint method was able to further reduce the random errors.

Cerebellum-Inspired Model-Free Tracking Control and Visual Servoing of a Rigid-Flexible Hybrid Robotic Endoscope With RCM Constraints

This article designs a model-free control scheme based on a cerebellum-inspired network model for tracking control and visual servoing of a surgical rigid-flexible hybrid robotic endoscope. The hybrid robot is composed of a rigid robot manipulator and a flexible endoscope as the end-effector. Inspired by the cerebellum associated with human motion control, we proposed a cerebellum-inspired model-free control scheme for the visual servoing and tracking control of the robotic endoscope with RCM (remote center of motion) constraints, which does not need to know the kinematic model of the robot. This scheme combines liquid state machines (LSM) and zeroing neural network (ZNN), where the LSM is able to generate control signals and the ZNN is used as the training signal for the LSM. Simulations and physical experiments demonstrate that the proposed cerebellum-inspired control scheme is effective to handle the tracking task under RCM constraints as well as the visual servoing task.

Assisted teleoperation control of robotic endoscope with visual feedback for nasotracheal intubation

Endoscopic operation is one of the most difficult parts of nasotracheal intubation (NTI), a surgical procedure used to secure the airway of a patient. Due to the requirement of eye–hand coordination, manual operation of a flexible endoscope is challenging, even for an experienced surgeon. To enhance intubation efficiency, this paper developed a master–slave robotic nasotracheal intubation system (RNIS) for endoscopic operation. Movements in three degrees of freedom of the endoscope are controlled by the RNIS. An assisted teleoperation control strategy is designed to assist the operator in remotely controlling the pose, i.e., position and orientation, of the endoscope tip via a joystick. To ensure the efficiency of intubation, visual feedback assistance is hereby proposed, which fine-tunes the orientation of the endoscope tip as needed. The proposed system and methods are experimentally validated on a motion simulator and a phantom. The results demonstrated that the master–slave RNIS can successfully insert the flexible endoscope into the trachea of a phantom through the nasal cavity.

Sensing Mucus Physiological Property In Situ by Wireless Millimeter‐Scale Soft Robots

AbstractThe physiological property of mucus is an important biomarker for monitoring the human health conditions and helping understand disease development, as mucus property such as viscosity is highly correlated with inflammation and other diseases. However, it remains challenging to sense mucus viscosity using pure medical imaging. Collecting and analyzing mucus sample in vitro using flexible endoscopes and capsule endoscope robots is also challenging due to their difficulty of accessing very confined, tortuous, and small spaces, and the sample may not reflect the real mucus property. Here a novel method is proposed to enable sensing mucus viscosity in situ by wireless miniature sensors actuated by magnetic fields and tracked by medical imaging. These miniature viscosity sensors can be delivered with minimal invasion using a novel sensor delivery mechanism by controlling a magnetically actuated millimeter‐scale soft climbing robot. As the soft robot can access confined and narrow spaces, and reliably deploy the sensor on soft tissue surfaces, multiple sensors can be delivered on soft biological tissues to sense biofluid viscosity spatiotemporally. The proposed minimally invasive robotic delivery and viscosity sensing method thus paves the way toward sensing biofluid properties deep inside the body for future disease monitoring and early diagnosis functions.