Invasive Surgical System Research Articles

Developments in robotic teleoperation

PurposeThe aim of this article is to provide details of recent technological developments in robotic teleoperation.Design/methodology/approachFollowing a short introduction, the two main sections of this article provide examples of recent research involving the application of virtual reality and haptic technologies, respectively, to robotic teleoperation. Brief conclusions are drawn.FindingsTeleoperation systems are being developed which incorporate virtual reality and haptic feedback technologies. Those using virtual reality seek to enhance the operator’s feeling of immersion in the scene and improve their situation awareness and trials involving diverse tasks illustrate that the technology can achieve these aims and overcome many limitations of traditional systems. Haptic feedback further enhances the degree of operator involvement and control and is now being adopted in commercial minimally invasive surgical systems. Systems which combine virtual reality with haptic feedback are being developed and have the potential to allow operators to conduct increasingly complex tasks.Originality/valueThrough reference to recent research, this illustrates how virtual reality and haptic technologies are enhancing the capabilities of robotic teleoperation.

Enhanced grip force estimation in robotic surgery: A sparrow search algorithm-optimized backpropagation neural network approach.

The absence of an effective gripping force feedback mechanism in minimally invasive surgical robot systems impedes physicians' ability to accurately perceive the force between surgical instruments and human tissues during surgery, thereby increasing surgical risks. To address the challenge of integrating force sensors on minimally invasive surgical tools in existing systems, a clamping force prediction method based on mechanical clamp blade motion parameters is proposed. The interrelation between clamping force, displacement, compression speed, and the contact area of the clamp blade indenter was analyzed through compression experiments conducted on isolated pig kidney tissue. Subsequently, a prediction model was developed using a backpropagation (BP) neural network optimized by the Sparrow Search Algorithm (SSA). This model enables real-time prediction of clamping force, facilitating more accurate estimation of forces between instruments and tissues during surgery. The results indicate that the SSA-optimized model outperforms traditional BP networks and genetic algorithm-optimized (GA) BP models in terms of both accuracy and convergence speed. This study not only provides technical support for enhancing surgical safety and efficiency, but also offers a novel research direction for the design of force feedback systems in minimally invasive surgical robots in the future.

Intraoperative laparoscopic photoacoustic image guidance system in the da Vinci surgical system.

This paper describes a framework allowing intraoperative photoacoustic (PA) imaging integrated into minimally invasive surgical systems. PA is an emerging imaging modality that combines the high penetration of ultrasound (US) imaging with high optical contrast. With PA imaging, a surgical robot can provide intraoperative neurovascular guidance to the operating physician, alerting them of the presence of vital substrate anatomy invisible to the naked eye, preventing complications such as hemorrhage and paralysis. Our proposed framework is designed to work with the da Vinci surgical system: real-time PA images produced by the framework are superimposed on the endoscopic video feed with an augmented reality overlay, thus enabling intuitive three-dimensional localization of critical anatomy. To evaluate the accuracy of the proposed framework, we first conducted experimental studies in a phantom with known geometry, which revealed a volumetric reconstruction error of 1.20 ± 0.71 mm. We also conducted an ex vivo study by embedding blood-filled tubes into chicken breast, demonstrating the successful real-time PA-augmented vessel visualization onto the endoscopic view. These results suggest that the proposed framework could provide anatomical and functional feedback to surgeons and it has the potential to be incorporated into robot-assisted minimally invasive surgical procedures.

A Transparent Teleoperated Robotic Surgical System with Predictive Haptic Feedback and Force Modelling.

In recent years, robotic minimally invasive surgery has transformed many types of surgical procedures and improved their outcomes. Implementing effective haptic feedback into a teleoperated robotic surgical system presents a significant challenge due to the trade-off between transparency and stability caused by system communication time delays. In this paper, these time delays are mitigated by implementing an environment estimation and force prediction methodology into an experimental robotic minimally invasive surgical system. At the slave, an exponentially weighted recursive least squares (EWRLS) algorithm estimates the respective parameters of the Kelvin-Voigt (KV) and Hunt-Crossley (HC) force models. The master then provides force feedback by interacting with a virtual environment via the estimated parameters. Palpation experiments were conducted with the slave in contact with polyurethane foam during human-in-the-loop teleoperation. The experimental results indicated that the prediction RMSE of error between predicted master force feedback and measured slave force was reduced to 0.076 N for the Hunt-Crossley virtual environment, compared to 0.356 N for the Kelvin-Voigt virtual environment and 0.560 N for the direct force feedback methodology. The results also demonstrated that the HC force model is well suited to provide accurate haptic feedback, particularly when there is a delay between the master and slave kinematics. Furthermore, a haptic feedback approach that incorporates environment estimation and force prediction improve transparency during teleoperation. In conclusion, the proposed bilateral master-slave robotic system has the potential to provide transparent and stable haptic feedback to the surgeon in surgical robotics procedures.

Enhanced Accuracy in Magnetic Actuation: Closed-loop Control of a Magnetic Agent with Low-Error Numerical Magnetic Model Estimation.

Magnetic actuation holds promise for wirelessly controlling small, magnetic surgical tools and may enable the next generation of ultra minimally invasive surgical robotic systems. Precise torque and force exertion are required for safe surgical operations and accurate state control. Dipole field estimation models perform well far from electromagnets but yield large errors near coils. Thus, manipulations near coils suffer from severe (10×) field modeling errors. We experimentally quantify closed-loop magnetic agent control performance by using both a highly erroneous dipole model and a more accurate numerical magnetic model to estimate magnetic forces and torques for any given robot pose in 2D. We compare experimental measurements with estimation errors for the dipole model and our finite element analysis (FEA) based model of fields near coils. With five different paths designed for this study, we demonstrate that FEA-based magnetic field modeling reduces positioning root-mean-square (RMS) errors by 48% to 79% as compared with dipole models. Models demonstrate close agreement for magnetic field direction estimation, showing similar accuracy for orientation control. Such improved magnetic modelling is crucial for systems requiring robust estimates of magnetic forces for positioning agents, particularly in force-sensitive environments like surgical manipulation.

Human-Machine Interaction Methods for Minimally Invasive Surgical Robotic Arms.

Minimally invasive surgery has a smaller incision area than traditional open surgery, which can greatly reduce damage to the human body and improve the utilization of medical devices. However, minimally invasive surgery also has disadvantages such as limited flexibility and operational characteristics. The interactive minimally invasive surgical robot system not only improves the stability, safety, and accuracy of minimally invasive surgery but also introduces force feedback in controlling the surgical robot, which is a new development direction in the field of minimally invasive surgery. This paper reviews the development status of interactive minimally invasive surgical robotic systems and key technologies to achieve human-robot interaction and finally provides an outlook and summary of its development. Fuzzy theory and reinforcement learning are introduced into the parameter adjustment process of the variable guide control model, and a human-robot interaction method for minimally invasive surgical robot posture adjustment is proposed.

Preoperative positioning planning for a robot-assisted minimally invasive surgical system based on accuracy and safety.

Minimally invasive robot-assisted techniques are being used increasingly in surgical procedures, and the preoperative preparation for this kind of operation is more complicated than traditional operations. Taking the placement requirements of minimally invasive thyroid surgery as the research object, taking the kinematic performance and safety of the manipulator as the optimization goal, a set of indexes and pendulums for evaluating the accuracy and safety of the manipulators are proposed accordingly. The optimal preoperative positioning scheme is obtained based on Non dominated sorting genetic algorithm- II multi-objective genetic algorithm. Meanwhile, compared with the optimization results of Gaot genetics and the positional scheme given by the surgeon based on experience, the advantages of the scheme are proved to be obvious. The scheme showed good performance in the comparison and proved the feasibility of the positional scheme in preoperative planning.

A novel low cost minimally invasive surgical system allows for the translation of modern pediatric surgical technology to low- and middle-income nations

BackgroundThere is a critical need for pediatric surgical humanitarian care. The role of minimally invasive surgery (MIS) in these environments with its reduction in pain and wound care, operative time, length of hospitalization, and morbidity is logical. However, the costs, logistics and feasibility of MIS in Low- and Middle-Income Countries (LMIC) can be challenging. Our goal was to develop a new low cost rapidly deployable minimally invasive surgical system (RDMIS) for use during remote pediatric general surgical (GPS) missions in LMIC. MethodsRDMIS system components consist of a universal serial bus (USB) interfaced laparoscopic camera, portable computer and a battery powered wireless portable laparoscopic light source. The surgeon transports the RDMIS in a single standard carry-on luggage. Utilizing prepositioning logistics from prior World Pediatric GPS missions, a standard MIS tower system (sMIS) was maintained on site. ResultsThe RDMIS was utilized to carry out procedural components of laparoscopic appendicostomy and laparoscopic cholecystectomy. Both sMIS and RDMIS were interchanged during the cases to allow for subjective comparison of surgical exposure and visualization. The RDMIS system allowed for safe and effective visualization and dissection of surgical structures. All children recovered uneventfully and were discharged the following day and have had no complications. The RDMIS system costs were significantly less than those of sMIS. ConclusionsRDMIS appears to be a safe, inexpensive option that will allow for the translation of modern MIS technology during GPS in remote locations. Further studies validating the RDMIS are indicated, however, the lower costs, ease of transport and potential benefit to children in the LMIC may be significant.

Friction modeling and compensation for haptic master manipulator based on deep Gaussian process

This paper investigates friction modeling and compensation for haptic master manipulator used in robot-assisted minimally invasive surgical system. Friction modeling and compensation is based on deep Gaussian process (DGP) and it does not require the utilization of explicit friction models. Therefore, the proposed friction modeling and compensation algorithm can circumvent the drawbacks associated with model-based methods. Through the adoption of implicit posterior variational inference, the proposed algorithm can accurately predict and compensate friction even when there exists parametric uncertainty in the dynamics of the haptic master manipulator. The effectiveness and feasibility of the proposed approach is validated experimentally through the robot-assisted minimally invasive surgical system. Experimental results demonstrate that the proposed method outperforms several existing alternatives in representing and compensating friction effects.

Non-Linear Hysteresis Compensation of a Tendon-Sheath-Driven Robotic Manipulator Using Motor Current

Tendon-sheath-driven manipulators (TSM) are widely used in minimally invasive surgical systems due to their long, thin shape, flexibility, and compliance making them easily steerable in narrow or tortuous environments. Many commercial TSM-based medical devices have non-linear phenomena resulting from their composition such as backlash hysteresis and dead zone, which lead to a considerable challenge for achieving precise control of the end effector pose. However, many recent works in the literature do not consider the combined effects and compensation of these phenomena, and less focus on practical ways to identify model parameters in realistic conditions. This paper proposes a simplified piecewise linear model to construct both backlash hysteresis and dead zone compensators together. Further, a practical method is introduced to identify model parameters using motor current from a robotic controller for the TSM. Our proposed methods are validated with multiple Intra-cardiac Echocardiography (ICE) catheters, which are typical commercial example of TSM, by periodic and non-periodic motions. Our results show that the errors from backlash hysteresis and dead zone are considerably reduced and therefore the accuracy of robotic control is improved when applying the presented methods.

Biologically Inspired Surgical Needle Steering: Technology and Application of the Programmable Bevel-Tip Needle.

Percutaneous interventions via minimally invasive surgical systems can provide patients with better outcomes and faster recovery times than open surgeries. Accurate needle insertions are vital for successful procedures, and actively steered needles can increase system precision. Here, we describe how biology inspired the design of a novel Programmable Bevel-Tip Needle (PBN), mimicking the mechanics and control methods of certain insects ovipositors. Following an overview of our unique research and development journey, this paper explores our latest, biomimetic control of PBNs and its application to neurosurgery, which we validate within a simulated environment. Three modalities are presented, namely a Direct Push Controller, a Cyclic Actuation Controller, and a newly developed Hybrid Controller, which have been integrated into a surgical visual interface. The results of open loop, expert human-in-the-loop and a non-expert user study show that the Hybrid Controller is the best choice when considering system performance and the ability to lesson strain on the surrounding tissue which we hypothesis will result in less damage along the insertion tract. Over representative trajectories for neurosurgery using a Hybrid Controller, an expert user could reach a target along a 3D path with an accuracy of mm, and non-expert users mm, both clinically viable results and equivalent or better than the state-of-the-art actively steered needles over 3D paths. This paper showcases a successful example of a biologically inspired, actively steered needle, which has been integrated within a clinical interface and designed for seamless integration into the neurosurgical workflow.

Open Reduction Internal Fixation of Condylar Head/Diacapitular Fracture of Mandible: Case Series.

Surgical management of condylar head is largely deferred due to the lack of appropriate armamentarium or instrumentation, restricted surgical access and risk of iatrogenic complications. Here we delineate open reduction internal fixation of condylar head fracture with various fixation modalities using specialized instrumentation for visualization and providing access for reduction with minimal complications. A total of 21 patients were reported with condylar head fracture of mandible to the Department of Oral and Maxillofacial Surgery from January 2017 to June 2018. Three patients had bilateral condylar head fracture, making it a total of 24 fractures. All patients had clinical symptoms including deranged occlusion, limited mouth opening, jaw deviation and restricted mandibular movements. The radiological findings were dislocated or displaced condylar head medially or laterally. All patients were treated by open reduction internal fixation using lag screws or standard long screws. Among condylar head fractures, 19 of the study population were male and 2 were female. Distribution of age among the condylar head fractures ranges from 19 years to 40 years with the mean being 22 years. At the end of three-month follow-up, all patients had satisfactory results, both clinically and radiologically. The functional outcome of this study was found to be superior. We recommend open reduction internal fixation of condylar head for patients with high risk of ankylosis, and it is possible without complications due to the availability of minimally invasive surgical access system.

Incidence of nonunion after surgery of distal femoral fractures using contemporary fixation device: a meta-analysis.

Nonunion is the most frequent cause of reoperation and is associated with high morbidity after distal femur fracture (DFF). We examined the rates of nonunion requiring reoperation after fixation for DFF using a locking compression plate (LCP) or retrograde intramedullary nail (RIMN). We included four studies comparing LCP and RIMN and 38 single-cohort studies reporting LCP or RIMN. In total, 2156 femurs were included and 166 non-unions were detected. We conducted a pair-wise meta-analysis (with a fixed-effects model) on the four comparative studies and a proportional meta-analysis on the 38 articles to estimate the nonunion rate. We performed sensitivity analysis by comparing studies using LCP with less invasive surgical systems (LISS) with those that used RIMN. The pairwise meta-analysis showed a similar nonunion rate between the groups [odds ratio: 1.02; 95% confidence interval (CI) 0.94-1.11, p = 0.633]. According to proportional meta-analysis, the pooled prevalence of nonunion was 5% (95% CI 4-7) totally, 6% (95% CI 4-8) in the LCP group, and 4% (95% CI 2-6) in the RIMN group (heterogeneity: p = 0.105). According to the sensitivity analysis, there was no difference in the union rate. The pooled prevalence of nonunion from sensitivity analysis was 4 % (95% CI, 3-5); it was 4% (95% CI, 3-6) in LCP with LISS and was 4% (95% CI, 2-6) in RIMN group (heterogeneity: p = 0.941). Approximately 5% of patients who underwent LCP or RIMN fixation developed nonunion. Therefore, LCP and RIMN are effective DFF techniques and mastering one of them is essential.

Microinstrument contact force sensing based on cable tension using BLSTM–MLP network

Minimally invasive surgical robotic systems established the foundation for precise and refined surgery, and the intelligentization of robotic systems is an important direction for future development. Among the methods of intelligentization, microinstrument external force sensing is an open and challenging research area. Force sensing information is used not only to ensure that surgeons apply the appropriate amount of force but also to prevent unintentional tissue damage. Because a microinstrument is a compact and small-sized construction, indirect force sensing method instead of the integration of sensors into the microinstrument is used, yielding better biocompatibility, sterilizability and monetary cost savings. This paper focuses on microinstrument-tissue contact force sensing, and the microinstrument used is a three degrees of freedom cable-driven manipulator. A contact force estimation strategy based on the differences in cable tension is established with consideration of the kinematics, dynamics and friction of the manipulator. A principle prototype of a surgical microinstrument force measurement system is developed, and then zero-drift, hysteresis and force loading experiments are studied. Based on the experimental data of the force loading experiments, the relationship between cable tension and contact forces is established by using a bidirectional long short-term memory plus multilayer perceptron network. The results show that the L2 cost of the network in the training set converges to 0.006 and that the RMSE of the network in the testing set converges to 0.053, and the network can meet the measurement requirements without overfitting. Therefore, the indirect force estimation method is a viable method of measuring forces of cable-driven microinstrument and can be used to integrate force sensing information into surgical robotic systems to improve the operability of surgical robots.

Modeling of a joint-type flexible endoscope based on elastic deformation and internal friction

Flexible endoscopes are widely used in minimally invasive surgical robot systems. Various kinematic models have been developed for describing the deformation of such endoscopes. For joint-type flexible endoscopes, most existing models neglect the effect of internal friction and cannot precisely show the shape.In this paper, we propose a new nonlinear bending model. The rubber tube and metal net at each joint are approximated as a tube under elastic deformation and are assigned an equivalent bending stiffness. The internal friction force is also taken into account to build the moment balance equation at each joint. Groups of experiments were performed to validate the nonlinear model. The results closely confirm the model’s predictions. The model’s tip position error during the bending and unbending phases are 1.48 ± 0.99 mm and 1.68 ± 0.91 mm respectively; the bending angle errors are −5.50 ± 2.54° and 1.68 ± 3.66°, respectively The model can also take account of the hysteresis effect of the bending, which is quite common for cable-driven flexible robots. Moreover, the model has good computational efficiency, making it suitable for real-time control.

ReTouchImg: Fusioning from-local-to-global context detection and graph data structures for fully-automatic specular reflection removal for endoscopic images

Minimally invasive surgical and diagnostic systems are commonly used in clinical practices. However, the accuracy and robustness of these systems depend heavily on computer based processes such as tracking, detecting or segmenting clinically meaningful regions of interest, which are significantly affected by the inherent specular reflections that appear on the organs’ surfaces. Restoration of the acquired data for clinical purposes still presents challenges because of the high texture and color variations across the image. In this work, we propose a novel fully-automated solution for endoscopic image restoration, which we call ReTouchImg. Our approach is designed as a two-step scheme. The first is a detection step that is based on the synergy of a set of color variations and gradient information conditions. For the second step, we introduce an inpainting process which is based on graph data structures for recovering the missing information. We exhaustively evaluate our approach on real endoscopic datasets and compare it against some works from the body of literature. We also demonstrate that our solution deals with complex cases such as strong illumination variation and large affected areas through a careful quantitative evaluation of a range of numerical results.

Image-guided, Minimally Invasive Evacuation of Intracerebral Hematoma: A Matched Cohort Study Comparing the Endoscopic and Tubular Exoscopic Systems.

IntroductionNovel image-guided, minimally invasive techniques to evacuate intracerebral hematomas represent a promising new avenue in the management of this disease entity. To our knowledge, a direct comparison of the Penumbra Apollo (Penumbra Inc, Alameda, California, US) and Nico BrainPath (Indianapolis, Indiana, US) system has not yet been performed.MethodsA retrospective review of image-guided, minimally invasive evacuation of intracerebral hematomas performed at one academic institution in the United States between July 2015 and July 2017 was performed. Cases performed with the Apollo and BrainPath system were matched based on age, gender, hematoma location and laterality, and volume.ResultsTwenty-four patients underwent surgery using either of the two minimally invasive surgical systems and five cases in each group were matched for age, gender, hematoma location and laterality, and volume. Median time from symptom onset to evacuation was two days with a mean distance from the brain surface to the clot of approximately 40 millimeters in both groups. Both techniques achieved comparable clot evacuation. The functional outcome was poor with either technique with the majority of patients dependent or dead at last follow-up.ConclusionsIn the present, small, matched cohort study, both the Apollo and BrainPath techniques achieved satisfactory clot evacuation. Nevertheless, the functional outcome in this patient population remains poor in the majority of cases.

Hysteresis analysis of a notched continuum manipulator driven by tendon

Abstract. Continuum manipulators are widely used in minimally invasive surgical robot systems (MISRS) because of their flexibility and compliance, while their modelling and control are relatively difficult and complex. This paper proposes an improved hysteresis model of a notched continuum manipulator based on the classical Bouc–Wen model, which can reduce errors and increase the accuracy of the kinematic-mechanics coupled model. Then parameters are identified by the mean of genetic algorithm (GA). Hysteresis phenomenon of the mentioned manipulator is actually caused by many factors such as the hysteresis property of Hyperelastic Nitinol Alloy (HNA), the elastic deformation of tendon and the friction between the tendon and the tube. The results of both static and dynamic experiments show that the introduced hysteresis model can eliminate the positional difference between forward and reverse bending processes, and thus improve the forecast precision of deformation during motion. This model can also be used to compensate modelling errors caused by hysteresis of other similar systems.

Toward the Design of Personalized Continuum Surgical Robots.

Robot-assisted minimally invasive surgical systems enable procedures with reduced pain, recovery time, and scarring compared to traditional surgery. While these improvements benefit a large number of patients, safe access to diseased sites is not always possible for specialized patient groups, including pediatric patients, due to their anatomical differences. We propose a patient-specific design paradigm that leverages the surgeon's expertise to design and fabricate robots based on preoperative medical images. The components of the patient-specific robot design process are a virtual reality design interface enabling the surgeon to design patient-specific tools, 3-D printing of these tools with a biodegradable polyester, and an actuation and control system for deployment. The designed robot is a concentric tube robot, a type of continuum robot constructed from precurved, elastic, nesting tubes. We demonstrate the overall patient-specific design workflow, from preoperative images to physical implementation, for an example clinical scenario: nonlinear renal access to a pediatric kidney. We also measure the system's behavior as it is deployed through real and artificial tissue. System integration and successful benchtop experiments in ex vivo liver and in a phantom patient model demonstrate the feasibility of using a patient-specific design workflow to plan, fabricate, and deploy personalized, flexible continuum robots.

Modular Instrument for a Haptically-Enabled Robotic Surgical System (HeroSurg)

To restore the sense of touch in robotic surgical systems, a modular force feedback-enabled laparoscopic instrument is developed and employed in a robotic-assisted minimally invasive surgical system (HeroSurg). Strain gauge technology is incorporated into the instrument to measure tip/tissue lateral interaction forces. The modularity feature of the proposed instrument makes it interchangeable between various tip types of different functionalities, e.g., cutter, grasper, and dissector, without losing force sensing capability. Series of experiments are conducted and results are reported to evaluate force sensing capability of the instrument. The results reveal mean errors of 1.32 g and 1.98° in the measurements of tip/tissue load magnitude and direction across all experiments, respectively.