Tip Position Error Research Articles

A temporal enhanced semi-supervised training framework for needle segmentation in 3D ultrasound images

Objective. Automated biopsy needle segmentation in 3D ultrasound images can be used for biopsy navigation, but it is quite challenging due to the low ultrasound image resolution and interference similar to the needle appearance. For 3D medical image segmentation, such deep learning networks as convolutional neural network and transformer have been investigated. However, these segmentation methods require numerous labeled data for training, have difficulty in meeting the real-time segmentation requirement and involve high memory consumption. Approach. In this paper, we have proposed the temporal information-based semi-supervised training framework for fast and accurate needle segmentation. Firstly, a novel circle transformer module based on the static and dynamic features has been designed after the encoders for extracting and fusing the temporal information. Then, the consistency constraints of the outputs before and after combining temporal information are proposed to provide the semi-supervision for the unlabeled volume. Finally, the model is trained using the loss function which combines the cross-entropy and Dice similarity coefficient (DSC) based segmentation loss with mean square error based consistency loss. The trained model with the single ultrasound volume input is applied to realize the needle segmentation in ultrasound volume. Main results. Experimental results on three needle ultrasound datasets acquired during the beagle biopsy show that our approach is superior to the most competitive mainstream temporal segmentation model and semi-supervised method by providing higher DSC (77.1% versus 76.5%), smaller needle tip position (1.28 mm versus 1.87 mm) and length (1.78 mm versus 2.19 mm) errors on the kidney dataset as well as DSC (78.5% versus 76.9%), needle tip position (0.86 mm versus 1.12 mm) and length (1.01 mm versus 1.26 mm) errors on the prostate dataset. Significance. The proposed method can significantly enhance needle segmentation accuracy by training with sequential images at no additional cost. This enhancement may further improve the effectiveness of biopsy navigation systems.

Toward Extending Concentric Tube Robot Kinematics for Large Clearance and Impulse Curvature.

Concentric Tube Robots (CTRs) have been proposed to operate within the unstructured environment for minimally invasive surgeries. In this letter, we consider the operation scenario where the tubes travel inside the channels with a large clearance or large curvature, such as aortas or industrial pipes. Accurate kinematic modeling of CTRs is required for the development of motion planning and control within these operation scenarios. To this end, we extended the conventional CTR kinematics model to a more general case with large tube-to-tube clearance and large centerline curvature. Numerical simulations and experimental validations are conducted to compare our model with respect to the conventional CTR kinematic model. In the physical experiments, our proposed model achieved a tip position error of 1.53 mm in the 2D planer case and 3.86 mm in 3D case, outperforming the state-of-the-art model by 71% and 61%, respectively.

Data-driven Shape Sensing of Continuum Dexterous Manipulators Using Embedded Capacitive Sensor.

We propose a novel inexpensive embedded capacitive sensor (ECS) for sensing the shape of Continuum Dexterous Manipulators (CDMs). Our approach addresses some limitations associated with the prevalent Fiber Bragg Grating (FBG) sensors, such as temperature sensitivity and high production costs. ECSs are calibrated using a vision-based system. The calibration of the ECS is performed by a recurrent neural network that uses the kinematic data collected from the vision-based system along with the uncalibrated data from ECSs. We evaluated the performance on a 3D printed prototype of a cable-driven CDM with multiple markers along its length. Using data from three ECSs along the length of the CDM, we computed the angle and position of its tip with respect to its base and compared the results to the measurements of the visual-based system. We found a 6.6% tip position error normalized to the length of the CDM. The work shows the early feasibility of using ECSs for shape sensing and feedback control of CDMs and discusses potential future improvements.

Model-based Design of the COAST Guidewire Robot for Large Deflection.

Minimally invasive endovascular procedures involve the manual placement of a guidewire, which is made difficult by vascular tortuosity and the lack of precise tip control. Steerable guidewire systems have been developed with tendon-driven, magnetic, and concentric tube actuation strategies to enable precise tip control, however, selecting machining parameters for such robots does not have a strict procedure. In this paper, we develop a systematic design procedure for selecting the tube pairs of the COaxially Aligned STeerable (COAST) guidewire robot. This includes the introduction of a mechanical model that accounts for micromachining-induced pre-curvatures with the goal of determining design parameters that reduce combined distal tip pre-curvature and minimize abrupt changes in actuated tip position for the COAST guidewire robot through selection of the best flexural rigidity between the tube pairs. We present adjustments in the kinematics modeling of COAST robot tip bending motion, and use these to characterize the bending behavior of the COAST robot for varying geometries of the micromachined tubes, with an average RMSE value for the tip position error of 0.816 mm in the validation study.

Design and kinematics of a lightweight cruciform continuum robot

Abstract. The design of new lightweight and dexterous configurations is a major research focus for continuum robotics. This work proposes a cruciform continuum robot. Its unique feature is that it is formed by multiple cruciform-arranged elastic sheets with a single dimension of motion connected in series, and thus it has low-coupling motion characteristics. In addition, the cruciform continuum robot has the advantages of lighter weight (65 g), better dexterity, and higher motion accuracy. In this paper, the forward and inverse kinematics models of the cruciform continuum robot are established by geometric methods based on the assumption of constant curvature, and its workspace is analysed. It is experimentally verified that the tip position errors are less than 1 mm, and the cable length errors are less than 0.4 mm. Further, the cruciform continuum robot is successfully used for the nucleic acid detection simulation experiment, which confirms its good dexterity and man–machine safety. The main contribution of this paper is to provide a new configuration for the lightweight and dexterous continuum robots, and to further provide a reference method for improving their modelling accuracy from the perspective of structure.

感度解析に基づくロボット関節誤差拡大軌道の設計と動画による関節原点誤差計測

For industrial welding robots using the teaching-playback method, repeatability is more important than absolute accuracy. However, when a robot is replaced due to failure or other reasons, the tip position is shifted because the error factors related to tip positioning are different between the robot before and after the replacement. This can impede recovery. To overcome this problem at production sites where industrial robots are used, easy measurement without the use of expensive sensors is required. On the other hand, when the robot is replaced with another robot of the same model, the mechanical errors such as joint stiffness, arm stiffness and link length error are almost the same and their difference can be assumed to be extremely small, but the joint origin error remains as an individual difference and its effect on the tip misalignment is not small. Therefore, it is the joint origin error that has a large influence on tip misalignment as an individual difference. In recent years, inexpensive devices with high image resolution, such as consumer-use cameras and smartphones have become available at low cost and their use is effective to reduce tip misalignment caused by individual differences in robots at manufacturing sites. In this paper, we focus on the sensitivity of the tip position error with respect to the joint origin error and propose a method to estimate the joint origin error by designing a trajectory in which the tip error is magnified so that the joint origin error can be easily measured using movies of the tip position taken by a digital camera. We verify the effectiveness of this method using an experimental robot.

Three-Dimensional Curve Reconstruction Based on Material Frame and Twisted Multicore Fiber

We propose a curve reconstruction model based on material frame and twisted multicore fiber (MCF), which can reconstruct the position and orientation of MCF simultaneously. The twisted MCF is approximated as a ribbon, whose centerline and surface characteristics are used to characterize the position and orientation of MCF. Material frame corresponding to the ribbon is established, and quantitative analysis is realized by the differential equations between frame basis vectors and fiber deformation parameters. Bending, twisting-strain model is established according to the approximate development of the twisted MCF, and the deformation parameters are obtained. Further, position and orientation of MCF can be reconstructed under the interference of twisting deformation. In simulation, the influence of curvature, torsion of target curves and twist bias, twisting deformation of MCF on curve reconstruction is analyzed. The results indicate that the proposed model is applicable to curves with zero curvature and variable curvature. With 10 rad/m sinusoidal twisting deformation, the deterioration of reconstruction accuracy can be effectively suppressed by introducing twist bias into MCF. For S-shaped curve with curvature and torsion within 1–50 m <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">−1</sup> and −10-10 rad/m, the tip position error and tip orientation error are less than 5 mm and 5 mrad over the length of 1 m.

Kalman-Filter-Based, Dynamic 3-D Shape Reconstruction for Steerable Needles With Fiber Bragg Gratings in Multicore Fibers

Steerable needles are a promising technology to provide safe deployment of tools through complex anatomy in minimally invasive surgery, including tumor-related diagnoses and therapies. For the 3-D localization of these instruments in soft tissue, fiber Bragg gratings (FBGs) based reconstruction methods have gained in popularity because of the inherent advantages of optical fibers in a clinical setting, such as flexibility, immunity to electromagnetic interference, nontoxicity, and the absence of line-of-sight issues. However, methods proposed thus far focus on shape reconstruction of the steerable needle itself, where accuracy is susceptible to errors in interpolation and curve fitting methods used to estimate the curvature vectors along the needle. In this study, we propose reconstructing the shape of the path created by the steerable needle tip based on the follow-the-leader nature of many of its variants. By assuming that the path made by the tip is equivalent to the shape of the needle, this novel approach paves the way for shape reconstruction through a single set of FBGs at the needle tip, which provides curvature information about every section of the path during navigation. We propose a Kalman-filter-based sensor fusion method to update the curvature information about the sections as they are continually estimated during the insertion process. The proposed method is validated through simulation, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">in vitro</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ex vivo</i> experiments employing a programmable bevel-tip steerable needle (PBN). The results show clinically acceptable accuracy, with 2.87-mm mean PBN tip position error, and a standard deviation of 1.63 mm for a 120-mm 3-D insertion.

Application of bidirectional rapidly exploring random trees (BiRRT) algorithm for collision-free trajectory planning of free-floating space manipulator

AbstractOn-orbit servicing and active debris removal missions will rely on the use of unmanned satellite equipped with a manipulator. Capture of the target object will be the most challenging phase of these missions. During the capture manoeuvre, the manipulator must avoid collisions with elements of the target object (e.g., solar panels). The dynamic equations of the satellite-manipulator system must be used during the trajectory planning because the motion of the manipulator influences the position and orientation of the satellite. In this paper, we propose application of the bidirectional rapidly exploring random trees (BiRRT) algorithm for planning a collision-free trajectory of a manipulator mounted on a free-floating satellite. A new approach based on pseudo-velocities method (PVM) is used for construction of nodes of the trajectory tree. Initial nodes of the second tree are selected from the set of potential final configurations of the system. The proposed method is validated in numerical simulations performed for a planar case (3-DoF manipulator). The obtained results are compared with the results obtained with two other trajectory planning methods based on the RRT algorithm. It is shown that in a simple test scenario, the proposed BiRRT PVM algorithm results in a lower manipulator tip position error. In a more difficult test scenario, only the proposed method was able to find a solution. Practical applicability of the BiRRT PVM method is demonstrated in experiments performed on a planar air-bearing microgravity simulator where the trajectory is realised by a manipulator mounted on a mock-up of the free-floating servicing satellite.

Maneuver and formation control of a pair of soft pneumatic bending fingers by a cooperative strategy for planar in-hand object manipulation

In this paper, we seek performing dynamic in-hand object manipulation with two-segmented soft fingers in 2-D plane. Thereby, after the dynamic behavior of the soft fingers was defined, we evaluate the influence of external tip force on the finger. It is shown that the tip position error, under the effect of tip force, can be represented by uncertainties in kinematic parameters of the finger. As a result, we design a sliding-adaptive controller with compensation of kinematic uncertainties to fulfill our expectation that is controlling the tip position of the finger in vertical plane while interacting with the environment. Finally, inspired by the way that human fingers manipulate an object, a cooperative control system for manipulation is proposed which incorporates the states and tip positions of the fingers in its feedback signal. Combination of this controller with the adaptive control system yields more promising results than does the PID controller. In order to evaluate performance of the proposed controller, Simulation results for manipulation of a cubic rigid object are demonstrated.

Fluoroscopic Navigation for a Surgical Robotic System Including a Continuum Manipulator.

We present an image-based navigation solution for a surgical robotic system with a Continuum Manipulator (CM). Our navigation system uses only fluoroscopic images from a mobile C-arm to estimate the CM shape and pose with respect to the bone anatomy. The CM pose and shape estimation is achieved using image intensity-based 2D/3D registration. A learning-based framework is used to automatically detect the CM in X-ray images, identifying landmark features that are used to initialize and regularize image registration. We also propose a modified hand-eye calibration method that numerically optimizes the hand-eye matrix during image registration. The proposed navigation system for CM positioning was tested in simulation and cadaveric studies. In simulation, the proposed registration achieved a mean error of 1.10±0.72 mm between the CM tip and a target entry point on the femur. In cadaveric experiments, the mean CM tip position error was 2.86±0.80 mm after registration and repositioning of the CM. The results suggest that our proposed fluoroscopic navigation is feasible to guide the CM in orthopedic applications.

A Magnetic Continuum Robot With Multi-Mode Control Using Opposite-Magnetized Magnets

Magnetic field-controlled continuum robots can be steered dexterously within a small dimension. However, their dexterity, generally being restricted to C-shape, is not sufficient for most practical applications, e.g., avoiding obstacles to reach the target region, fixing the tip orientation, or passing through sharp access with the lowest damage to the human body. In this letter, a novel magnetic continuum robot (MCR) with multi-mode control using opposite-magnetized magnets is proposed and studied. A prototype robot containing a pair of opposite-magnetized magnets and a hollow elastic tube was designed and constructed. This simple structure does not hinder the miniaturization of robots. The robot is steered by an external mobile magnet, which can generate an inhomogeneous field to offer a varying magnetic torque and force on opposite-magnetized magnets to help achieve multi-mode deformation. A practical mathematical model, coupling point-dipole field model and energy-based kinematics, is developed to describe its multi-mode deformation with a mean error of 0.72 ± 0.52 mm along the arc length, and a mean tip position error of 1.44 ± 0.74 mm. Experimental results show that the opposite-magnetized magnetic continuum robot has a larger tip reachable workspace and improved dexterity compared to other uniform-magnetized MCRs.

Needle artifact characteristics and insertion accuracy using a 1.2 T open MRI scanner: A phantom study

PurposeTo evaluate the characteristics of needle artifacts and the accuracy of needle insertion using a 1.2 Tesla open magnetic resonance imaging (MRI) system in a phantom. Materials and methodsFirst, the apparent width of the needle on the MRI and the needle tip position error of 16- and 18-gauge MRI-compatible introducer needles and a 17-gauge cryoneedle were examined with different needle angles (0°, 30°, 45°, 60°, and 90°) to the main magnetic field (B0), sequence types (balanced steady-state acquisition with rewound gradient echo [BASG] and T2-weighted fast spin echo [FSE] sequence), and frequency encoding directions. Second, the accuracy of needle insertion was evaluated after 10 MRI fluoroscopy-guided insertions in a phantom. ResultsThe apparent needle widths was larger when the angle of the needle axis relative to B0 was larger. The needles appeared larger on BASG than on T2-weighted FSE images, with the largest apparent widths of 16-, 17-, and 18-gauge needles of 14.3, 11.6, and 11.0mm, respectively. The apparent needle tip position was always more distal than the actual position on BASG images, with the largest longitudinal error of 4.0mm. Meanwhile, the 16- and 18-gauge needle tips appeared more proximal on T2-weighted FSE images with right-to-left frequency encoding direction. The mean accuracy of MRI fluoroscopy-guided needle insertion was 3.1mm. ConclusionThese experiments clarify the characteristics of needle artifacts in a 1.2 Tesla open MRI. With this system, the MRI fluoroscopy-guided needle insertion demonstrated an acceptable accuracy for clinical use.

Shape Estimation of Soft Manipulator Using Stretchable Sensor.

The soft robot manipulator is attracting attention in the surgical fields with its intrinsic softness, lightness in its weight, and safety toward the human organ. However, it cannot be used widely because of its difficulty of control. To control a soft robot manipulator accurately, shape sensing is essential. This paper presents a method of estimating the shape of a soft robot manipulator by using a skin-type stretchable sensor composed of a multiwalled carbon nanotube (MWCNT) and silicone (p7670). The sensor can be easily fabricated and applied by simply attaching it to the surface of the soft manipulator. In its fabrication, MWCNT is sprayed on a teflon sheet, and liquid-state silicone is poured on it. After curing, we turn it over and cover it with another silicone layer. The sensor is fabricated with a sandwich structure to decrease the hysteresis of the sensor. After calibration and determining the relationship between the resistance of the sensor and the strain, three sensors are attached at 120° intervals. Using the obtained data, the curvature of the manipulator is calculated, and the entire shape is reconstructed. To validate its accuracy, the estimated shape is compared with the camera data. We experiment with three, six, and nine sensors attached, and the result of the error of shape estimation is compared. As a result, the minimum tip position error is approximately 8.9 mm, which corresponded to 4.45% of the total length of the manipulator when using nine sensors.

Experimental investigation of a twisted string actuation for usage in active catheter.

The improvement of the steerability and controllability of catheters is highly required because catheter-based interventional diagnosis and therapy have become increasingly popular. However, active catheters are often complex and expensive. This study attempts to control catheter tip by a twisted string actuator. Experiments are carried out to analyze the full actuation and bending characteristics of the active catheter. It is indicated that the expected bending motion can be achieved by twisting the strings. Hysteresis behavior in backward and forward motion is also discussed. High repetitive positional accuracy and reverse position accuracy can be obtained when the number of rotation is low. When the number of turns is 25, the maximum tip position errors are 4, 1.8 and 1 mm in X, Y and Z directions, respectively. It is expected that the presented method will prove useful for the application of twisted string active catheter.

High-Accuracy Distributed Bend Sensor Eligible for High-Curvature Structures Based on Brillouin Optical Correlation Domain Analysis

We present a high-accuracy distributed bend sensor suitable for high-curvature structures based on Brillouin frequency measurement using a multi-fiber probe. Brillouin optical correlation domain analysis (BOCDA) with differential measurement is adopted for distributed strain sensing with a cm-order spatial resolution (minimum 1 cm), and a specially designed sensing probe embedding three optical fibers is used to enhance the minimum measurable bending radius and accuracy of bend sensing. Analysis on the reconstruction accuracy is performed for various spatial resolutions (10, 20, 30, 40 mm) and steps (1, 5, 10, 15, 20 mm) of the BOCDA system. Our results show that the maximum measurable curvature of three-dimensional bending is 143 m <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-1</sup> , which is the highest curvature so far measured by Brillouin sensors in three-dimensional space, with about 0.7% of tip position error. Additionally, both pseudo-discrete sensing and fully distributed sensing are performed together to analyze the relation between the methods of sensing (discrete or distributed) and the accuracy of probing multi-bend curvatures according to the measurement step.

Video-based augmented reality combining CT-scan and instrument position data to microscope view in middle ear surgery

The aim of the study was to develop and assess the performance of a video-based augmented reality system, combining preoperative computed tomography (CT) and real-time microscopic video, as the first crucial step to keyhole middle ear procedures through a tympanic membrane puncture. Six different artificial human temporal bones were included in this prospective study. Six stainless steel fiducial markers were glued on the periphery of the eardrum, and a high-resolution CT-scan of the temporal bone was obtained. Virtual endoscopy of the middle ear based on this CT-scan was conducted on Osirix software. Virtual endoscopy image was registered to the microscope-based video of the intact tympanic membrane based on fiducial markers and a homography transformation was applied during microscope movements. These movements were tracked using Speeded-Up Robust Features (SURF) method. Simultaneously, a micro-surgical instrument was identified and tracked using a Kalman filter. The 3D position of the instrument was extracted by solving a three-point perspective framework. For evaluation, the instrument was introduced through the tympanic membrane and ink droplets were injected on three middle ear structures. An average initial registration accuracy of 0.21 ± 0.10 mm (n = 3) was achieved with a slow propagation error during tracking (0.04 ± 0.07 mm). The estimated surgical instrument tip position error was 0.33 ± 0.22 mm. The target structures’ localization accuracy was 0.52 ± 0.15 mm. The submillimetric accuracy of our system without tracker is compatible with ear surgery.

Modeling of a joint-type flexible endoscope based on elastic deformation and internal friction

Flexible endoscopes are widely used in minimally invasive surgical robot systems. Various kinematic models have been developed for describing the deformation of such endoscopes. For joint-type flexible endoscopes, most existing models neglect the effect of internal friction and cannot precisely show the shape.In this paper, we propose a new nonlinear bending model. The rubber tube and metal net at each joint are approximated as a tube under elastic deformation and are assigned an equivalent bending stiffness. The internal friction force is also taken into account to build the moment balance equation at each joint. Groups of experiments were performed to validate the nonlinear model. The results closely confirm the model’s predictions. The model’s tip position error during the bending and unbending phases are 1.48 ± 0.99 mm and 1.68 ± 0.91 mm respectively; the bending angle errors are −5.50 ± 2.54° and 1.68 ± 3.66°, respectively The model can also take account of the hysteresis effect of the bending, which is quite common for cable-driven flexible robots. Moreover, the model has good computational efficiency, making it suitable for real-time control.

Modeling Tube Clearance and Bounding the Effect of Friction in Concentric Tube Robot Kinematics.

The shape of a concentric tube robot depends not only on the relative rotations and translations of its constituent tubes, but also on the history of relative tube displacements. Existing mechanics-based models neglect all history-dependent phenomena with the result that when calibrated on experimental data collected over a robot's workspace, the maximum tip position error can exceed 8 mm for a 200-mm-long robot. In this paper, we develop a model that computes the bounding kinematic solutions in which Coulomb friction is acting either to maximize or minimize the relative twisting between each pair of contacting tubes. The path histories associated with these limiting cases correspond to first performing all tube translations and then performing relative tube rotations of sufficient angle so that the maximum Coulomb friction force is obtained along the interface of each contacting tube pair. The robot tip configurations produced by these path histories are shown experimentally to bound position error with respect to the estimated frictionless model compared to path histories comprised of translation or mixed translation and rotation. Intertube friction forces and torques are computed as proportional to the intertube contact forces. To compute these contact forces, the standard zero-clearance assumption that constrains the concentrically combined tubes to possess the same centerline is relaxed. The effects of clearance and friction are explored through numerical and physical experiments and it is shown that friction can explain much of the prediction error observed in existing models. This model is not intended for real-time control, but rather for path planning-to provide error bounds and to inform how the ordering of tube rotations and translations can be used to reduce the effect of friction.

A feedrate scheduling algorithm to constrain tool tip position and tool orientation errors of five-axis CNC machining under cutting load disturbances

This paper proposes a model to keep the cutting load induced tool tip position and tool orientation errors of five-axis CNC machines within the desired tolerance limit by scheduling the feed along the tool paths. The cutting forces at the tool tip are transmitted to three translational and two rotary servo drives of the machine tool using its kinematic model. Tracking errors of each axis contributed by cutting forces are modeled using the disturbance transfer function of servo drives. The tracking errors are mapped to the workpiece coordinate system to evaluate the tool tip position and tool axis orientation errors at discrete tool path locations. The cutting forces, therefore the tracking errors, are modeled as a linear function of tangential feed velocity. The feed is adjusted to constrain the tool tip position and tool axis orientation errors at the desired tolerance levels along the tool path. The proposed algorithm has been experimentally validated by milling a curved tool path on a five-axis commercial CNC machine tool. It is shown that the proposed algorithm can significantly improve the part machining accuracy without increasing the machining time.