Real-time Visual Guidance Research Articles

A Vision-Guided Robotic System for Safe Dental Implant Surgery

Background: Recent advancements in dental implantology have significantly improved outcomes, with success rates of 90–95% over a 10-year period. Key improvements include enhanced preplanning processes, such as precise implant positioning, model selection, and optimal insertion depth. However, challenges remain, particularly in achieving correct spatial positioning and alignment of implants for optimal occlusion. These challenges are pronounced in patients with reduced bone substance or complex anthropometric features, where even minor misalignments can result in complications or defects. Methods: This paper introduces a vision-guided robotic system designed to improve spatial positioning accuracy during dental implant surgery. The system incorporates advanced force-feedback control to regulate the pressure applied to bone, minimizing the risk of bone damage. A preoperative CBCT scan, combined with real-time images from a robot-mounted camera, guides implant positioning. A personalized marker holder guide, developed from the initial CBCT scan, is used for patient–robot calibration. The robot-mounted camera provides continuous visual feedback of the oral cavity during surgery, enabling precise registration of the patient with the robotic system. Results: Initial experiments were conducted on a 3D-printed mandible using a personalized marker holder. Following successful patient–robot registration, the robotic system autonomously performed implant drilling. To evaluate the accuracy of the robotic-assisted procedure, further tests were conducted on 40 identical molds, followed by measurements of implant positioning. The results demonstrated improved positioning accuracy compared to the manual procedure. Conclusions: The vision-guided robotic system significantly enhances the spatial accuracy of dental implants compared to traditional manual methods. By integrating advanced force-feedback control and real-time visual guidance, the system addresses key challenges in implant positioning, particularly for patients with complex anatomical structures. These findings suggest that robotic-assisted implant surgery could offer a safer and more precise alternative to manual procedures, reducing the risk of implant misalignment and associated complications.

The minimally invasive sacrospinous fixation under visual guidance: An anatomical study

ObjectiveSacrospinous fixation is the gold standard procedure for management of apical pelvic organ prolapse by the vaginal route. However, there may be a relevant risk of neurovascular injury due to the proximity of neurovascular structures. We propose an anatomical study concerning the sacrospinous ligament with a new innovative minimally invasive technology using both a suture capturing device and a chip-on-the-tip endoscope to perform sacropinous fixation.Study design: Bilateral sacrospinous fixation was performed in three female cadavers, in the course of the anatomical study conducted with a specific device (the Suture Capturing I Stitch™ Device) under real time visual guidance with a chip-on -the-tip endoscope, the NanoScope™ system. ResultsIdentification of ischial spine and sacrospinous ligament as well as feasibility of sacrospinous fixation under NanoScope™ control were always possible on both sides. ConclusionsThis new innovative minimally invasive technology using both a suture capturing device and a chip-on-the-tip endoscope is relevant and could be an advantage in terms of safety and better placement of the suture on the sacrospinous ligament.

Visual-triggered contextual guidance for lithium battery disassembly: a multi-modal event knowledge graph approach

Lithium battery disassembly is a complex task that presents numerous challenges, including the wide variety of battery types, intricate manufacturing processes, and the absence of standardised procedures. These challenges pose significant obstacles in achieving efficient and accurate disassembly operations. Consequently, there is a clear need for guidance and support throughout the disassembly process. Conventional strategies for providing disassembly guidance often struggle to adapt to the dynamic changes that occur during the process. Therefore, it is essential to develop an approach that offers flexibility, efficiency, and adaptability to real-time visual cues in order to effectively address these challenges. This paper introduces a novel real-time visual guidance scheme that utilises a sophisticated multi-modal event knowledge graph. By leveraging outputs from computer vision models, the proposed scheme monitors the disassembly process in real-time and provides visually triggered, context-sensitive guidance through the multi-modal event knowledge graph. Additionally, this paper presents an auxiliary training method for visual state detection, guided by the large-scale visual model known as the Segment Anything Model (SAM), which helps mitigate the costs associated with data annotation during model development. The efficacy of the proposed framework is validated through experimental evaluations, demonstrating its potential in enhancing disassembly efficiency.

Usability, acceptability and feasibility of a novel technology with visual guidance with video and audio recording during newborn resuscitation: a pilot study

ObjectiveInadequate adherence to resuscitation for non-crying infants will have poor outcome and thus rationalise a need for real-time guidance and quality improvement technology. This study assessed the usability, feasibility and...

State of the art: percutaneous tracheostomy in the intensive care unit.

Percutaneous tracheostomy is a commonly performed procedure for patients in the intensive care unit (ICU) and offers many benefits, including decreasing ICU length of stay and need for sedation while improving patient comfort, effective communication, and airway clearance. However, there is no consensus on the optimal timing of tracheostomy in ICU patients. Ultrasound (US) and bronchoscopy are useful adjunct tools to optimize procedural performance. US can be used pre-procedurally to identify vascular structures and to select the optimal puncture site, intra-procedurally to assist with accurate placement of the introducer needle, and post-procedurally to evaluate for a pneumothorax. Bronchoscopy provides real-time visual guidance from within the tracheal lumen and can reduce complications, such as paratracheal puncture and injury to the posterior tracheal wall. A step-by-step detailed procedural guide, including preparation and procedural technique, is provided with a team-based approach. Technical aspects, such as recommended equipment and selection of appropriate tracheostomy tube type and size, are discussed. Certain procedural considerations to minimize the risk of complications should be given in circumstances of patient obesity, coagulopathy, or neurologic illness. Herein, we provide a practical state of the art review of percutaneous tracheostomy in ICU patients. Specifically, we will address pre-procedural preparation, procedural technique, and post-tracheostomy management.

A New Visual Transient Elastography Technique for Grading Liver Fibrosis in Patients With Chronic Hepatitis B.

Liver fibrosis is evaluated to assess the prognosis and guide the treatment of chronic hepatitis B (CHB). To compare the efficiency of 2 transient elastography techniques for grading liver fibrosis in CHB: visual transient elastography (ViTE) with real-time image guidance and FibroScan (FS) with no image guidance. All of the CHB patients in this study underwent both FS and ViTE examinations. The final diagnosis was based on the histological findings of a liver biopsy. According to the severity of liver fibrosis (based on the Scheuer criteria), the area under the receiver operating characteristic curve values for diagnostic efficiency were calculated for the 2 elastography techniques. This study enrolled 227 patients (79 [39.1%] women; mean age, 45.8 ± 16.8 years). The ViTE and FS liver elasticity measurements were highly correlated with liver fibrosis stage (r = 0.852 and r = 0.813, respectively). The area under the receiver operating characteristic curve value was larger for ViTE compared with FS, with respect to differentiating liver fibrosis stage, but not significantly (P > 0.05). The ViTE and FS can be used to detect and stage liver fibrosis. ViTE, easier and quicker to perform with superior interoperator reproducibility, is a stable and reliable elastography technique that benefits from real-time visual guidance.

Microsphere-Based Super-Resolution Imaging for Visualized Nanomanipulation.

Nanomanipulation provides high operating accuracy and has been successfully applied in many fields such as nanoparticle assembly, nanowire alignment, and semiconductor device manufacturing. However, because of the limits of optical diffraction, the use of nanomanipulation is challenged by a lack of visual feedback at the nanoscale, and thus, its efficiency is difficult to be improved. In this study, we developed a novel method of microlens-enhanced nanomanipulation capable of real-time super-resolution imaging. Nanomanipulation was performed using the atomic force microscopy (AFM) mechanism by coupling a microlens to an AFM probe, and optical imaging with a minimum characteristic size of 80 nm is realized by combining the microlens with the optical imaging system. Under the conditions of fluorescent illumination and white light illumination, nanomanipulations were achieved under real-time visual guidance for fluorescent nanoparticles with a diameter of 100 nm and silver nanowires with a diameter of 80 nm, respectively. This method enables the possibility of in situ observation and manipulation, which can potentially be used for biological samples.

Real-Time Thermoacoustic Imaging-Guidance for Breast Tumor Resection

In situ image guidance is one of the key techniques for accurate diagnosis and treatment in clinical medicine. Conventional imaging techniques such as CT, MRI provide critical information for diagnosis and treatment planning, but are less practical due to either technical complexity or ionizing radiation-related safety. Microwave-pumped thermoacoustic imaging (MTAI) is a promising alternative to real-time bioimaging due to its deep imaging depth, high resolution and minimal biological hazards. More importantly, by mapping the distribution of microwave absorbers, this method can also provide a new perspective for noninvasively observing the anatomy and functional structure of the target. In this work, we introduce a concept of in situ MTAI guidance by combining MTAI and an accurate spatial definition of invasive apparatus used in potential procedures. By mounting tiny MTAI beacons on the apparatus, it is possible to accurately reconstruct the relative position of the device to the target tissue for either real time visual guidance or as the foundation for potential automated procedures. The image quality including its applicable depth and resolution, the accuracy of the device localization is analyzed and tested with simulated breast tumor to demonstrate the capacity of the proposed technique. With refined adaptation for potential applications, this technique may provide a novel approach for a broad range of medical and biological applications.

Neuroblastoma-targeting triangular gadolinium oxide nanoplates for precise excision of cancer

Neuroblastoma accounts for 8–10% of malignancies in infants and children. It is urgent to develop an appropriate theranostic agent for effective diagnosis and therapy of neuroblastoma. Herein, we constructed RVG peptide and IRDye800-conjugated bovine serum albumin-coated triangular gadolinium oxide nanoplates (RVG&IRDye800-Gd2O3 TNs) as a targeting MRI agent for the diagnosis of neuroblastoma preoperation and a fluorescence imaging agent for the guidance of the precise excision of the neuroblastoma in surgery. RVG&IRDye800-Gd2O3 TNs have uniform edge length. The RVG&IRDye800-Gd2O3 TNs show remarkably enhanced affinity to both mouse- and human-derived neuroblastoma cells compared with IRDye800-Gd2O3 TNs (3.07-fold and 3.02-fold, respectively). Because of the increased accumulation in tumor cells, RVG&IRDye800-Gd2O3 TNs exhibit signals threefold to fivefold higher than the surrounding normal tissues, which is propitious to the diagnosis of neuroblastoma preoperation and provides real-time visual guidance of the precise excision of the neuroblastoma. Most importantly, with the guidance of the fluorescence imaging agent, the survival rate increased from 0% to 80% 42 days after surgery compared with that in conventional surgery. These findings indicated that the RVG peptide combined with IRDye800-Gd2O3 TNs has the potential to improve the diagnosis and treatment of patients with neuroblastoma. Statement of significanceIn this study, we prepared RVG peptide and IRDye800-conjugated bovine serum albumin-coated triangular gadolinium oxide nanoplates (RVG&IRDye800-Gd2O3 TNs) as a targeting MRI agent for the diagnosis of neuroblastoma preoperation and a fluorescence imaging agent for the guidance of the precise excision of the neuroblastoma during surgery.Neuroblastoma was accurately located by MRI imaging, and the tumor margin could be real-time monitored through near-infrared fluorescence imaging.The RVG&IRDye800-Gd2O3 TNs exhibit signals threefold to fivefold higher than those in the surrounding normal tissues, which is propitious to the diagnosis of the neuroblastoma preoperation and provides real-time visual guidance of the precise excision of the neuroblastoma.With the guidance of the fluorescence imaging agent in surgery, the survival rate increased from 0% to 80% 42 days after surgery compared with that in conventional surgery.

Anti-EGFR Peptide-Conjugated Triangular Gold Nanoplates for Computed Tomography/Photoacoustic Imaging-Guided Photothermal Therapy of Non-Small Cell Lung Cancer.

Non-small cell lung cancer (NSCLC) is difficult to cure because of the high recurrence rate and the side effects of current treatments. It is urgent to develop a new treatment that is safer and more effective than current treatments against NSCLC. Herein, we constructed anti-epidermal growth factor receptor (EGFR) peptide-conjugated PEGylated triangular gold nanoplates (TGN-PEG-P75) as a targeting photothermal therapy (PTT) agent to treat NSCLC under the guidance of computed tomography (CT) and photoacoustic (PA) imaging. The surface of TGNs is successfully conjugated with a novel peptide P75 that has the specific affinity to epidermal growth factor receptor (EGFR). It is found that the EGFR is overexpressed in NSCLC cells. The TGN-PEG-P75 has uniform edge length (77.9 ± 7.0 nm) and neutrally charged surface. The cell uptake experiments demonstrate remarkable affinity of the TGN-PEG-P75 to high EGFR expression cells than low EGFR expression cells (5.1-fold). Thanks to the strong near-infrared absorbance, high photothermal conversion efficiency, and the increased accumulation in tumor cells via the interaction of P75 and EGFR, TGN-PEG-P75 exhibits 3.8-fold superior therapeutic efficacy on HCC827 cells than TGN-PEG. The in vivo CT/PA dual-modal imaging of the TGN-PEG-P75 is helpful in selecting the optimal treatment time and providing real-time visual guidance of PTT. Furthermore, treatments on HCC827 tumor-bearing mouse model demonstrate that the growth of NSCLC cells can be effectively inhibited by the TGN-PEG-P75 through PTT, indicating the great promise of the nanoplatform for treating NSCLC in vivo.

Technical note: real-time web-based wireless visual guidance system for radiotherapy.

Describe a Web-based wireless visual guidance system that mitigates issues associated with hard-wired audio-visual aided patient interactive motion management systems that are cumbersome to use in routine clinical practice. Web-based wireless visual display duplicates an existing visual display of a respiratory-motion management system for visual guidance. The visual display of the existing system is sent to legacy Web clients over a private wireless network, thereby allowing a wireless setting for real-time visual guidance. In this study, active breathing coordinator (ABC) trace was used as an input for visual display, which captured and transmitted to Web clients. Virtual reality goggles require two (left and right eye view) images for visual display. We investigated the performance of Web-based wireless visual guidance by quantifying (1) the network latency of visual displays between an ABC computer display and Web clients of a laptop, an iPad mini 2 and an iPhone 6, and (2) the frame rate of visual display on the Web clients in frames per second (fps). The network latency of visual display between the ABC computer and Web clients was about 100ms and the frame rate was 14.0fps (laptop), 9.2fps (iPad mini 2) and 11.2fps (iPhone 6). In addition, visual display for virtual reality goggles was successfully shown on the iPhone 6 with 100ms and 11.2fps. A high network security was maintained by utilizing the private network configuration. This study demonstrated that a Web-based wireless visual guidance can be a promising technique for clinical motion management systems, which require real-time visual display of their outputs. Based on the results of this study, our approach has the potential to reduce clutter associated with wired-systems, reduce space requirements, and extend the use of medical devices from static usage to interactive and dynamic usage in a radiotherapy treatment vault.

Intraoperative Real-Time Localization of Normal Parathyroid Glands With Autofluorescence Imaging.

This biomedical investigation is valuable for identification and localization of parathyroid glands during thyroidectomy, which can provide an intraoperative real-time visual guidance. The objective of the study was to investigate the feasibility of real-time autofluorescence imaging of the parathyroid glands without exogenous contrast dye for their localization and demonstration of relation to the background tissues. This research was undertaken at Kosin University Gospel Hospital. Sixteen normal parathyroid glands from eight patients with papillary thyroid carcinoma were enrolled. Photo images of the surgical field including the parathyroid and background tissues were taken with a digital camera, 780 nm light-emitting diode to excite the parathyroid, and infrared illuminator to visualize the entire neck. The area-averaged autofluorescence intensity of parathyroid over the area-averaged fluorescence intensity of background tissues was measured. The location of the parathyroid gland was verified with a single image. The area-averaged autofluorescence intensity of parathyroid over the area-averaged fluorescence intensity of background tissues for all parathyroid glands was higher than 1, with a minimum of 1.95 and a maximum of 5.20 (average 2.76, SD 0.79). By our technique, all 16 parathyroid glands were detected (positive predictive value of 100%), and the entire surgical field including the parathyroid and background tissues was visualized as well. The parathyroid glands that were exposed or even covered by connective tissues or blood vessels could be detected with strong emission. This method showed the precise localization of the parathyroid glands and demonstrated their relation to background tissue. We believe that this simple, nonexogenous dye technique of anatomical guidance can aid surgeons to preserve parathyroid glands during thyroidectomy.

Hardware Implementation of a Spline-Based Genetic Algorithm for Embedded Stereo Vision Sensor Providing Real-Time Visual Guidance to the Visually Impaired

Many image and signal processing techniques have been applied to medical and health care applications in recent years. In this paper, we present a robust signal processing approach that can be used to solve the correspondence problem for an embedded stereo vision sensor to provide real-time visual guidance to the visually impaired. This approach is based on our new one-dimensional (1D) spline-based genetic algorithm to match signals. The algorithm processes image data lines as 1D signals to generate a dense disparity map, from which 3D information can be extracted. With recent advances in electronics technology, this 1D signal matching technique can be implemented and executed in parallel in hardware such as field-programmable gate arrays (FPGAs) to provide real-time feedback about the environment to the user. In order to complement (not replace) traditional aids for the visually impaired such as canes and Eyes dogs, vision systems that provide guidance to the visually impaired must be affordable, easy to use, compact, and free from attributes that are awkward or embarrassing to the user. Seeing Eye Glasses, an embedded stereo vision system utilizing our new algorithm, meets all these requirements.

Fluoroscopic-Based Navigation–Assisted Placement of the Tibial Tunnel in Revision Anterior Cruciate Ligament Reconstruction

With the increasing number of primary anterior cruciate ligament (ACL) reconstructions, revisions are more frequent. The most common cause of technical failure in primary ACL reconstruction is the bone tunnel misplacement. An incorrectly positioned bone tunnel or tunnel enlargement interferes with correct tunnel placement, which is critical for successful restoration of the knee kinematics. We describe a technique using fluoroscopic-based navigation systems to place the tibial tunnel in an ideal position with accuracy and reproducibility in the revision ACL reconstruction. With a reference frame attached to the middle part of the tibia, the conventional tibial guide with a tracking device provides surgeons with real-time visual guidance in multiple image planes. In 3 revision cases with incorrect tibial tunnels and without bone enlargement, the new tibial tunnel could be created in the ideal position under the control of the navigation system. Computer navigation and virtual ligament reconstruction enable surgeons to control bone tunnel positioning during technically demanding revision ACL cases.

Introducing navigated transcranial magnetic stimulation as a refined brain mapping methodology.

A major intrinsic limitation of transcranial magnetic stimulation (TMS) to map the human brain lies in the unclear relationship between the position of the stimulating coil on the scalp and the underlying stimulated cortex. The relationship between structure and function as the major feature constituting a brain mapping modality can therefore not be established. Recent advances in image processing allowed us to refine TMS by combining magnetic resonance imaging (MRI) modalities with TMS using a neuronavigation system to measure the position of the stimulating coil and map this position onto a MRI data set. This technique has several advantages over recent TMS mapping strategies. The position of the coil on the scalp can be held constant as verified by real time visual guidance. When evaluating higher cortical functions, the relationship between underlying cortical anatomy and the scalp stimulation site can be accurately assessed. Cortical motor output maps can be easily obtained for preoperative planning and decision making for mass lesions near rolandic cortex in patients. In conclusion, navigated TMS is a reliable alternative for localizing cortical functions and therefore may be a useful adjunct or in selected patients even a helpful alternative to other functional neuroimaging methods.

Multimodality neuroimaging: research and clinical applications.

This manuscript reviews the current stance and the pertinent problems of transcranial magnetic stimulation (TMS) and functional magnetic resonance imaging (fMRI) as brain mapping methodologies. The missing structure-function relation limits the use of TMS, whereas the uncertainty about the functional significance of activated cortical regions might render interpretation of fMRI studies difficult. Advances in image processing, however, allowed for 3- dimensional real-time visual guidance of TMS and integration with fMRI data. We describe the method used to coregister TMS and fMRI and present examples where a multimodality neuroimaging approach might add to our understanding of normal and pathological brain function.