Image-guided Interventions Research Articles

Magnetic tracking using a modular C++ environment for image-guided interventions

ABSTRACT Magnetic tracking enables instrument tracking for image-guided interventions when no line of sight is available. This paper describes the first steps towards a more cost-effective, modular, and adaptable approach that builds upon prior work in open hardware architectures for magnetic tracking in image-guided interventions. An exemplary C++ framework is implemented and demonstrated with the open-hardware Anser EMT system. System performance in speed, accuracy, and precision of the C++ implementation is analysed. Static positioning accuracy and precision are calculated within the Region of Interest (ROI) and an average position error of 1.0 0.1 mm is demonstrated. Results show an indicative increase in the update rate using the C++ framework and substantially lower memory requirements, compared to the previously optimised Python and Matlab solvers. These preliminary results provide the basis for future development which will integrate the C++ framework in a 3D Slicer module, greatly extending the adaptability of the platform for customisation in advanced image-guided procedures.

In Vivo Micro-Computerized Tomography Tracking of Human Periodontal Ligament Stem Cells Labeled with Gold Nanocomplexes.

Gold nanocomplexes have been proposed as contrast agents for computerized tomography (CT) and cell tracking, which is especially useful in stem cell therapy. However, their potential for long-term in vivo cell detection is still unknown. This study proposes an optimized approach to labeling human periodontal ligament stem cells (hPDLSCs) with gold nanocomplexes to evaluate their detection with micro-CT after transplantation at four different rat tissues. The gold nanocomplexes of 0.05mgmL-1 do not affect cell viability nor osteogenic differentiation capacity, but render fluorescent and radiopaque hPDLSCs. Excellent linear correlation with the number of labeled cells is shown over a wide range (r=0.99, P<0.01), with a detection limit of ≈1.2×103 cells/µL. In vivo, strong, and durable detection of transplanted labeled cells within 5days at all investigated areas is seen by micro-CT and immunohistochemical assay. This approach confirms the potential of gold nanocomplexes in longitudinal in vivo cell tracking, which may facilitate their application in CT image-guided interventions commonly used in oromaxillofacial or systemic applications of stem cell therapy.

Minimally invasive magnetic resonance image-guided prostate interventions.

Whole gland prostate cancer treatment, i.e. radical prostatectomy or radiation therapy, is highly effective but also comes with a significant impact on quality of life and possible overtreatment in males with low to intermediate risk disease. Minimal-invasive treatment strategies are emerging techniques. Different sources of energy are used to aim for targeted treatment in order to reduce treatment-related complications and morbidity. Imaging plays an important role in targeting and monitoring of treatment approaches preserving parts of the prostatic tissue. Multiparametric magnetic resonance imaging (mpMRI) is widely used during image-guided interventions due to the multiplanar and real-time anatomical imaging while providing an improved treatment accuracy. This review evaluates the available image-guided prostate cancer treatment options using MRI or magnetic resonance imaging/transrectal ultrasound (MRI/TRUS)-fusion guided imaging. The discussed minimal invasive image-guided prostate interventions may be considered as safe and feasible partial gland ablation in patients with (recurrent) prostate cancer. However, most studies focusing on minimally invasive prostate cancer treatments only report early stages of research and subsequent high-level evidence is still needed. Ensuring a safe and appropriate utilization in patients that will benefit the most, and applied by physicians with relevant training, has become the main challenge in minimally invasive prostate cancer treatments.

Deep Interactive Denoiser (DID) for X-Ray Computed Tomography

Low-dose computed tomography (LDCT) is desirable for both diagnostic imaging and image-guided interventions. Denoisers are widely used to improve the quality of LDCT. Deep learning (DL)-based denoisers have shown state-of-the-art performance and are becoming mainstream methods. However, there are two challenges to using DL-based denoisers: 1) a trained model typically does not generate different image candidates with different noise-resolution tradeoffs, which are sometimes needed for different clinical tasks; and 2) the model's generalizability might be an issue when the noise level in the testing images differs from that in the training dataset. To address these two challenges, in this work, we introduce a lightweight optimization process that can run on top of any existing DL-based denoiser during the testing phase to generate multiple image candidates with different noise-resolution tradeoffs suitable for different clinical tasks in real time. Consequently, our method allows users to interact with the denoiser to efficiently review various image candidates and quickly pick the desired one; thus, we termed this method deep interactive denoiser (DID). Experimental results demonstrated that DID can deliver multiple image candidates with different noise-resolution tradeoffs and shows great generalizability across various network architectures, as well as training and testing datasets with various noise levels.

X-ray-induced acoustic computed tomography (XACT) imaging with single-shot nanosecond x-ray.

X-ray-induced acoustic computed tomography (XACT) has emerged as a promising imaging modality with broad applications in both biomedicine and nondestructive testing. The previous XACT imaging systems require thousands of averages to achieve reasonable images. Here, we report the experimental demonstration of single-shot XACT imaging of a metal object using a single-shot 50 ns x-ray pulse. A two-stage dedicated amplification and a 128-channel parallel data acquisition configuration were introduced for XACT imaging to enable sufficient acoustic signal amplification and maintain an overall low noise level for single-shot XACT imaging. Details of the system design are presented; the improved signal-to-noise ratio (>23 dB) and image reconstruction have been demonstrated with a ring ultrasound transducer array imaging system. The study paves the way for realizing real-time XACT imaging and its potential applications in image-guided intervention.

Integrating Real-time Video View with Pre-operative Models for Image-guided Renal Navigation: An in vitro Evaluation Study.

To provide a complete picture of a scene sufficient to conduct a minimally invasive, image-guided renal intervention, real-time laparoscopic video needs to be integrated with underlying anatomy information typically available from pre- or intra-operative images. Here we present a simple and efficient hand-eye calibration method for an optically tracked camera, which only requires the acquisition of several poses of a Polaris stylus featuring 4 markers automatically localized by both the camera and the optical tracker. We evaluate the calibration using both the Polaris stylus, as well as a patient-specific 3D printed kidney phantom in terms of the number of poses acquired, as well as the depth of the imaged scene into the field of view of the camera, by projecting the several landmarks on the imaged object at known location in the 3D world onto the camera image. The RMS projection error decreases with increasing distance from the camera to the imaged object from 7 pixels at 15-18 mm, to under 2 pixels at 28-30 mm, which corresponds to a 2 mm and 1 mm error, respectively, in 3D space.

Novel-View X-Ray Projection Synthesis Through Geometry-Integrated Deep Learning

X-ray imaging is a widely used approach to view the internal structure of a subject for clinical diagnosis, image-guided interventions and decision-making. The X-ray projections acquired at different view angles provide complementary information of patient's anatomy and are required for stereoscopic or volumetric imaging of the subject. In reality, obtaining multiple-view projections inevitably increases radiation dose and complicates clinical workflow. Here we investigate a strategy of obtaining the X-ray projection image at a novel view angle from a given projection image at a specific view angle to alleviate the need for actual projection measurement. Specifically, we propose a deep learning-based geometry-integrated projection synthesis framework for generating novel-view X-ray projections.We for the first time investigate the novel-view projection synthesis problem for X-ray imaging. We propose a deep learning-based geometry-integrated projection synthesis model to generate novel-view X-ray projections through feature disentanglement and geometry transformation. The proposed deep learning model extracts geometry and texture features from a source-view projection, and then conducts geometry transformation on the geometry features to accommodate the change of view angle. Then the X-ray projection in target view is synthesized from the transformed geometry and shared texture features via an image generator. We validate the proposed approach by experimenting on one-to-one and multi-to-multi X-ray projection synthesis using lung imaging cases across various patients. We conduct experiments on a public dataset containing 1018 lung CT images, where 80% and 20% data are used for training and testing. The projection images are digitally produced from CT images using geometry consistent with a clinical on-board cone-beam CT system for radiation therapy.We deploy the trained model on the held-out testing set for novel-view projection synthesis. The generated results are compared with ground truth qualitatively and quantitatively. For AP-to-lateral and lateral-to-AP view synthesis, average NRMSE / SSIM / PSNR values over all testing data are 0.185 / 0.880 / 22.646 and 0.173 / 0.911 / 23.908, respectively. Promising results are also obtained for multi-to-multi view synthesis. By visualizing the generated projections, we observe the proposed model can generate images closely to the targets despite the various anatomic structures of different patients, indicating the potential of the proposed model for X-ray projection synthesis.This work investigates the synthesis of novel-view X-ray projections and presents a robust model combining the deep learning and geometry transformation. It is shown that the generated X-ray projections reveal the internal anatomy distribution from new viewpoints, which potentially provides a new paradigm for volumetric imaging with substantially reduced efforts in data acquisition.

Enabling Few-View 3D Tomographic Image Reconstruction by Geometry-Informed Deep Learning

<h3>Purpose/Objective(s)</h3> Deep learning affords enormous opportunities to augment the armamentarium of biomedical imaging, albeit its design and implementation have potential flaws. Fundamentally, most deep learning models are driven entirely by data without consideration of any physical priors, which dramatically increases the complexity of neural networks and limits the application scope and generalizability of the resultant models. Here we establish a geometry-informed deep learning framework for ultra-sparse tomographic image reconstruction. We introduce a novel mechanism for the integration of geometric priors of the imaging system. We demonstrate that the seamless inclusion of known geometric priors is essential to enhance the performance of volumetric computed tomography imaging with ultra-sparse sampling. <h3>Materials/Methods</h3> We propose a geometry-informed deep learning framework for 3D tomographic image reconstruction and illustrate the way that the known imaging geometry is integrated into the dual-domain deep learning. Specifically, the proposed framework consists of three modules: a) 2D projection generation network is developed to learn to generate novel-view projections from the given sparse views; b) geometric back-projection operator transforms the 2D projections to 3D images, referred to as geometry preserving images (GPI), which geometrically relates the pixelated 2D input data to the corresponding ray lines in 3D space; and c) 3D image refinement network learns to refine the GPI to generate the final 3D images. To evaluate feasibility of the proposed approach, we conduct experiments on a dataset containing 1018 lung CT images, where 80% and 20% data are used for training and testing. The projection images are digitally produced from CT images using geometry consistent with a clinical on-board cone-beam CT system for radiation therapy. <h3>Results</h3> We deploy the trained model on the held-out testing set for few-view 3D image reconstruction. The reconstructed results are compared with ground truth qualitatively and quantitatively. For single-/two-/three-view reconstruction, the average NRMSE / SSIM / PSNR values over all testing data are 0.368 / 0.734 / 20.770, 0.300 / 0.807 / 22.687 and 0.274 / 0.838 / 23.669, respectively. By visualizing the reconstructed CT images, we observe the proposed model can generate images closely to the targets although the anatomic structures of different patients have a large variance, indicating the potential of the proposed model for volumetric imaging even with few views. Moreover, experiments also show the proposed model can also be generalized to multi-view 3D image reconstruction and outperform deep models without geometry priors. <h3>Conclusion</h3> We present a novel geometry-integrated deep learning model for volumetric imaging with ultra-sparse sampling. The study opens new avenues for data-driven biomedical imaging and promises to provide substantially improved imaging tools for various clinical imaging and image-guided interventions.

Factors Influencing Stent Failure in Chronic Total Occlusion Coronary Intervention.

Stent failure remains one of the greatest challenges for interventional cardiologists. Despite the evolution to superior second- and third-generation drug-eluting stent designs, increasing use of intracoronary imaging and the adoption of more potent antiplatelet regimens, registries continue to demonstrate a prevalence of stent failure or target lesion revascularisation of 15–20%. Predisposition to stent failure is consistent across both chronic total occlusion (CTO) and non-CTO populations and includes patient-, lesion- and procedure-related factors. However, histological and pathophysiological properties specific to CTOs, alongside complex strategies to treat these lesions, may potentially render percutaneous coronary interventions in this cohort more vulnerable to failure. Prevention requires recognition and mitigation of the precipitants of stent failure, optimisation of interventional techniques, including image-guided precision percutaneous coronary intervention, and aggressive modification of a patient’s cardiovascular risk factors. Management of stent failure in the CTO population is technically challenging and itself begets recurrence. We aim to provide a comprehensive review of factors influencing stent failure in the CTO population and strategies to attenuate these.

Occupational radiation exposure in doctors: an analysis of exposure rates over 25 years.

Healthcare professionals' occupational exposure to ionising radiation may be increasing due to increasing use of imaging and image-guided intervention. This study aims to assess the occupational exposure of doctors over a 25-year period at an NHS teaching hospital. Dosemeter measurements were collected prospectively from 1995 to 2019. Two retrospective analyses were performed over time (first including all measurements, second excluding "zero-dose" measurements), and by speciality. Group comparisons were undertaken using multilevel linear regression; a p-value <0.05 was deemed significant. 8,892 measurements (3,983 body, 1,514 collar, 649 eye, 2,846 hand), of which 3,350 were non-zero measurements (1,541 body, 883 collar, 155 eye, 771 hand), were included. Whole dataset analysis found a significant decrease in exposure for radiologists and cardiologists, as measured by body, hand and collar dosemeters over the last 25 years (p < 0.01 for all). The non-zero readings reflect the whole cohort analysis except in the case of eye dosemeters, which showed a significant decrease in exposure for cardiologists (p < 0.01), but a significant increase for radiologists and surgeons/anaesthetists (p < 0.01 for both). Whilst ionising radiation remains an occupational risk for doctors, the overall decreasing trend in occupational exposure is reassuring. However, a significant rise in eye dose for radiologists, surgeons and anaesthetists is concerning, and close monitoring is required to prevent future issues. This paper is one of few evaluating the occupational radiation exposure to doctors over a 25-year period, showing that although most dosemeter measurements reflect decreasing exposure, the increase in eye exposure warrants caution.

Current updates in image-guided musculoskeletal interventions

Image-guided musculoskeletal interventions are frequently done in clinical practice. Even then, the literature regarding their effectiveness is relatively scarce. Image guidance adds value over the conventional landmark-based approach and should be preferred. We hereby try to list the commonly performed procedures along with the current practice guidelines regarding their clinical indications and periprocedural care.

Modality-agnostic self-supervised deep feature learning and fast instance optimisation for multimodal fusion in ultrasound-guided interventions

Background and Objective: Fast and robust alignment of pre-operative MRI planning scans to intra-operative ultrasound is an important aspect for automatically supporting image-guided interventions. Thus far, learning-based approaches have failed to tackle the intertwined objectives of fast inference computation time and robustness to unexpectedly large motion and misalignment. In this work, we propose a novel method that decouples deep feature learning and the computation of long ranging local displacement probability maps from fast and robust global transformation prediction. Methods: In our approach, we firstly train a convolutional neural network (CNN) to extract modality-agnostic features with sub-second computation times for both 3D volumes during inference. Using sparsity-based network weight pruning, the model complexity and computation times can be substantially reduced. Based on these features, a large discretized search range of 3D motion vectors is explored to compute a probabilistic displacement map for each control point. These 3D probability maps are employed in our newly proposed, computationally efficient, instance optimisation that robustly estimates the most likely globally linear transformation that best reflects the local displacement beliefs subject to outlier rejection. Results: Our experimental validation demonstrates state-of-the-art accuracy on the challenging CuRIOUS dataset with average target registration errors of 2.50 mm, model size of only 1.2 MByte and run times of approx. 3 seconds for a full 3D multimodal registration. Conclusion: We show that a significant improvement in accuracy and robustness can be gained with instance optimisation and our fast self-supervised deep learning model can achieve state-of-the-art accuracy on challenging registration task in only 3 seconds.

Bone Metastases: State of the Art in Minimally Invasive Interventional Oncology.

Bone is the third most common site involved by cancer metastases, and skeleton-related events such as intractable pain due to direct osseous tumor involvement, pathologic fracture, and neurologic deficits as a consequence of nerve or spinal cord compression often affect patients' functional independence and quality of life unfavorably. The annual medical-economic burden related to bone metastases is a substantial component of the total direct medical cost estimated by the National Institutes of Health. There have been substantial recent advances in percutaneous image-guided minimally invasive musculoskeletal oncologic interventions for the management of patients with osseous metastatic disease. These advances include thermal ablation, cementation with or without osseous reinforcement with implants, osteosynthesis, thermal and chemical neurolyses, and palliative injections, which are progressively incorporated into the management paradigm for such patients. These interventions are performed in conjunction with or are supplemented by adjuvant radiation therapy, systemic therapy, surgery, or analgesic agents to achieve durable pain palliation, local tumor control, or cure, and they provide a robust armamentarium for interventional radiologists to achieve safe and effective treatment in a multidisciplinary setting. In addition, these procedures are shifting the patient management paradigm in modern-era practice. The authors detail the state of the art in minimally invasive percutaneous image-guided musculoskeletal oncologic interventions and the role of radiologists in managing patients with skeletal metastases. ©RSNA, 2021.

The Impact of Machine Learning on 2D/3D Registration for Image-Guided Interventions: A Systematic Review and Perspective.

Image-based navigation is widely considered the next frontier of minimally invasive surgery. It is believed that image-based navigation will increase the access to reproducible, safe, and high-precision surgery as it may then be performed at acceptable costs and effort. This is because image-based techniques avoid the need of specialized equipment and seamlessly integrate with contemporary workflows. Furthermore, it is expected that image-based navigation techniques will play a major role in enabling mixed reality environments, as well as autonomous and robot-assisted workflows. A critical component of image guidance is 2D/3D registration, a technique to estimate the spatial relationships between 3D structures, e.g., preoperative volumetric imagery or models of surgical instruments, and 2D images thereof, such as intraoperative X-ray fluoroscopy or endoscopy. While image-based 2D/3D registration is a mature technique, its transition from the bench to the bedside has been restrained by well-known challenges, including brittleness with respect to optimization objective, hyperparameter selection, and initialization, difficulties in dealing with inconsistencies or multiple objects, and limited single-view performance. One reason these challenges persist today is that analytical solutions are likely inadequate considering the complexity, variability, and high-dimensionality of generic 2D/3D registration problems. The recent advent of machine learning-based approaches to imaging problems that, rather than specifying the desired functional mapping, approximate it using highly expressive parametric models holds promise for solving some of the notorious challenges in 2D/3D registration. In this manuscript, we review the impact of machine learning on 2D/3D registration to systematically summarize the recent advances made by introduction of this novel technology. Grounded in these insights, we then offer our perspective on the most pressing needs, significant open problems, and possible next steps.

Clinical indications for image-guided interventional procedures in the musculoskeletal system: a Delphi-based consensus paper from the European Society of Musculoskeletal Radiology (ESSR)\u2014part VI, foot and ankle

ObjectivesClarity regarding accuracy and effectiveness for interventional procedures around the foot and ankle is lacking. Consequently, a board of 53 members of the Ultrasound and Interventional Subcommittees of the European Society of Musculoskeletal Radiology (ESSR) reviewed the published literature to evaluate the evidence on image-guided musculoskeletal interventional procedures around this anatomical region.MethodsWe report the results of a Delphi-based consensus of 53 experts from the European Society of Musculoskeletal Radiology who reviewed the published literature for evidence on image-guided interventional procedures offered around foot and ankle in order to derive their clinical indications. Experts drafted a list of statements and graded them according to the Oxford Centre for evidence-based medicine levels of evidence. Consensus was considered strong when > 95% of experts agreed with the statement or broad when > 80% but < 95% agreed. The results of the Delphi-based consensus were used to write the paper that was shared with all panel members for final approval.ResultsA list of 16 evidence-based statements on clinical indications for image-guided musculoskeletal interventional procedures in the foot and ankle were drafted after a literature review. The highest level of evidence was reported for four statements, all receiving 100% agreement.ConclusionAccording to this consensus, image-guided interventions should not be considered a first-level approach for treating Achilles tendinopathy, while ultrasonography guidance is strongly recommended to improve the efficacy of interventional procedures for plantar fasciitis and Morton’s neuroma, particularly using platelet-rich plasma and corticosteroids, respectively.Key Points• The expert panel of the ESSR listed 16 evidence-based statements on clinical indications of image-guided musculoskeletal interventional procedures in the foot and ankle.• Strong consensus was obtained for all statements.• The highest level of evidence was reached by four statements concerning the effectiveness of US-guided injections of corticosteroid for Morton’s neuroma and PRP for plantar fasciitis.

Limited Chest Ultrasound to Replace CXR in Diagnosis of Pneumothorax Post Image-Guided Transthoracic Interventions.

To assess the diagnostic accuracy of limited chest ultrasound in detecting pneumothorax following percutaneous transthoracic needle interventions using chest X-ray (CXR) as the reference standard. With IRB approval, after providing consent, asymptomatic patients after percutaneous transthoracic needle interventions were enrolled to undergo limited chest ultrasound in addition to CXR. A chest Radiologist blinded to the patient's prior imaging performed a bedside ultrasound, scanning only the first 3 anterior intercostal spaces. Pneumothorax diagnosed on CXR was categorized as small or large and on ultrasound as grades 1, 2, or 3 when detected in 1, 2, or 3 intercostal spaces, respectively. 38 patients underwent 36 biopsies (34 lungs, 1 pleura, and 1 mediastinum) and 2 coil localizations. CXR showed pneumothorax in 13 patients. Ultrasound was positive in 10 patients, with 9 true-positives, 1 false-positive, 4 false-negatives, and 24 true-negatives. The false positive results were due to apical subpleural bullae. The false-negative results occurred in 2 small apical and 2 focal pneumothoraces at the needle entry sites. Four pneumothoraces were categorized as large on CXR, all of which were categorized as grade 3 on ultrasound. Sensitivity and specificity of US for detection of pneumothorax of any size were 69.23% (95%CI 38.6%, 90.1%) and 96.0% (95%CI 79.6%, 99.9%), and for detection of large pneumothorax were 100% (95%CI 39.8%, 100%) and 100% (95%CI 89.7%, 100%). Results of this prospective study is promising. Limited chest ultrasound could potentially replace CXR in the management of postpercutaneous transthoracic needle intervention patients.

FPSN-FNCC: an accurate and fast motion tracking algorithm in 3D ultrasound for image-guided interventions

The uncertain motions of a target caused by the breath, heartbeat and body drift of a patient can increase the target locating error during image-guided interventions, and that may cause additional surgery trauma. A surgery navigation system with accurate motion tracking is important for improving the operation accuracy and reducing trauma. In this work, we propose an accurate and fast tracking algorithm in three-dimensional (3D) ultrasound (US) sequences for US-guided surgery to achieve moving object tracking. The idea of this algorithm is as follows. Firstly, feature pyramid architecture is introduced into a Siamese network to extract multiscale convolutional features. Secondly, to improve the network discriminative power and the robustness to ultrasonic noise and gain variation, we use the normalized cross correlation (NCC) to calculate the similarity between template block and search block. Thirdly, a fast NCC (FNCC) is proposed, which can perform the real-time tracking. Finally, a density peaks clustering approach is used to compensate the motion of the target and further improve the tracking accuracy. The proposed algorithm is evaluated on a CLUST dataset that includes 22 sets of 3D US sequences, and the mean error of 1.60 0.97 mm compared with manual annotations is obtained. After comparing with other published works, the results show that our algorithm achieves the comparable performance. The ablation study proves that the results benefit from the feature pyramid architecture and FNCC. These findings show that our algorithm may improve the motion tracking accuracy in image-guided interventions.

Diffeomorphic respiratory motion estimation of thoracoabdominal organs for image-guided interventions.

Percutaneous image-guided interventions are commonly used for the diagnosis and treatment of cancer. In practice, physiological breathing-induced motion increases the difficulty of accurately inserting needles into tumors without impairing the surrounding vital structures. In this work, we propose a data-driven patient-specific hierarchical respiratory motion estimation framework to accurately estimate the position of a tumor and surrounding vital tissues in real time. The motion of optical markers attached to the chest or abdomen skin is used as a surrogate signal to estimate tumor motion based on ɛ-support vector regression (ɛ-SVR). With the estimated tumor motion as the input, a novel respiratory motion model is developed to estimate the diffeomorphic deformation field of the whole organ (liver or lung) without intraoperative, iterative optimization computations. The respiratory motion model of the whole organ is established in Lie algebra space based on the kriging algorithm to ensure that the estimated deformation field is diffeomorphic, optimal, and unbiased. Preoperative prior knowledge for modeling the motion of whole organs is obtained by deformation registration between four-dimensional computed tomography (4D CT) images using a hybrid diffeomorphic registration method. Experimental results on an invivo beagle dog show that the minimum value of the determinant of the Jacobian of the estimated deformation field is greater than zero, so the estimated deformation field of the whole liver with our method is diffeomorphic. The mean position error of the tumor is 1.2mm corresponding to a mean accuracy improvement of 76.5%, and the mean position error of the whole liver is 2.1mm, corresponding to a mean accuracy improvement of 37.9%. The experimental results based on public human subject data show that the mean position error of the tumor is 1.1mm, corresponding to a mean accuracy improvement of 83.1%, and the mean position error of the whole lung is 2.1mm, corresponding to a mean accuracy improvement of 41.4%. The positioning errors for the tumor and whole organ are hierarchical and consistent with clinical demand.

Medical Data and Mathematically Modeled Implicit Surface Real-Rime Visualization in Web Browsers

Raycasting can display volumetric medical data in fine details and reveal crucial inner imaging information, while implicit surface is able to effectively model complex objects with high flexibility, combining these two rendering modalities together will provide comprehensive information of the scene and has wide applications in surgical simulation, image-guided intervention, and medical training. However, medical data rendering is based on texture depth at every sampling point, while mathematically modeled implicit surfaces do not have geometric information in texture space. It is a challenging task to visualize both physical scalar data and virtual implicit surfaces simultaneously. To address this issue, in this paper, we present a new dual-casting ray-based double modality data rendering algorithm and web-based software platform to visualize volumetric medical data and implicit surface in the same browser. The algorithm runs on graphics processing unit and casts two virtual rays from camera to each pixel on the display panel, where one ray travels through the mathematically defined scene for implicit surface rendering and the other one passes the 3D texture space for volumetric data visualization. The proposed algorithm can detect voxel depth information and algebraic surface models along each casting ray and dynamically enhance the visualized dual-modality data with the improved lighting model and transparency adjustment function. Moreover, auxiliary innovative techniques are also presented to enhance the shading and rendering features of interest. Our software platform can seamlessly visualize volumetric medical data and implicit surfaces in the same web browser over Internet.

Robot-Assisted Image-Guided Interventions.

Image guidance is a common methodology of minimally invasive procedures. Depending on the type of intervention, various imaging modalities are available. Common imaging modalities are computed tomography, magnetic resonance tomography, and ultrasound. Robotic systems have been developed to enable and improve the procedures using these imaging techniques. Spatial and technological constraints limit the development of versatile robotic systems. This paper offers a brief overview of the developments of robotic systems for image-guided interventions since 2015 and includes samples of our current research in this field.