Mean Target Registration Error Research Articles

Intraoral Scanning Enables Virtual-Splint-Based Non-Invasive Registration Protocol for Maxillofacial Surgical Navigation.

Background/Objectives: Surgical navigation has advanced maxillofacial surgery since the 1990s, bringing benefits for various indications. Traditional registration methods use fiducial markers that are either invasively bone-anchored or attached to a dental vacuum splint and offer high accuracy but necessitate additional imaging with increased radiation exposure. We propose a novel, non-invasive registration protocol using a CAD/CAM dental splint based on high-resolution intraoral scans. Methods: The effectiveness of this method was experimentally evaluated with an ex vivo 3D-printed skull measuring the target registration error (TRE). Surgical application is demonstrated in two clinical cases. Results: In the ex vivo model, the new CAD/CAM-splint-based method achieved a mean TRE across the whole facial skull of 0.97 ± 0.29 mm, which was comparable to traditional techniques like using bone-anchored screws (1.02 ± 0.23 mm) and dental vacuum splints (1.01 ± 0.33 mm), while dental anatomical landmarks showed a lower accuracy with a mean TRE of 1.84 ± 0.44 mm. Multifactorial ANOVA confirmed significant differences in TRE based on the registration method and the navigated level of the facial skull (p < 0.001). In clinical applications, the presented method demonstrated high accuracy for both midfacial and mandibular surgeries. Conclusions: Our results suggest that this non-invasive CAD/CAM-splint-based method is a viable alternative to traditional fiducial marker techniques, with the potential for broad application in maxillofacial surgery. This approach retains high accuracy while eliminating the need for supplementary imaging and reduces patient radiation exposure. Further clinical trials are necessary to confirm these findings and optimize splint design for enhanced navigational accuracy.

D2BGAN: Dual Discriminator Bayesian Generative Adversarial Network for Deformable MR-Ultrasound Registration Applied to Brain Shift Compensation.

During neurosurgical procedures, the neuro-navigation system's accuracy is affected by the brain shift phenomenon. One popular strategy is to compensate for brain shift using intraoperative ultrasound (iUS) registration with pre-operative magnetic resonance (MR) scans. This requires a satisfactory multimodal image registration method, which is challenging due to the low image quality of ultrasound and the unpredictable nature of brain deformation during surgery. In this paper, we propose an automatic unsupervised end-to-end MR-iUS registration approach named the Dual Discriminator Bayesian Generative Adversarial Network (D2BGAN). The proposed network consists of two discriminators and a generator optimized by a Bayesian loss function to improve the functionality of the generator, and we add a mutual information loss function to the discriminator for similarity measurements. Extensive validation was performed on the RESECT and BITE datasets, where the mean target registration error (mTRE) of MR-iUS registration using D2BGAN was determined to be 0.75 ± 0.3 mm. The D2BGAN illustrated a clear advantage by achieving an 85% improvement in the mTRE over the initial error. Moreover, the results confirmed that the proposed Bayesian loss function, rather than the typical loss function, improved the accuracy of MR-iUS registration by 23%. The improvement in registration accuracy was further enhanced by the preservation of the intensity and anatomical information of the input images.

Deformable anthropomorphic pelvis phantom for dose accumulation verification

Objective. The validation of deformable image registration (DIR) for contour propagation is often done using contour-based metrics. Meanwhile, dose accumulation requires evaluation of voxel mapping accuracy, which might not be accurately represented by contour-based metrics. By fabricating a deformable anthropomorphic pelvis phantom, we aim to (1) quantify the voxel mapping accuracy for various deformation scenarios, in high- and low-contrast regions, and (2) identify any correlation between dice similarity coefficient (DSC), a commonly used contour-based metric, and the voxel mapping accuracy for each organ. Approach. Four organs, i.e. pelvic bone, prostate, bladder and rectum (PBR), were 3D printed using PLA and a Polyjet digital material, and assembled. The latter three were implanted with glass bead and CT markers within or on their surfaces. Four deformation scenarios were simulated by varying the bladder and rectum volumes. For each scenario, nine DIRs with different parameters were performed on RayStation v10B. The voxel mapping accuracy was quantified by finding the discrepancy between true and mapped marker positions, termed the target registration error (TRE). Pearson correlation test was done between the DSC and mean TRE for each organ. Main results. For the first time, we fabricated a deformable phantom purely from 3D printing, which successfully reproduced realistic anatomical deformations. Overall, the voxel mapping accuracy dropped with increasing deformation magnitude, but improved when more organs were used to guide the DIR or limit the registration region. DSC was found to be a good indicator of voxel mapping accuracy for prostate and rectum, but a comparatively poorer one for bladder. DSC > 0.85/0.90 was established as the threshold of mean TRE ⩽ 0.3 cm for rectum/prostate. For bladder, extra metrics in addition to DSC should be considered. Significance. This work presented a 3D printed phantom, which enabled quantification of voxel mapping accuracy and evaluation of correlation between DSC and voxel mapping accuracy.

Technical note: Minimizing CIED artifacts on a 0.35 T MRI-Linac using deep learning.

Artifacts from implantable cardioverter defibrillators (ICDs) are a challenge to magnetic resonance imaging (MRI)-guided radiotherapy (MRgRT). This study tested an unsupervised generative adversarial network to mitigate ICD artifacts in balanced steady-state free precession (bSSFP) cine MRIs and improve image quality and tracking performance for MRgRT. Fourteen healthy volunteers (Group A) were scanned on a 0.35 T MRI-Linac with and without an MR conditional ICD taped to their left pectoral to simulate an implanted ICD. bSSFP MRI data from 12 of the volunteers were used to train a CycleGAN model to reduce ICD artifacts. The data from the remaining two volunteers were used for testing. In addition, the dataset was reorganized three times using a Leave-One-Out scheme. Tracking metrics [Dice similarity coefficient (DSC), target registration error (TRE), and 95 percentile Hausdorff distance (95% HD)] were evaluated for whole-heart contours. Image quality metrics [normalized root mean square error (nRMSE), peak signal-to-noise ratio (PSNR), and multiscale structural similarity (MS-SSIM) scores] were evaluated. The technique was also tested qualitatively on three additional ICD datasets (Group B) including a patient with an implanted ICD. For the whole-heart contour with CycleGAN reconstruction: 1) Mean DSC rose from 0.910 to 0.935; 2) Mean TRE dropped from 4.488 to 2.877mm; and 3) Mean 95% HD dropped from 10.236 to 7.700mm. For the whole-body slice with CycleGAN reconstruction: 1) Mean nRMSE dropped from 0.644 to 0.420; 2) Mean MS-SSIM rose from 0.779 to 0.819; and 3) Mean PSNR rose from 18.744 to 22.368. The three Group B datasets evaluated qualitatively displayed a reduction in ICD artifacts in the heart. CycleGAN-generated reconstructions significantly improved both tracking and image quality metrics when used to mitigate artifacts from ICDs.

VOARiability: Interobserver and intermodality variability analysis in OAR contouring from head and neck CT and MR images.

Accurate and consistent contouring of organs-at-risk (OARs) from medical images is a key step of radiotherapy (RT) cancer treatment planning. Most contouring approaches rely on computed tomography (CT) images, but the integration of complementary magnetic resonance (MR) modality is highly recommended, especially from the perspective of OAR contouring, synthetic CT and MR image generation for MR-only RT, and MR-guided RT. Although MR has been recognized as valuable for contouring OARs in the head and neck (HaN) region, the accuracy and consistency of the resulting contours have not been yet objectively evaluated. To analyze the interobserver and intermodality variability in contouring OARs in the HaN region, performed by observers with different level of experience from CT and MR images of the samepatients. In the final cohort of 27 CT and MR images of the same patients, contours of up to 31 OARs were obtained by a radiation oncology resident (junior observer, JO) and a board-certified radiation oncologist (senior observer, SO). The resulting contours were then evaluated in terms of interobserver variability, characterized as the agreement among different observers (JO and SO) when contouring OARs in a selected modality (CT or MR), and intermodality variability, characterized as the agreement among different modalities (CT and MR) when OARs were contoured by a selected observer (JO or SO), both by the Dice coefficient (DC) and 95-percentile Hausdorff distance (HD ). The mean (±standard deviation) interobserver variability was 69.0 ± 20.2% and 5.1 ± 4.1mm, while the mean intermodality variability was 61.6 ± 19.0% and 6.1 ± 4.3mm in terms of DC and HD , respectively, across all OARs. Statistically significant differences were only found for specific OARs. The performed MR to CT image registration resulted in a mean target registration error of 1.7 ± 0.5mm, which was considered as valid for the analysis of intermodalityvariability. The contouring variability was, in general, similar for both image modalities, and experience did not considerably affect the contouring performance. However, the results indicate that an OAR is difficult to contour regardless of whether it is contoured in the CT or MR image, and that observer experience may be an important factor for OARs that are deemed difficult to contour. Several of the differences in the resulting variability can be also attributed to adherence to guidelines, especially for OARs with poor visibility or without distinctive boundaries in either CT or MR images. Although considerable contouring differences were observed for specific OARs, it can be concluded that almost all OARs can be contoured with a similar degree of variability in either the CT or MR modality, which works in favor of MR images from the perspective of MR-only and MR-guidedRT.

A Fully Differentiable Framework for 2D/3D Registration and the Projective Spatial Transformers.

Image-based 2D/3D registration is a critical technique for fluoroscopic guided surgical interventions. Conventional intensity-based 2D/3D registration approa- ches suffer from a limited capture range due to the presence of local minima in hand-crafted image similarity functions. In this work, we aim to extend the 2D/3D registration capture range with a fully differentiable deep network framework that learns to approximate a convex-shape similarity function. The network uses a novel Projective Spatial Transformer (ProST) module that has unique differentiability with respect to 3D pose parameters, and is trained using an innovative double backward gradient-driven loss function. We compare the most popular learning-based pose regression methods in the literature and use the well-established CMAES intensity-based registration as a benchmark. We report registration pose error, target registration error (TRE) and success rate (SR) with a threshold of 10mm for mean TRE. For the pelvis anatomy, the median TRE of ProST followed by CMAES is 4.4mm with a SR of 65.6% in simulation, and 2.2mm with a SR of 73.2% in real data. The CMAES SRs without using ProST registration are 28.5% and 36.0% in simulation and real data, respectively. Our results suggest that the proposed ProST network learns a practical similarity function, which vastly extends the capture range of conventional intensity-based 2D/3D registration. We believe that the unique differentiable property of ProST has the potential to benefit related 3D medical imaging research applications. The source code is available at https://github.com/gaocong13/Projective-Spatial-Transformers.

Transformer-based 2D/3D medical image registration for X-ray to CT via anatomical features.

2D/3D medical image registration is one of the key technologies for surgical navigation systems to perform pose estimation and achieve accurate positioning, which still remains challenging. The purpose of this study is to introduce a new method for X-ray to CT 2D/3D registration and conduct a feasibility study. In this study, a 2D/3D affine registration method based on feature point detection is investigated. It combines the morphological and edge features of spinal images to accurately extract feature points from the images, and uses graph neural networks to aggregate anatomical features of different points to increase the local detail information. Meanwhile, global and positional information are extracted by the Swin Transformer. The results indicate that the proposed method has shown improvements in both accuracy and success ratio compared with other methods. The mean target registration error value reached up to 0.31mm; meanwhile, the runtime overhead was much lower, achieving an average runtime of about 0.6s. This ultimately improves the registration accuracy and efficiency, demonstrating the effectiveness of the proposed method. The proposed method can provide more comprehensive image information and shows good prospects for pose estimation and achieving accurate positioning in surgical navigation systems.

A high-dimensional respiratory motion modeling method based on machine learning

PurposeRespiratory motion of the human body remains a major challenge in procedures for percutaneous puncture intervention. A solution is to develop a respiratory motion model. The aim of this study is to construct a respiratory motion model that can accommodate extreme respiratory conditions. MethodsPrincipal component analysis (PCA) and the support vector regression (SVR) method were employed as the framework to build the respiratory motion model. In the first stage, internal respiratory signals and respiratory surrogate signals are extracted. In the second stage, the respiratory motion model based on SVR is established. This study proposes a unique data augmentation method to improve the model robustness and the corresponding response capacity under extreme respiratory conditions. Finally, relevant evaluation indicators are used to evaluate the motion prediction accuracy of a reference model and the model in this study. ResultsWhen estimating the motion of the end-inspiratory phase (T00), the mean target registration error (TRE) of the motion estimate of the model after data augmentation was 1.28 ± 0.24 mm, while that of the model before data augmentation and that of the reference model were 1.67 ± 0.55 mm and 2.29 ± 1.06 mm, respectively. When estimating the motion of all phases, the mean TRE of the model after data augmentation was 1.26 ± 0.23 mm, which was also lower than that of the model before data augmentation. The data augmentation method improves the accuracy of the model and is statistically significant. ConclusionDisplaying excellent performance under extrapolation conditions, the proposed model effectively improves both accuracy and robustness.

P07.04.A ELASTIC IMAGE FUSION OF INTRAOPERATIVE ULTRASOUND AND PREOPERATIVE MRI FOR IMPROVING NAVIGATION ACCURACY IN BRAIN TUMOR SURGERY

Abstract BACKGROUND In brain surgery, neuronavigation based on preoperative MRI is limited by various physical and operational factors, such as surgery-induced brain shift. Using navigated intraoperative ultrasound (iUS) imaging and 3D iUS image reconstruction may help improve neuronavigation. However, the registration of MRI and US scans can lead to spatial inconsistencies between the two modalities. This study aims to evaluate the clinical integration of navigated iUS imagining and its potential to improve the accuracy of resection during brain tumor surgery by comparing the performance of rigid and elastic MRI-iUS image fusion algorithms during retrospective evaluation. MATERIAL AND METHODS N = 50 patients with brain tumors were prospectively enrolled in this trial. The neuronavigation system (BrainLab AG) used in the study allowed for the reconstruction and integration of 3D iUS imaging, which was acquired multiple times throughout the surgery using a tracked ultrasound probe (BK Medical). Prototypes of automatic rigid and elastic MRI-iUS image fusion algorithms (Brainlab Elements Image Fusion) were retrospectively applied to improve co-registration accuracy between the MRI and iUS data. To evaluate the image co-registration accuracy, anatomical landmarks were defined in the MRI and iUS scans and used to measure Euclidean distances between each landmark pair. RESULTS Improvements in the image quality over the course of the study indicated that skills in US handling are essential. Contrast behavior and tumor structure influence the visibility of the tumor in the US. Artifacts caused by hemostatic agents challenge the US image quality and the fluid-filled resection cavity requires experience in probe handling. In terms of image fusion, significant mismatches of the iUS and MRI data were reported. For scans acquired before resection, rigid fusion using a novel co-registration algorithm may enhance the registration accuracy with mean target registration errors for two representative patients of 6.5/8.9 mm and 3.7/3.4 mm before and after registration update, respectively. For iUS scans acquired after tumor resection, rigid fusion is inherently limited due to non-linear brain shifts. Therefore, elastic image fusion has been applied to provide deformable MR-iUS image co-registration. CONCLUSION Navigated iUS imaging can be integrated into clinical routine and the use of MR-iUS image co-registration for updating navigation is a promising tool to compensate brainshift during surgical resection. However, further studies are needed to quantify the accuracy and assess whether iUS could potentially replace the use of iMRI in resection control of selected cases.

Evaluating similarity measure for multimodal 3D to 2D registration

The 3D to 2D registration technique in spine surgery is vital to aid surgeons in avoiding the wrong site surgery by estimating the vertebral pose. The vertebral poses are estimated by generating the spatial correspondence relationship between pre-operative MR with intra-operative x-ray images, then evaluated using a similarity measure. Different similarity measures are used in 3D to 2D registration techniques to assess the spatial correspondence between the pre-operative and intra-operative images. However, to evaluate the registration performance of the similarity measures, the proposed framework employs three different similarity measures: Binary Image Matching, Dice Coefficients, and Normalized Cross-correlation technique to compare the images based on pixel positions. The registration accuracy of the proposed similarity measures is compared based on the mean Target Registration Error, mean Iteration Times, and success rate. In the absence of simulated test images, the experiment is conducted on the simulated AP and Lateral test images. The experiment conducted on the simulated test images shows that all three similarity measures work well for the feature based 3D to 2D registration in that BIM gives better results. The experiment also indicates high registration accuracy when the initial displacements are varied up to ±20 mm and ±100 of the translational and rotational parameters, respectively, for three similarity measures.

The Effect of Fiducial Marker Number and Configuration on Registration Error in Dynamic Implant Surgery.

Objective: To verify the effect of fiducial marker number and configuration on target registration error (TRE) for dynamic computer-aided zygomatic implant surgery. Material and Methods: All patients who underwent zygomatic implant surgery with navigation from January 2018 to December 2021 were enrolled. For each patient, 6 to 8 miniscrews were placed intraorally as fiducial markers before the surgery. After the registration procedure, the TRE, which represents the distance between the target of the image space and the real position of the fiducial markers, was calculated. SPSS (22.0) was used for statistical analysis. Results: A total of 325 titanium miniscrews were placed in 47 patients who underwent zygomatic implant placement by navigation. The lowest TRE was 0.2 mm, compared to the highest TRE of 1.9 mm. There was no significant difference in the mean TRE value among the different titanium miniscrew groups (P = .07). A total of 8 miniscrews in 7 patients were lost in the maxillary tuberosity area prior to and during navigation surgery, which resulted in an irregular polygonal distribution of fiducial markers. However, there was no statistically significant difference in TRE between a polygonal distribution (0.62 ± 0.35 mm) and an irregular polygonal distribution (0.68 ± 0.33 mm) (P = .35). Conclusion: A scattered, polygonal distribution with of a minimum of five fiducial markers in an edentulous maxilla could achieve acceptable TRE values in registration. It seems that the registration error was not influenced by the absence of one corner in a polygon distribution.

Integration of statistical shape modeling and alternating interpolation-based model tracking technique for measuring knee kinematics in vivo using clinical interleaved bi-plane fluoroscopy.

A 2D fluoroscopy/3D model-based registration with statistical shape modeling (SSM)-reconstructed subject-specific bone models will help reduce radiation exposure for 3D kinematic measurements of the knee using clinical alternating bi-plane fluoroscopy systems. The current study aimed to develop such an approach and evaluate in vivo its accuracy and identify the effects of the accuracy of SSM models on the kinematic measurements. An alternating interpolation-based model tracking (AIMT) approach with SSM-reconstructed subject-specific bone models was used for measuring 3D knee kinematics from dynamic alternating bi-plane fluoroscopy images. A two-phase optimization scheme was used to reconstruct subject-specific knee models from a CT-based SSM database of 60 knees using one, two, or three pairs of fluoroscopy images. Using the CT-reconstructed model as a benchmark, the performance of the AIMT with SSM-reconstructed models in measuring bone and joint kinematics during dynamic activity was evaluated in terms of mean target registration errors (mmTRE) for registered bone poses and the mean absolute differences (MAD) for each motion component of the joint poses. The mmTRE of the femur and tibia for one image pair were significantly greater than those for two and three image pairs without significant differences between two and three image pairs. The MAD was 1.16 to 1.22° for rotations and 1.18 to 1.22 mm for translations using one image pair. The corresponding values for two and three image pairs were 0.75 to 0.89° and 0.75 to 0.79 mm; and 0.57 to 0.79° and 0.6 to 0.69 mm, respectively. The MAD values for one image pair were significantly greater than those for two and three image pairs without significant differences between two and three image pairs. An AIMT approach with SSM-reconstructed models was developed, enabling the registration of interleaved fluoroscopy images and SSM-reconstructed models from more than one asynchronous fluoroscopy image pair. This new approach had sub-millimeter and sub-degree measurement accuracy when using more than one image pair, comparable to the accuracy of CT-based methods. This approach will be helpful for future kinematic measurements of the knee with reduced radiation exposure using 3D fluoroscopy with clinically alternating bi-plane fluoroscopy systems.

Feasibility of tracked ultrasound registration for pelvic-abdominal tumor navigation: a patient study.

Surgical navigation techniques can guide surgeons in localizing pelvic-abdominal malignancies. For abdominal navigation, accurate patient registration is crucial and is generally performed using an intra-operative cone-beam CT (CBCT). However, this method causes 15-min surgical preparation workflow interruption and radiation exposure, and more importantly, it cannot be repeated during surgery to compensate for large patient movement. As an alternative, the accuracy and feasibility of tracked ultrasound (US) registration are assessed in this patient study. Patients scheduled for surgical navigation during laparotomy of pelvic-abdominal malignancies were prospectively included. In the operating room, two percutaneous tracked US scans of the pelvic bone were acquired: one in supine and one in Trendelenburg patient position. Postoperatively, the bone surface was semiautomatically segmented from US images and registered to the bone surface on the preoperative CT scan. The US registration accuracy was computed using the CBCT registration as a reference and acquisition times were compared. Additionally, both US measurements were compared to quantify the registration error caused by patient movement into Trendelenburg. In total, 18 patients were included and analyzed. US registration resulted in a mean surface registration error of 1.2 ± 0.2mm and a mean target registration error of 3.3 ± 1.4mm. US acquisitions were 4 × faster than the CBCT scans (two-sample t-test P < 0.05) and could even be performed during standard patient preparation before skin incision. Patient repositioning in Trendelenburg caused a mean target registration error of 7.7 ± 3.3mm, mainly in cranial direction. US registration based on the pelvic bone is accurate, fast and feasible for surgical navigation. Further optimization of the bone segmentation algorithm will allow for real-time registration in the clinical workflow. In the end, this would allow intra-operative US registration to correct for large patient movement. This study is registered in ClinicalTrials.gov (NCT05637359).

Automatic rigid image Fusion of preoperative MR and intraoperative US acquired after craniotomy

BackgroundNeuronavigation of preoperative MRI is limited by several errors. Intraoperative ultrasound (iUS) with navigated probes that provide automatic superposition of pre-operative MRI and iUS and three-dimensional iUS reconstruction may overcome some of these limitations. Aim of the present study is to verify the accuracy of an automatic MRI – iUS fusion algorithm to improve MR-based neuronavigation accuracy.MethodsAn algorithm using Linear Correlation of Linear Combination (LC2)-based similarity metric has been retrospectively evaluated for twelve datasets acquired in patients with brain tumor. A series of landmarks were defined both in MRI and iUS scans. The Target Registration Error (TRE) was determined for each pair of landmarks before and after the automatic Rigid Image Fusion (RIF). The algorithm has been tested on two conditions of the initial image alignment: registration-based fusion (RBF), as given by the navigated ultrasound probe, and different simulated course alignments during convergence test.Results Except for one case RIF was successfully applied in all patients considering the RBF as initial alignment. Here, mean TRE after RBF was significantly reduced from 4.03 (± 1.40) mm to (2.08 ± 0.96 mm) (p = 0.002), after RIF. For convergence test, the mean TRE value after initial perturbations was 8.82 (± 0.23) mm which has been reduced to a mean TRE of 2.64 (± 1.20) mm after RIF (p < 0.001).ConclusionsThe integration of an automatic image fusion method for co-registration of pre-operative MRI and iUS data may improve the accuracy in MR-based neuronavigation.

Promising results of an non-invasive measurement of knee implant loosening using a loading device, CT-scans and 3D image analysis

BackgroundAfter total knee arthroplasty up to 13% requires revision surgery to address loosening. No current diagnostic modalities have a sensitivity or specificity higher than 70–80% to detect loosening, leading to 20–30% of patients undergoing unnecessary, risky and expensive revision surgery. A reliable imaging modality is required to diagnose loosening. This study presents a new and non-invasive method and evaluates its reproducibility and reliability in a cadaveric study. MethodsTen cadaveric specimens were implanted with a loosely fitted tibial components and CT scanned under load towards valgus and varus using a loading device. Advanced three-dimensional imaging software was used to quantify displacement. Subsequently, the implants were fixed to the bone and scanned to determine the differences between the fixed and the loose state. Reproducibility errors were quantified using a frozen specimen in which displacement was absent. FindingsReproducibility errors, expressed as mean target registration error, screw-axis rotation and maximum total point motion were 0.073 mm (SD 0.033), 0.129 degrees (SD 0.039) and 0.116 mm (SD 0.031), respectively. In the loose condition, all displacements and rotation changes were larger than the reported reproducibility errors. Comparing the mean target registration error, screw axis rotation and maximum total point motion in the loose condition to the fixed condition resulted in mean differences of 0.463 mm (SD 0.279; p = 0.001), 1.769 degrees (SD 0.868; p < 0.001) and 1.339 mm (SD 0.712; p < 0.001), respectively. InterpretationThe results of this cadaveric study show that this non-invasive method is reproducible and reliable for detection of displacement differences between fixed and loose tibial components.

Validation of a surface-based deformable MRI-3D ultrasound image registration algorithm toward clinical implementation for interstitial prostate brachytherapy

PURPOSEThe purpose of this study was to evaluate and clinically implement a deformable surface-based magnetic resonance imaging (MRI) to three-dimensional ultrasound (US) image registration algorithm for prostate brachytherapy (BT) with the aim to reduce operator dependence and facilitate dose escalation to an MRI-defined target. METHODS AND MATERIALSOur surface-based deformable image registration (DIR) algorithm first translates and scales to align the US- and MR-defined prostate surfaces, followed by deformation of the MR-defined prostate surface to match the US-defined prostate surface. The algorithm performance was assessed in a phantom using three deformation levels, followed by validation in three retrospective high-dose-rate BT clinical cases. For comparison, manual rigid registration and cognitive fusion by physician were also employed. Registration accuracy was assessed using the Dice similarity coefficient (DSC) and target registration error (TRE) for embedded spherical landmarks. The algorithm was then implemented intraoperatively in a prospective clinical case. RESULTSIn the phantom, our DIR algorithm demonstrated a mean DSC and TRE of 0.74 ± 0.08 and 0.94 ± 0.49 mm, respectively, significantly improving the performance compared to manual rigid registration with 0.64 ± 0.16 and 1.88 ± 1.24 mm, respectively. Clinical results demonstrated reduced variability compared to the current standard of cognitive fusion by physicians. CONCLUSIONSWe successfully validated a DIR algorithm allowing for translation of MR-defined target and organ-at-risk contours into the intraoperative environment. Prospective clinical implementation demonstrated the intraoperative feasibility of our algorithm, facilitating targeted biopsies and dose escalation to the MR-defined lesion. This method provides the potential to standardize the registration procedure between physicians, reducing operator dependence.

Database-driven patient-specific registration error compensation method for image-guided laparoscopic surgery

A surgical navigation system helps surgeons understand anatomical structures in the operative field during surgery. Patient-to-image registration, which aligns coordinate systems between the CT volume and a positional tracker, is vital for accurate surgical navigation. Although a point-based rigid registration method using fiducials on the body surface is often utilized for laparoscopic surgery navigation, precise registration is difficult due to such factors as soft tissue deformation. We propose a method that compensates a transformation matrix computed using fiducials on the body surface based on the analysis of positional information in the database. We built our database by measuring the positional information of the fiducials and the guidance targets in both the CT volume and positional tracker coordinate systems through previous surgeries. We computed two transformation matrices: using only the fiducials and using only the guidance targets in all the data in the database. We calculated the differences between the two transformation matrices in each piece of data. The compensation transformation matrix was computed by averaging these difference matrices. In this step, we selected the data from the database based on the similarity of the fiducials and the configuration of the guidance targets. We evaluated our proposed method using 20 pieces of data acquired during laparoscopic gastrectomy for gastric cancer. The locations of blood vessels were used as guidance targets for computing target registration error. The mean target registration errors significantly decreased from 33.0 to 17.1 mm before and after the compensation. This paper described a registration error compensation method using a database for image-guided laparoscopic surgery. Since our proposed method reduced registration error without additional intraoperative measurements during surgery, it increases the accuracy of surgical navigation for laparoscopic surgery.

Augmented Reality to Compensate for Navigation Inaccuracies.

This study aims to report on the capability of microscope-based augmented reality (AR) to evaluate registration and navigation accuracy with extracranial and intracranial landmarks and to elaborate on its opportunities and obstacles in compensation for navigation inaccuracies. In a consecutive single surgeon series of 293 patients, automatic intraoperative computed tomography-based registration was performed delivering a high initial registration accuracy with a mean target registration error of 0.84 ± 0.36 mm. Navigation accuracy is evaluated by overlaying a maximum intensity projection or pre-segmented object outlines within the recent focal plane onto the in situ patient anatomy and compensated for by translational and/or rotational in-plane transformations. Using bony landmarks (85 cases), there was two cases where a mismatch was seen. Cortical vascular structures (242 cases) showed a mismatch in 43 cases and cortex representations (40 cases) revealed two inaccurate cases. In all cases, with detected misalignment, a successful spatial compensation was performed (mean correction: bone (6.27 ± 7.31 mm), vascular (3.00 ± 1.93 mm, 0.38° ± 1.06°), and cortex (5.31 ± 1.57 mm, 1.75° ± 2.47°)) increasing navigation accuracy. AR support allows for intermediate and straightforward monitoring of accuracy, enables compensation of spatial misalignments, and thereby provides additional safety by increasing overall accuracy.

Predicting 4D liver MRI for MR-guided interventions.

Organ motion poses an unresolved challenge in image-guided interventions like radiation therapy, biopsies or tumor ablation. In the pursuit of solving this problem, the research field of time-resolved volumetric magnetic resonance imaging (4D MRI) has evolved. However, current techniques are unsuitable for most interventional settings because they lack sufficient temporal and/or spatial resolution or have long acquisition times. In this work, we propose a novel approach for real-time, high-resolution 4D MRI with large fields of view for MR-guided interventions. To this end, we propose a network-agnostic, end-to-end trainable, deep learning formulation that enables the prediction of a 4D liver MRI with respiratory states from a live 2D navigator MRI. Our method can be used in two ways: First, it can reconstruct high quality fast (near real-time) 4D MRI with high resolution (209×128×128 matrix size with isotropic 1.8mm voxel size and 0.6s/volume) given a dynamic interventional 2D navigator slice for guidance during an intervention. Second, it can be used for retrospective 4D reconstruction with a temporal resolution of below 0.2s/volume for motion analysis and use in radiation therapy. We report a mean target registration error (TRE) of 1.19±0.74mm, which is below voxel size. We compare our results with a state-of-the-art retrospective 4D MRI reconstruction. Visual evaluation shows comparable quality. We compare different network architectures within our formulation. We show that small training sizes with short acquisition times down to 2 min can already achieve promising results and 24 min are sufficient for high quality results. Because our method can be readily combined with earlier time reducing methods, acquisition time can be further decreased while also limiting quality loss. We show that an end-to-end, deep learning formulation is highly promising for 4D MRI reconstruction.

Quantitative evaluation of the influence of multiple MRI sequences and of pathological tissues on the registration of longitudinal data acquired during brain tumor treatment.

Registration methods facilitate the comparison of multiparametric magnetic resonance images acquired at different stages of brain tumor treatments. Image-based registration solutions are influenced by the sequences chosen to compute the distance measure, and the lack of image correspondences due to the resection cavities and pathological tissues. Nonetheless, an evaluation of the impact of these input parameters on the registration of longitudinal data is still missing. This work evaluates the influence of multiple sequences, namely T1-weighted (T1), T2-weighted (T2), contrast enhanced T1-weighted (T1-CE), and T2 Fluid Attenuated Inversion Recovery (FLAIR), and the exclusion of the pathological tissues on the non-rigid registration of pre- and post-operative images. We here investigate two types of registration methods, an iterative approach and a convolutional neural network solution based on a 3D U-Net. We employ two test sets to compute the mean target registration error (mTRE) based on corresponding landmarks. In the first set, markers are positioned exclusively in the surroundings of the pathology. The methods employing T1-CE achieves the lowest mTREs, with a improvement up to 0.8 mm for the iterative solution. The results are higher than the baseline when using the FLAIR sequence. When excluding the pathology, lower mTREs are observable for most of the methods. In the second test set, corresponding landmarks are located in the entire brain volumes. Both solutions employing T1-CE obtain the lowest mTREs, with a decrease up to 1.16 mm for the iterative method, whereas the results worsen using the FLAIR. When excluding the pathology, an improvement is observable for the CNN method using T1-CE. Both approaches utilizing the T1-CE sequence obtain the best mTREs, whereas the FLAIR is the least informative to guide the registration process. Besides, the exclusion of pathology from the distance measure computation improves the registration of the brain tissues surrounding the tumor. Thus, this work provides the first numerical evaluation of the influence of these parameters on the registration of longitudinal magnetic resonance images, and it can be helpful for developing future algorithms.