Landmark Registration Error Research Articles

RMSim: controlled respiratory motion simulation on static patient scans

Objective. This work aims to generate realistic anatomical deformations from static patient scans. Specifically, we present a method to generate these deformations/augmentations via deep learning driven respiratory motion simulation that provides the ground truth for validating deformable image registration (DIR) algorithms and driving more accurate deep learning based DIR. Approach. We present a novel 3D Seq2Seq deep learning respiratory motion simulator (RMSim) that learns from 4D-CT images and predicts future breathing phases given a static CT image. The predicted respiratory patterns, represented by time-varying displacement vector fields (DVFs) at different breathing phases, are modulated through auxiliary inputs of 1D breathing traces so that a larger amplitude in the trace results in more significant predicted deformation. Stacked 3D-ConvLSTMs are used to capture the spatial-temporal respiration patterns. Training loss includes a smoothness loss in the DVF and mean-squared error between the predicted and ground truth phase images. A spatial transformer deforms the static CT with the predicted DVF to generate the predicted phase image. 10-phase 4D-CTs of 140 internal patients were used to train and test RMSim. The trained RMSim was then used to augment a public DIR challenge dataset for training VoxelMorph to show the effectiveness of RMSim-generated deformation augmentation. Main results. We validated our RMSim output with both private and public benchmark datasets (healthy and cancer patients). The structure similarity index measure (SSIM) for predicted breathing phases and ground truth 4D CT images was 0.92 ± 0.04, demonstrating RMSim’s potential to generate realistic respiratory motion. Moreover, the landmark registration error in a public DIR dataset was improved from 8.12 ± 5.78 mm to 6.58mm ± 6.38 mm using RMSim-augmented training data. Significance. The proposed approach can be used for validating DIR algorithms as well as for patient-specific augmentations to improve deep learning DIR algorithms. The code, pretrained models, and augmented DIR validation datasets will be released at https://github.com/nadeemlab/SeqX2Y.

Inter-Phase 4D Cardiac MRI Registration With a Motion Prior Derived From CTA.

Registration between phases in 4D cardiac MRI is essential for reconstructing high-quality images and appreciating the dynamics. Complex motion and limited image quality make it challenging to design regularization functionals. We propose to introduce a motion representation model (MRM) into a registration network to impose customized, site-specific, and spatially variant prior for cardiac motion. We propose a novel approach to regularize deep registration with a deformation vextor field (DVF) representation model using computed tomography angiography (CTA). In the form of a convolutional auto-encoder, the MRM was trained to capture the spatially variant pattern of feasible DVF Jacobian. The CTA-derived MRM was then incorporated into an unsupervised network to facilitate MRI registration. In the experiment, 10 CTAs were used to derive the MRM. The method was tested on 10 0.35 T scans in long-axis view with manual segmentation and 15 3 T scans in short-axis view with tagging-based landmarks. Introducing the MRM improved registration accuracy and achieved 2.23, 7.21, and 4.42 mm 80% Hausdorff distance on left ventricle, right ventricle, and pulmonary artery, respectively, and 2.23 mm landmark registration error. The results were comparable to carefully tuned SimpleElastix, but reduced the registration time from 40 s to 0.02 s. The MRM presented good robustness to different DVF sample generation methods. The model enjoys high accuracy as meticulously tuned optimization model and the efficiency of deep networks. The method enables model to go beyond the quality limitation of MRI. The robustness to training DVF generation scheme makes the method attractive to adapting to the available data and software resources in various clinics.

Tissue-specific deformable image registration using a spatial-contextual filter.

Intensity-based deformable registration with spatial-invariant regularization generally fails when distinct motion exists across different types of tissues. The purpose of this work was to develop and validate a new regularization approach for deformable image registration that is tissue-specific and able to handle motion discontinuities. Our approach was built upon a Demons registration framework, and used the image context supplementing the original spatial constraint to regularize displacement vector fields in iterative image registration process. The new regularization was implemented as a spatial-contextual filter, which favors the motion vectors within the same tissue type but penalizes the motion vectors from different tissues. This approach was validated using five public lung cancer patients, each with 300 landmark pairs identified by a thoracic radiation oncologist. The mean and standard deviation of the landmark registration errors were 1.3 ± 0.8 mm, compared with those of 2.3 ± 2.9 mm using the original Demons algorithm. Particularly, for the case with the largest initial landmark displacement of 15 ± 9 mm, the modified Demons algorithm had a registration error of 1.3 ± 1.1 mm, while the original Demons algorithm had a registration error of 3.6 ± 5.9 mm. We also qualitatively evaluated the modified Demons algorithm using two difficult cases in our routine clinic: one lung case with large sliding motion and one head and neck case with large anatomical changes in air cavity. Visual evaluation on the deformed image created by the deformable image registration showed that the modified Demons algorithm achieved reasonable registration accuracy, but the original Demons algorithm produced distinct registration errors.

Accuracy of Deformable Image Registration for Reirradiation of Lung Cancer Following Stereotactic Body Radiotherapy

Stereotactic body radiotherapy (SBRT) for lung cancer typically causes radiographic lung changes that can affect the registration accuracy at the time of reirradiation. In this study, we investigate the accuracy of deformable image registration (DIR) in the presence of post SBRT lung changes using anatomical landmarks. We hypothesize that DIR accuracy is worse in regions near radiographic changes than in normal lung tissue. Planning, 3-, 6-, and 12-month follow-up CT images were deformably registered for 10 patients exhibiting radiographic lung changes following SBRT. Several multi-pass b-spline based DIRs were performed using a commercially available algorithm by varying the target intensity window and the structures with overridden image intensities. The structures included: the planning target volume (PTV) and ground glass (GG), and consolidated (C) radiographic changes, while the intensity windows investigated included: full image contrast, lung window (-1024HU to 200 HU), and mediastinum window (-135HU to 215HU). Landmark points were manually located in both the treated lung (mean 26.5 points, range 17-30 per scan within 5cm of the PTV) and contralateral normal lung (mean 29.7 points, range 29-30). Accuracy was evaluated by calculating the average 3D landmark registration error. The Pearson correlation coefficient was used to investigate the influence of the PTV, GG, and C lung volumes and normal lung registration error on the treated region registration error. For all time points considered together, the most accurate registration near treated tissue involved masking both the PTV and GG lung changes and using full image contrast. The average landmark error near the treated region with this registration was 7.7 +/- 4.9 mm and 5.2 +/- 3.5mm in the normal lung. The landmark error for the individual time points, 3-, 6-, and 12-month respectively, were 3.7mm, 9.8mm, and 9.8mm near the treated region and 4.8mm, 5.7 and 5mm in the normal lung tissue. Landmark errors at 3-months were strongly correlated with the volume of the GG (rˆ2 = 0.84) and C (rˆ2 = 0.84) lung changes. The 6- and 12-month landmark errors were moderately correlated with GG (rˆ2 = 0.29, rˆ2 = 0.43) and C (rˆ2 = 0.31, rˆ2 = 0.47) volumes. There was moderate correlation between landmark errors near the treated region and normal tissue that increases over time from rˆ2 = 0.28 at 3 months to rˆ2 = 0.69 at 12 months. No strong correlation was found with the PTV volume. Landmark registration errors in the treated region can be a few millimeters larger than in normal tissue and appear to increase with time, before stabilizing. There is also an increasing correlation over time between the normal tissue registration errors and those in the treated region. DIR uncertainty in the treated and normal lung needs to be considered during retreatment composite planning.

Open Source Platform for Transperineal In-Bore MRI-Guided Targeted Prostate Biopsy.

Accurate biopsy sampling of the suspected lesions is critical for the diagnosis and clinical management of prostate cancer. Transperineal in-bore MRI-guided prostate biopsy (tpMRgBx) is a targeted biopsy technique that was shown to be safe, efficient, and accurate. Our goal was to develop an open source software platform to support evaluation, refinement, and translation of this biopsy approach. We developed SliceTracker, a 3D Slicer extension to support tpMRgBx. We followed modular design of the implementation to enable customization of the interface and interchange of image segmentation and registration components to assess their effect on the processing time, precision, and accuracy of the biopsy needle placement. The platform and supporting documentation were developed to enable the use of software by an operator with minimal technical training to facilitate translation. Retrospective evaluation studied registration accuracy, effect of the prostate segmentation approach, and re-identification time of biopsy targets. Prospective evaluation focused on the total procedure time and biopsy targeting error (BTE). Evaluation utilized data from 73 retrospective and ten prospective tpMRgBx cases. Mean landmark registration error for retrospective evaluation was 1.88 ± 2.63 mm, and was not sensitive to the approach used for prostate gland segmentation. Prospectively, we observed target re-identification time of 4.60 ± 2.40 min and BTE of 2.40 ± 0.98 mm. SliceTracker is modular and extensible open source platform for supporting image processing aspects of the tpMRgBx procedure. It has been successfully utilized to support clinical research procedures at our site.

Error estimation of deformable image registration of pulmonary CT scans using convolutional neural networks.

Error estimation in nonlinear medical image registration is a nontrivial problem that is important for validation of registration methods. We propose a supervised method for estimation of registration errors in nonlinear registration of three-dimensional (3-D) images. The method is based on a 3-D convolutional neural network that learns to estimate registration errors from a pair of image patches. By applying the network to patches centered around every voxel, we construct registration error maps. The network is trained using a set of representative images that have been synthetically transformed to construct a set of image pairs with known deformations. The method is evaluated on deformable registrations of inhale-exhale pairs of thoracic CT scans. Using ground truth target registration errors on manually annotated landmarks, we evaluate the method's ability to estimate local registration errors. Estimation of full domain error maps is evaluated using a gold standard approach. The two evaluation approaches show that we can train the network to robustly estimate registration errors in a predetermined range, with subvoxel accuracy. We achieved a root-mean-square deviation of 0.51mm from gold standard registration errors and of 0.66mm from ground truth landmark registration errors.

Landmark constellation models for medical image content identification and localization.

Many medical imaging tasks require the detection and localization of anatomical landmarks, for example for the initialization of model-based segmentation or to detect anatomical regions present in an image. A large number of landmark and object localization methods have been described in the literature. The detection of single landmarks may be insufficient to achieve robust localization across a variety of imaging settings and subjects. Furthermore, methods like the generalized Hough transform yield the most likely location of an object, but not an indication whether or not the landmark was actually present in the image. For these reasons, we developed a simple and computationally efficient method combining localization results from multiple landmarks to achieve robust localization and to compute a localization confidence measure. For each anatomical region, we train a constellation model indicating the mean relative locations and location variability of a set of landmarks. This model is registered to the landmarks detected in a test image via point-based registration, using closed-form solutions. Three different outlier suppression schemes are compared, two using iterative re-weighting based on the residual landmark registration errors and the third being a variant of RANSAC. The mean weighted residual registration error serves as a confidence measure to distinguish true from false localization results. The method is optimized and evaluated on synthetic data, evaluating both the localization accuracy and the ability to classify good from bad registration results based on the residual registration error. Two application examples are presented: the identification of the imaged anatomical region in trauma CT scans and the initialization of model-based segmentation for C-arm CT scans with different target regions. The identification of the target region with the presented method was in 96% of the cases correct. The presented method is a simple solution for combining multiple landmark localization results. With appropriate parameters, outlier suppression clearly improves the localization performance over model registration without outlier suppression. The optimum choice of method and parameters depends on the expected level of noise and outliers in the application at hand, as well as on the focus on localization, classification, or both. The method allows detecting and localizing anatomical fields of view in medical images and is well suited to support a wide range of applications comprising image content identification, anatomical navigation and visualization, or initializing the pose of organ shape models.

Multimodal image-guided prostate fusion biopsy based on automatic deformable registration.

Transrectal ultrasound (TRUS)-guided random prostate biopsy is, in spite of its low sensitivity, the gold standard for the diagnosis of prostate cancer. The recent advent of PET imaging using a novel dedicated radiotracer, [Formula: see text]-labeled prostate-specific membrane antigen (PSMA), combined with MRI provides improved pre-interventional identification of suspicious areas. This work proposes a multimodal fusion image-guided biopsy framework that combines PET-MRI images with TRUS, using automatic segmentation and registration, and offering real-time guidance. The prostate TRUS images are automatically segmented with a Hough transform-based random forest approach. The registration is based on the Coherent Point Drift algorithm to align surfaces elastically and to propagate the deformation field calculated from thin-plate splines to the whole gland. The method, which has minimal requirements and temporal overhead in the existing clinical workflow, is evaluated in terms of surface distance and landmark registration error with respect to the clinical ground truth. Evaluations on agar-gelatin phantoms and clinical data of 13 patients confirm the validity of this approach. The system is able to successfully map suspicious regions from PET/MRI to the interventional TRUS image.

Image registration for targeted MRI-guided transperineal prostate biopsy.

To develop and evaluate image registration methodology for automated re-identification of tumor-suspicious foci from preprocedural MR exams during MR-guided transperineal prostate core biopsy. A hierarchical approach for automated registration between planning and intra-procedural T2-weighted prostate MRI was developed and evaluated on the images acquired during 10 consecutive MR-guided biopsies. Registration accuracy was quantified at image-based landmarks and by evaluating spatial overlap for the manually segmented prostate and sub-structures. Registration reliability was evaluated by simulating initial mis-registration and analyzing the convergence behavior. Registration precision was characterized at the planned biopsy targets. The total computation time was compatible with a clinical setting, being at most 2 min. Deformable registration led to a significant improvement in spatial overlap of the prostate and peripheral zone contours compared with both rigid and affine registration. Average in-slice landmark registration error was 1.3 ± 0.5 mm. Experiments simulating initial mis-registration resulted in an estimated average capture range of 6 mm and an average in-slice registration precision of ±0.3 mm. Our registration approach requires minimum user interaction and is compatible with the time constraints of our interventional clinical workflow. The initial evaluation shows acceptable accuracy, reliability and consistency of the method.

Deformable registration of abdominal kilovoltage treatment planning CT and tomotherapy daily megavoltage CT for treatment adaptation

In adaptive radiation therapy the treatment planning kilovoltage CT (kVCT) images need to be registered with daily CT images. Daily megavoltage CT (MVCT) images are generally noisier than the kVCT images. In addition, in the abdomen, low image contrast, differences in bladder filling, differences in bowel, and rectum filling degrade image usefulness and make deformable image registration very difficult. The authors have developed a procedure to overcome these difficulties for better deformable registration between the abdominal kVCT and MVCT images. The procedure includes multiple image preprocessing steps and a two deformable registration steps. The image preprocessing steps include MVCT noise reduction, bowel gas pockets detection and painting, contrast enhancement, and intensity manipulation for critical organs. The first registration step is carried out in the local region of the critical organs (bladder, prostate, and rectum). It requires structure contours of these critical organs on both kVCT and MVCT to obtain good registration accuracy on these critical organs. The second registration step uses the first step results and registers the entire image with less intensive computational requirement. The two-step approach improves the overall computation speed and works together with these image preprocessing steps to achieve better registration accuracy than a regular single step registration. The authors evaluated the procedure on multiple image datasets from prostate cancer patients and gynecological cancer patients. Compared to rigid alignment, the proposed method improves volume matching by over 60% for the critical organs and reduces the prostate landmark registration errors by 50%.

Establishing a Normative Atlas of the Human Lung: Intersubject Warping and Registration of Volumetric CT Images

Rationale and Objectives To establish the range of normal values for quantitative CT-based measures of lung structure and function, the authors developed a method for matching pulmonary structures across individuals and creating a normative human lung atlas. Materials and Methods A computerized human lung atlas was synthesized from computed tomographic (CT) images from six subjects by means of three-dimensional image registration. The authors identified a set of reproducible feature points for each CT image and used these points to establish correspondences across subjects, used a landmark- and intensity-based consistent image registration algorithm to register a template image volume from the population to the rest of the pulmonary CT volumes in the population, averaged these transformations, and constructed an atlas by deforming the template with the average transformation. Results The effectiveness of the authors' method was evaluated and visualized by means of both gray-level and segmented CT images. The method reduced the average landmark registration error from 10.5 mm to 0.4 mm and the average relative volume overlap error from 0.7 to 0.11 for the six data sets studied. Conclusion The method, and the computerized human lung atlas constructed and visualized by the authors with this method, provides a basis for establishing regional ranges of normative values for structural and functional measures of the human lung.