Image-based Registration Research Articles

Assessment of image registration accuracy in three‐dimensional transrectal ultrasound guided prostate biopsy

Prostate biopsy, performed using two-dimensional (2D) transrectal ultrasound (TRUS) guidance, is the clinical standard for a definitive diagnosis of prostate cancer. Histological analysis of the biopsies can reveal cancerous, noncancerous, or suspicious, possibly precancerous, tissue. During subsequent biopsy sessions, noncancerous regions should be avoided, and suspicious regions should be precisely rebiopsied, requiring accurate needle guidance. It is challenging to precisely guide a needle using 2D TRUS due to the limited anatomic information provided, and a three-dimensional (3D) record of biopsy locations for use in subsequent biopsy procedures cannot be collected. Our tracked, 3D TRUS-guided prostate biopsy system provides additional anatomic context and permits a 3D record of biopsies. However, targets determined based on a previous biopsy procedure must be transformed during the procedure to compensate for intraprocedure prostate shifting due to patient motion and prostate deformation due to transducer probe pressure. Thus, registration is a critically important step required to determine these transformations so that correspondence is maintained between the prebiopsied image and the real-time image. Registration must not only be performed accurately, but also quickly, since correction for prostate motion and deformation must be carried out during the biopsy procedure. The authors evaluated the accuracy, variability, and speed of several surface-based and image-based intrasession 3D-to-3D TRUS image registration techniques, for both rigid and nonrigid cases, to find the required transformations. Our surface-based rigid and nonrigid registrations of the prostate were performed using the iterative-closest-point algorithm and a thin-plate spline algorithm, respectively. For image-based rigid registration, the authors used a block matching approach, and for nonrigid registration, the authors define the moving image deformation using a regular, 3D grid of B-spline control points. The authors measured the target registration error (TRE) as the postregistration misalignment of 60 manually marked, corresponding intrinsic fiducials. The authors also measured the fiducial localization error (FLE), the effect of segmentation variability, and the effect of fiducial distance from the transducer probe tip. Lastly, the authors performed 3D principal component analysis (PCA) on the x, y, and z components of the TREs to examine the 95% confidence ellipsoids describing the errors for each registration method. Using surface-based registration, the authors found mean TREs of 2.13 +/- 0.80 and 2.09 +/- 0.77 mm for rigid and nonrigid techniques, respectively. Using image-based rigid and non-rigid registration, the authors found mean TREs of 1.74 +/- 0.84 and 1.50 +/- 0.83 mm, respectively. Our FLE was 0.21 mm and did not dominate the overall TRE. However, segmentation variability contributed substantially approximately50%) to the TRE of the surface-based techniques. PCA showed that the 95% confidence ellipsoid encompassing fiducial distances between the source and target registra- tion images was reduced from 3.05 to 0.14 cm3, and 0.05 cm3 for the surface-based and image-based techniques, respectively. The run times for both registration methods were comparable at less than 60 s. Our results compare favorably with a clinical need for a TRE of less than 2.5 mm, and suggest that image-based registration is superior to surface-based registration for 3D TRUS-guided prostate biopsies, since it does not require segmentation.

Direct Estimation of Nonrigid Registrations with Image-Based Self-Occlusion Reasoning

The registration problem for images of a deforming surface has been well studied. External occlusions are usually well handled. In 2D image-based registration, self-occlusions are more challenging. Consequently, the surface is usually assumed to be only slightly self-occluding. This paper is about image-based nonrigid registration with self-occlusion reasoning. A specific framework explicitly modeling self-occlusions is proposed. It is combined with an intensity-based, "direct" data term for registration. Self-occlusions are detected as shrinkage areas in the 2D warp. Experimental results on several challenging data sets show that our approach successfully registers images with self-occlusions while effectively detecting the self-occluded regions.

Direct Estimation of Nonrigid Registrations with Image-Based Self-Occlusion Reasoning

The registration problem for images of a deforming surface has been well studied. External occlusions are usually well handled. In 2D image-based registration, self-occlusions are more challenging....

Parallel computation of mutual information on the GPU with application to real-time registration of 3D medical images

Due to processing constraints, automatic image-based registration of medical images has been largely used as a pre-operative tool. We propose a novel method named sort and count for efficient parallelization of mutual information (MI) computation designed for massively multi-processing architectures. Combined with a parallel transformation implementation and an improved optimization algorithm, our method achieves real-time (less than 1 s) rigid registration of 3D medical images using a commodity graphics processing unit (GPU). This represents a more than 50-fold improvement over a standard implementation on a CPU. Real-time registration opens new possibilities for development of improved and interactive intraoperative tools that can be used for enhanced visualization and navigation during an intervention.

A computational framework for image-based constrained registration

Adding external knowledge improves the results for ill-posed problems. In this paper we present a new computational framework for image registration when adding constraints on the transformation. We demonstrate that unconstrained registration can lead to ambiguous and non-physical results. Adding appropriate constraints introduces prior knowledge and contributes to reliability and uniqueness of the registration. Particularly, we consider recently proposed locally rigid transformations and volume preserving constraints as examples.

White matter abnormalities revealed by diffusion tensor imaging in non-demented and demented HIV+ patients

HIV associated dementia (HAD) is the most advanced stage of central nervous system disease caused by HIV infection. Previous studies have demonstrated that patients with HAD exhibit greater cerebral and basal ganglia atrophy than non-demented HIV+ (HND) patients. However, the extent to which white matter is affected in HAD patients compared to HND patients remains elusive. This study is designed to address the potential white matter abnormalities through the utilization of diffusion tensor imaging (DTI) in both HND and HAD patients. DTI and T1-weighted images were acquired from 18 healthy controls, 21 HND and 8 HAD patients. T1 image-based registration was performed to 1) parcellate the whole brain white matter into major white matter regions, including frontal, parietal, temporal and occipital white matter, corpus callosum and internal capsule for statistical comparisons of the mean DTI values, and 2) warp all DTI parametric images towards the common template space for voxel-based analysis. The statistical comparisons were performed with four DTI parameters including fractional anisotropy (FA), mean (MD), axial (AD), and radial (RD) diffusivities. With Whitney U tests on the mean DTI values, both HND and HAD demonstrated significant differences from the healthy control in multiple white matter regions. In addition, HAD patients exhibited significantly elevated MD and RD in the parietal white matter when compared to HND patients. In the voxel-based analysis, widespread abnormal regions were identified for both HND and HAD patients, although a much larger abnormal volume was observed in HAD patients for all four DTI parameters. Furthermore, both region of interest (ROI) based and voxel-based analyses revealed that RD was affected to a much greater extent than AD by HIV infection, which may suggest that demyelination is the prominent disease progression in white matter.

Acquisition-related motion compensation for digital subtraction angiography

Subtraction methods in angiography are generally applied in order to enhance the visualization of blood vessels by eliminating bones and surrounding tissues from X-ray images. The main limitation of these methods is the sensitivity to patient movement, which leads to artifacts and reduces the clinical value of the subtraction images. In this paper we present a novel method for rigid motion compensation with primary application to road mapping, frequently used in image-guided interventions. Using the general concept of image-based registration, we optimize the physical position and orientation of the C-arm X-ray device, thought of as the rigid 3D transformation accounting for the patient movement. The registration is carried out using a hierarchical optimization strategy and a similarity measure based on the variance of intensity differences, which has been shown to be most suitable for fluoroscopic images. Performance evaluation demonstrated the capabilities of the proposed approach to compensate for potential intra-operative patient motion, being more resilient to the fundamental problems of pure image-based registration.

Inter-Image Statistics for 3D Environment Modeling

In this article we present a method for automatically recovering complete and dense depth maps of an indoor environment by fusing incomplete data for the 3D environment modeling problem. The geometry of indoor environments is usually extracted by acquiring a huge amount of range data and registering it. By acquiring a small set of intensity images and a very limited amount of range data, the acquisition process is considerably simplified, saving time and energy consumption. In our method, the intensity and partial range data are registered first by using an image-based registration algorithm. Then, the missing geometric structures are inferred using a statistical learning method that integrates and analyzes the statistical relationships between the visual data and the available depth on terms of small patches. Experiments on real-world data on a variety of sampling strategies demonstrate the feasibility of our method.

Ultrasound of the Fingers for Human Identification Using Biometrics

It was hypothesized that the use of internal finger structure as imaged using commercially available ultrasound (US) scanners could act as a supplement to standard methods of biometric identification, as well as a means of assessing physiological and cardiovascular status. Anatomical structures in the finger including bone contour, tendon and features along the interphalangeal joint were investigated as potential biometric identifiers. Thirty-six pairs of three-dimensional (3D) gray-scale images of second to fourth finger (index, middle and ring) data taken from 20 individuals were spatially registered using MIAMI-Fuse © software developed at our institution and also visually matched by four readers. The image-based registration met the criteria for matching successfully in 14 out of 15 image pairs on the same individual and did not meet criteria for matching in any of the 12 image pairs from different subjects, providing a sensitivity and specificity of 0.93 and 1.00, respectively. Visual matching of all image pairs by four readers yielded 96% successful match. Power Doppler imaging was performed to calculate the change in color pixel density due to physical exercise as a surrogate of stress level and to provide basic physiological information. (E-mail: gnarayan@umich.edu)

SU-FF-J-116: Image Registration for Change Detection and Quatification in Multimodality Breast Tomosynthesis and Ultrasound

Purpose: To evaluate tomosynthesis‐to‐tomosynthesis and ultrasound‐to‐ultrasound image based registration (IBaR) of whole breast image volumes acquired at different times. These two modalities are probably the most immediately promising for routine breast cancer screening and diagnosis. Successful IBaR should aid more rapid and detailed detectio of change in response to treatment or tumor growth over time. Method and Materials: A system combining automated whole breast ultrasound (ABU) and digital tomosynthesis mammography(DTM) is being utilized in several studies of breast mass classification and assessment of response to neoadjuvant chemotherapy. Much attention was given to making the breast stable for the duration of each exam with the breast compressed between mammographic‐style plates and with acoustic coupling for reproducible, large area scanning through the plates. Four DTM pairs and many ABU pairs were acquired after reasonable stability and coverage techniques were developed. Registration was performed with MIAMI Fuse™ nonrigid registration based on the mutual information cost function and thin plate spline interpolation.Results: Registration was successful on 5 of 8 recent ABU image volume pairs with a mean registration error (MRE) of 3±2 mm and on all 4 DTM pairs, MRE = 4 ± 2 mm, as determined from visually selected homologous points. The lowest DTM mean error was obtained from studies 1 year apart, with minimal change in the breast. Conclusion: Both ABU and DTM offer angle dependent artifacts, with registration being perhaps more difficult in ABU. The variable breast distortion during compression offers similar difficulties, probably greater than actual breast change. No special effort was directed toward reproducible positioning. These results suggest that, usually, it should be possible to display in the same slice of two separate studies, a breast mass of > 5 mm, or its preceding tissues. Conflict of Interest: Work performed in cooperation with xxx, Inc.

A quantitative comparison of two methods to correct eddy current-induced distortions in DT-MRI

Eddy current-induced geometric distortions of single-shot, diffusion-weighted, echo-planar (DW-EP) images are a major confounding factor to the accurate determination of water diffusion parameters in diffusion tensor MRI (DT-MRI). Previously, it has been suggested that these geometric distortions can be removed from brain DW-EP images using affine transformations determined from phantom calibration experiments using iterative cross-correlation (ICC). Since this approach was first described, a number of image-based registration methods have become available that can also correct eddy current-induced distortions in DW-EP images. However, as yet no study has investigated whether separate eddy current calibration or image-based registration provides the most accurate way of removing these artefacts from DT-MRI data. Here we compare how ICC phantom calibration and affine FLIRT (http://www.fmrib.ox.ac.uk), a popular image-based multi-modal registration method that can correct both eddy current-induced distortions and bulk subject motion, perform when registering DW-EP images acquired with different slice thicknesses (2.8 and 5 mm) and b-values (1000 and 3000 s/mm(2)). With the use of consistency testing, it was found that ICC was a more robust algorithm for correcting eddy current-induced distortions than affine FLIRT, especially at high b-value and small slice thickness. In addition, principal component analysis demonstrated that the combination of ICC phantom calibration (to remove eddy current-induced distortions) with rigid body FLIRT (to remove bulk subject motion) provided a more accurate registration of DT-MRI data than that achieved by affine FLIRT.

Interscan registration using navigator echoes

A common problem in clinical MRI is anatomic misalignment of imaging slices across successive examinations. This unnecessarily complicates the radiologic assessment of anatomic change over time on serial MRI studies. To address this problem, spherical navigator echoes, which can detect rigid body motion in all six degrees of freedom, were used to guide spatial location and orientation adjustments to an exam prescription to match the reference frame of images acquired in an earlier exam. An initial linear navigator echo is also necessary to effect coarse Z translation adjustments prior to fine six degrees of freedom adjustment with a spherical navigator echo. Results of this technique are presented for head image volumes of five volunteers. Each volunteer was imaged on two scanners. In all cases, the reference frame adjustments provided by the navigator echoes substantially improved the alignment of the latter exam and performed well compared to retrospective image-based registration.

Prospective registration of human head magnetic resonance images for reproducible slice positioning using localizer images.

To facilitate assessing brain tumor growth and progression of stroke lesions by reproducible slice positioning in human head magnetic resonance (MR) images, a method for prospective registration is proposed that adjusts the image slice position without moving the patient and with no additional scans. The gradient reference frame of follow-up examinations was adjusted to achieve the same image slice positioning relative to the patient as in the previous examination. The three-dimensional geometrical transformation parameters for the gradients were determined using two-dimensional image registration of three orthogonal localizer images. The method was developed and evaluated using a phantom with arbitrarily adjustable position. Feasibility for in vivo applications was demonstrated with brain MR imaging (MRI) of healthy volunteers. Standard retrospective registration was used for assessing the quality of the method. The accuracy of the realignment was 0.0 mm +/- 1.2 mm and -0.2 degrees +/- 0.9 degrees (mean +/- SD) in phantom experiments. In 10 examinations of volunteers, misalignments up to 49.2 mm and 21 degrees were corrected. The accuracy of the realignment after prospective registration was 0.1 mm +/- 1.5 mm and 0.2 degrees +/- 1.5 degrees. Image-based prospective registration using localizer images of the pre- and postexaminations is a robust method for reproducible slice positioning.

Evaluation of two methods for soft tissue registration in online CT guided prostate cancer radiotherapy

Our clinic is conducting a feasibility study for CT-guided hypo-fractionated treatment of prostate cancer. Patients will receive a CT scan on our cone-beam CT-equipped linac immediately prior to treatment, the displacement of the prostate gland with respect to its planning position will be determined, and a correction will be applied accordingly. The purpose of this study is to evaluate two methods of performing the soft tissue registration. To simulate the entire course of the treatment we used serial CT data from 28 prostate cancer patients. In addiction to the planning CT, each patient had a pelvic CT scan (3 mm slice thickness) bi-weekly over the entire course of the treatment for a total of 15 CTs. Using a commercial treatment planning system, each daily CT was fused with the planning CT based on bony anatomy. This step eliminates the setup error component of the inter-fraction geometric variation. The prostate was contoured on each scan by a single physician. Subsequently prostatic image-based registration (IBR) was performed. This was accomplished by translating each daily prostate CT image to match the reference CT prostate contour, allowing sagittal rotation only. The registration was then repeated using only the contour of the daily prostate (CBR). A single investigator performed the two types of registration for all 28 patients. Each registration induces a transformation (Δdx,Δdy,Δdz,) that characterizes the motion of the daily prostate per a rigid body formalism, where Θ is the rotation in the sagittal plane. We assessed the IBR accuracy and stability by comparing each parameter of the 420 transformations between IBR and CBR. The average time to perform the correction was on the order of 3 minutes for the IBR and 1 minute for the CBR. While both techniques have uncertainties, we adopted the CBR as a reference both because there is a greater body of experience in contour definition than in soft tissue registration, and because there is less ambiguity in performing the CBR. The average shift magnitude from the AP perspective was 0.18 ± 0.18cm for CBR and 0.12 ± 0.13cm for IBR. From the lateral perspective, the results for the CBR and IBR are 0.37 ± 0.30cm and 0.25 ± 0.25cm, respectively. The analysis shows a larger shift in the AP direction then in any other direction, both in the IBR and CBR, a result consistent with previous studies. The average rotation angle, allowed only in the sagittal view, was 0.7o ± 2.0o for the CBR and 1.3o ± 2.4o for the IBR. The per-patient average discrepancy (Δdx,Δdy,Δdz) between the CBR and IBR results varies with perspective. While the average difference in the LAT direction is within 1mm for all but 6 patients, the results for the AP and SI directions have considerably more variation between patients. Furthermore, a systematic bias of more than 1mm is observed in the AP direction for 20 of the patients; in particular, the IBR indicates a larger posterior shift of the patient than does the CBR. The composite result over all patients reiterates these observations: (Δdx,Δdy,Δdz) =( 0.04 ± 0.12, 0.16 ± 0.22, 0.0 ± 0.26) cm. The CBR-IBR agreement was within 2mm for 69% of results, between 2 and 4mm for 23%, and greater than 4mm for 8%. The CBR was faster and easier to perform. The IBR showed an overestimation of the sagittal rotation angle and an underestimation of the shift magnitude for both the AP and LAT perspectives when compared to the CBR. The discrepancy between IBR and CBR is a measure of the intraobserver uncertainty of the online CT based correction. Due to the potential for significant discrepancies between the CBR and IBR results, we strongly recommend performing offline CBR verification of an online IBR-based correction. The observed IBR-CBR gap will possibly be narrowed by the use of on board KV conebeam device capable of good quality images in the coronal and sagittal views

Anatomical image-based rigid registration between fluoroscopic X-ray and CT: methods comparison and experimental results

This paper describes a practical, whole system approach to rigid registration between preoperative CT and intraoperative X-rays obtained with a tracked C-arm and demonstrates its efficacy experimentally. Our approach is generic, targeted to orthopaedic surgery, and includes fluoroscopic X-ray calibration, image distortion correction, optical tracking, and 2D/3D image registration. For 2D/3D image registration, we use two new gradient-based and intensity-based algorithms which are fast, robust, and accurate in realistic setups. Our experiments on simulated, in vitro, and cadaver show that an overall mean target registration error of 1–1.5 mm (2 mm worst case), which succeeds on the first try 95% or more of the time in less than 2 min, is practically feasible.

Correction of probe pressure artifacts in freehand 3D ultrasound

We present an algorithm which combines non-rigid image-based registration and conventional position sensing to correct probe-pressure-induced registration errors in freehand three-dimensional (3D) ultrasound volumes. The local accuracy of image-based registration enables the accurate freehand acquisition of high resolution (>15 MHz) 3D ultrasound data, opening the way for 3D musculoskeletal examinations. External position sensor readings guarantee the large-scale positional accuracy of the data. Pressure correction is shown to dramatically increase the perceived quality of extended-field-of-view data sets and reslices through volumetric data sets, while quantitative comparisons of multiple in vivo volumes demonstrate the superior precision of the corrected data.

Comparative In Vitro Study of Contact-and Image-Based Rigid Registration for Computer-Aided Surgery

We present an in vitro study of rigid registration methods for computer-aided surgery. The goals of the study were to obtain accuracy measures empirically under optimal laboratory conditions, and to identify the weak links in the registration chain. Specifically, we investigated two common registration methods (contact-based registration and image-based landmark registration) and established a framework for comparing the accuracy of both methods. The phantoms, protocols, and algorithms for tool tip calibration, contact-based registration with an optical tracker, fluoroscopic X-ray camera calibration, and fluoroscopic X-ray image-based landmark registration are described. Average accuracies of 0.5 mm (1.5 mm maximum) and 2.75 mm (3.4 mm maximum) were found for contact-based and image-based landmark registration, respectively. Based on these findings, the camera calibration was identified as being the main source of error in image-based landmark registration. Protocol improvements and algorithmic refinements to improve the accuracy of image-based landmark registration are proposed.

3D ultrasound measurement of large organ volume

Freehand 3D ultrasound is particularly appropriate for the measurement of organ volumes. For small organs, which can be fully examined with a single sweep of the ultrasound probe, the results are known to be much more accurate than those using conventional 2D ultrasound. However, large or complex shaped organs are difficult to quantify in this manner because multiple sweeps are required to cover the entire organ. Typically, there are significant registration errors between the various sweeps, which generate artifacts in an interpolated voxel array, making segmentation of the organ very difficult. This paper describes how sequential freehand 3D ultrasound, which does not employ an interpolated voxel array, can be used to measure the volume of large organs. Partial organ cross-sections can be segmented in the original B-scans, and then combined, without the need for image-based registration, to give the organ volume. The inherent accuracy (not including position sensor and segmentation errors) is demonstrated in simulation to be within ±2%. The in vivo precision of the complete system is demonstrated (by repeated observations of a human liver) to be ±5%.

System for Robotically Assisted Percutaneous Procedures with Computed Tomography Guidance

We present the prototype of an image-guided robotic system for accurate and consistent placement of percutaneous needles in soft-tissue targets under CT guidance inside the gantry of a CT scanner. The couch-mounted system consists of a seven-degrees-of-freedom passive mounting arm, a remote center-of-motion robot, and a motorized needle-insertion device. Single- image-based coregistration of the robot and image space is achieved by stereotactic localization using a miniature version of the BRW head frame built into the radiolucent needle driver. The surgeon plans and controls the intervention in the scanner room on a desktop computer that receives DICOM images from the scanner. The system does not need calibration, employs pure image-based registration, and does not utilize any vendor-specific hardware or software features. In the open air, where there is no needle-tissue interaction, we systematically achieved an accuracy better than 1 mm in hitting targets at 5-8 cm from the fulcrum point. In the phantom, the orientation accuracy was 0.6°, and the distance between the needle tip and the target was 1.04 mm. Experiments indicated that this robotic system is suitable for a variety of percutaneous clinical applications.

High-quality texture reconstruction from multiple scans

The creation of three-dimensional digital content by scanning real objects has become common practice in graphics applications for which visual quality is paramount, such as animation, e-commerce, and virtual museums. While a lot of attention has been devoted recently to the problem of accurately capturing the geometry of scanned objects, the acquisition of high-quality textures is equally important, but not as widely studied. In this paper, we focus on methods to construct accurate digital models of scanned objects by integrating high-quality texture and normal maps with geometric data. These methods are designed for use with inexpensive, electronic camera-based systems in which low-resolution range images and high-resolution intensity images are acquired. The resulting models are well-suited for interactive rendering on the latest-generation graphics hardware with support for bump mapping. Our contributions include new techniques for processing range, reflectance, and surface normal data, for image-based registration of scans, and for reconstructing high-quality textures for the output digital object.