Image Registration Applications Research Articles

Deformable image registration for contour propagation from CT to cone-beam CT scans in radiotherapy of prostate cancer

Background and purpose. Daily organ motion occurring during the course of radiotherapy in the pelvic region leads to uncertainties in the doses delivered to the tumour and the organs at risk. Motion patterns include both volume and shape changes, calling for deformable image registration (DIR), in approaches involving dose accumulation and adaptation. In this study, we tested the performance of a DIR application for contour propagation from the treatment planning computed tomography (pCT) to repeat cone-beam CTs (CBCTs) for a set of prostate cancer patients. Material and methods. The prostate, rectum and bladder were delineated in the pCT and in six to eight repeat CBCTs for each of five patients. The pCT contours were propagated onto the corresponding CBCT using the Multi-modality Image Registration and Segmentation application, resulting in 36 registrations. Prior to the DIR, a rigid registration was performed. The algorithm used for the DIR was based on a ‘demons’ algorithm and the performance of it was examined quantitatively using the Dice similarity coefficient (DSC) and qualitatively as visual slice-by-slice scoring by a radiation oncologist grading the deviations in shape and/or distance relative to the anatomy. Results. The average DSC (range) for the DIR over all scans and patients was 0.80 (0.65–0.87) for prostate, 0.77 (0.63–0.87) for rectum and 0.73 (0.34–0.91) for bladder, while the corresponding DSCs for the rigid registrations were 0.77 (0.65–0.86), 0.71 (0.55–0.82) and 0.64 (0.33–0.87). The percentage of propagated contours of good/acceptable quality was 45% for prostate; 20% for rectum and 33% for bladder. For the bladder, there was an association between the average DSC and the different scores of the qualitative evaluation. Conclusions. DIR improved the performance of pelvic organ contour propagation from the pCT to CBCTs as compared to rigid registration only. Still, a large fraction of the propagated rectum and bladder contours were unacceptable. The image quality of the CBCTs was sub-optimal and the usability of CBCTs for dose accumulation and adaptation purposes is therefore likely to benefit from improved image quality and improvements of the DIR algorithm.

SU-E-T-605: Validation of a GPU-Based Fast Deformable Image Registration Platform for 4D Treatment Planning Application

Purpose: Computational speed and geometric accuracy are important considerations in the application of deformable image registration (DIR) in radiotherapy. The goal was to use clinical data to validate for geometric accuracy, a DIR and atlas‐based auto‐segmentation (ABAS, CMS‐Elekta) platform programmable on a Graphics Processing Unit (GPU) which is capable of improving registration speeds up to 100 fold. Methods: We restricted our validation to intra‐patient registration application although the platform can be used for inter‐patient registration. Ten respiratory phase‐ dependent 3DCT image‐sets per patient for 9 patients were used. Demon's algorithm, as implemented in the research release of ABAS, was used to propagate clinician‐contoured regions of interests (ROIs) from the reference phase, chosen as the exhale, to the rest of the phases via deformation vector fields (DVFs) generated by deformably registering each image‐set to the reference. These ROIs shall be called DVF‐generated ROIs. Using the same window/level setting, clinician‐contoured ROIs were created on the other phases. To compare two ROIs for overlap, we defined an overlap index (OI) as the ratio of overlapping volume to the overlapping plus non‐overlapping volume, a related but more conservative index compared to the commonly used dice coefficient (overlapping/average volume). Results: The average OI computed for the DVF‐generated ROIs and the clinician‐contoured ROIs on the inhale phase across all patients were tumor (80% +/−6.3%); esophagus (72.0% +/−5.7%); heart (87.1% +/−3.9%); liver (90.6% +/−1.2%); cord (77.1% +/−5.9%); right lung (94.5% +/−1.5%); left lung (94.6% +/− 1.7%). Further qualitative analysis via visual inspection by the same clinician resulted in the verification of the DVF‐generated ROIs across all phases with a total of 592 out of 594 ROIs receiving satisfactory accuracy scores. Conclusions: This work verifies that in addition to high registration speed capability, the ABAS‐platform programmed on the GPU can achieve accuracy levels required for intra‐patient image registration related applications.

Application of Elastic Image Registration and Refraction Correction for Non‐Contact Underwater Strain Measurement

Abstract: The refraction‐induced image distortion introduces large errors in the deformation measurement of fluid submerged specimens using digital image correlation (DIC). This study provides a review of the nature of the refraction‐induced image distortion, assesses experimental conditions that interact with refraction and proposes an elastic image registration technique to correct the refraction distortion of underwater images. In the elastic image registration technique, control points are selected on reference and refracted images of a template object and locally sensitive transformation functions that overlay the two images are obtained. The transformation functions so obtained are then used to reconstruct undistorted images from underwater images and the former are used as input to a DIC system. The proposed approach has shown to improve the refraction error in the order of 5–8% for typical material test samples undergoing deformation inside a water‐filled glass chamber.

The best-so-far selection in Artificial Bee Colony algorithm

The Artificial Bee Colony (ABC) algorithm is inspired by the behavior of honey bees. The algorithm is one of the Swarm Intelligence algorithms explored in recent literature. ABC is an optimization technique, which is used in finding the best solution from all feasible solutions. However, ABC can sometimes be slow to converge. In order to improve the algorithm performance, we present a modified method for solution update of the onlooker bees in this paper. In our method, the best feasible solutions found so far are shared globally among the entire population. Thus, the new candidate solutions are more likely to be close to the current best solution. In other words, we bias the solution direction toward the best-so-far position. Moreover, in each iteration, we adjust the radius of the search for new candidates using a larger radius earlier in the search process and then reduce the radius as the process comes closer to converging. Finally, we use a more robust calculation to determine and compare the quality of alternative solutions. We empirically assess the performance of our proposed method on two sets of problems: numerical benchmark functions and image registration applications. The results demonstrate that the proposed method is able to produce higher quality solutions with faster convergence than either the original ABC or the current state-of-the-art ABC-based algorithm.

Propagation of target and organ at risk contours in radiotherapy of prostate cancer using deformable image registration

Background. Successful deformable image registration is an essential component of both dose accumulation and plan adaptation in radiotherapy. The aim of this study was to evaluate the performance of a deformable image registration application for propagation of contours using repeat CT scans of the pelvis, a region where considerable deformations are expected. Material and methods. The study involved four prostate cancer patients, each with 9–11 repeat CT scans. An oncologist contoured bladder, rectum, clinical target volume of pelvic lymph nodes (CTV-ln) and prostate (CTV-p) in all CT scans. The reference CT was retrospectively registered to the repeat CT scans with both rigid and deformable registration using a recently released commercial clinical software application. Two different diffusion-based ‘demons’ deformable registration algorithms were applied, differing in the amount of deformations being allowed, with algorithm A being more generous than algorithm B. The evaluation of the propagated structures included both quantitative measures and qualitative scoring. Results. We found the differences between the algorithms to be most evident for bladder and rectum. An increase in mean Dice similarity coefficient relative the rigid registrations of 12% and 13% was obtained with algorithm A for bladder and rectum, compared to 2% with algorithm B. For bladder the mean sensitivity and positive predictive value was 0.92 and 0.87 with algorithm A and 0.82 and 0.83 with algorithm B. Corresponding values for rectum was 0.81 and 0.76 with algorithm A and 0.75 and 0.69 with algorithm B. This translated into 57% and 26% passing the clinical evaluation for bladder and rectum, with algorithm A, compared to 17% and 14% with algorithm B. For CTV-ln and CTV-p both algorithms performed well by all measures, e.g. with 86% of the target structures passing the clinical evaluation. Conclusions. Deformable image registration improved contour propagation in the pelvis for all organs investigated. Differences in the performance of the algorithms were seen which became more pronounced for the highly deformable organs of bladder and rectum.

On the Suitability of SIFT Technique to Deal with Image Modifications Specific to Confocal Scanning Laser Microscopy

Computer vision tasks such as recognition and classification of objects and structures or image registration and retrieval can provide significant information when applied to microscopy images. Recently developed techniques for the detection and description of local features make the extraction and description of local image features that are invariant to various changes possible. The invariance and robustness of feature detection and description techniques play a key role in the design and implementation of object recognition, image registration, or image mosaicing applications. The scale-invariant feature transform (SIFT) technique is a widely used method for the detection, description, and matching of image features. In this article we present the results of our experiments regarding the repeatability of SIFT features, and to the precision of the SIFT feature matching, under image modifications specific to confocal scanning laser microscopy (CSLM). We have analyzed the behavior of SIFT while changing the pinhole aperture, photomultiplier gain, laser beam power, and electronic zoom. Our experiments, conducted on CSLM images, show that the SIFT technique is able to match detected key points between images acquired at different values of the acquisition parameters with good precision and represents a consistent tool for computer vision applications in CSLM.

SU-GG-I-112: Evaluation of Deformable Registration in CT Images with Limited Feature Visibility

Purpose: Applications of deformable image registration (DIR) often assume deformations to be correct everywhere in the patient volume of interest, but prior studies of DIR accuracy have been limited mostly to high‐contrast features in lung. This study examines DIR accuracy with respiration‐correlated CT (RCCT) images of software and physical deformable phantoms and with patient images in abdomen. Methods and Materials: DIR uses a fast free‐form algorithm, which minimizes an energy function consisting of pixel grayscale similarity and smoothness terms. We evaluate its accuracy in 3 types of RCCT images: an anthropomorphic software phantom that models diaphragmatic contraction and chest expansion and whose known voxel displacements provide ground truth; a physical deformable abdominal phantom with implanted radiopaque markers; and patient data in which the motion between end expiration (EE) and end inspiration (EI) images of small low‐contrast features in or near the liver are compared to rigid registration as an approximate ground truth. Results: The mean 3D vector EE‐to‐EI discrepancy between DIR and ground truth displacement in software phantom is 4.5 mm in lung and 5.0 mm in abdomen. This is larger than that observed by comparing visible features in images alone. Mean 3D vector discrepancy is 6.1mm in physical phantom. In both phantoms, larger discrepancy appears to correlate with regions that lack visible features. In patient images, visible features in liver show good agreement between DIR prediction and ground truth, with mean 3D vector EE‐to‐EI discrepancy of 2.0 mm. Conclusions: Visible internal features in CTimages are important for guiding deformable registration. Radiotherapy applications that assume DIR is accurate throughout the volume may underestimate errors in regions lacking visible features. Results in patient images show that reasonable results can be obtained for features in abdomen. Research supported by NIH/NCI grant R01CA126993.

A Spatial-Feature-Enhanced MMI Algorithm for Multimodal Airborne Image Registration

Maximization of mutual information (MI) (MMI) has a wide application in multimodal image registration. The conventional MMI algorithm is based on the statistics derived from image sampling and does not use spatial features. In this paper, we proposed an improved MMI registration algorithm that combines the spatial information through a feature-based selection mechanism. A Harris corner filter is chosen to classify the pixels. The classification results are used to focus sample selection, joint entropy estimation, and weighted MI information calculation. Although features play an important role, conventional feature matching is not used, which removes an important challenge to the use of feature-based information. The advantages of the proposed algorithm are improved robustness by incorporation of spatial features and significant reduction in computing time for registration. Experiments with challenging synthetic images and multimodal airborne infrared images are provided to demonstrate the improvement.

SU‐GG‐J‐154: Developing Multimodality Image Analysis Software for Target Definition and Therapy Response Assessment

Purpose: Multimodality functional imaging in cancer chemo‐radiotherapy holds promise for improved determination of tumor involvement in tissues and quantification of spatial and temporal changes in tumor pathophysiology in response to therapy. To address urgent needs for efficient analysis tools, we have developed an integrated software suite for retrospective research, prospectively evaluating the efficacy of functional imaging and facilitating streamlined data analysis for large‐scale clinical trials. Methods: Our software was developed using the MATLAB programming language version 7.6. The software runs on a Dell computer server with a 3‐GHz Intel Quad Core Xeon processor and 16‐gigabytes of memory. The software was designed to allow data exchange either directly with a patient image database or via a file system that organizes the data files hierarchically by institutional protocol, patient medical record number, imaging modality, study date, and series description. The spatial transformation matrix files generated in Siemens image registration applications can be read into our system and applied to image datasets in their original format. Results: This software allows simultaneous visualization of spatially registered images and derived parametric maps from various functional and anatomic imaging modalities at multiple time points. DICOM‐RT structure and dose files imported from treatment planning systems can be related to these multimodality images. Quantitative tools were implemented to perform image segmentation, histogram analysis of volumes of interest, and voxel‐wise correlation across modalities. Image and dosimetry data from example patients were processed to illustrate the clinical applicability of the software. Conclusion: We present an efficient image analysis software suite capable of simultaneously analyzing longitudinal patient images and radiation dose map with user‐defined volumes of interest or structures previously drawn for treatment planning. We hope this study illustrates well the unique needs of multimodality imaging in radiation therapy practice and encourages collaborative research between medical imaging sciences and radiation oncology.

A grid workflow environment for brain imaging analysis on distributed systems

AbstractScientific applications like neuroscience data analysis are usually compute and data‐intensive. With the use of the additional capacity offered by distributed resources and suitable middlewares, we can achieve much shorter execution time, distribute compute and storage load, and add greater flexibility to the execution of these scientific applications than we could ever achieve in a single compute resource. In this paper, we present the processing of image registration (IR) for functional magnetic resonance imaging studies on Global Grids. We characterize the application, list its requirements and then transform it to a workflow. We use Gridbus Broker and Gridbus Workflow Engine technologies for executing the neuroscience application on the Grid. We developed a complete web‐based portal integrating GUI‐based workflow editor, execution management, monitoring and visualization of tasks and resources. We describe each component of the system in detail. We then execute the application on Grid'5000 platform and present extensive performance results. We show that the IR application can have (1) significantly improved makespan, (2) distribution of compute and storage load among resources used, and (3) flexibility when executing multiple times on Grid resources. Copyright © 2009 John Wiley & Sons, Ltd.

Investigating Hologram‐Based Route Planning

AbstractIt is often assumed that three‐dimensional topographic maps provide more effective route planning, navigation, orientation, and way‐finding results than traditional two‐dimensional representations. The research reported here investigates whether three‐dimensional spatial mappings provide better support for route planning than two‐dimensional representations. In a set of experiments performed as part of this research, human subjects were randomly shown either a two‐ or three‐dimensional hologram of San Francisco and were asked to plan a bicycling route between an origin and a destination point. In a second task, participants used these holograms to identify the highest elevation point in the displayed area. The eye‐movements of the participants, throughout the process of looking at the geospatial holograms and executing the tasks, were recorded. The eye‐tracking metrics analysis indicates with a high statistical level of confidence that three‐dimensional holographic maps enable more efficient route planning. In addition, the research group is developing a new algorithm to analyze the differences between participant‐selected routes and a set of “good routes.” The algorithm employs techniques used to represent the boundary of objects and methods for assessing the difference between objects in modern digital image recognition, image registration, and image alignment applications. The overall goal is to create a theoretical framework for investigating and quantifying route planning effectiveness.

Uncertainty and sensitivity analysis for camera calibration

PurposeThe purpose of this paper is to present a unified approach to uncertainty and sensitivity analysis for camera calibration.Design/methodology/approachThe approach is based on the fact that camera calibration is a problem of parameter estimation and the parameters of interest are given by the optimal solution of a least‐squares problem.FindingsA system of linear equations relating the errors of the extracted feature points to the errors of the estimated parameters is derived, and the expression of the covariance matrix of the estimated parameters is given. Also, a system of linear equations characterizing the influence of the uncalibrated parameters on the calibrated ones is presented. Simulation results show that the camera's position and orientation are less sensitive to the lens distortion than the offset of the image center.Research limitations/implicationsThe models developed can be applied to optimize the layout of the calibration marks and design multiple‐stage calibration algorithm. The rationale can also find applications in image registration and robot calibration.Originality/valueA generic approach is proposed to perform uncertainty and sensitivity analysis for camera calibration. By using the optimality condition for an unconstrained optimization problem, two mathematical models are developed in a unified framework.

Robust detection of perceptually salient features on 3D meshes

Crest lines, curves on a surface along which the surface bends sharply, are powerful shape descriptors. Crest lines and their subsets have numerous applications in image analysis, face recognition, analysis and registration of anatomical structures, surface segmentation and non-photorealistic rendering. In this paper, a method is proposed for robust detection of crest lines. The proposed method is based on contextual information, the attributes of neighboring points. So it provides a basis of robustly detecting salient crest lines corresponding to potentially important features. Consequently, the algorithm is immune to noisy mesh and textured mesh with repeated bumps. Comparative results indicate that our algorithm yields favorable detection results and is effective.

Coherent Multiscale Image Processing Using Dual-Tree Quaternion Wavelets

The dual-tree quaternion wavelet transform (QWT) is a new multiscale analysis tool for geometric image features. The QWT is a near shift-invariant tight frame representation whose coefficients sport a magnitude and three phases: two phases encode local image shifts while the third contains image texture information. The QWT is based on an alternative theory for the 2-D Hilbert transform and can be computed using a dual-tree filter bank with linear computational complexity. To demonstrate the properties of the QWT's coherent magnitude/phase representation, we develop an efficient and accurate procedure for estimating the local geometrical structure of an image. We also develop a new multiscale algorithm for estimating the disparity between a pair of images that is promising for image registration and flow estimation applications. The algorithm features multiscale phase unwrapping, linear complexity, and sub-pixel estimation accuracy.

Model-based mapping of reconfigurable image registration on FPGA platforms

Image registration is a computationally intensive application in the medical imaging domain that places stringent requirements on performance and memory management efficiency. This paper develops techniques for mapping rigid image registration applications onto configurable hardware under real-time performance constraints. Building on the framework of homogeneous parameterized dataflow, which provides an effective formal model of design and analysis of hardware and software for signal processing applications, we develop novel methods for representing and exploring the hardware design space when mapping image registration algorithms onto configurable hardware. Our techniques result in an efficient framework for trading off performance and configurable hardware resource usage based on the constraints of a given application. Based on trends that we have observed when applying these techniques, we also present a novel architecture that enables dynamically-reconfigurable image registration. This proposed architecture has the ability to tune its parallel processing structure adaptively based on relevant characteristics of the input images.

A Theory of Automatic Parameter Selection for Feature Extraction With Application to Feature-Based Multisensor Image Registration

This paper generalizes the previously developed automated edge-detection parameter selection algorithm of Yitzhaky and Peli. We generalize the approach to arbitrary multidimensional, continuous or discrete parameter spaces, and feature spaces. This generalization enables use of the parameter selection approach with more general image features, for use in feature-based multisensor image registration applications. We investigate the problem of selecting a suitable parameter space sampling density in the automated parameter selection algorithm. A real-valued sensitivity measure is developed which characterizes the effect of parameter space sampling on feature set variability. Closed-form solutions of the sensitivity measure for special feature set relationships are derived. We conduct an analysis of the convergence properties of the sensitivity measure as a function of increasing parameter space sampling density. For certain parameter space sampling sequence types, closed-form expressions for the sensitivity measure limit values are presented. We discuss an approach to parameter space sampling density selection which uses the sensitivity measure convergence behavior. We provide numerical results indicating the utility of the sensitivity measure for selecting suitable parameter values.

The Use of the Hopfield Neural Network to Measure Sea-Surface Velocities From Satellite Images

The knowledge of ocean surface circulation is of major importance for many applications, including the understanding of global climate, resources exploitation, and containment of chemical spills. In this letter, sea-surface feature tracking based on the Hopfield neural network (NN) is described. The method is based on the minimization of an energy function that represents the feature tracking problem. A Hopfield NN is used to merge cross-correlation information with prior knowledge of sea-surface flows and image contextual information. It has been tested on real satellite images. A set of five Advanced Very High Resolution Radiometer thermal images of the coastal zone of California, along with a data set of coincident surface drifters positions, was used to test the method. Results of the new analysis are compared with in situ data and previous results using other techniques. The method can be used on various kinds of images for tracking and also find other applications in image registration and pattern recognition.

Accelerating reconstruction of reference digital tomosynthesis using graphics hardware

The successful implementation of digital tomosynthesis (DTS) for on-board image guided radiation therapy (IGRT) requires fast DTS image reconstruction. Both target and reference DTS image sets are required to support an image registration application for IGRT. Target images are usually DTS image sets reconstructed from on-board projections, which can be accomplished quickly using the conventional filtered backprojection algorithm. Reference images are DTS image sets reconstructed from digitally reconstructed radiographs (DRRs) previously generated from conventional planning CT data. Generating a set of DRRs from planning CT is relatively slow using the conventional ray-casting algorithm. In order to facilitate DTS reconstruction within a clinically acceptable period of time, we implemented a high performance DRR reconstruction algorithm on a graphics processing unit of commercial PC graphics hardware. The performance of this new algorithm was evaluated and compared with that which is achieved using the conventional software-based ray-casting algorithm. DTS images were reconstructed from DRRs previously generated by both hardware and software algorithms. On average, the DRR reconstruction efficiency using the hardware method is improved by a factor of 67 over the software method. The image quality of the DRRs was comparable to those generated using the software-based ray-casting algorithm. Accelerated DRR reconstruction significantly reduces the overall time required to produce a set of reference DTS images from planning CT and makes this technique clinically practical for target localization for radiation therapy.

2371: Clinical Application of Deformable Image Registration in Head and Neck Cancer Radiotherapy: Does Planned Dose to Tumor and Normal Structures Change During Treatment?

2371: Clinical Application of Deformable Image Registration in Head and Neck Cancer Radiotherapy: Does Planned Dose to Tumor and Normal Structures Change During Treatment?

SU‐FF‐J‐35: Clinical Applications of 3D and 4D Deformable Image Registration for Image Guided Radiotherapy

Purpose: To develop a framework of 3D and 4D deformable image registration and investigate its applications in image guided radiotherapy (IGRT). Method and Materials: We have developed 3D and 4D deformable image registration methods to register fan beam CT (FBCT) and cone beam CT (CBCT) images. Applications of the 3D and 4D deformable registration include: (1) Inter‐subject atlas‐based image segmentation to automatically contour normal tissue regions of interest (ROI) in treatment planning CT images; (2) Intra‐subject atlas‐based image segmentation to delineate daily ROIs by registering daily FBCT or CBCT images to treatment planning CT images. This method is an ideal ROI delineation method for online plan adjustments; (3) 4D deformable image registration to extract average voxel trajectories for real‐time tumor tracking and for 4D tomotherapy treatment planning; (4) Deformable fusion of 4D‐PET images, which is based on deformable registration of 4D‐CT images, to minimize motion blurring in PET images without increase of committed doses or data acquisition time. Results: The deformable image registration system was tested using clinical images. Registrations of inter‐ and intra‐subject images show acceptable results. The corresponding results are reported in our supporting document. Quantitative validation of intra‐subject atlas‐based image segmentation showed agreements with physicians' manual contours. 4D registration yields voxel trajectories smooth in both spatial and temporal spaces. Problems were noticed in the cases where the registration image pairs are intrinsically different, such as gas bubbles in rectums, or deformations are extremely large, such as empty vs. full bladders. Conclusions: We have developed a general framework of applying 3D & 4D deformable image registration in image guided radiotherapy. Our clinical applications showed the positive and convincing results.