Image Registration Results Research Articles

Error analysis of corner and center points for image registration

Image registration is an active research topic in the fields of remote sensing, photogrammetry, computer vision, pattern recognition, and medical image processing. Both corner and geometric center points are used for image registration, but not much attention has been paid to date to the differences in the outcome of image registration when these different types of points are used. Such differences have the potential to directly affect the success of multiresolution or multimodal image registration. The goal of this paper is to compare the characteristics of corner and center points in image registration and to quantify the differences in the registration accuracy when different features are used. Corners and geometric centers were compared in two cases: registration of images with the same resolution and registration of images with different resolutions. The results show that geometric centers provide more accurate results for image registration in both cases. Quantitative analysis revealed that the position error of the centers is only about 47%–63% of the position error of the corners. Experiments confirmed this finding.

KW-SIFT descriptor for remote-sensing image registration

A technique to construct an affine invariant descriptor for remote-sensing image registration based on the scale invariant features transform (SIFT) in a kernel space is proposed. Affine invariant SIFT descriptor is first developed in an elliptical region determined by the Hessian matrix of the feature points. Thereafter, the descriptor is mapped to a feature space induced by a kernel, and a new descriptor is constructed by whitening the mapped descriptor in the feature space, with the transform called KW-SIFT. In a final step, the new descriptor is used to register remote-sensing images. Experimental results for remote-sensing image registration indicate that the proposed method improves the registration performance as compared with other related methods.

Combination of intensity-based image registration with 3D simulation in radiation therapy

Modern techniques of radiotherapy like intensity modulated radiation therapy (IMRT) make it possible to deliver high dose to tumors of different irregular shapes at the same time sparing surrounding healthy tissue. However, internal tumor motion makes precise calculation of the delivered dose distribution challenging. This makes analysis of tumor motion necessary. One way to describe target motion is using image registration. Many registration methods have already been developed previously. However, most of them belong either to geometric approaches or to intensity approaches. Methods which take account of anatomical information and results of intensity matching can greatly improve the results of image registration. Based on this idea, a combined method of image registration followed by 3D modeling and simulation was introduced in this project. Experiments were carried out for five patients 4DCT lung datasets. In the 3D simulation, models obtained from images of end-exhalation were deformed to the state of end-inhalation. Diaphragm motions were around −25 mm in the cranial–caudal (CC) direction. To verify the quality of our new method, displacements of landmarks were calculated and compared with measurements in the CT images. Improvement of accuracy after simulations has been shown compared to the results obtained only by intensity-based image registration. The average improvement was 0.97 mm. The average Euclidean error of the combined method was around 3.77 mm. Unrealistic motions such as curl-shaped deformations in the results of image registration were corrected. The combined method required less than 30 min. Our method provides information about the deformation of the target volume, which we need for dose optimization and target definition in our planning system.

Wavelet-based image registration technique for high-resolution remote sensing images

Image registration is the process of geometrically aligning one image to another image of the same scene taken from different viewpoints at different times or by different sensors. It is an important image processing procedure in remote sensing and has been studied by remote sensing image processing professionals for several decades. Nevertheless, it is still difficult to find an accurate, robust, and automatic image registration method, and most existing image registration methods are designed for a particular application. High-resolution remote sensing images have made it more convenient for professionals to study the Earth; however, they also create new challenges when traditional processing methods are used. In terms of image registration, a number of problems exist in the registration of high-resolution images: (1) the increased relief displacements, introduced by increasing the spatial resolution and lowering the altitude of the sensors, cause obvious geometric distortion in local areas where elevation variation exists; (2) precisely locating control points in high-resolution images is not as simple as in moderate-resolution images; (3) a large number of control points are required for a precise registration, which is a tedious and time-consuming process; and (4) high data volume often affects the processing speed in the image registration. Thus, the demand for an image registration approach that can reduce the above problems is growing. This study proposes a new image registration technique, which is based on the combination of feature-based matching (FBM) and area-based matching (ABM). A wavelet-based feature extraction technique and a normalized cross-correlation matching and relaxation-based image matching techniques are employed in this new method. Two pairs of data sets, one pair of IKONOS panchromatic images from different times and the other pair of images consisting of an IKONOS panchromatic image and a QuickBird multispectral image, are used to evaluate the proposed image registration algorithm. The experimental results show that the proposed algorithm can select sufficient control points semi-automatically to reduce the local distortions caused by local height variation, resulting in improved image registration results.

SU-GG-J-09: A DICOM Screen Dumper That Links Tomotherapy Units and R&V Systems

Purpose: To present a DICOM screen dumper that captures 3D image registration results of MVCT vs. KVCT on Tomotherapy delivery console and sends them to R&V system to make remote image reviewing feasible. Method and Materials: Helical Tomotherapy is a true IGRT modality. Its on-line imaging feature allows us to produce patient's position accurately by means of 3D image registration of MVCT vs. KVCT prior to treatment so that inter-fraction motion is compensated. Unfortunately, this system lacks the feature to communicate with R&V systems due to its unique design. The image registration results are either approved by radiation oncologists at the console or on the printouts. To smooth the workflow, a software utility, ScreenBee, was developed for Tomotherapy system. Instead of acquiring entire 3D registration results and plan parameters that currently are not supported by any R&V system, it takes snapshots of either entire screen or a specific region, i.e., the graphical representations of 3D image registration and plan, encodes them in DICOM RT format and transfers them to R&V system. Results: It has been extensively tested and evaluated for about 8 months in our clinic. More than 2,000 images have been successfully sent to IMPAC. It also works on any computer with Microsoft Windows NT/2000/XP wherever the screen is desired to be seen and reviewed remotely in a timely fashion. So the treatment prescription and QA results can also be sent to R&V system if it is installed on the Tomotherapy planning workstation. Conclusion: The technical limitation on DICOM RT implementation of Tomotherapy system can be remedied by dedicated software without extra hardware capital investment. It helps radiation therapy clinics with cutting edge technologies move towards digital solutions so that the clinic workflows become more efficient and smooth with permanent records.

WE-E-AUD A-06: Image and Dose Processing for Image Guided Adaptive Radiation Therapy and Outcome Research

Purpose: To develop a general image processing and dose computation procedure that allows for the accumulation of accurate dose distributions for purposes of both outcome research and IG‐ART (image guided adaptive radiation therapy). We applied the procedure to cone‐beam CT head‐neck cancertreatment cases, as well as Tomotherapy MVCT (mega‐voltage CT) GYN and prostate cancertreatment cases. Method and Materials: The image processing procedures include preprocessing, rigid and deformable registration, mesh‐ based structure contour propagation, and image volume composition for dose re‐computation. The dose processing includes dose recomputation on the daily CT or the composed CT volume, deforming and register the daily dose to the planning dose space, dose accumulation, evaluating of the accumulated dose against the planned dose. Accuracy of image registration is very important for the entire procedure because it determines the accuracy of all later subsequent steps. In order to improve the accuracy, we preprocessed the CTimages before the registration step by using various methods, including edge‐preserving smoothing, Gaussian lowpass smoothing, contrast enhancement, window‐level intensity transformation, and bowel gas pocket painting for abdominal regions. We used MI (mutual information) based algorithm for rigid image registration, and applied the Horn‐Schunck optical flow algorithm for deformable image registration. We used a mesh‐based algorithm for structure contour deformation, interpolation and smoothing. Results: By applying these preprocessing procedures we were able to achieve improved image registration results. The computed image deformation fields were then used to propagate the structure contours from the kVCT to the current daily CTimage, which could be used for dose deformation or re‐computation. Conclusion: We have developed a procedure of image processing and dose computation to support the accumulation of ‘true’ dose distributions. These methods could be used in both radiotherapy outcome research and adaptive radiotherapy applications. Partially supported by NIH R01 grant CA85181.

Development of a Novel Post-processing Treatment Planning Platform for 4D Radiotherapy

The aim of this study is to develop an Automatic Post-processing Tool for four-dimensional (4D) treatment planning (APT4D) that enables the user to perform some necessary procedures related to 4D treatment planning, such as automated image registration, automatic propagation of regions of interest, and dose distribution transformation. Demons-based deformable registrations were performed to map the moving phase images (such as the end-inhalation phase or 0% phase) to the reference phase (typically the end-exhalation fixed phase or 50% phase). Contours were automatically propagated into the moving phase using the image registration results. The dose distribution of each moving phase was transformed to the fixed phase and subsequently was summed as an average with equal weighting factor. To validate the application of APT4D utility, the 4D computed tomography (CT) images of a lung cancer patient and an abdominal cancer patient were acquired and resorted into ten respiratory phases. 4D plans based on the 4D CT images were developed. The correlation coefficient ranged from 0.992 to 0.999 for the re-sampled deformed moving phase image against the fixed phase image for the lung patient plan and from 0.977 to 0.999 for the abdominal patient plan. For all the organs, the match indices between the manual contours and automatic contour propagation results were around 0.92 to 0.95. The 4D composite dose-volume histogram showed dosimetric reductions for liver and kidneys in the high dose region. The APT4D adds automation, efficiency, and functionality, while integrating the whole process of 4D treatment planning.

TH‐D‐M100F‐08: Development and Evaluation of An Automatic Contour Propagation Method for 4D Radiotherapy

Purpose: To develop and evaluate automatic contour propagation method for 4D radiotherapy Method and Materials: 4D CT images of one lung cancer patient and dynamic phantom images were acquired and resorted into ten respiratory phases. Demons deformable registration was performed to find the deformation field from the end‐exhale fixed phase to moving phases such as the end‐inhale phase. Regions of interest (ROIs) were contoured in the fixed phase with the Pinnacle3 treatment planning system. The planar parallel contours were reconstructed as the 3D binary mask image data which has zero value outside and non‐zero value inside of the 3D outline of contouring. The 3D contour image data were transformed to another moving phase using the deformation field from the image registration result. On each 2D plane the binary image boundary was found to trace the closed intersection points. To assess the quality of the automatic contour propagation method, the contour for the moving phase was delineated manually. Visual inspection between the contour propagation and manual contour was performed as the qualitative evaluation. Two indices were used as the quantitative examination. One was the match index to indicate how well two contour sets match each other. The other was volume difference evaluation between two contour sets. Results: The demons deformable registration results show good alignment for both the phantom and the patient. The match index between manual and automatic contouring ranged from 0.96 to 0.99 for the dynamic phantom and from 0.95 to 0.99 for the lung patient. The volume differences were between 0.75 and 2.97 percent for the dynamic phantom and between 1.50 and 7.32 percent for the lung patient. Conclusions: The proposed automatic contour propagation method is simple and feasible without surface construction. It is a variable method for defining the ROIs for large amount of 4DCT data sets.

Automatic Target Localization and Verification for On-Line Image-Guided Stereotactic Body Radiotherapy of the Spine

To facilitate image-guided stereotactic body radiotherapy (IG-SBRT) of spinal and paraspinal tumors, the authors have developed an on-line image registration system for automated target localization and patient position verification with high precision. When rotations are present in a patient's daily setup position, a setup error of a few millimeters can be introduced in localization of the isocenter by using surrounding bony structures. This setup error not only will deteriorate the dose coverage of the tumor, more importantly it will overdose the spinal cord. To resolve this issue, the image registration program developed by the authors detects translational shifts as well as rotational shifts using 3D CT image registration. Unacceptable rotations were corrected by either repositioning the patient or adjusting the treatment couch that was capable of rotational corrections when such a couch was available for clinical use. One pair of orthogonal digitally reconstructed radiographs (DRR) were generated from the daily pretreatment CT scan to compare with the corresponding DRRs generated from the planning CT scan to confirm the target shift correction. After the patient's position was corrected a pair of orthogonal portal images were taken for final verification. The accuracy of the image registration result was found to be within 0.1 mm on a head and neck phantom. Target shifts of a fraction of a millimeter were readily visible in our DRR comparison and portal image verification. The time needed to complete the image registration and DRR comparison was about 3 minutes. An integrated system that combines a high-speed CT scanner and a linear accelerator was used for imaging and treatment delivery. Application of the program in actual IG-SBRT cases demonstrated that it was accurate, fast, and reliable. It serves as a useful tool for image-guided radiotherapy where high precision of target localization is required.

Fusion of color and infrared video for moving human detection

We approach the task of human silhouette extraction from color and thermal image sequences using automatic image registration. Image registration between color and thermal images is a challenging problem due to the difficulties associated with finding correspondence. However, the moving people in a static scene provide cues to address this problem. In this paper, we propose a hierarchical scheme to automatically find the correspondence between the preliminary human silhouettes extracted from synchronous color and thermal image sequences for image registration. Next, we discuss strategies for probabilistically combining cues from registered color and thermal images for improved human silhouette detection. It is shown that the proposed approach achieves good results for image registration and human silhouette extraction. Experimental results also show a comparison of various sensor fusion strategies and demonstrate the improvement in performance over non-fused cases for human silhouette extraction.

Multi-modal image registration using local frequency representation and computer-aided design (CAD) models

Given two images of roughly the same scene, image registration is the process of determining the transformation that nearly maps one image to another. While some efficient procedures have been developed lately, the registration of images acquired from sensors operating in different modalities is still a challenging problem. In general, such images have different gray level characteristics and the features in the two images to be registered are often not well preserved, rendering registration techniques such as those based on feature extraction and area correlation generally not efficient, and hence not feasible in all cases. In this paper, we propose a new algorithm for multi-sensor image registration based on using the local frequency representation of the images together with image representation of computer-aided design (CAD) models that permit region-of-interest-to-region-of-interest, ROI-to-ROI, space to solve image registration problem (instead of relying only on the captured images to solve image registration problem using image-to-image space). The key point underlying the proposed approach is the employment of local frequency representations of CAD models images to efficiently determine sets of matching points from the images to be registered, which in turn enables obtaining correspondence between these sets for estimating the transformation parameters. Performance evaluation results reported here indicate that the proposed technique is robust and yields promising results for multi-modal image registration.

TU-C-J-6B-05: A Novel Volume Subdivision-Based Algorithm to Register Respiration Phase-Correlated Lung CT Images

Purpose: To develop a deformable image registration algorithm that will allow for temporal interpolation of phase-correlated 3D CT data sets and test it on lung CT images. Method and Materials: We have developed a new deformable image registration algorithm that starts with a global rigid-body matching between two (volumetric) images, progressively subdivides one image, and refines the spatial matching locally for each of the subvolumes. When subvolumes reach a user-defined size, the algorithm interpolates all local solutions to generate a smooth deformation field. The above algorithm was applied to two lung CT image sets, each with three frames corresponding to end-inhalation (IN), mid-exhalation (MID) and end-exhalation (EX). These images were acquired in conjunction with a spirometer and using a spiral CT scanner. In each case, the IN image was registered to the MID image and the MID image to the EX image. An interactive multi-planar reformatting tool was used to evaluate the result of image registration quantitatively. For quantitative evaluation, reduction in root-mean squared (RMS) error was computed for each image pair before and after image registration. Results: Visual inspection indicated an improved matching of anatomical features following registration. For the four image registrations performed (two for each case), the RMS error reduced from 13.8 to 7.2, 19.9 to 12.7, 13.5 to 8.0, and 12.2 to 7.0, indicating improved spatial matching following image registration. Conclusions: We have developed a new deformable image registration and shown its application to gated lung CT images. The algorithm effectively aligned phase-correlated CT images. Full development of this deformable registration capability will allow tumor trajectory mapping and dose registration.

Surface Curvature-Based MR-PET Image Registration and Hybrid Hippocampus Modeling

The correlations between the anatomical shape of brain subsystems and brain diseases have been widely researched in order to diagnose and prevent the diseases. In order to study the anatomical and the metabolic correlations, a correct image registration between the anatomical image and the metabolic image is required. In this study, we present a multi-modality image registration based on surface distance and curvature information. The proposed scheme enhances the stability and accuracy of registration results. In the first step, we extract the surface voxels and object-centered coordinate systems from reference and test volume data sets, respectively. In order to guarantee a stable registration result that is independent of the initial position or direction of the test object, two object-centered coordinate systems are overlapped before the fine registration process. In the fine registration step, we minimize the cost function to be defined by the surface distance and surface curvature difference between reference and test objects. The proposed cost function enhanced registration accuracy, which was verified through the registration error ratio and 2D/3D visual inspection of the registration results. Furthermore, we suggest a reconstruction of a hybrid hippocampus model that includes anatomical and functional information using a multi-modality image registration result.

3D deformable image processing and integration

Medical images from different modalities, such as X-ray computerized tomography (CT), magnetic resonance imaging (MRI), single photon emission computed tomography (SPECT) and positron emission tomography (PET) are simultaneously used for clinical diagnosis and research. The intention to integrate medical images includes (1) complementing functional information from functional images with anatomical information from CT and MRI, (2) finding changes over time or under different conditions, or (3) integrating information taken from different imaging modalities. In this paper, we present a three-dimensional (3D) image deformation technique to integrate multi-modality medical images. The 3D optical flow (OF) estimation and thin-plate spline (TPS) transformation are applied on landmarks tracing and geometric deformation, respectively, for inter- and intra-subject medical image integration. The proposed technique is separated into two parts—coarse and fine registration. In the coarse registration, the 3D OF method iteratively estimates moving vectors between corresponding pixels in two image datasets. The desired landmarks are obtained according to the estimated vectors, which relate to the corresponding image dataset, estimated by 3D OF method. As the pairs of landmarks are obtained, these reference landmarks are input into the 3D TPS transformation matrix to generate desired deformation results of coarse image registration. The coarse registration result is applied for fine adjustment by 3D OF estimation by iterative multi-resolution procedures at different levels. To validate the integration performance, the normal brain templates, including MRI and PET volume images, are applied as standard datasets in this study, and the clinical physician selects the standard landmarks from the standard images used for 3D TPS deformation. The results show that RMSE between standard templates and deformation results of MRI and PET studies are less than 3% on average, respectively. In conclusion, the automatic landmarks tracing technique (using 3D OF) and image deformation (using TPS transformation) can minimize human interaction, and integrate medical images effectively.

Developing structural constraints for accurate registration of overlapping range images

Automatic image registration is an attractive and unresolved problem in the machine vision literature. This paper presents two novel structural constraints, namely proximity and closeness constraints, that improve both the accuracy and robustness of an existing motion consistency based algorithm (GICP) for automatic image registration. It also defines the conditions when such constraints are fired at specific points of the registration of two overlapping range images. The proximity constraint says that neighbouring points should be neighbouring before and after a rigid motion. The closeness constraint implements a local closest point search from the second image to the first image. While the GICP algorithm uses an exhaustive search for the closest points, in this paper, we employ the optimised k– d tree data structure to accelerate the closest point search. A large number of experiments based on real range images demonstrate that the combination of rigid motion constraints with these novel proximity and closeness constraints leads to a more accurate and robust evaluation of possible correspondences leading, in turn, to more accurate, robust and efficient automatic image registration results.

LIQUID CRYSTAL PHOTOSENSITIVE CELLS AS A MEDIUM FOR REAL-TIME DIGITAL PROJECTED HOLOGRAMS

The possibility of diffraction pattern generation in pure and dye-doped LC (Liquid Crystal) cells as a media was proven. Newly developed projection method of optoelectronic reconstruction of holograms expands possible applications of liquid crystals in dynamic holography. The presented works goal is to check the physical and optical parameters of isothiocyanate nematic liquid crystal mixtures and cell construction in reference to projection set-up of hologram presentation. The experimental results of holographic image registration are discussed. Sample reconstructions of synthetic holograms are shown.

Objective assessment of image registration results using statistical confidence intervals

Precision of medical image registration is very important in clinical diagnosis and treatment, which is usually assessed by visual inspection or by referring to other methods that require special expertise and extensive experience. In this study, the authors proposed a novel automatic approach based on statistical theory to estimate confidence intervals of the registration parameters and allow the precision of registration results to be objectively assessed. Under the assumption of local linearity, statistical confidence intervals of model fitting (regression) can be used to evaluate registration precision. Monte Carlo simulations using the Hoffman brain phantom with various amounts of displacement, noise and spatial filtering were conducted to evaluate the formula for estimating the confidence intervals in 2D image registrations. Monte Carlo simulation results are consistent with the calculated confidence intervals, and the agreement is applicable to different amounts of translation, angular rotation and spatial smoothing. The estimated parameter values fall within the predicted 90%, 95% and 99% confidence intervals with less than /spl plusmn/1% of errors. The present results indicate that the use of statistical confidence intervals can provide an objective assessment of individual image registration results.

Image analysis in chemistry I. Properties of images, greylevel operations, the multivariate image

The use of imaging and images in chemical and technological applications is treated in two parts. Part I focuses on the generation and properties of images. The result of image registration is an analog or digital image, a representation of intensities as a function of position coordinates x, y (and sometimes also z) in a non-homogeneous material. Certain univariate operations for reducing noise and errors, improving contrast, measuring and counting particles etc. can be carried out on digitized images. A simple example from bioenergy research is given. In this first part a multivariate image is defined and an explanation is given of how it can be obtained. Operations on multivariate images and interpretations of multivariate results are presented in part II.