Image Registration Approach Research Articles

Automatic Registration of Images With Inconsistent Content Through Line-Support Region Segmentation and Geometrical Outlier Removal.

The implementation of automatic image registration is still difficult in various applications. In this paper, an automatic image registration approach through line-support region segmentation and geometrical outlier removal is proposed. This new approach is designed to address the problems associated with the registration of images with affine deformations and inconsistent content, such as remote sensing images with different spectral content or noise interference, or map images with inconsistent annotations. To begin with, line-support regions, namely a straight region whose points share roughly the same image gradient angle, are extracted to address the issues of inconsistent content existing in images. To alleviate the incompleteness of line segments, an iterative strategy with multi-resolution is employed to preserve global structures that are masked at full resolution by image details or noise. Then, geometrical outlier removal is developed to provide reliable feature point matching, which is based on affine-invariant geometrical classifications for corresponding matches initialized by scale invariant feature transform. The candidate outliers are selected by comparing the disparity of accumulated classifications among all matches, instead of conventional methods which only rely on local geometrical relations. Various image sets have been considered in this paper for the evaluation of the proposed approach, including aerial images with simulated affine deformations, remote sensing optical and synthetic aperture radar images taken at different situations (multispectral, multisensor, and multitemporal), and map images with inconsistent annotations. Experimental results demonstrate the superior performance of the proposed method over the existing approaches for the whole data set.

Model-Based Sparse-to-Dense Image Registration for Realtime Respiratory Motion Estimation in Image-Guided Interventions.

Intra-interventional respiratory motion estimation is becoming a vital component in modern radiation therapy delivery or high intensity focused ultrasound systems. The treatment quality could tremendously benefit from more accurate dose delivery using real-time motion tracking based on magnetic-resonance (MR) or ultrasound (US) imaging techniques. However, current practice often relies on indirect measurements of external breathing indicators, which has an inherently limited accuracy. In this work, we present a new approach that is applicable to challenging real-time capable imaging modalities like MR-Linac scanners and 3D-US by employing contrast-invariant feature descriptors. We combine GPU-accelerated image-based realtime tracking of sparsely distributed feature points and a dense patient-specific motion-model for regularisation and sparse-to-dense interpolation within a unified optimization framework. We achieve highly accurate motion predictions with landmark errors of ≈ 1 mm for MRI (and ≈ 2 mm for US) and substantial improvements over classical template tracking strategies. Our technique can model physiological respiratory motion more realistically and deals particularly well with the sliding of lungs against the rib cage. Our model-based sparse-to-dense image registration approach allows for accurate and realtime respiratory motion tracking in image-guided interventions.

Accurate Segmentation and Registration of Skin Lesion Images to Evaluate Lesion Change.

Skin cancer is a major health problem. There are several techniques to help diagnose skin lesions from a captured image. Computer-aided diagnosis (CAD) systems operate on single images of skin lesions, extracting lesion features to further classify them and help the specialists. Accurate feature extraction, which later on depends on precise lesion segmentation, is key for the performance of these systems. In this paper, we present a skin lesion segmentation algorithm based on a novel adaptation of superpixels techniques and achieve the best reported results for the ISIC 2017 challenge dataset. Additionally, CAD systems have paid little attention to a critical criterion in skin lesion diagnosis: the lesion's evolution. This requires operating on two or more images of the same lesion, captured at different times but with a comparable scale, orientation, and point of view; in other words, an image registration process should first be performed. We also propose in this work, an image registration approach that outperforms top image registration techniques. Combined with the proposed lesion segmentation algorithm, this allows for the accurate extraction of features to assess the evolution of the lesion. We present a case study with the lesion-size feature, paving the way for the development of automatic systems to easily evaluate skin lesion evolution.

An Image Registration Approach Based on 3D Geometric Projection Similarity of the Human Head

This study aims to develop a novel brain image registration algorithm, the geometry projection similarity (GPS) algorithm, which utilizes the anatomical feature of human head and simplifies the objective function for optimization and search for registration parameters in a step-by-step fashion, largely reducing the solution space and improving the efficiency. The algorithm was tested and compared with existing algorithms in simulation environment and tested on computed tomography/magnetic resonance imaging (CT/MRI) clinical datasets pre- and post-craniotomy surgery. Compared with the maximal mutual information algorithm, the GPS algorithm showed significantly smaller errors (1.2 ± 1.1 vs. 4.5 ± 3.5, 1.0 ± 1.0 vs. 3.5 ± 2.5, 0.8 ± 1.1 vs. 2.8 ± 2.4 pixels, at cluster size of 100, 150 and 200, respectively) and shorter computational durations. Compared with the iterative closest point algorithm, the GPS algorithm showed a greater robustness to the perturbed initial mismatch between registering images (registration failure rate 0 vs. 29 out of 200 trials), while keeping a satisfactorily low registration error (0.71 ± 0.83 pixels). Overall results demonstrated that registration accuracy, efficiency, and robustness of this new algorithm surpass those of existing methods. Pilot test on CT/MRI clinical datasets also showed a satisfactory outcome, demonstrating great potentials for future applications of registration of brain images across modalities.

Influence of deformable image registration on 4D dose simulation for extracranial SBRT: A multi-registration framework study

To evaluate the influence of deformable image registration approaches on correspondence model-based 4D dose simulation in extracranial SBRT by means of open source deformable image registration (DIR) frameworks. Established DIR algorithms of six different open source DIR frameworks were considered and registration accuracy evaluated using freely available 4D image data. Furthermore, correspondence models (regression-based correlation of external breathing signal measurements and internal structure motion field) were built and model accuracy evaluated. Finally, the DIR algorithms were applied for motion field estimation in radiotherapy planning 4D CT data of five lung and five liver lesion patients, correspondence model formation, and model-based 4D dose simulation. Deviations between the original, statically planned and the 4D-simulated VMAT dose distributions were analyzed and correlated to DIR accuracy differences. Registration errors varied among the DIR approaches, with lower DIR accuracy translating into lower correspondence modeling accuracy. Yet, for lung metastases, indices of 4D-simulated dose distributions widely agreed, irrespective of DIR accuracy differences. In contrast, liver metastases 4D dose simulation results strongly vary for the different DIR approaches. Especially in treatment areas with low image contrast (e.g. the liver), DIR-based 4D dose simulation results strongly depend on the applied DIR algorithm, drawing resulting dose simulations and indices questionable.

An Individualized Continuing Education Approach for Soft Tissue Prostate Image Registration

An Individualized Continuing Education Approach for Soft Tissue Prostate Image Registration

A novel image registration approach via combining local features and geometric invariants.

Image registration is widely used in many fields, but the adaptability of the existing methods is limited. This work proposes a novel image registration method with high precision for various complex applications. In this framework, the registration problem is divided into two stages. First, we detect and describe scale-invariant feature points using modified computer vision-oriented fast and rotated brief (ORB) algorithm, and a simple method to increase the performance of feature points matching is proposed. Second, we develop a new local constraint of rough selection according to the feature distances. Evidence shows that the existing matching techniques based on image features are insufficient for the images with sparse image details. Then, we propose a novel matching algorithm via geometric constraints, and establish local feature descriptions based on geometric invariances for the selected feature points. Subsequently, a new price function is constructed to evaluate the similarities between points and obtain exact matching pairs. Finally, we employ the progressive sample consensus method to remove wrong matches and calculate the space transform parameters. Experimental results on various complex image datasets verify that the proposed method is more robust and significantly reduces the rate of false matches while retaining more high-quality feature points.

Hybrid multiscale affine and elastic image registration approach towards wireless capsule endoscope localization

Abstract Wireless capsule endoscope (WCE) enables the visualization of the interior of the gastrointestinal (GI) tract. In particular it is very important for the examination of regions in the small bowel that cannot be reached by conventional endoscopy techniques. However, when an abnormality is found in WCE images of the small bowel, it is unknown how far is this abnormality from an anatomical reference point. The primary objective of the present paper is to give a contribution to WCE localization, using image-based methods. The main focus of this work is on the description of a hybrid multiscale affine and elastic image registration approach, its experimental application on WCE videos, and comparison with a multiscale affine registration. The proposed approach intends to track the WCE motion, by using the successive WCE frames that image the walls of the elastic small intestine. It includes registrations that capture both rigid-like and non-rigid deformations, due respectively to the rigid-like WCE movement and the elastic deformation of the small intestine originated by the GI peristaltic movement. Furthermore, the proposed approach enables the extraction of two parameters (scale and rotation) from which the relative displacement and orientation of the WCE inside the GI tract can be derived, via projective geometry. Under this approach an indicator of the WCE speed can be inferred, which can be clinically useful for video interpretation. The results of the experimental tests with real WCE video frames show the good performance of the proposed approach, when elastic deformations of the small intestine are involved in successive frames, and its superiority with respect to a multiscale affine image registration, which accounts for rigid-like deformations only and discards elastic deformations.

Image registration for InISAR based on joint translational motion compensation

The registration of inverse synthetic aperture radar (ISAR) images is a key process in interferometric ISAR (InISAR) imaging. According to published literature, there are mainly two solutions for ISAR image registration, one is correlation coefficient based method and the other is parameter estimation of target angular motion. Though some simulation results are effective, defects of these two methods still need to be studied and solved. Different from precedent methods, the authors try new approach for ISAR image registration in this study and better results are obtained. The authors' new approach is based on joint translational motion compensation, which consists of two steps namely joint range alignment and joint phase autofocus. The first step realises registration along range direction and the second one realises registration along cross-range direction. The new method achieves ISAR image registration along with the translational motion compensation, thus no extra computation is needed. In addition to high computational efficiency, the new method is more precise compared with precedent methods and works well even under strong noise. Simulation results show the advantages of the proposed method in computing efficiency, precision, robustness and practicability.

Remote sensing image registration based on Gaussian-Hermite moments and the Pseudo-RANSAC algorithm

ABSTRACTThis study deals with an important issue that is often encountered with the registration of remote sensing images which are obtained at different times and/or through inter/intra sensors. Remote sensing images may differ significantly in gray-level characteristics and contrast, among other aspects. Thus, it may be difficult to apply directly area-based approaches which are dependent on image intensity values. In this work, a novel image registration approach based on Gaussian-Hermite moments and the Pseudo-RANSAC algorithm is proposed. The problem of intensity difference commonly incurred in multi-temporal or multimodal remote sensing image registration is tackled using features that are invariant to intensity mapping during the feature point matching process. In particular, the feature points are herein represented by a range of newly introduced Gaussian-Hermite moments, and the corresponding feature points in a certain reference image are sought with the Euclidean distance measure. Moreover, an improved RANdom SAmple Consensus (RANSAC) algorithm is presented, reducing computational time complexity while improving performance in stability and accuracy. The final warping of images according to their refined feature points is conducted with bilinear interpolation. The proposed approach has been successfully applied to register synthetic and real remote sensing images, demonstrating its efficacy with systematic experimental evaluations.

Non-Rigid Contour-Based Registration of Cell Nuclei in 2-D Live Cell Microscopy Images Using a Dynamic Elasticity Model

The analysis of the pure motion of subnuclear structures without influence of the cell nucleus motion and deformation is essential in live cell imaging. In this paper, we propose a 2-D contour-based image registration approach for compensation of nucleus motion and deformation in fluorescence microscopy time-lapse sequences. The proposed approach extends our previous approach, which uses a static elasticity model to register cell images. Compared with that scheme, the new approach employs a dynamic elasticity model for the forward simulation of nucleus motion and deformation based on the motion of its contours. The contour matching process is embedded as a constraint into the system of equations describing the elastic behavior of the nucleus. This results in better performance in terms of the registration accuracy. Our approach was successfully applied to real live cell microscopy image sequences of different types of cells including image data that was specifically designed and acquired for evaluation of cell image registration methods. An experimental comparison with the existing contour-based registration methods and an intensity-based registration method has been performed. We also studied the dependence of the results on the choice of method parameters.

An experimental evaluation of the accuracy of keypoints-based retinal image registration.

This work regards an investigation of the accuracy of a state-of-the-art, keypoint-based retinal image registration approach, as to the type of keypoint features used to guide the registration process. The employed registration approach is a local method that incorporates the notion of a 3D retinal surface imaged from different viewpoints and has been shown, experimentally, to be more accurate than competing approaches. The correspondences obtained between SIFT, SURF, Harris-PIIFD and vessel bifurcations are studied, either individually or in combinations. The combination of SIFT features with vessel bifurcations was found to perform better than other combinations or any individual feature type, alone. The registration approach is also comparatively evaluated against representative methods of the state-of-the-art in retinal image registration, using a benchmark dataset that covers a broad range of cases regarding the overlap of the acquired images and the anatomical characteristics of the imaged retinas.

A level-set approach to joint image segmentation and registration with application to CT lung imaging

Automated analysis of structural imaging such as lung Computed Tomography (CT) plays an increasingly important role in medical imaging applications. Despite significant progress in the development of image registration and segmentation methods, lung registration and segmentation remain a challenging task. In this paper, we present a novel image registration and segmentation approach, for which we develop a new mathematical formulation to jointly segment and register three-dimensional lung CT volumes. The new algorithm is based on a level-set formulation, which merges a classic Chan–Vese segmentation with the active dense displacement field estimation. Combining registration with segmentation has two key advantages: it allows to eliminate the problem of initializing surface based segmentation methods, and to incorporate prior knowledge into the registration in a mathematically justified manner, while remaining computationally attractive. We evaluate our framework on a publicly available lung CT data set to demonstrate the properties of the new formulation. The presented results show the improved accuracy for our joint segmentation and registration algorithm when compared to registration and segmentation performed separately.

A stabilized multigrid solver for hyperelastic image registration

SummaryImage registration is a central problem in a variety of areas involving imaging techniques and is known to be challenging and ill‐posed. Regularization functionals based on hyperelasticity provide a powerful mechanism for limiting the ill‐posedness. A key feature of hyperelastic image registration approaches is their ability to model large deformations while guaranteeing their invertibility, which is crucial in many applications. To ensure that numerical solutions satisfy this requirement, we discretize the variational problem using piecewise linear finite elements, and then solve the discrete optimization problem using the Gauss–Newton method. In this work, we focus on computational challenges arising in approximately solving the Hessian system. We show that the Hessian is a discretization of a strongly coupled system of partial differential equations whose coefficients can be severely inhomogeneous. Motivated by a local Fourier analysis, we stabilize the system by thresholding the coefficients. We propose a Galerkin‐multigrid scheme with a collective pointwise smoother. We demonstrate the accuracy and effectiveness of the proposed scheme, first on a two‐dimensional problem of a moderate size and then on a large‐scale real‐world application with almost 9 million degrees of freedom.

Dynamic elasticity model for inter-subject non-rigid registration of 3D MRI brain scans

Abstract The purpose of this paper is to present a new approach for inter-subject non-rigid registration of 3D MR brain images which could assist in automatic labeling of brain structures. A physical dynamic elasticity model (DEM) is developed which tends to represent the complex non-linear deformations as elastic waves. The transformation is governed by elastodynamics wave equation. The registration process ensues in a hierarchical fashion, thus reducing the risk of obtaining a local optimal transformation. Along with the correction of local misalignments, it also removes global shape differences without any prior initialization. The proposed scheme was compared against high ranking registration methods including: DROP, SyN, ART and DRAMMS. The results were quantitatively analyzed by computing and testing the statistical significance of the volume overlap measures and Hausdorff distance for segmented structures with DROP, SyN, ART, DRAMMS and DEM registration methods. Experimental results demonstrate that the proposed DEM registration method leads to very promising results when applied to the problem of inter-subject registration and that favorably compares against classical registration approaches. Since DEM registration method is able to reduce registration errors significantly, hence it could be used to automatically label the anatomical structures for clinical studies.

A GPU-Accelerated Deformable Image Registration Algorithm With Applications to Right Ventricular Segmentation

Delineation of the cardiac right ventricle is essential in generating clinical measurements such as ejection fraction and stroke volume. Given manual segmentation on the first frame, one approach to segment right ventricle from all of the magnetic resonance images is to find point correspondence between the sequence of images. Finding the point correspondence with non-rigid transformation requires a deformable image registration algorithm, which often involves computationally expensive optimization. The central processing unit (CPU)-based implementation of point correspondence algorithm has been shown to be accurate in delineating organs from a sequence of images in recent studies. The purpose of this study is to develop computationally efficient approaches for deformable image registration. We propose a graphics processing unit (GPU) accelerated approach to improve the efficiency. The proposed approach consists of two parallelization components: Parallel compute unified device architecture (CUDA) version of the deformable registration algorithm; and the application of an image concatenation approach to further parallelize the algorithm. Three versions of the algorithm were implemented: 1) CPU; 2) GPU with only intra-image parallelization (sequential image registration); and 3) GPU with inter and intra-image parallelization (concatenated image registration). The proposed methods were evaluated over a data set of 16 subjects. CPU, GPU sequential image, and GPU concatenated image methods took an average of 113.13, 16.50, and 5.96 s to segment a sequence of 20 images, respectively. The proposed parallelization approach offered a computational performance improvement of around $19\times $ in comparison to the CPU implementation while retaining the same level of segmentation accuracy. This paper demonstrated that the GPU computing could be utilized for improving the computational performance of a non-rigid image registration algorithm without compromising the accuracy.

Bag-of-Spatial Words(BoSW)Framework for Predicting SAR Image Registration in Real Time Applications

SAR Image registration is a precursor for several remote sensing applications, which need precise spatial transformation between the real time moving image and fixed off-line image. In such applications, the processing time in finding whether the moving images can be registered with fixed image constitute an overhead. Hence we have approached the problem by trying to predict if the given SAR images can be registered or not even without registering them. The proposed image registration approach incorporates a classifier into the standard pipeline of feature based image registration. The attributes for the classifier model are derived from fusing the spatial parameters of the feature detector to the descriptor vector in bag of visual words framework.

A hybrid approach for brain image registration with local constraints

A hybrid approach for brain image registration with local constraints

Robust image alignment for cryogenic transmission electron microscopy

Cryo-electron microscopy recently experienced great improvements in structure resolution due to direct electron detectors with improved contrast and fast read-out leading to single electron counting. High frames rates enabled dose fractionation, where a long exposure is broken into a movie, permitting specimen drift to be registered and corrected. The typical approach for image registration, with high shot noise and low contrast, is multi-reference (MR) cross-correlation. Here we present the software package Zorro, which provides robust drift correction for dose fractionation by use of an intensity-normalized cross-correlation and logistic noise model to weight each cross-correlation in the MR model and filter each cross-correlation optimally. Frames are reliably registered by Zorro with low dose and defocus. Methods to evaluate performance are presented, by use of independently-evaluated even- and odd-frame stacks by trajectory comparison and Fourier ring correlation. Alignment of tiled sub-frames is also introduced, and demonstrated on an example dataset. Zorro source code is available at github.com/CINA/zorro.

A Two-Step Approach for Longitudinal Registration of Retinal Images.

This paper presents a novel two step approach for longitudinal (over time) registration of retinal images. Longitudinal registration is an important preliminary step to analyse longitudinal changes on the retina including disease progression. While potential overlap and minimal geometric distortion are likely in longitudinal images, identification of reliable features over time is a potential challenge for longitudinal registration. Relying on the widely accepted phenomenon that retinal vessels are more reliable over time, the proposed method aims to accurately match bifurcation and cross-over points between different timestamp images. Binary robust independent elementary features (BRIEF) are computed around bifurcation points which are then matched based on Hamming distance. Prior to computing BRIEF descriptors, a preliminary registration is performed relying on SURF key-point matching. Experiments are conducted on different image datasets containing 109 longitudinal image pairs in total. The proposed method has been found to produce accurate registration (i.e. registration with zero alignment error) for 97% cases, which is significantly higher than the other methods in comparison. The paper also reveals the finding that both the number and distributions of accurately matching key-points pairs are important for successful registration of image pairs.