Multi-modal Image Registration Algorithm Research Articles

Multimodal Image Registration Using a Viscous Fluid Model with the Bhattacharyya Distance.

Image registration is an elementary task in medical image processing and analysis, which can be divided into monomodal and multimodal. Direct 3D multimodal registration in volumetric medical images can provide more insight into the interpretation of subsequent image processing applications than 2D methods. This paper is dedicated to the development of a 3D multimodal image registration algorithm based on a viscous fluid model associated with the Bhattacharyya distance. In our approach, a modified Navier-Stoke's equation is exploited as the foundation of the multimodal image registration framework. The hopscotch method is numerically implemented to solve the velocity field, whose values at the explicit locations are first computed and the values at the implicit positions are solved by transposition. The differential of the Bhattacharyya distance is incorporated into the body force function, which is the main driving force for deformation, to enable multimodal registration. A variety of simulated and real brain MR images were utilized to assess the proposed 3D multimodal image registration system. Preliminary experimental results indicated that our algorithm produced high registration accuracy in various registration scenarios and outperformed other competing methods in many multimodal image registration tasks.Clinical Relevance- This facilitates the disease diagnosis and treatment planning that requires accurate 3D multimodal image registration without massive image data and extensive training regardless of the imaging modality.

Registration of multi-modal images under a complex background combining multiscale features extraction and semantic segmentation.

Multi-modal imaging technology has a very broad application value in target recognition and other fields, and image registration is one of its key technologies. In this paper, a multi-modal image registration algorithm that combines multiscale features extraction and semantic segmentation is proposed to achieve accurate registration of polarized images and near-infrared images under complex backgrounds. A classical convolutional neural network ResNet is employed to capture the robust feature descriptors, and a convolutional neural network with an attention mechanism is trained to filter out the irrelevant feature points. Further, the two multi-modal images can be further registered. The experimental results show the feasibility and effectiveness of the proposed method.

Multimodal Remote Sensing Image Registration Algorithm Based on a New Edge Descriptor

Image registration of multimodal remote sensing images plays a vital role in remote sensing image analysis. However, there are significant nonlinear intensity differences between multimodal remote sensing image pairs, making it difficult for most traditional image registration algorithms to meet the registration requirements. In this paper, we propose a novel edge descriptor utilizing edge information, which has not only affine invariance but is also insensitive to nonlinear intensity differences. Moreover, we utilize the proposed descriptor to design a multimodal image registration algorithm. We use several different multimodal image pairs to evaluate the proposed algorithm. The experimental results show that the proposed algorithm holds a stable performance and can still achieve accurate spatial alignment even with the huge nonlinear intensity differences.

Generation of annotated multimodal ground truth datasets for abdominal medical image registration

PurposeSparsity of annotated data is a major limitation in medical image processing tasks such as registration. Registered multimodal image data are essential for the diagnosis of medical conditions and the success of interventional medical procedures. To overcome the shortage of data, we present a method that allows the generation of annotated multimodal 4D datasets.MethodsWe use a CycleGAN network architecture to generate multimodal synthetic data from the 4D extended cardiac–torso (XCAT) phantom and real patient data. Organ masks are provided by the XCAT phantom; therefore, the generated dataset can serve as ground truth for image segmentation and registration. Realistic simulation of respiration and heartbeat is possible within the XCAT framework. To underline the usability as a registration ground truth, a proof of principle registration is performed.ResultsCompared to real patient data, the synthetic data showed good agreement regarding the image voxel intensity distribution and the noise characteristics. The generated T1-weighted magnetic resonance imaging, computed tomography (CT), and cone beam CT images are inherently co-registered. Thus, the synthetic dataset allowed us to optimize registration parameters of a multimodal non-rigid registration, utilizing liver organ masks for evaluation.ConclusionOur proposed framework provides not only annotated but also multimodal synthetic data which can serve as a ground truth for various tasks in medical imaging processing. We demonstrated the applicability of synthetic data for the development of multimodal medical image registration algorithms.

Combining Sparse and Dense Features to Improve Multi-Modal Registration for Brain DTI Images.

A new solution to overcome the constraints of multimodality medical intra-subject image registration is proposed, using the mutual information (MI) of image histogram-oriented gradients as a new matching criterion. We present a rigid, multi-modal image registration algorithm based on linear transformation and oriented gradients for the alignment of T2-weighted (T2w) images (as a fixed reference) and diffusion tensor imaging (DTI) (b-values of 500 and 1250 s/mm2) as floating images of three patients to compensate for the motion during the acquisition process. Diffusion MRI is very sensitive to motion, especially when the intensity and duration of the gradient pulses (characterized by the b-value) increases. The proposed method relies on the whole brain surface and addresses the variability of anatomical features into an image stack. The sparse features refer to corners detected using the Harris corner detector operator, while dense features use all image pixels through the image histogram of oriented gradients (HOG) as a measure of the degree of statistical dependence between a pair of registered images. HOG as a dense feature is focused on the structure and extracts the oriented gradient image in the x and y directions. MI is used as an objective function for the optimization process. The entropy functions and joint entropy function are determined using the HOGs data. To determine the best image transformation, the fiducial registration error (FRE) measure is used. We compare the results against the MI-based intensities results computed using a statistical intensity relationship between corresponding pixels in source and target images. Our approach, which is devoted to the whole brain, shows improved registration accuracy, robustness, and computational cost compared with the registration algorithms, which use anatomical features or regions of interest areas with specific neuroanatomy. Despite the supplementary HOG computation task, the computation time is comparable for MI-based intensities and MI-based HOG methods.

Anatomically-adaptive multi-modal image registration for image-guided external-beam radiotherapy

Image-guided radiotherapy (IGRT) allows observation of the location and shape of the tumor and organs-at-risk (OAR) over the course of a radiation cancer treatment. Such information may in turn be used for reducing geometric uncertainties during therapeutic planning, dose delivery and response assessment. However, given the multiple imaging modalities and/or contrasts potentially included within the imaging protocol over the course of the treatment, the current manual approach to determining tissue displacement may become time-consuming and error prone. In this context, variational multi-modal deformable image registration (DIR) algorithms allow automatic estimation of tumor and OAR deformations across the acquired images. In addition, they require short computational times and a low number of input parameters, which is particularly beneficial for online adaptive applications, which require on-the-fly adaptions with the patient on the treatment table.However, the majority of such DIR algorithms assume that all structures across the entire field-of-view (FOV) undergo a similar deformation pattern. Given that various anatomical structures may behave considerably different, this may lead to the estimation of anatomically implausible deformations at some locations, thus limiting their validity. Therefore, in this paper we propose an anatomically-adaptive variational multi-modal DIR algorithm, which employs a regionalized registration model in accordance with the local underlying anatomy. The algorithm was compared against two existing methods which employ global assumptions on the estimated deformations patterns.Compared to the existing approaches, the proposed method has demonstrated an improved anatomical plausibility of the estimated deformations over the entire FOV as well as displaying overall higher accuracy. Moreover, despite the more complex registration model, the proposed approach is very fast and thus suitable for online scenarios. Therefore, future adaptive IGRT workflows may benefit from an anatomically-adaptive registration model for precise contour propagation and dose accumulation, in areas showcasing considerable variations in anatomical properties.

Algorithm Selection in Multimodal Medical Image Registration

Over the past few decades, fast<strong>-</strong>growth has occurred in the area of medical image acquisition devices, and physicians now rely on the utilization of medical images for the diagnosis, treatment plans, and surgical guidance. Researchers have classified medical images according to two structures: anatomical and functional structures. Due to this classification, the data obtained from two or more images of the same object frequently provide complementary and more abundant information through a process known as multimodal medical model registration. Image registration is spatially mapping the coordinate system of the two images obtained from a different viewpoint and utilizing various sensors. Several automatic multimodal medical image registration algorithms have been introduced based on types of medical images and their applications to increase the reliability, robustness, and accuracy. Due to the diversity in imaging and the different demands<strong> </strong>for applications, there is no single registration algorithm that can do that. This paper introduces a novel method for developing a multimodal medical image registration system that can select the most accepted registration algorithm from a group of registration algorithms for a variety of input datasets. The method described here is based on a machine learning technique that selects the most promising candidate. Several experiments have been conducted, and the results reveal that the novel approach leads to considerably faster reliability, accuracy, and more robustness registration algorithm selection.

Evaluation Method for the Optimization of 3D Rigid Image Registration on Multimodal Image Datasets

Optimization based three dimensional (3D) rigid image registration (RIR) is one of the most commonly used methods of image registration in radiotherapy. Interpolator and similarity metric plays a crucial role in optimization image registration process. In this paper, the efficiency of image registration algorithm is analyzed by using various combinations of interpolators and similarity metric in terms of quantitative measures and is compared with commercially available image registration algorithm in radiotherapy. Computed Tomography (CT) and Cone Beam Computed Tomography (CBCT) image datasets were registered by image registration algorithm written in python language using simple image tool kit (SITK). Different combinations of similarity metric and interpolator such as mean square difference (MSD), mutual information (MI), demons and nearest neighbor (NN), linear, B- spline respectively were used in this study. The efficiency of the algorithm was quantified in terms of mean square error (MSE), structural similarity index (SSI), normalized cross correlation (NCC) and mutual information (MI). The image registration algorithm with most efficient combination of similarity metric and interpolator was selected for comparison with the commercially available image registration algorithm. The algorithm for multimodal (CT-CBCT) 3D image registration with NN interpolator and MI similarity metric showed the highest values of SSI, NCC and MI as 0.865, 0.933, 1.223 respectively among other combination of interpolator and similarity metric. Further this algorithm when compared and statistically analyzed with commercially available image registration algorithm of Treatment Planning System (TPS. most commonly used for radiotherapy treatment) resulted in no significant difference (F value NCC-3.18, MI-4.010, SSI-2.776) in their quantitative measures. The present study is limited to 3D RIR and can be extended for deformable image registration.

Fast Algorithm for Multimodal Image Registration with DTCWT

Objective: To perform image registration with sub band features computed using dual tree complex wavelets and reducing the computation time in registration. Methods/Statistical Analysis: The input images are preprocessed with Gaussian filters; features are extracted from dual tree sub bands. The dominant features are retained with feature selection process to minimize complexity in registration. Translation of images based on similar features is carried out using singular value decomposition. A refinement phase further registers the images considering similarities in features. The proposed algorithm is modeled in MATLAB, registration processes are modeled from fundamental equations. Test images are considered for validation. The test images are geometrically transformed with knows sets of translation, scale and rotational parameters. The two images are registered and the corresponding transformation parameters are compared. Critical points in the images are considered as reference points for validation. Findings: The proposed algorithm performs registration of all images with less than 10% error. The PSNR computed for all sets of data base is greater than 32dB. With dual tree sub bands, the unique features with 150, 450 and 750 orientations are captured thus the features are twice more compared with DWT features. The feature selection algorithm considers the orientations in all three directions thus improving registration process. SVD algorithm optimizes the complexity in transformation process and refinement phase. Elimination of false features and use of appropriate sub band features has reduced the registration time by 76% thus meeting the requirement for real time applications in micro air vehicles. Application/Improvements: The improvement in computation time and registration accuracy of this algorithm is suitable for MAVs moving with speed of 20-30kmph. Multilevel decomposition will further improve the registration process, hardware accelerators can reduce computation time to few milliseconds.

Fast Algorithm for Multimodal Image Registration with DTCWT

Fast Algorithm for Multimodal Image Registration with DTCWT

EVolution: an edge-based variational method for non-rigid multi-modal image registration

Image registration is part of a large variety of medical applications including diagnosis, monitoring disease progression and/or treatment effectiveness and, more recently, therapy guidance. Such applications usually involve several imaging modalities such as ultrasound, computed tomography, positron emission tomography, x-ray or magnetic resonance imaging, either separately or combined.In the current work, we propose a non-rigid multi-modal registration method (namely EVolution: an edge-based variational method for non-rigid multi-modal image registration) that aims at maximizing edge alignment between the images being registered. The proposed algorithm requires only contrasts between physiological tissues, preferably present in both image modalities, and assumes deformable/elastic tissues. Given both is shown to be well suitable for non-rigid co-registration across different image types/contrasts (T1/T2) as well as different modalities (CT/MRI). This is achieved using a variational scheme that provides a fast algorithm with a low number of control parameters. Results obtained on an annotated CT data set were comparable to the ones provided by state-of-the-art multi-modal image registration algorithms, for all tested experimental conditions (image pre-filtering, image intensity variation, noise perturbation). Moreover, we demonstrate that, compared to existing approaches, our method possesses increased robustness to transient structures (i.e. that are only present in some of the images).

Design of a multimodal (1H/23Na MR/CT) anthropomorphic thorax phantom

ObjectivesThis work proposes a modular, anthropomorphic MR and CT thorax phantom that enables the comparison of experimental studies for quantitative evaluation of deformable, multimodal image registration algorithms and realistic multi-nuclear MR imaging techniques. MethodsA human thorax phantom was developed with insertable modules representing lung, liver, ribs and additional tracking spheres. The quality of human tissue mimicking characteristics was evaluated for 1H and 23Na MR as well as CT imaging. The position of landmarks in the lung lobes was tracked during CT image acquisition at several positions during breathing cycles. 1H MR measurements of the liver were repeated after seven months to determine long term stability. ResultsThe modules possess HU, T1 and T2 values comparable to human tissues (lung module: −756±148HU, artificial ribs: 218±56HU (low CaCO3 concentration) and 339±121 (high CaCO3 concentration), liver module: T1=790±28ms, T2=65±1ms). Motion analysis showed that the landmarks in the lung lobes follow a 3D trajectory similar to human breathing motion. The tracking spheres are well detectable in both CT and MRI. The parameters of the tracking spheres can be adjusted in the following ranges to result in a distinct signal: HU values from 150 to 900HU, T1 relaxation time from 550ms to 2000ms, T2 relaxation time from 40ms to 200ms. ConclusionThe presented anthropomorphic multimodal thorax phantom fulfills the demands of a simple, inexpensive system with interchangeable components. In future, the modular design allows for complementing the present set up with additional modules focusing on specific research targets such as perfusion studies, 23Na MR quantification experiments and an increasing level of complexity for motion studies.

A Robust and Accurate Two-Step Auto-Labeling Conditional Iterative Closest Points (TACICP) Algorithm for Three-Dimensional Multi-Modal Carotid Image Registration.

Atherosclerosis is among the leading causes of death and disability. Combining information from multi-modal vascular images is an effective and efficient way to diagnose and monitor atherosclerosis, in which image registration is a key technique. In this paper a feature-based registration algorithm, Two-step Auto-labeling Conditional Iterative Closed Points (TACICP) algorithm, is proposed to align three-dimensional carotid image datasets from ultrasound (US) and magnetic resonance (MR). Based on 2D segmented contours, a coarse-to-fine strategy is employed with two steps: rigid initialization step and non-rigid refinement step. Conditional Iterative Closest Points (CICP) algorithm is given in rigid initialization step to obtain the robust rigid transformation and label configurations. Then the labels and CICP algorithm with non-rigid thin-plate-spline (TPS) transformation model is introduced to solve non-rigid carotid deformation between different body positions. The results demonstrate that proposed TACICP algorithm has achieved an average registration error of less than 0.2mm with no failure case, which is superior to the state-of-the-art feature-based methods.

3D CT to 2D low dose single-plane fluoroscopy registration algorithm for in-vivo knee motion analysis.

A limitation to accurate automatic tracking of knee motion is the noise and blurring present in low dose X-ray fluoroscopy images. For more accurate tracking, this noise should be reduced while preserving anatomical structures such as bone. Noise in low dose X-ray images is generated from different sources, however quantum noise is by far the most dominant. In this paper we present an accurate multi-modal image registration algorithm which successfully registers 3D CT to 2D single plane low dose noisy and blurred fluoroscopy images that are captured for healthy knees. The proposed algorithm uses a new registration framework including a filtering method to reduce the noise and blurring effect in fluoroscopy images. Our experimental results show that the extra pre-filtering step included in the proposed approach maintains higher accuracy and repeatability for in vivo knee joint motion analysis.

Robust initialisation for single-plane 3D CT to 2D fluoroscopy image registration

Gradient descent-based automatic image registration algorithms typically fail when the initial misalignment between objects is large. This is a major limitation for routine clinical applications. The registration task is even more difficult for multi-modal images because of the nonlinear relationship between the pixel intensities in the images to be aligned. In this paper, we present a fast and accurate multi-modal image registration algorithm which successfully registers three-dimensional (3D) computed tomography to two-dimensional single-plane fluoroscopy data for large initial displacements between the images. Our experimental results show that the proposed approach can increase the range of initial displacements up to ± 20 mm for all translations and up to ± 20° for rotation while maintaining high precision and small bias in all six 3D rigid body transform parameters.

基于Shape Context和尺度不变特征变换的多模图像自动配准方法

基于Shape Context和尺度不变特征变换的多模图像自动配准方法

A Spatial-Feature-Enhanced MMI Algorithm for Multimodal Airborne Image Registration

Maximization of mutual information (MI) (MMI) has a wide application in multimodal image registration. The conventional MMI algorithm is based on the statistics derived from image sampling and does not use spatial features. In this paper, we proposed an improved MMI registration algorithm that combines the spatial information through a feature-based selection mechanism. A Harris corner filter is chosen to classify the pixels. The classification results are used to focus sample selection, joint entropy estimation, and weighted MI information calculation. Although features play an important role, conventional feature matching is not used, which removes an important challenge to the use of feature-based information. The advantages of the proposed algorithm are improved robustness by incorporation of spatial features and significant reduction in computing time for registration. Experiments with challenging synthetic images and multimodal airborne infrared images are provided to demonstrate the improvement.

Image registration using Markov random coefficient and geometric transformation fields

Image registration is central to different applications such as medical analysis, biomedical systems, and image guidance. In this paper we propose a new algorithm for multimodal image registration. A Bayesian formulation is presented in which a likelihood term is defined using an observation model based on coefficient and geometric fields. These coefficients, which represent the local intensity polynomial transformations, as the local geometric transformations, are modeled as prior information by means of Markov random fields. This probabilistic approach allows one to find optimal estimators by minimizing an energy function in terms of both fields, making the registration between the images possible.