Multimodal Image Registration Technique Research Articles

Effect of TMJ disc position on condylar bone remodeling after arthroscopic disc repositioning surgery

PurposeThe objective of this study was to analyze the effect of TMJ disc position on condylar bone remodeling after arthroscopic disc repositioning surgery. MethodsNine patients with anterior disc displacement without reduction (ADDWoR, 15 sides) who underwent arthroscopic disc repositioning surgery were included. Three-dimensional (3D) reconstruction of the articular disc and the condyle in the closed-mouth position was performed using cone-beam computed tomography (CBCT) and magnetic resonance imaging (MRI) data. Then, the CBCT and MRI images were fused and displayed together by multimodal image registration techniques. Morphological changes in the articular disc and condyle, as well as changes in their spatial relationship, were studied by comparing preoperative and 3-month postoperative CBCT-MRI fused images. ResultsThe volume and superficial area of the articular disc, as well as the area of the articular disc surface in the subarticular cavity, were significantly increased compared to that before the surgical treatment（P < 0.01）. There was also a significant increase in the volume of the condyle (P < 0.001). All condyles showed bone remodeling after surgery that could be categorized as one of two types depending on the position of the articular disc, suggesting that the location of the articular disc was related to the new bone formation. ConclusionsThe morphology of the articular disc and condyle were significantly changed after arthroscopic disc repositioning surgery. The 3D changes in the position of the articular disc after surgery tended to have an effect on condylar bone remodeling and the location of new bone formation.

Multi-Modal Imaging to Assess the Interaction Between Inflammation and Bone Damage Progression in Inflammatory Arthritis.

Combining results from multiple imaging techniques (i.e., multi-modal imaging) through image registration can result in the better characterization of joint tissue characteristics. In the context of inflammatory arthritis conditions, high-resolution peripheral quantitative computed tomography (HR-pQCT) provides excellent bone contrast while magnetic resonance imaging (MRI) provides superior contrast and resolution of soft tissue and inflammatory characteristics. Superimposing these imaging results upon each other provides a robust characterization of the joint. In a preliminary study of nine rheumatoid arthritis (RA) participants in clinical remission, we acquired HR-pQCT and MR images of their 2nd and 3rd metacarpophalangeal (MCP) joints at two timepoints 6 months apart. We present the benefits of a multi-modal imaging approach, in which we demonstrate the ability to localize regions of inflammation with subtle changes in bone erosion volume. Using HR-pQCT and MRI to visualize bone damage and inflammation, respectively, will improve our understanding of the impact that subclinical inflammation has on bone damage progression, and demonstrating if bone repair occurs where inflammation is resolved. The presented multi-modal imaging technique has the potential to study the progression of bone damage in relation to inflammation that otherwise would not be possible with either imaging technique alone. The multi-modal image registration technique will be helpful to understanding the development and pathogenesis of RA-associated bone erosions. Additionally, multi-modal imaging may provide a technique to probe the tissue-level changes that occur as a result of treatment regimes.

Three-dimensional acquisition technologies for facial soft tissues – Applications and prospects in orthognathic surgery

The management of patients with dento-maxillofacial deformities is based on assessments of the dental occlusion – facial skeleton – soft tissues triad. As societal demands and surgical practices have evolved, facial soft tissues have moved to the forefront of considerations in orthognathic surgery. Techniques are therefore required to analyze facial soft tissues objectively and reproducibly, for diagnosis, preoperative planning, and follow-up. Several technologies are currently capable of providing three-dimensional (3D) models of the face, either by 3D reconstruction of traditional computed tomography or cone beam computed tomography data, or directly by stereophotogrammetry, laser scanning or structured light scanning. Multimodal image registration techniques allow bone base, dental occlusion and facial soft tissue information to be combined in a 3D virtual patient. Three-dimensional cephalometric analysis of the facial skeleton and skin is now perfectly integrated in virtual planning and is gradually gaining in automation and accuracy. Photorealistic 3D simulations allow optimal soft tissue planning and facilitate physician-patient communication. Finally, these facial modeling techniques facilitate post-operative studies of soft tissues, which generally involve comparisons of volumetric data. There are many research avenues to pursue and technical improvements are to be expected, particularly through the development of big data and artificial intelligence approaches.

Self-Similarity and Symmetry With SIFT for Multi-Modal Image Registration

This paper presents a novel feature-based multi-modal image registration technique called self-similarity and symmetry with a scale invariant feature transform (3S-SIFT). The proposed technique has the following two main components. First, a ubiquitous problem existing in registering multi-modal images, gradient reversal, is well studied and addressed. Second, the proposed technique takes into account self-similarity information between keypoint triangles, which is conducive to enhancing the registration accuracy. Moreover, a simplified version of 3S-SIFT called 4S-SIFT is proposed as a pruning technique for feature matching. The proposed techniques are generally applicable to the registration of multi-modal images with changes in scale, rotation, and translation. The experiments have been conducted on a set of benchmark datasets in the domain of image registration, demonstrating that the proposed techniques achieve the state-of-the-art performance in both matching accuracy and recall.

Co-Sparse Analysis Model Based Image Registration to Compensate Brain Shift by Using Intra-Operative Ultrasound Imaging.

Notwithstanding the widespread use of image guided neurosurgery systems in recent years, the accuracy of these systems is strongly limited by the intra-operative deformation of the brain tissue, the so-called brain shift. Intra-operative ultrasound (iUS) imaging as an effective solution to compensate complex brain shift phenomena update patients coordinate during surgery by registration of the intra-operative ultrasound and the pre-operative MRI data that is a challenging problem.In this work a non-rigid multimodal image registration technique based on co-sparse analysis model is proposed. This model captures the interdependency of two image modalities; MRI as an intensity image and iUS as a depth image. Based on this model, the transformation between the two modalities is minimized by using a bimodal pair of analysis operators which are learned by optimizing a joint co-sparsity function using a conjugate gradient.Experimental validation of our algorithm confirms that our registration approach outperforms several of other state-of-the-art registration methods quantitatively. The evaluation was performed using seven patient dataset with the mean registration error of only 1.83 mm. Our intensity-based co-sparse analysis model has improved the accuracy of non-rigid multimodal medical image registration by 15.37% compared to the curvelet based residual complexity as a powerful registration method, in a computational time compatible with clinical use.

COREG: a corner based registration technique for multimodal images

This paper presents a COrner based REGistration technique for multimodal images (referred to as COREG). The proposed technique focuses on addressing large content and scale differences in multimodal images. Unlike traditional multimodal image registration techniques that rely on intensities or gradients for feature representation, we propose to use contour-based corners. First, curvature similarity between corners are for the first time explored for the purpose of multimodal image registration. Second, a novel local descriptor called Distribution of Edge Pixels Along Contour (DEPAC) is proposed to represent the edges in the neighborhood of corners. Third, a simple yet effective way of estimating scale difference is proposed by making use of geometric relationships between corner triplets from the reference and target images. Using a set of benchmark multimodal images and multimodal microscopic images, we will demonstrate that our proposed technique outperforms a state-of-the-art multimodal image registration technique.

A detector of structural similarity for multi-modal microscopic image registration

This paper presents a Detector of Structural Similarity (DSS) to minimize the visual differences between brightfield and confocal microscopic images. The context of this work is that it is very challenging to effectively register such images due to a low structural similarity in image contents. To address this issue, DSS aims to maximize the structural similarity by utilizing the intensity relationships among red-green-blue (RGB) channels in images. Technically, DSS can be combined with any multi-modal image registration technique in registering brightfield and confocal microscopic images. Our experimental results show that DSS significantly increases the visual similarity in such images, thereby improving the registration performance of an existing state-of-the-art multi-modal image registration technique by up to approximately 27%.

Multimodal non-rigid image registration based on elastodynamics

In this paper, we present a new multimodal image registration technique established on elastodynamics notion. The main idea behind this concept is the progression of waves on an elastic body as soon as it is disturbed from its initial rest state. We propose to solve the multimodal registration problem by modeling the non-linear deformations as elastic waves and iteratively solving the elastodynamics wave equation to estimate the transformation. The inertial force in elastodynamics model is computed as the gradient of mutual information which considers the statistical relationship between the intensities of the images acquired using different imaging modalities. We tested our method on T1–T2 weighted MR brain image pairs and MR-CT brain image pairs. The proposed registration technique was compared against a variant of demons method proposed for multimodal images. The registration results were analyzed by examining the overlay images and by computing the normalized mutual information. The qualitative and quantitative analysis proved that our proposed method registers the images better than the compared method.

Multimodal image registration technique based on improved local feature descriptors

Multimodal image registration has received significant research attention over the past decade, and the majority of the techniques are global in nature. Although local techniques are widely used for general image registration, there are only limited studies on them for multimodal image registration. Scale invariant feature transform (SIFT) is a well-known general image registration technique. However, SIFT descriptors are not invariant to multimodality. We propose a SIFT-based technique that is modality invariant and still retains the strengths of local techniques. Moreover, our proposed histogram weighting strategies also improve the accuracy of descriptor matching, which is an important image registration step. As a result, our proposed strategies can not only improve the multimodal registration accuracy but also have the potential to improve the performance of all SIFT-based applications, e.g., general image registration and object recognition.

Multimodal Medical Image registration using Discrete Wavelet Transform

In image processing, image registration develops a relationship between two images using optimal transformation where the images could have been taken at various times, sources or devices, or from different perspectives. It aligns the reference and moving image using geometric transformations. This research study evaluates the performance of multimodal(images acquired from different sources)image registration technique using Discrete Wavelet Transform (DWT).The reference and the target images are decomposed into their respective DWT coefficients and then are processed for image registration. After registration, the resultant DWT coefficients are transformed back using Inverse DWT into their spatial coordinates in order to retrieve the registered image. The similarity of the two input images for image registration is calculated and investigated using a similarity metric known as Mutual Information (MI) which is maximized. The quality of registration is measured using cross-correlation coefficient (CCC) of the registered image with respect to the reference image. Finally the time taken for image registration in wavelet domain is analyzed and compared with the image registration taking place in spatial domain.

Multi-Modal Image Registration Based on Gradient Orientations of Minimal Uncertainty

In this paper, we propose a new multi-scale technique for multi-modal image registration based on the alignment of selected gradient orientations of reduced uncertainty. We show how the registration robustness and accuracy can be improved by restricting the evaluation of gradient orientation alignment to locations where the uncertainty of fixed image gradient orientations is minimal, which we formally demonstrate correspond to locations of high gradient magnitude. We also embed a computationally efficient technique for estimating the gradient orientations of the transformed moving image (rather than resampling pixel intensities and recomputing image gradients). We have applied our method to different rigid multi-modal registration contexts. Our approach outperforms mutual information and other competing metrics in the context of rigid multi-modal brain registration, where we show sub-millimeter accuracy with cases obtained from the retrospective image registration evaluation project. Furthermore, our approach shows significant improvements over standard methods in the highly challenging clinical context of image guided neurosurgery, where we demonstrate misregistration of less than 2 mm with relation to expert selected landmarks for the registration of pre-operative brain magnetic resonance images to intra-operative ultrasound images.

A multi-resolution area-based technique for automatic multi-modal image registration

To allow remotely sensed datasets to be used for data fusion, either to gain additional insight into the scene or for change detection, reliable spatial referencing is required. With modern remote sensing systems, reliable registration can be gained by applying an orbital model for spaceborne data or through the use of global positioning (GPS) and inertial navigation (INS) systems in the case of airborne data. Whilst, individually, these datasets appear well registered when compared to a second dataset from another source (e.g., optical to LiDAR or optical to radar) the resulting images may still be several pixels out of alignment. Manual registration techniques are often slow and labour intensive and although an improvement in registration is gained, there can still be some misalignment of the datasets. This paper outlines an approach for automatic image-to-image registration where a topologically regular grid of tie points was imposed within the overlapping region of the images. To ensure topological consistency, tie points were stored within a network structure inspired from Kohonen’s self-organising networks [24]. The network was used to constrain the motion of the tie points in a manner similar to Kohonen’s original method. Using multiple resolutions, through an image pyramid, the network structure was formed at each resolution level where connections between the resolution levels allowed tie point movements to be propagated within and to all levels. Experiments were carried out using a range of manually registered multi-modal remotely sensed datasets where known linear and non-linear transformations were introduced against which our algorithm’s performance was tested. For single modality tests with no introduced transformation a mean error of 0.011 pixels was identified increasing to 3.46 pixels using multi-modal image data. Following the introduction of a series of translations a mean error of 4.98 pixels was achieve across all image pairs while a mean error of 7.12 pixels was identified for a series of non-linear transformations. Experiments using optical reflectance and height data were also conducted to compare the manually and automatically produced results where it was found the automatic results out performed the manual results. Some limitations of the network data structure were identified when dealing with very large errors but overall the algorithm produced results similar to, and in some cases an improvement over, that of a manual operator. We have also positively compared our method to methods from two other software packages: ITK and ITT ENVI.

Real-time multi-modal rigid registration based on a novel symmetric-SIFT descriptor

The purpose of image registration is to spatially align two or more single-modality images taken at different times, or several images acquired by multiple imaging modalities. Intensity-based registration usually requires optimization of the similarity metric between the images. However, global optimization techniques are too time-consuming, and local optimization techniques frequently fail to search the global transformation space because of the large initial misalignment of the two images. Moreover, for large non-overlapping area registration, the similarity metric cannot reach its optimum value when the two images are properly registered. In order to solve these problems, we propose a novel Symmetric Scale Invariant Feature Transform (symmetric-SIFT) descriptor and develop a fast multi-modal image registration technique. The proposed technique automatically generates a lot of highly distinctive symmetric-SIFT descriptors for two images, and the registration is performed by matching the corresponding descriptors over two images. These descriptors are invariant to image scale and rotation, and are partially invariant to affine transformation. Moreover, these descriptors are symmetric to contrast, which makes it suitable for multi-modal image registration. The proposed technique abandons the optimization and similarity metric strategy. It works with near real-time performance, and can deal with the large non-overlapping and large initial misalignment situations. Test cases involving scale change, large non-overlapping, and large initial misalignment on computed tomography (CT) and magnetic resonance (MR) datasets show that it needs much less runtime and achieves better accuracy when compared to other algorithms.

표면거리 및 표면곡률 최적화 기반 다중모달리티 뇌영상 정합

서로 다른 종류의 영상을 정확하게 연관시켜 복합적인 정보를 제공하는 다중모달리티 의료 영상정합기법 중 표면정보 기반 영상정합에서는 일반적으로 동일 대상에 대한 서로 다른 모달리티에서 추출된 표면 윤곽정보 사이의 거리를 최소화함으로써 매칭이 이루어진다. 그런데 동일대상에 대해 취득되는 서로 다른 두 모달리티는 관심 영역 상의 표면 특성이 서로 유사하다. 그러므로 다중모달리티 영상정합에서 표면거리와 함께 표면의 형태 특성을 고려하여 두 영상을 매칭하는 방법이 정합결과의 정확도를 향상시킬 수 있다. 본 연구에서는 동일 대상의 서로 다른 두 모달리티 뇌영상 간의 표면거리와 표면곡률을 최적화하는 정합기법을 제안한다. 영상정합은 참조영상과 테스트영상에 대한 표면정보 생성과 이 두 개의 표면정보를 최적화하는 단계로 구성된다. 표면정보 생성 단계에서는 두 모달리티로부터 관심영역의 윤곽선을 추출하고, 이 중 참조 볼륨의 윤곽선에 대해서는 표면거리맵과 표면곡률맵을 구성하게 된다. 최적화 단계에서는 표면거리맵과 표면곡률맵을 참조하는 최적화 평가함수(cost function)에 의해 두 객체의 표면거리 차이와 표면곡률 차이를 최소화하는 정합 변환 값이 결정되고, 이것이 테스트영상의 변환에 적용되어 결과적으로 두 영상이 정합 되게 된다. 제안된 최적화 평가함수는 표면거리 정보만을 사용하는 평가함수에 비해 보다 견고한 정합 정확도를 보였으며 또한 본 연구는 정합결과의 볼륨 가시화를 통해 효율적인 영상 분석 수단을 제공하고자 하였다. Within multimodal medical image registration techniques, which correlate different images and Provide integrated information, surface registration methods generally minimize the surface distance between two modalities. However, the features of two modalities acquired from one subject are similar. So, it can improve the accuracy of registration result to match two images based on optimization of both surface distance and shape feature. This research proposes a registration method which optimizes surface distance and surface curvature of two brain modalities. The registration process has two steps. First, surface information is extracted from the reference images and the test images. Next, the optimization process is performed. In the former step, the surface boundaries of regions of interest are extracted from the two modalities. And for the boundary of reference volume image, distance map and curvature map are generated. In the optimization step, a transformation minimizing both surface distance and surface curvature difference is determined by a cost function referring to the distance map and curvature map. The applying of the result transformation makes test volume be registered to reference volume. The suggested cost function makes possible a more robust and accurate registration result than that of the cost function using the surface distance only. Also, this research provides an efficient means for image analysis through volume visualization of the registration result.