Multimodal Image Matching Research Articles

Multimodal image matching based on Multimodality Robust Line Segment Descriptor

Although a number of local feature-based methods have been proposed, the multimodality matching is still a challenging problem in object recognition, remote sensing and medical image processing where the image contrast is significantly different. The local feature-based multimodality matching method is usually intensity-based, so the matching performance is not good enough because intensity-based method is sensitive to contrast variations. In order to solve these problems, we propose a novel Multimodality Robust Line Segment Descriptor (MRLSD) and develop a MRLSD matching method. The proposed method generates MRLSD descriptors based on extracted highly equivalent corners and line segments for two multimodal images, and then performs image matching by measuring the similarity of corresponding descriptors over two images. The proposed corner and line segment extraction method is based on local phase and direction information, and is insensitive to contrast variations, so the MRLSD descriptor is robust to modality variations. The MRLSD descriptor is rotation invariant by selecting circular feature sub-regions and projecting feature vectors to radial direction. The MRLSD descriptor achieves scale invariance by adjusting the radius of circular feature region according to the scale. Experimental results indicate that the proposed method achieves higher precision and repeatability than several state-of-the-art local feature-based multimodality matching methods, and also demonstrate its robustness to multimodal images.

Multimodal image matching via dual‐codebook‐based self‐similarity hypercube feature descriptor and voting strategy

An effective feature descriptor is proposed for multimodal local-image patch matching. The conventional self-similarity hypercube (SSH) fails in multimodal image matching due to different intensities of multimodal images. To mitigate this problem, a dual-codebook clustering is proposed for generating the descriptors. It is based on extracting a codebook, respectively, from visible and thermal images but sharing the same k -means clustering index of the local features of visible and thermal image patches. The experimental results show that the proposed approach effectively solves the multimodal image quantisation problem. Moreover, a voting strategy based on the proposed similarity family function facilitates the multimodal image matching more robustly compared with the conventional state-of-the-art methods.

Multi-modal image matching based on local frequency information

This paper challenges the issue of matching between multi-modal images with similar physical structures but different appearances. To emphasize the common structural information while suppressing the illumination and sensor-dependent information between multi-modal images, two image representations namely Mean Local Phase Angle (MLPA) and Frequency Spread Phase Congruency (FSPC) are proposed by using local frequency information in Log-Gabor wavelet transformation space. A confidence-aided similarity (CAS) that consists of a confidence component and a similarity component is designed to establish the correspondence between multi-modal images. The two representations are both invariant to contrast reversal and non-homogeneous illumination variation, and without any derivative or thresholding operation. The CAS that integrates MLPA with FSPC tightly instead of treating them separately can more weight the common structures emphasized by FSPC, and therefore further eliminate the influence of different sensor properties. We demonstrate the accuracy and robustness of our method by comparing it with those popular methods of multi-modal image matching. Experimental results show that our method improves the traditional multi-modal image matching, and can work robustly even in quite challenging situations (e.g. SAR & optical image).

Dense deformation field estimation for atlas-based segmentation of pathological MR brain images

Atlas registration is a recognized paradigm for the automatic segmentation of normal MR brain images. Unfortunately, atlas-based segmentation has been of limited use in presence of large space-occupying lesions. In fact, brain deformations induced by such lesions are added to normal anatomical variability and they may dramatically shift and deform anatomically or functionally important brain structures. In this work, we chose to focus on the problem of inter-subject registration of MR images with large tumors, inducing a significant shift of surrounding anatomical structures. First, a brief survey of the existing methods that have been proposed to deal with this problem is presented. This introduces the discussion about the requirements and desirable properties that we consider necessary to be fulfilled by a registration method in this context: To have a dense and smooth deformation field and a model of lesion growth, to model different deformability for some structures, to introduce more prior knowledge, and to use voxel-based features with a similarity measure robust to intensity differences. In a second part of this work, we propose a new approach that overcomes some of the main limitations of the existing techniques while complying with most of the desired requirements above. Our algorithm combines the mathematical framework for computing a variational flow proposed by Hermosillo et al. [G. Hermosillo, C. Chefd’Hotel, O. Faugeras, A variational approach to multi-modal image matching, Tech. Rep., INRIA (February 2001).] with the radial lesion growth pattern presented by Bach et al. [M. Bach Cuadra, C. Pollo, A. Bardera, O. Cuisenaire, J.-G. Villemure, J.-Ph. Thiran, Atlas-based segmentation of pathological MR brain images using a model of lesion growth, IEEE Trans. Med. Imag. 23 (10) (2004) 1301–1314.]. Results on patients with a meningioma are visually assessed and compared to those obtained with the most similar method from the state-of-the-art.

A modular registration platform for multimodal image matching using ITK—first clinical application: localizing epileptogenic foci

The resolution in 3D reconstructions from tilt series is limited to the information below the first zero of the contrast transfer function unless the signal is corrected computationally. The restoration is usually based on the assumption of a linear space-invariant system and a linear relationship between object mass density and observed image contrast. The space-invariant model is no longer valid when applied to tilted micrographs because the defocus varies in a direction perpendicular to the tilt axis and with it the shape of the associated point spread function. In this paper, a method is presented for determining the defocus gradient in thin specimens such as sections and 2D crystals, and for restoration of the images subsequently used for 3D reconstruction. The alignment procedure for 3D reconstruction includes area matching and tilt geometry refinement. A map with limited resolution computed from uncorrected micrographs is compared to a volume computed from corrected micrographs with extended resolution.

Variational, geometric, and statistical methods for modeling brain anatomy and function

We survey the recent activities of the Odyssée Laboratory in the area of the application of mathematics to the design of models for studying brain anatomy and function. We start with the problem of reconstructing sources in MEG and EEG, and discuss the variational approach we have developed for solving these inverse problems. This motivates the need for geometric models of the head. We present a method for automatically and accurately extracting surface meshes of several tissues of the head from anatomical magnetic resonance (MR) images. Anatomical connectivity can be extracted from diffusion tensor magnetic resonance images but, in the current state of the technology, it must be preceded by a robust estimation and regularization stage. We discuss our work based on variational principles and show how the results can be used to track fibers in the white matter (WM) as geodesics in some Riemannian space. We then go to the statistical modeling of functional magnetic resonance imaging (fMRI) signals from the viewpoint of their decomposition in a pseudo-deterministic and stochastic part that we then use to perform clustering of voxels in a way that is inspired by the theory of support vector machines and in a way that is grounded in information theory. Multimodal image matching is discussed next in the framework of image statistics and partial differential equations (PDEs) with an eye on registering fMRI to the anatomy. The paper ends with a discussion of a new theory of random shapes that may prove useful in building anatomical and functional atlases.

Variational Methods for Multimodal Image Matching

Matching images of different modalities can be achieved by the maximization of suitable statistical similarity measures within a given class of geometric transformations. Handling complex, nonrigid deformations in this context turns out to be particularly difficult and has attracted much attention in the last few years. The thrust of this paper is that many of the existing methods for nonrigid monomodal registration that use simple criteria for comparing the intensities (e.g. SSD) can be extended to the multimodal case where more complex intensity similarity measures are necessary. To this end, we perform a formal computation of the variational gradient of a hierarchy of statistical similarity measures, and use the results to generalize a recently proposed and very effective optical flow algorithm (L. Alvarez, J. Weickert, and J. Sanchez, 2000, Technical Report, and IJCV 39(1):41–56) to the case of multimodal image registration. Our method readily extends to the case of locally computed similarity measures, thus providing the flexibility to cope with spatial non-stationarities in the way the intensities in the two images are related. The well posedness of the resulting equations is proved in a complementary work (O.D. Faugeras and G. Hermosillo, 2001, Technical Report 4235, INRIA) using well established techniques in functional analysis. We briefly describe our numerical implementation of these equations and show results on real and synthetic data.

Medical image matching-a review with classification

A classification scheme for multimodal image matching is considered. The scope of the classification is restricted to methods that register data after acquisitions. The classification scheme may be used for any modality; not only for (2-D) projection images and (3-D) tomographic images, but also for other signal modalities that provide spatial insight into function or anatomy, e.g., EEG (electroencephalography) or MEG (magnetoencephalography) and for the real physical patient. The available literature on image matching is discussed and classified.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>