Multimodal Similarity Measure Research Articles

An efficient hybrid method for 3D to 2D medical image registration

PurposeThe purpose of this paper is to present a method for registration of 3D computed tomography to 2D single-plane fluoroscopy knee images to provide 3D motion information for knee joints. This 3D kinematic information has unique utility for examining joint kinematics in conditions such as ligament injury, osteoarthritis and after joint replacement.MethodsWe proposed a non-invasive rigid body image registration method which is based on two different multimodal similarity measures. This hybrid registration method helps to achieve a trade-off among different challenges including, time complexity and accuracy.ResultsWe performed a number of experiments to evaluate the performance of the proposed method. The experimental results show that the proposed method is as accurate as one of the most recent registration methods while it is several times faster than that method.ConclusionThe proposed method is a non-invasive, fast and accurate registration method, which can provide 3D information for knee joint kinematic measurements. This information can be very helpful in improving the accuracy of diagnosis and providing targeted treatment.

Local structure orientation descriptor based on intra-image similarity for multimodal registration of liver ultrasound and MR images

PurposeUltrasound (US)-magnetic resonance (MR) fusion imaging is a profitable tool for image-guided abdominal diagnosis and biopsy. However, the automatic registration of liver US and MR images remains a challenging task. An effective local structure orientation descriptor (LSOD) for use in registering multimodal images is proposed in this study. MethodsLSOD utilizes a normalized similarity distance vector of intra-image patch pairs to extract intensity change orientations from intensity value changes in a local area. The multimodal similarity measure is then derived using the LSOD vector difference. Experiments were performed on simulated US and liver 2D US–3D MR images from a phantom, two healthy volunteers, and seven patients. ResultsUsing the LSOD-based method, the root-mean-square target registration errors (RMS-TREs) were 1.76±1.90mm/2.03±0.84mm in phantom/clinical experiments. All of the results outperformed those obtained using modality independent neighborhood descriptor (MIND)- and linear correlation of linear combination (LC2)-based methods (phantom/clinical: 5.23±3.35mm/4.32±3.63mm and 9.79±5.03mm/6.29±3.85mm, respectively). The registration cover range for all subjects of the LSOD-based method was 9.16mm, which was larger than those of the MIND- and LC2-based methods (5.06 and 5.12mm, respectively). ConclusionsThe results demonstrated that the LSOD-based registration method could robustly register 2D US and 3D MR images of different liver sections with acceptable accuracy for clinical requirements. This approach is useful for the practical clinical application of the US–MR fusion imaging technique.

Image-assisted non-invasive and dynamic biomechanical analysis of human joints

Kinematic analysis provides a strong link between musculoskeletal injuries, chronic joint conditions, treatment planning/monitoring and prosthesis design/outcome. However, fast and accurate 3D kinematic analysis still remains a challenge in order to translate this procedure into clinical scenarios. 3D computed tomography (CT) to 2D single-plane fluoroscopy registration is a promising non-invasive technology for biomechanical examination of human joints. Although this technique has proven to be very precise in terms of in-plane translation and rotation measurements, out-of-plane motion estimations have been a difficulty so far. Therefore, to enable this technology into clinical translation, precise and fast estimation of both in-plane and out-of-plane movements is crucial, which is the aim of this paper. Here, a fast and accurate 3D/2D registration technique is proposed to evaluate biomechanical/kinematic analysis. The proposed algorithm utilizes a new multi-modal similarity measure called ‘sum of conditional variances’, a coarse-to-fine Laplacian of Gaussian filtering approach for robust gradient-descent optimization and a novel technique for the analytic calculation of the required gradients for out-of-plane rotations. Computer simulations and in vitro experiments showed that the new approach was robust in terms of the capture range, required significantly less iterations to converge and achieved good registration and kinematic accuracy when compared to existing techniques and to the ‘gold-standard’ Roentgen stereo analysis.

Local self-similarity-based registration of human ROIs in pairs of stereo thermal-visible videos

For several years, mutual information (MI) has been the classic multimodal similarity measure. The robustness of MI is closely restricted by the choice of MI window sizes. For unsupervised human monitoring applications, obtaining appropriate MI window sizes for computing MI in videos with multiple people in different sizes and different levels of occlusion is problematic. In this work, we apply local self-similarity (LSS) as a dense multimodal similarity metric and show its adequacy and strengths compared to MI for a human ROIs registration. We also propose an LSS-based registration of thermal-visible stereo videos that addresses the problem of multiple people and occlusions in the scene. Our method improves the accuracy of the state-of-the-art disparity voting (DV) correspondence algorithm by proposing a motion segmentation step that approximates depth segments in an image and enables assigning disparity to each depth segment using larger matching window while keeping registration accuracy. We demonstrate that our registration method outperforms the recent state-of-the-art MI-based stereo registration for several realistic close-range indoor thermal-visible stereo videos of multiple people.

Leveraging community metadata for multimodal image ranking

Searching for relevant images given a query term is an important task in nowadays large-scale community databases. The image ranking approach presented in this work represents an image collection as a graph that is built using a multimodal similarity measure based on visual features and user tags. We perform a random walk on this graph to find the most common images. Further we discuss several scalability issues of the proposed approach and show how in this framework queries can be answered fast. Experimental results validate the effectiveness of the presented algorithm.

Low-dimensional Multimodal Deformable Registration of MRI Brain Images in Stereotaxic Space

Deformable image registration is a fundamental technique in computational neuroanatomy. An iterative multilevel block matching technique with the use of several recent inventions is proposed here. A symmetric multimodal similarity measure allows to register subject images to an arbitrary digital brain atlas. Smooth deformations produced by scattered data interpolation based on compactly supported radial basis functions suppress gross inter-subject differences and preserve the localized anatomical variability which may be further studied with selected automated morphometry methods. Four similarity measures are tested in an experiment with image data obtained from the Simulated Brain Database and a quantitative evaluation of the algorithm is presented.

Point similarity measures for non-rigid registration of multi-modal data

High-dimensional non-rigid registration of multi-modal data requires similarity measures with two important properties: multi-modality and locality. Unfortunately all commonly used multi-modal similarity measures are inherently global and cannot operate on small image regions. In this paper, we propose a new class of multi-modal similarity measures, which are constructed from information of the whole images but can be applied pointwise. Due to their capability of measuring correspondence for individual image points we call them point similarity measures. Point similarity measures can be derived from global measures and enable detailed relative comparison of local image correspondence. We present a set of multi-modal point similarity measures based on joint intensity distribution and test them as an integral part of non-rigid multi-modal registration system. The comparison results show that segmentation-based measure, which models the joint distribution as a sum of intensity classes, performs best. When intensity classes do not exist or cannot be accurately modeled, each intensity pair can be treated as a separate class, which results in a more general measure, suitable for various non-rigid registration tasks.