Image Similarity Measure Research Articles

A deep learning based method for large-scale classification, registration, and clustering of in-situ hybridization experiments in the mouse olfactory bulb

BackgroundThe Allen Mouse Brain Atlas allows study of the brain’s molecular anatomy at cellular scale, for thousands genes. To fully leverage this resource, one must register histological images of brain tissue – a task made challenging by the brain’s structural complexity and heterogeneity, as well as inter-experiment variability. New methodWe have developed a deep-learning based methodology for classification and registration of thousands of sections of brain tissue, using the mouse olfactory bulb (OB) as a case study. ResultsWe trained a convolutional neural network (CNN) to derive an image similarity measure for in-situ hybridization experiments, and embedded these in a low-dimensional feature space to guide the design of registration templates. We then compiled a high quality, registered atlas of gene expression for the OB (the first such atlas for the OB, to our knowledge). As proof-of-principle, the atlas was clustered using non-negative matrix factorization to reveal canonical expression motifs, and to identify novel, lamina-specific marker genes. Comparison with existing methodsOur method leverages virtues of CNNs for a set of important problems in molecular neuroanatomy, with performance comparable to existing methods. ConclusionThe atlas we have complied allows for intra- and inter-laminar comparisons of gene expression patterns in the OB across thousands of genes, as well identification of canonical expression profiles through clustering. We anticipate that this will be a useful resource for investigators studying the bulb’s development and functional topography. Our methods are publicly available for those interested in extending them to other brain areas.

Measurement of Copy-Move Forensic Image Similarity Using Distance Function

Measurement of Copy-Move Forensic Image Similarity Using Distance Function

On feature selection in network flow based traffic sign tracking models

Abstract We use the network flow based tracking model to process videos recorded by taxicabs to collect information, such as the locations and types, of traffic signs. One of the key questions in this type of tracking models is the feature selection problem, which determines the features and their associated weights to construct the costs on the edges of the underlying network. However, in existing studies, the features and their weights are set mostly by intuition and there are no studies that systematically investigate this problem. We investigate the feature selection problem by considering a wide range of features related to traffic sign candidates, including its position, size, frame index, color histogram, speed up robust feature (SURF) descriptors, and template matching similarities. We propose to use the multinomial logit model to identify features that have significant influence on the edge costs and to estimate their corresponding weights. We test the performance of our selected features using real videos recorded by taxicabs in the city of Beijing. We find that the position and frame index of the traffic sign candidates are the top two features that influence the tracking results. We also find that image similarity measures are helpful features, but their contribution is not as large as the previous two features.

Automatic Functionality Verification of Hybrid Set-Top Boxes With Dynamic User Interface

Modern set-top-box (STB) user interfaces (UI) contain increased amount of animations/effects compared to standard UIs, thus demanding a new approach for accurate and cost-efficient STB functionality verification. In this paper, a fully automated system for functional failure detection in hybrid STBs with dynamic UI is proposed. It consists of image capturing unit, testing tool, and hybrid STB being verified. The automatic verification of hybrid STB functionality is performed by comparing the captured image of the dynamic UI to the referent image captured from referent STB. The system incorporates a new algorithm for image similarity measurement, adjusted for image alterations which often appear in dynamic UIs. Algorithm utilizes image feature extraction and comparison to determine captured and referent image similarity and to make decision regarding STB functionality. The proposed system was compared with existing fully automated system for functionality verification of an STB device with static UI. Tests were performed on both computer generated and live stream cases. For this purpose a new freely available computer generated image alterations database is created. The results show that the proposed system achieves higher performance and robustness, in terms of measuring image similarity despite alterations caused by dynamic UI effects.

A Filter of Minhash for Image Similarity Measures

Image similarity measures play an important role in nearest neighbor search and duplicate detection for large-scale image datasets. Recently, Minwise Hashing (or Minhash) and its related hashing algorithms have achieved great performances in large-scale image retrieval systems. However, there are a large number of comparisons for image pairs in these applications, which may spend a lot of computation time and affect the performance. In order to quickly obtain the pairwise images that theirs similarities are higher than the specific thresholdT(e.g., 0.5), we propose a dynamic threshold filter of Minwise Hashing for image similarity measures. It greatly reduces the calculation time by terminating the unnecessary comparisons in advance. We also find that the filter can be extended to other hashing algorithms, on when the estimator satisfies the binomial distribution, such as b-Bit Minwise Hashing, One Permutation Hashing, etc. In this pager, we use the Bag-of-Visual-Words (BoVW) model based on the Scale Invariant Feature Transform (SIFT) to represent the image features. We have proved that the filter is correct and effective through the experiment on real image datasets.

Multi-segmental spine image registration supporting image-guided interventions of spinal metastases

BackgroundRadiofrequency ablation was introduced recently to treat spinal metastases, which are among the most common metastases. These minimally-invasive interventions are most often image-guided by flat-panel CT scans, withholding soft tissue contrast like MR imaging. Image fusion of diagnostic MR and operative CT images could provide important and useful information during interventions. MethodDiagnostic MR and interventional flat-panel CT scans of 19 patients, who underwent radiofrequency ablations of spinal metastases were obtained. Our presented approach piecewise rigidly registers single vertebrae using normalized gradient fields and embeds them within a fused image. Registration accuracy was determined via Euclidean distances between corresponding landmark pairs of ground truth data. ResultsOur method resulted in an average registration error of 2.35mm. An optimal image fusion performed by landmark registrations achieved an average registration error of 1.70mm. Additionally, intra- and inter-reader variability was determined, resulting in mean distances of corresponding landmark pairs of 1.05mm (MRI) and 1.03mm (flat-panel CT) for the intra-reader variability and 1.36mm and 1.28mm for the inter-reader variability, respectively. ConclusionsOur multi-segmental approach with normalized gradient fields as image similarity measure can handle spine deformations due to patient positioning and avoid time-consuming manually performed registration. Thus, our method can provide practical and applicable intervention support without significantly delaying the clinical workflow or additional workload.

A similarity measure method combining location feature for mammogram retrieval.

Breast cancer, the most common malignancy among women, has a high mortality rate in clinical practice. Early detection, diagnosis and treatment can reduce the mortalities of breast cancer greatly. The method of mammogram retrieval can help doctors to find the early breast lesions effectively and determine a reasonable feature set for image similarity measure. This will improve the accuracy effectively for mammogram retrieval. This paper proposes a similarity measure method combining location feature for mammogram retrieval. Firstly, the images are pre-processed, the regions of interest are detected and the lesions are segmented in order to get the center point and radius of the lesions. Then, the method, namely Coherent Point Drift, is used for image registration with the pre-defined standard image. The center point and radius of the lesions after registration are obtained and the standard location feature of the image is constructed. This standard location feature can help figure out the location similarity between the image pair from the query image to each dataset image in the database. Next, the content feature of the image is extracted, including the Histogram of Oriented Gradients, the Edge Direction Histogram, the Local Binary Pattern and the Gray Level Histogram, and the image pair content similarity can be calculated using the Earth Mover's Distance. Finally, the location similarity and content similarity are fused to form the image fusion similarity, and the specified number of the most similar images can be returned according to it. In the experiment, 440 mammograms, which are from Chinese women in Northeast China, are used as the database. When fusing 40% lesion location feature similarity and 60% content feature similarity, the results have obvious advantages. At this time, precision is 0.83, recall is 0.76, comprehensive indicator is 0.79, satisfaction is 96.0%, mean is 4.2 and variance is 17.7. The results show that the precision and recall of this method have obvious advantage, compared with the content-based image retrieval.

Color-Based Image Retrieval Using Proximity Space Theory

The goal of object retrieval is to rank a set of images by their similarity compared with a query image. Nowadays, content-based image retrieval is a hot research topic, and color features play an important role in this procedure. However, it is important to establish a measure of image similarity in advance. The innovation point of this paper lies in the following. Firstly, the idea of the proximity space theory is utilized to retrieve the relevant images between the query image and images of database, and we use the color histogram of an image to obtain the Top-ranked colors, which can be regard as the object set. Secondly, the similarity is calculated based on an improved dominance granule structure similarity method. Thus, we propose a color-based image retrieval method by using proximity space theory. To detect the feasibility of this method, we conducted an experiment on COIL-20 image database and Corel-1000 database. Experimental results demonstrate the effectiveness of the proposed framework and its applications.

Attribute-enhanced metric learning for face retrieval

Metric learning is a significant factor for media retrieval. In this paper, we propose an attribute label enhanced metric learning model to assist face image retrieval. Different from general cross-media retrieval, in the proposed model, the information of attribute labels are embedded in a hypergraph metric learning framework for face image retrieval tasks. The attribute labels serve to build a hypergraph, in which each image is abstracted as a vertex and is contained in several hyperedges. The learned hypergraph combines the attribute label to reform the topology of image similarity relationship. With the mined correlation among multiple facial attributes, the reformed metrics incorporates the semantic information in the general image similarity measure. We apply the metric learning strategy to both similarity face retrieval and interactive face retrieval. The proposed metric learning model effectively narrows down the semantic gap between human and machine face perception. The learned distance metric not only increases the precision of similarity retrieval but also speeds up the convergence distinctively in interactive face retrieval.

A Survey on Local Textural Patterns for Facial Feature Extraction

Over the last two decades retrieving an accurate image has become a challenging task. Regardless, texture patterns address this problem by decreasing the significant gap between the actual image over the user expectation rather than other low-level features. This article represents the comprehensive survey of the recent achievements and relevant publications investigated in different directions of the textural areas in CBIR. These consist of triggered methods for image local texture feature extraction, numerical illustration and similarity measurement. In addition, challenges are discussed in comparisons of textural patterns. Retrospectively, concluded with a few recommendations based on generic survey and demand from the

Weibull statistical modeling for textured image retrieval using nonsubsampled contourlet transform

In this paper, we proposed a new framework for textured image retrieval, which is based on Weibull statistical distribution and nonsubsampled contourlet transform. Firstly, the image is decomposed into one lowpass subband and several highpass subbands by using nonsubsampled contourlet transform (NSCT). Secondly, Weibull probability distribution is employed to describe the statistical characteristics of the highpass NSCT coefficients, and the Weibull model parameters are utilized to construct a compact texture image feature space. Finally, image similarity measurement is accomplished by using closed-form solutions for the Kullback–Leibler divergences between the Weibull statistical models. Experimental results demonstrate the high efficiency of our textured image retrieval scheme, which can provide better retrieval rates and lower computational cost, in comparison with the state-of-the-art approaches recently proposed in the literature.

Metrics for Assessing the Similarity of Microwave Breast Imaging Scans of Healthy Volunteers.

Microwave radar imaging is promising as a complementary medical imaging modality. However, the unique nature of the images means interpretation can be difficult. As a result, it is important to understand the sources of image differences, and how much variability is inherent in the imaging system itself. To address this issue, we compare the effectiveness of six different measures of image similarity for quantifying the similarity (or difference) between two microwave radar images. The structural similarity index has become the de facto standard for image comparison, but we propose that useful information can be acquired from a measure known as the Modified Hausdorff Distance. We apply each measure to image pairs from sequential scans of both phantoms and volunteers. We find that rather than using a single value to quantify the image similarity, by computing a number of values that are designed to capture different image aspects, we can better assess the ways in which the images differ.

Real-time identification of the singleness of a trapped bead in optical tweezers.

Beads trapped in optical tweezers are aligned along the optical propagation direction, which makes it difficult to determine the number of beads with bright-field microscopy. This problem also dramatically influences the measurement of the optical trapping based single-molecule force spectroscopy. Here, we propose a video processing approach to count the number of trapped micro-objects in real time. The approach uses a normalized cross-correlation algorithm and image enhancement techniques to amplify a slight change of the image induced by the entry of an exotic object. As tested, this method introduces a ∼10% change per bead to the image similarity, and up to four beads, one-by-one falling into the trap, are identified. Moreover, the feasibility of the above analysis in a moving trap is investigated. A movement of the trap leads to a fluctuation of less than 2% for the similarity signal and can be ignored in most cases. The experimental results prove that image similarity measurement is a sensitive way to monitor the interruption, which is very useful, especially during experiments. In addition, the approach is easy to apply to an existing optical tweezers system.

A variational approach to non-rigid image registration with Bregman divergences and multiple features

Variational methods register images by minimizing functionals that balance measures of smoothness and image similarity. However, the specificity of image-similarity measures can limit the application of variational methods. To address this issue, we propose a new energy functional that generalizes the classical variational method for nonrigid image registration. Our generalization uses Bregman divergences as the similarity measure of the registration functional. These divergences include a number of important similarity measures, such as the Squared Loss distance, the KL divergence, the Logistic Loss, the Mahalanobis distance, the Itakura–Saito divergence, and the I-divergence. We derive the Euler Lagrange and gradient-flow equations associated with the new functional. By using Bregman divergences, our registration method can combine various types of image characterization as well as spatial and statistical information. Our experiments show that the use of Bregman divergences improves registration quality.

Learning non-linear patch embeddings with neural networks for label fusion.

In brain structural segmentation, multi-atlas strategies are increasingly being used over single-atlas strategies because of their ability to fit a wider anatomical variability. Patch-based label fusion (PBLF) is a type of such multi-atlas approaches that labels each target point as a weighted combination of neighboring atlas labels, where atlas points with higher local similarity to the target contribute more strongly to label fusion. PBLF can be potentially improved by increasing the discriminative capabilities of the local image similarity measurements. We propose a framework to compute patch embeddings using neural networks so as to increase discriminative abilities of similarity-based weighted voting in PBLF. As particular cases, our framework includes embeddings with different complexities, namely, a simple scaling, an affine transformation, and non-linear transformations. We compare our method with state-of-the-art alternatives in whole hippocampus and hippocampal subfields segmentation experiments using publicly available datasets. Results show that even the simplest versions of our method outperform standard PBLF, thus evidencing the benefits of discriminative learning. More complex transformation models tended to achieve better results than simpler ones, obtaining a considerable increase in average Dice score compared to standard PBLF.

44. Deformable image registration for dose accumulation: Principle and evaluation

Most Treatment Planning Systems for adaptive radiotherapy today propose Deformable Image Registration (DIR) tools which enable contour propagation or dose accumulation between images acquired over the course of treatment. While the results of DIR for contour propagation can be easily assessed and eventually manually corrected, the evaluation of DIR for dose accumulation relies on a knowledge of anatomical point correspondences and is therefore much more challenging. This talk will review the methodological principle of DIR for dose accumulation and current evaluation methods. Accumulating dose in a tissue element contained in a voxel of the reference image requires to determine the location of that element in the considered longitudinal images. The role of DIR is to automatically establish those spatial correspondences usually in the form of displacement vector fields (DVF). The first natural way to evaluate DIR results is to deform the longitudinal image onto the reference image based on the DVF and to measure the resulting similarity between the images. However, as demonstrated in previous studies, popular DIR algorithms for contour propagation can lead to high similarity between the deformed and reference images and critically wrong anatomical point correspondences. For this reason, image similarity measures or visual assessment with image fusion tools should be considered in clinical practice only after the local accuracy (in millimeters) has been quantified for a significant number of cases. Among DIR validation methods for dose accumulation, the most reliable one consists of manually identifying pairs of landmarks in the images and to relate the misalignment of those landmark pairs after DIR to the spatial resolution of the dose distributions. This approach can be easily implemented for some anatomical localizations but identifying correspondences on the surface of smooth organs or in large homogeneous tissues is not always feasible. In that case, DIR validation will have to verify that the estimated DVF respect physical considerations such as a reasonable range of local volume variations measured with the Jacobian determinant of the DVF. DIR algorithms can also be tested with images derived from numerical or physical phantoms which provide a ground truth deformation. Phantoms have demonstrated that traditional DIR algorithms could fail to estimate complex deformations such as a sliding motion between organs and justified the need to develop anatomical site-specific models. Finally, a limited number of studies have addressed DIR of longitudinal images presenting inconsistent content (i.e. in case of tumor growth or erosion) and a special attention should be paid in those situations. In parallel of efforts to develop more accurate DIR methods, it is important today to define a commissioning strategy for available DIR solutions to be used in clinical trials involving dose accumulation. In that sense, a Task Group of the American Association of Medical Physicists recently reported its recommendations.

Robust Image Similarity Measurement based on MR Physical Information

Robust Image Similarity Measurement based on MR Physical Information

Quality of biological images, reconstructed using localization microscopy data.

MotivationFluorescence localization microscopy is extensively used to study the details of spatial architecture of subcellular compartments. This modality relies on determination of spatial positions of fluorophores, labeling an extended biological structure, with precision exceeding the diffraction limit. Several established models describe influence of pixel size, signal-to-noise ratio and optical resolution on the localization precision. The labeling density has been also recognized as important factor affecting reconstruction fidelity of the imaged biological structure. However, quantitative data on combined influence of sampling and localization errors on the fidelity of reconstruction are scarce. It should be noted that processing localization microscopy data is similar to reconstruction of a continuous (extended) non-periodic signal from a non-uniform, noisy point samples. In two dimensions the problem may be formulated within the framework of matrix completion. However, no systematic approach has been adopted in microscopy, where images are typically rendered by representing localized molecules with Gaussian distributions (widths determined by localization precision).ResultsWe analyze the process of two-dimensional reconstruction of extended biological structures as a function of the density of registered emitters, localization precision and the area occupied by the rendered localized molecule. We quantify overall reconstruction fidelity with different established image similarity measures. Furthermore, we analyze the recovered similarity measure in the frequency space for different reconstruction protocols. We compare the cut-off frequency to the limiting sampling frequency, as determined by labeling density.Availability and implementationThe source code used in the simulations along with test images is available at https://github.com/blazi13/qbioimages.Supplementary information Supplementary data are available at Bioinformatics online.

A Work Area Visualization by Multi-View Camera-Based Diminished Reality

Hand-held tools are indispensable for efficient manual working in fields ranging from woodworking to surgery. In this paper, we present a diminished reality (DR) method to visualize areas occluded by hands and tools in various hand-working scenarios. We propose a redesigned existing arbitrary viewpoint image generation method for DR applications as a core DR background rendering to recover views without undesirable objects. We conducted quantitative and qualitative experiments using real data to validate the performance of the proposed method. The experimental results showed that our method runs in real-time (40.1 fps) and surpasses conventional methods (including image inpainting-based and geometry-based approaches) in terms of image similarity measures.

A Feature-Based Structural Measure: An Image Similarity Measure for Face Recognition

Facial recognition is one of the most challenging and interesting problems within the field of computer vision and pattern recognition. During the last few years, it has gained special attention due to its importance in relation to current issues such as security, surveillance systems and forensics analysis. Despite this high level of attention to facial recognition, the success is still limited by certain conditions; there is no method which gives reliable results in all situations. In this paper, we propose an efficient similarity index that resolves the shortcomings of the existing measures of feature and structural similarity. This measure, called the Feature-Based Structural Measure (FSM), combines the best features of the well-known SSIM (structural similarity index measure) and FSIM (feature similarity index measure) approaches, striking a balance between performance for similar and dissimilar images of human faces. In addition to the statistical structural properties provided by SSIM, edge detection is incorporated in FSM as a distinctive structural feature. Its performance is tested for a wide range of PSNR (peak signal-to-noise ratio), using ORL (Olivetti Research Laboratory, now AT&T Laboratory Cambridge) and FEI (Faculty of Industrial Engineering, São Bernardo do Campo, São Paulo, Brazil) databases. The proposed measure is tested under conditions of Gaussian noise; simulation results show that the proposed FSM outperforms the well-known SSIM and FSIM approaches in its efficiency of similarity detection and recognition of human faces.