Image Similarity Measure Research Articles

SIMILARITY CHECKING OF CCTV IMAGES USING PEARSON CORRELATION: IMPLEMENTATION WITH PYTHON

Video surveillance technology, such as CCTV, is increasingly common in various applications, including public safety and business surveillance. Analyzing and comparing images from CCTV systems is essential for ensuring safety and security. This research implements the Pearson Correlation method in Python to measure the similarity of CCTV images. Pearson Correlation, which assesses the linear relationship between two variables, is employed to compare the pixel values of two images, resulting in a coefficient that indicates the degree of similarity. We used a quantitative approach with experiments on two scenarios to test the program's effectiveness in measuring image similarity. The results demonstrate that Pearson Correlation is highly effective in distinguishing between identical and other images, providing a more accurate and comprehensive assessment of image similarity compared to histogram analysis. However, the findings are constrained by the specific scenarios and dataset utilized. Further research with more diverse empirical data is required to generalize these results across a broader range of CCTV conditions.

A Framework for Extensive Content-Based Image Retrieval System Incorporating Relevance Feedback and Query Suggestion

Over the last decade, there is a huge amount of images in which visual information has become increasingly important. If the images are to be utilized, they must be organized into databases, from there it can be searched based on different criteria. Content-based Image retrieval (CBIR) frameworks have turned into a mainstream subject of exploration; for their ability to retrieve images based on the real visual content rather than by manually connected textual descriptions. In the CBIR framework, analysis and interpretation of image information in large and diverse image databases is evidently complex because there is no prior information on the size or scale of individual structures within the images to be analyzed. In CBIR, retrieval is based on visual image features, which can be extracted automatically from the images with the help of human intervention, namely a technique called relevance feedback. Nonetheless, an efficient way of differentiating the visual content of images is complex to produce. Therefore, rather than a perfect solution CBIR systems must be able to exploit a partial solution to the problem of image understanding. In this paper, there is implementation of a CBIR framework is introduced that not only tries to efficiently capture the user intent based on the feedback but also provide query suggestions that can help its users to pose better queries in order to retrieve desired results in an efficient manner. The proposed technique is straightforward to implement and scopes efficiently to huge datasets. Extensive experiments on diverse real datasets with image similarity measures have revealed the dominance of the proposed method over original algorithms.

Disambiguity and Alignment: An Effective Multi-Modal Alignment Method for Cross-Modal Recipe Retrieval.

As a prominent topic in food computing, cross-modal recipe retrieval has garnered substantial attention. However, the semantic alignment across food images and recipes cannot be further enhanced due to the lack of intra-modal alignment in existing solutions. Additionally, a critical issue named food image ambiguity is overlooked, which disrupts the convergence of models. To these ends, we propose a novel Multi-Modal Alignment Method for Cross-Modal Recipe Retrieval (MMACMR). To consider inter-modal and intra-modal alignment together, this method measures the ambiguous food image similarity under the guidance of their corresponding recipes. Additionally, we enhance recipe semantic representation learning by involving a cross-attention module between ingredients and instructions, which is effective in supporting food image similarity measurement. We conduct experiments on the challenging public dataset Recipe1M; as a result, our method outperforms several state-of-the-art methods in commonly used evaluation criteria.

Proposal of a similarity measure for unified modeling language class diagram images using convolutional neural network

The unified modeling language (UML) represents an essential tool for modeling and visualizing software systems. UML diagrams provide a graphical representation of a system's components. Comparing and processing these diagrams, for instance, can be complicated, especially as software projects grow in size and complexity. In such contexts, deep learning techniques have emerged as a promising solution for solving complex problems. One of these crucial problems is the measurement of similarity between images, making it possible to compare and calculate the differences between two given diagrams. The present work intends to build a method for calculating the degree of similarity between two UML class diagrams. With a goal to provide teachers a helpful tool for assessing students' UML class diagrams.

Contrast-enhanced to non-contrast-enhanced image translation to exploit a clinical data warehouse of T1-weighted brain MRI

BackgroundClinical data warehouses provide access to massive amounts of medical images, but these images are often heterogeneous. They can for instance include images acquired both with or without the injection of a gadolinium-based contrast agent. Harmonizing such data sets is thus fundamental to guarantee unbiased results, for example when performing differential diagnosis. Furthermore, classical neuroimaging software tools for feature extraction are typically applied only to images without gadolinium. The objective of this work is to evaluate how image translation can be useful to exploit a highly heterogeneous data set containing both contrast-enhanced and non-contrast-enhanced images from a clinical data warehouse.MethodsWe propose and compare different 3D U-Net and conditional GAN models to convert contrast-enhanced T1-weighted (T1ce) into non-contrast-enhanced (T1nce) brain MRI. These models were trained using 230 image pairs and tested on 77 image pairs from the clinical data warehouse of the Greater Paris area.ResultsValidation using standard image similarity measures demonstrated that the similarity between real and synthetic T1nce images was higher than between real T1nce and T1ce images for all the models compared. The best performing models were further validated on a segmentation task. We showed that tissue volumes extracted from synthetic T1nce images were closer to those of real T1nce images than volumes extracted from T1ce images.ConclusionWe showed that deep learning models initially developed with research quality data could synthesize T1nce from T1ce images of clinical quality and that reliable features could be extracted from the synthetic images, thus demonstrating the ability of such methods to help exploit a data set coming from a clinical data warehouse.

A Regional Brightness Control Method for a Beam Projector to Avoid Human Glare

In this study, we proposed a system to reduce the speaker’s suffering from the strong light of a beam projector by applying regional brightness control over the screen. Since the original image and the projected one on the screen are quite different in area, brightness, and color, the proposed system first transforms them so that they have the same area and similar color tone. Then, to accurately determine the difference between those images, we have introduced a SSIM map, which is a perception-based method of measuring image similarity. Accordingly, an image segmentation model is used to determine the speaker’s silhouette from the SSIM map. We applied a couple of well-trained segmentation models, such as Selfie and DeepLab-v3, provided with MediaPipe. The experimental results showed the operability of the proposed system and that it determines most of a lecturer’s body area on the screen. To closely evaluate the system’s effectiveness, we have measured error rates consisting of false-positive and false-negative errors in the confusion matrix. With the measured results, the error rates appeared so insignificant and stable that the proposed system provides a practical effect for the speakers, especially in the case of applying DeepLab-v3. With the results, it is implied that an accurate segmentation model can considerably elevate the effectiveness of the system.

Self-supervised multicontrast super-resolution for diffusion-weighted prostate MRI.

This study addresses the challenge of low resolution and signal-to-noise ratio (SNR) in diffusion-weighted images (DWI), which are pivotal for cancer detection. Traditional methods increase SNR at high b-values through multiple acquisitions, but this results in diminished image resolution due to motion-induced variations. Our research aims to enhance spatial resolution by exploiting the global structure within multicontrast DWI scans and millimetric motion between acquisitions. We introduce a novel approach employing a "Perturbation Network" to learn subvoxel-size motions between scans, trained jointly with an implicit neural representation (INR) network. INR encodes the DWI as a continuous volumetric function, treating voxel intensities of low-resolution acquisitions as discrete samples. By evaluating this function with a finer grid, our model predicts higher-resolution signal intensities for intermediate voxel locations. The Perturbation Network's motion-correction efficacy was validated through experiments on biological phantoms and in vivo prostate scans. Quantitative analyses revealed significantly higher structural similarity measures of super-resolution images to ground truth high-resolution images compared to high-order interpolation (p 0.005). In blind qualitative experiments, of super-resolution images were assessed to have superior diagnostic quality compared to interpolated images. High-resolution details in DWI can be obtained without the need for high-resolution training data. One notable advantage of the proposed method is that it does not require a super-resolution training set. This is important in clinical practice because the proposed method can easily be adapted to images with different scanner settings or body parts, whereas the supervised methods do not offer such an option.

Numerical study of satellite image quality in the presence of Seidel aberrations

AbstractWe present estimated images of a hypothetical satellite system in the presence of Seidel aberrations. We exemplified the effects of defocus, spherical, coma, astigmatism, field curvature, and distortion aberrations. Our analysis methods allow the investigation of aberration effects individually and in any combined form. Thereby, we introduce a potentially rich analysis dimension for the imaging chain, image processing, and image quality analyses. For a number of degrees of aberrations, we provide image estimates, show and quantify how individual (field‐dependent) aberrations would introduce degradations on the satellite images. We got correlation coefficient values for defocus, spherical, coma, astigmatism, field curvature, and distortion aberrations as 0.69, 0.71, 0.66, 0.79, 0.76, and 0.60, respectively. Thereby, we identfied the significance order of Seidel aberrations for this system by using image simulations and image similarity measures.

Alignment of multimodal rigid cardiac angiography images with an improved particle swarm algorithm

Abstract In this paper, we conduct a preliminary study on the current development status in medical image alignment and build up a basic framework for image alignment. The feature space, search space, similarity measure, and search strategy of cardiac angiography images are calculated and studied. The DGVF model is utilized to process the traditional snake model for optimization search and is combined with B-splines to construct the B-spline DGVF model. Optimize the traditional MsFCM algorithm by using the PSO algorithm and propose an MsFCM-PSO image segmentation method. It is applied together with the B-spline DGVF model to segment the vascular lumen in cardiac angiography ultrasound images. Finally, the model of this paper is analyzed in terms of segmentation performance, alignment stability, and evaluation of alignment results. The mean values of Dice, IoU, and HD of this paper’s MsFCM-PSO model in image segmentation of cardiac vessels are 94.27%, 92.60%, and 1.06, respectively (all optimal performances). In the ablation experiments, the MsFCMPSO model in this paper shows an increase of 6.02% and 5.47% in the mean values of Dice and IoU compared to the benchmark model. The stability calibration percentage of this paper’s MsFCM-PSO algorithm is 31.13% when the Gaussian factor is 0.5, which is significantly better than other algorithms. The algorithm in this paper is better than other methods in terms of alignment stability and alignment results.

Automated filtering of façade defect images using a similarity method for enhanced inspection documentation

ABSTRACT Unmanned Aerial Vehicles (UAVs) are routinely utilized to capture images of building façades for structural health inspections. These images are often taken at regular time intervals, capturing overlapping views to ensure complete coverage of the façade. However, this practice generates numerous highly similar images, leading to inefficiencies in documentation and potentially biased analysis due to redundant data. Hence, this study proposes an automated image data filtering method based on defect similarity. Given façade images where defects are detected, this method applies key point detection and descriptor matching to measure image similarity concerning defect presence. Subsequently, a selection strategy is proposed to produce a minimal set of representative images covering all defects. This method was validated using a dataset of UAV-captured façade images across three façades, successfully deriving the set to six representative images. Overall, the proposed method offers a practical solution to reduce the quantity of images required for documentation, thereby enhancing the efficiency of UAV deployments for façade inspection.

CHALLENGES IN PREPARING DATASETS FOR SUPER-RESOLUTION ON THE EXAMPLE OF SENTINEL-2 AND PLANET SCOPE IMAGES

Abstract. Benchmark datasets is an significant aspect in in many areas such as computer vision, deep learning, geospatial data as they serve as standardized test sets for evaluating the performance of models. Among many techniques of image processing, there is super-resolution (SR) which is aimed at reconstructing a low-resolution (LR) image into a high-resolution (HR) image. For training and validation SR models as a dataset the pairs of HR and LR images are needed, which should be the same apart from resolution. There is a lot of benchmark datasets for super-resolution methods, but they usually include conventional photographs of an common objects, while remote sensing data have different characteristic in general. This paper focuses on the process of preparing datasets for super-resolution in satellite images, where high-resolution and low-resolution image data come from different sources. The case of the single-image super-resolution method was considered. The experiment was performed on Sentinel-2 and PlanetScope data, but the assumptions can also be transferred to data obtained from other satellites. The procedure on how to make the pairs of HR and LR images consistent in terms of time, location and spectral values was proposed. The impact of the processes carried out was measured using image similarity measurement methods such as PSNR, SSIM and SCC.

Explainable Image Similarity: Integrating Siamese Networks and Grad-CAM.

With the proliferation of image-based applications in various domains, the need for accurate and interpretable image similarity measures has become increasingly critical. Existing image similarity models often lack transparency, making it challenging to understand the reasons why two images are considered similar. In this paper, we propose the concept of explainable image similarity, where the goal is the development of an approach, which is capable of providing similarity scores along with visual factual and counterfactual explanations. Along this line, we present a new framework, which integrates Siamese Networks and Grad-CAM for providing explainable image similarity and discuss the potential benefits and challenges of adopting this approach. In addition, we provide a comprehensive discussion about factual and counterfactual explanations provided by the proposed framework for assisting decision making. The proposed approach has the potential to enhance the interpretability, trustworthiness and user acceptance of image-based systems in real-world image similarity applications.

Implementing the map task in applied linguistics research: What, how, and why

The “map task” is an interactive, goal-driven, real-time conversational task used to elicit semi-controlled natural language production data. We present recommendations for creating a bespoke map task that can be tailored to individual research projects and administered online using a chat interface. As proof of concept, we present a case study exemplifying our own implementation, designed to elicit informal written communication in either English or Spanish. Eight experimental maps were created, manipulating linguistic factors including lexical frequency, cognate status, and semantic ambiguity. Participants (N = 40) completed the task in pairs and took turns (i) providing directions based on a pre-traced route, or (ii) following directions to draw the route on an empty map. Computational measures of image similarity (e.g., structural similarity index) between pre-traced and participant-traced routes showed that participants completed the task successfully; we describe use of this method for measuring task success quantitatively. We also provide a comparative analysis of the language elicited in English and Spanish. The most frequently used words were roughly equivalent in both languages, encompassing primarily commands and items on the maps. Similarly, abbreviations, swear words, and slang present in both datasets indicated that the task successfully elicited informal communication. Interestingly, Spanish turns were longer and displayed a wider range of morphologically complex forms. English, conversely, displayed strategies mostly absent in Spanish, such as the use of cardinal directions as a communicative strategy. We consider the online map task as a promising method for examining a variety of phenomena in applied linguistics research.

Vision-based image similarity measurement for image search similarity

Vision-based image similarity measurement for image search similarity

Evaluating Text-to-Image GANs Performance: A Comparative Analysis of Evaluation Metrics

Generative Adversarial Networks (GANs) have emerged as powerful techniques for generating high-quality images in various domains but assessing how realistic the generated images are is a challenging task. To address this issue, researchers have proposed a variety of evaluation metrics for GANs, each with its own strengths and limitations. This paper presents a comprehensive analysis of popular GAN evaluation metrics, including FID, Mode Score, Inception Score, MMD, PSNR, and SSIM. The strengths, weaknesses, and calculation processes of these metrics are discussed, focusing on assessing image fidelity and diversity. Two approaches, pixel distance, and feature distance, are employed to measure image similarity, while the importance of evaluating individual objects using input captions is emphasized. Experimental results on a basic GAN trained on the MNIST dataset demonstrate improvement in various metrics across different epochs. The FID score decreases from 497.54594 at Epoch 0 to 136.91156 at Epoch 100, indicating improved differentiation between real and generated images. In addition, the Inception Score increases from 1.1533 to 1.6408, reflecting enhanced image quality and diversity. These findings highlight the effectiveness of the GAN model in generating more realistic and diverse images with training progression. However, when it comes to evaluating GANs on complex datasets, challenges arise, highlighting the need to combine evaluation metrics with visual inspection and subjective measures of image quality. By adopting a comprehensive evaluation approach, researchers can gain a deeper understanding of GAN performance and guide the development of advanced models.

Weakly supervised volumetric prostate registration for MRI-TRUS image driven by signed distance map

Image registration is a fundamental step for MRI-TRUS fusion targeted biopsy. Due to the inherent representational differences between these two image modalities, though, intensity-based similarity losses for registration tend to result in poor performance. To mitigate this, comparison of organ segmentations, functioning as a weak proxy measure of image similarity, has been proposed. Segmentations, though, are limited in their information encoding capabilities. Signed distance maps (SDMs), on the other hand, encode these segmentations into a higher dimensional space where shape and boundary information are implicitly captured, and which, in addition, yield high gradients even for slight mismatches, thus preventing vanishing gradients during deep-network training. Based on these advantages, this study proposes a weakly-supervised deep learning volumetric registration approach driven by a mixed loss that operates both on segmentations and their corresponding SDMs, and which is not only robust to outliers, but also encourages optimal global alignment. Our experimental results, performed on a public prostate MRI-TRUS biopsy dataset, demonstrate that our method outperforms other weakly-supervised registration approaches with a dice similarity coefficient (DSC), Hausdorff distance (HD) and mean surface distance (MSD) of 87.3 ± 11.3, 4.56 ± 1.95 mm, and 0.053 ± 0.026 mm, respectively. We also show that the proposed method effectively preserves the prostate gland's internal structure.

Analysis of different color unmixing methods for cMET-PDL1-EGFR multiplex assay.

e21080 Background: Multiplex brightfield imaging offers the advantage of simultaneously analyzing multiple biomarkers on a single slide, as opposed to single biomarker labeling on multiple consecutive slides. To accurately analyze multiple biomarkers localized to the same cellular compartment, a set of common lung panel markers were selected as a model, cMET-PDL1-EGFR. One of the most crucial preliminary stages for analyzing this assay is identifying the unique chromogens on individual cells. This is a challenging problem due to the co-localization of membranes from all the three biomarkers. It requires advanced color unmixing for creating equivalent singleplex images (referred as synthetic singleplexes) from each triplex image for each biomarker. Methods: We explored different unsupervised methods for unmixing cMET-PDL1-EGFR images. Initially we explored an unsupervised method based on Lee et al. [1], which formulated the color unmixing problem as a non-negative matrix factorization (NMF) method and proposed a system capable of performing the color decomposition in a fully automated manner, where no reference stain color selection is required. Next, we developed cycle-Generative Adversarial (cycle-GAN) [2] models, for each of the three stains – TAMRA (purple), QM-Dabsyl (yellow) and Green. The models were trained on 6 cases of lung having different intensities of the stains. Results: The NMF and cycle-GAN method was validated on 5400 lung images. On analysis of the results and on review with pathologists, we observed that NMF was able to produce reasonable results on synthetic PDL1 and EGFR, but the synthetic Green images did not replicate the adjacent Green images properly. On the other hand, cycle-GAN was able to generate similar synthetic images for all three stains as demonstrated by the similarity metrics in Table 1. Pathologist review of the cycle-GAN results determined that the difference in percentage scores between pairs of adjacent and synthetic singleplex images would be within the range 2%-5% only. Conclusions: In this project, we performed detailed analysis of two unsupervised methods for color unmixing on triplex images. On comparison of the methods based on image similarity measures and pathologist review, we conclude that the cycle-GAN method is competent in generating synthetic images.

A Novel Image Similarity Measure Based on Greatest and Smallest Eigen Fuzzy Sets

A novel image similarity index based on the greatest and smallest fuzzy set solutions of the max–min and min–max compositions of fuzzy relations, respectively, is proposed. The greatest and smallest fuzzy sets are found symmetrically as the min–max and max–min solutions, respectively, to a fuzzy relation equation. The original image is partitioned into squared blocks and the pixels in each block are normalized to [0, 1] in order to have a fuzzy relation. The greatest and smallest fuzzy sets, found for each block, are used to measure the similarity between the original image and the image reconstructed by joining the squared blocks. Comparison tests with other well-known image metrics are then carried out where source images are noised by applying Gaussian filters. The results show that the proposed image similarity measure is more effective and robust to noise than the PSNR and SSIM-based measures.

Mixture of multivariate generalized Gaussians for multi-band texture modeling and representation

We present a unified statistical model for multivariate and multi-modal texture representation. This model is based on the formalism of finite mixtures of multivariate generalized Gaussians (MoMGG) which enables to build a compact and accurate representation of texture images using multi-resolution texture transforms. The MoMGG model enables to describe the joint statistics of subbands in different scales and orientations, as well as between adjacent locations within the same subband, providing a precise description of the texture layout. It can also combine different multi-scale transforms to build a richer and more representative texture signature for image similarity measurement. We tested our model on both traditional texture transforms (e.g., wavelets, contourlets, maximum response filter) and convolution neural networks (CNNs) features (e.g., ResNet, SqueezeNet). Experiments on color-texture image retrieval have demonstrated the performance of our approach comparatively to state-of-the-art methods.

Fully automated radiologic identification focusing on the sternal bone

A crucial task in forensic investigations is the identification of unknown deceased. In general, secure identification methods rely on a comparison of ante mortem (AM) with post mortem (PM) data. However, available morphologic approaches are often dependent on the expertise and experience of the examiner, and often lack standardisation and statistical evidence. The objective of this study was therefore to overcome the current challenges via developing a fully automated radiologic identification (autoRADid) method based on the sternal bone. An anonymised AM data set consisting of 91 chest computed tomography (CT) scans, as well as an anonymised PM data set of 42 chest CT scans were included in this work. Out of the 91 available AM CT data sets, 42 AM scans corresponded to the 42 PM CT scans. For the fully automated identification analysis, a custom-made python pipeline was developed, which automatically registers AM data to the PM data in question using a two-step registration method. To evaluate the registration procedure and subsequent identification success, the image similarity measures Jaccard Coefficient, Dice Coefficient, and Mutual Information were computed. The highest value for each metric was retrieved in order to analyse the correspondence between AM and PM data. For all three similarity measures, 38 out of the 42 cases were matched correctly. This corresponds to an accuracy of 91.2%. The four unsuccessful cases incorporated surgical interventions taking place between the AM and the PM CT acquisition or poor CT scan quality preventing robust registration results. To conclude, the presented autoRADid method seems to be a promising fully automated tool for a reliable and facile identification of unknown deceased. A final pipeline combining all three similarity measures is open source and publicly available for efficient future identifications of unknown deceased.