Similarity Measure Research Articles

TransExION: a transformer based explainable similarity metric for comparing IONS in tandem mass spectrometry

Small molecule identification is a crucial task in analytical chemistry and life sciences. One of the most commonly used technologies to elucidate small molecule structures is mass spectrometry. Spectral library search of product ion spectra (MS/MS) is a popular strategy to identify or find structural analogues. This approach relies on the assumption that spectral similarity and structural similarity are correlated. However, popular spectral similarity measures, usually calculated based on identical fragment matches between the MS/MS spectra, do not always accurately reflect the structural similarity. In this study, we propose TransExION, a Transformer based Explainable similarity metric for IONS. TransExION detects related fragments between MS/MS spectra through their mass difference and uses these to estimate spectral similarity. These related fragments can be nearly identical, but can also share a substructure. TransExION also provides a post-hoc explanation of its estimation, which can be used to support scientists in evaluating the spectral library search results and thus in structure elucidation of unknown molecules. Our model has a Transformer based architecture and it is trained on the data derived from GNPS MS/MS libraries. The experimental results show that it improves existing spectral similarity measures in searching and interpreting structural analogues as well as in molecular networking.Scientific ContributionWe propose a transformer-based spectral similarity metrics that improves the comparison of small molecule tandem mass spectra. We provide a post hoc explanation that can serve as a good starting point for unknown spectra annotation based on database spectra.

Semantically Enriched Keyword Prefetching Based on Usage and Domain Knowledge

In intelligent web systems [2], web prefetching [27] plays a crucial role. In order to make accurate predictions for web prefetching, it is important but challenging to uncover valuable information from web use statistics [16]. Using statistics and domain expertise, this study presents a new approach dubbed SPUDK for efficient prefetching. In this paper, it is shown how web access logs can be used efficiently for browsing prediction. Our main focus is on the technique needed to manage the queries found in web access logs so that valuable information can be attained. We further process these access logs using a taxonomy and a thesaurus, WordNet, to find the semantics of queries. SPUDK, a system that organises use data into semantic clusters, is one example of this approach. Our contributions in this paper are as follows: (1) A technique to exploit query keywords from access logs. (2) An approach to enrich queries with semantic information. (3) A new similarity measure for finding similarity among URLs present in access logs. (4) A novel clustering technique to find semantic clusters of URLs. (5) Experimental evaluation of the proposed system. The proposed SPUDK system is evaluated using American Online (AOL) logs, which gives improvement of 39% in precision of prediction, 35% in hit ratio and reduction of 50.6% in latency on average as compared to other prediction techniques in the literature.

Career path discovery through bipartite graphs

ABSTRACT To address the complexity of career decision-making limitations, we propose a novel framework for calculating occupation similarity measures based on bipartite graphs constructed from public occupation-skills ontologies. The proposed occupation similarity measures were constructed exclusively utilising knowledge from the European standard classification of occupations (ESCO) ontology. The resulting occupation similarity measures are fully explainable and computationally efficient. Furthermore, they are are effortlessly transferable across regions and countries. This approach allows us to tailor the similarity measures to the specific needs, preferences, and career status of job seekers, providing a more personalised and comprehensive view of potential career paths. Our validation using an extensive dataset of over 450,000 job transitions in Slovenia confirms the effectiveness of our approach, demonstrating the value of employing multiple occupation similarity measures over a single measure.

Multi-perspective learning for enhanced user preferences for session-based recommendation

Due to the sparsity and anonymity of interactions within sessions, capturing user preferences by modeling item relationships poses a significant challenge. Previous methods have attempted to utilize collaborative information from adjacent sessions or item attribute knowledge as supplements. However, these methods are prone to introducing interfering information through their session similarity measurement strategies and attribute embedding learning processes, both requiring optimization. Moreover, they often fail to fully model global item relationships. To address these issues, we propose a novel method called Multi-perspective Learning for Enhanced User Preferences (MLEUP) for session-based recommendations. Specifically, we first construct reliable session groups by measuring the overlap and incompatibility of items across sessions to calculate session similarity. Then, by modeling attribute–item and attribute–attribute relationships, along with employing a neighbor-based contrastive self-supervised attribute learning task, we learn high-quality attribute embeddings that capture the user’s stable preferences. Next, leveraging the session groups, we model global item multi-granularity co-occurrence and high-order sequential relationships through designed attention layers, enabling us to learn informative item embeddings that represent the user’s dynamic preferences. Finally, we further model the correlated relationship between the two user preference representations. Our approach ensures that effective attribute knowledge and global collaborative information are learned to mitigate the data sparsity issue, while comprehensively modeling global item relationships to capture the user’s complex dynamic preferences. Experimental results demonstrate that MLEUP achieves state-of-the-art performance in terms of precision, MRR, and NDCG. The experimental datasets and source code of MLEUP are publicly available at https://github.com/lean030/MLEUP.

A Robust Mismatch Removal Method for Image Matching Based on the Fusion of the Local Features and the Depth

Feature point matching is a fundamental task in computer vision such as vision simultaneous localization and mapping (VSLAM) and structure from motion (SFM). Due to the similarity or interference of features, mismatches are often unavoidable. Therefore, how to eliminate mismatches is important for robust matching. Smoothness constraint is widely used to remove mismatch, but it cannot effectively deal with the issue in the rapidly changing scene. In this paper, a novel LCS-SSM (Local Cell Statistics and Structural Similarity Measurement) mismatch removal method is proposed. LCS-SSM integrates the motion consistency and structural similarity of a local image block as the statistical likelihood of matched key points. Then, the Random Sampling Consensus (RANSAC) algorithm is employed to preserve the isolated matches that do not satisfy the statistical likelihood. Experimental and comparative results on the public dataset show that the proposed LCS-SSM can effectively and reliably differentiate true and false matches compared with state-of-the-art methods, and can be used for robust matching in scenes with fast motion, blurs, and clustered noise.

Locally-Scaled Kernels and Confidence Voting

Classification, the task of discerning the class of an unlabeled data point using information from a set of labeled data points, is a well-studied area of machine learning with a variety of approaches. Many of these approaches are closely linked to the selection of metrics or the generalizing of similarities defined by kernels. These metrics or similarity measures often require their parameters to be tuned in order to achieve the highest accuracy for each dataset. For example, an extensive search is required to determine the value of K or the choice of distance metric in K-NN classification. This paper explores a method of kernel construction that when used in classification performs consistently over a variety of datasets and does not require the parameters to be tuned. Inspired by dimensionality reduction techniques (DRT), we construct a kernel-based similarity measure that captures the topological structure of the data. This work compares the accuracy of K-NN classifiers, computed with specific operating parameters that obtain the highest accuracy per dataset, to a single trial of the here-proposed kernel classifier with no specialized parameters on standard benchmark sets. The here-proposed kernel used with simple classifiers has comparable accuracy to the ‘best-case’ K-NN classifiers without requiring the tuning of operating parameters.

#1506 Uremic toxicity: gaining novel insights through AI-driven literature review

Abstract Background and Aims The rapidly growing scientific literature poses a significant challenge for researchers seeking to distill key insights. We utilized Retrieval-Augmented Generation (RAG), a novel AI-driven approach, to efficiently process and extract meaningful information from published literature on uremic toxins. RAG is a general AI framework for improving the quality of responses generated by Large Language Models (LLMs) by supplementing the LLM's internal representation of information with curated expert knowledge. Method First, we collected on PubMed all abstracts related to the topic of “uremic toxins” through Metapub, a Python library designed to facilitate fetching metadata from PubMed. Second, we set up a RAG system that comprises 2 steps. In a retrieval step, the questions on topic (“uremic toxins”) and the documents (=all collected abstracts and manuscripts) are encoded into vectors (i.e., high-dimensional numerical representations). Similarity measures are used to find the best matches between documents and the questions on topic. Second, in the augmented generation step, the LLM (e.g., ChatGPT) uses these best matches of documents to generate a coherent and informed response. Results We collected 3497 abstracts from the PubMed and 191 expert-curated publications in PDF format related to the topic “uremic toxin”. These 191 publications were broken down to 5756 documents, each with a manageable size of text. The final vector database comprised 9253 vectors. Using RAG, we requested responses from the LLM on multiple questions related to “uremic toxins”. Some examples are shown in Table 1. The first and second responses given by the LLM are reasonable. However, the third answer shows the phenomenon of ‘hallucination’—where models generate plausible and convincingly sounding yet factually incorrect information. Conclusion The use of RAG improves the capability of LLMs to answer questions by leveraging the information contained within curated abstracts and publications. Despite the improvements with RAG, the phenomenon of ‘hallucination’ persists. A concerning feature of hallucinations is their eloquent and convincing language. For the time being, LLM output—even when improved with RAG—requires scrutiny and human verification.

Neighboring-Aware Hierarchical Clustering

In this work, a simple yet robust neighboring-aware hierarchical-based clustering approach (NHC) is developed. NHC employs its dynamic technique to take into account the surroundings of each point when clustering, making it extremely competitive. NHC offers a straightforward design and reliable clustering. It comprises two key techniques, namely, neighboring- aware and filtering and merging. While the proposed neighboring-aware technique helps find the most coherent clusters, filtering and merging help reach the desired number of clusters during the clustering process. The NHC's performance, which includes all evaluation metrics and run time, has been thoroughly tested against nine clustering rivals using four similarity measures on several real-world numerical and textual datasets. The evaluation is done in two phases. First, we compare NHC to three common clustering methods and show its efficacy through empirical analysis. Second, a comparison with six relevant, contemporary competitors highlights NHC's extremely competitive performance.

Disambiguity and Alignment: An Effective Multi-Modal Alignment Method for Cross-Modal Recipe Retrieval.

As a prominent topic in food computing, cross-modal recipe retrieval has garnered substantial attention. However, the semantic alignment across food images and recipes cannot be further enhanced due to the lack of intra-modal alignment in existing solutions. Additionally, a critical issue named food image ambiguity is overlooked, which disrupts the convergence of models. To these ends, we propose a novel Multi-Modal Alignment Method for Cross-Modal Recipe Retrieval (MMACMR). To consider inter-modal and intra-modal alignment together, this method measures the ambiguous food image similarity under the guidance of their corresponding recipes. Additionally, we enhance recipe semantic representation learning by involving a cross-attention module between ingredients and instructions, which is effective in supporting food image similarity measurement. We conduct experiments on the challenging public dataset Recipe1M; as a result, our method outperforms several state-of-the-art methods in commonly used evaluation criteria.

A Belief Similarity Measure for Dempster-Shafer Evidence Theory and Application in Decision Making

How to effectively deal with uncertain and imprecise information in decision making is a complex task. Dempster-Shafer evidence theory (DSET) is widely used for handling such challenges due to its ability to model uncertainty and imprecision. However, Dempster's rule can sometimes yield counterintuitive results when dealing with highly conflicting evidence. In this paper, we introduce a novel belief sine similarity measure, called $BS^2M$, which effectively measures the discrepancy between different pieces of evidence. We also establish that $BS^2M$ possesses important properties such as boundedness, symmetry, and non-degeneracy. Building upon $BS^2M$, we present a new method for decision making. The proposed method considers both the credibility and the information volume of each evidence, providing a more comprehensive reflection of their importance. To validate our method, we conduct experiment in target recognition application, demonstrating the effectiveness and rationality of the proposed method.

Enhanced EDAS technique for quality evaluation of online games interface design based on three types of probabilistic linguistic similarity measures

Enhanced EDAS technique for quality evaluation of online games interface design based on three types of probabilistic linguistic similarity measures

Cervic cancer classification using quantum fuzzy set

In this sophisticated world living with CIN cervic cancer is much jeopardy. Cancer is a stochastic (random) process. So, in that CIN in initial stage is not jeopardy. It is totally because of hazardous malign cells. Using its modality in image is selected in existing system only automated classification depends on input image. But in proposed methodology the innovative alludes the jeopardy of CIN cancer is found using the size of /area of nucleus or cytoplasm. This proposed methodology was developed with an algorithm to find CIN area/size. This research work establishes a Cervic Cancer Classification Using Contour Based on Area of Nucleolus and Cytoplasm in Cells (CBANC) which classifies noise spread images into any one of five phases. A similarity measure produces 90% efficiency in proposed system as par with inefficient existing system which fetches us 50%. By pragmatic application it is proved that CBANC with fuzzy is better than Baye’s. This can be accomplished by removing well distinct consistency features and choosing preeminent classifier. Proposed work can extend with 3D input images for future research. It produces mightiest parameter shape and intensity which is very essential for 3D approach. The inference of proposed system can extend the latest classifier engines for more accuracy. It can easily predict more than 90% accuracy will be there. And also derive cancer growing and after therapy for cancer shrinking algorithm will be used for 2D or 3D CIN cancer classification. The outcomes of the proposed methodology CBANC shows that better when compared to the existing methodology like Bayes. It can be implemented in the real world environments of the medical field.

Robust Spherical Laplacian Embedding.

Laplacian embedding (LE) aims to project high-dimensional input data samples, which often contain nonlinear structures, into a low-dimensional space. However, existing distance functions used in the embedding space fail to provide discriminative representations for real-world datasets, especially those related to text analysis or image processing. Cosine similarity measurements are advantageous in dealing with sparse data but are fragile to the impact of outliers and noise from samples or data features. In this work, we propose robust spherical LE (RS-LE), a powerful method that builds the LE on a novel metric that unifies the Euclidean distance and cosine similarity in a spherical space using a robust lp,p -norm. We introduce an efficient iterative algorithm, the proximal alternating linearized minimization (ALM) method, to solve the difficult optimization problem that arises from the new metric, which is neither convex nor smooth. This algorithm allows the solution to achieve both global and objective convergence. Our findings are presented in rigorous theoretical analysis and validated in experiments.

Deformable registration of magnetic resonance images using unsupervised deep learning in neuro-/radiation oncology

PurposeAccurate deformable registration of magnetic resonance imaging (MRI) scans containing pathologies is challenging due to changes in tissue appearance. In this paper, we developed a novel automated three-dimensional (3D) convolutional U-Net based deformable image registration (ConvUNet-DIR) method using unsupervised learning to establish correspondence between baseline pre-operative and follow-up MRI scans of patients with brain glioma.MethodsThis study involved multi-parametric brain MRI scans (T1, T1-contrast enhanced, T2, FLAIR) acquired at pre-operative and follow-up time for 160 patients diagnosed with glioma, representing the BraTS-Reg 2022 challenge dataset. ConvUNet-DIR, a deep learning-based deformable registration workflow using 3D U-Net style architecture as a core, was developed to establish correspondence between the MRI scans. The workflow consists of three components: (1) the U-Net learns features from pairs of MRI scans and estimates a mapping between them, (2) the grid generator computes the sampling grid based on the derived transformation parameters, and (3) the spatial transformation layer generates a warped image by applying the sampling operation using interpolation. A similarity measure was used as a loss function for the network with a regularization parameter limiting the deformation. The model was trained via unsupervised learning using pairs of MRI scans on a training data set (n = 102) and validated on a validation data set (n = 26) to assess its generalizability. Its performance was evaluated on a test set (n = 32) by computing the Dice score and structural similarity index (SSIM) quantitative metrics. The model’s performance also was compared with the baseline state-of-the-art VoxelMorph (VM1 and VM2) learning-based algorithms.ResultsThe ConvUNet-DIR model showed promising competency in performing accurate 3D deformable registration. It achieved a mean Dice score of 0.975 ± 0.003 and SSIM of 0.908 ± 0.011 on the test set (n = 32). Experimental results also demonstrated that ConvUNet-DIR outperformed the VoxelMorph algorithms concerning Dice (VM1: 0.969 ± 0.006 and VM2: 0.957 ± 0.008) and SSIM (VM1: 0.893 ± 0.012 and VM2: 0.857 ± 0.017) metrics. The time required to perform a registration for a pair of MRI scans is about 1 s on the CPU.ConclusionsThe developed deep learning-based model can perform an end-to-end deformable registration of a pair of 3D MRI scans for glioma patients without human intervention. The model could provide accurate, efficient, and robust deformable registration without needing pre-alignment and labeling. It outperformed the state-of-the-art VoxelMorph learning-based deformable registration algorithms and other supervised/unsupervised deep learning-based methods reported in the literature.

Link Prediction in Complex Networks Using Average Centrality-Based Similarity Score.

Link prediction plays a crucial role in identifying future connections within complex networks, facilitating the analysis of network evolution across various domains such as biological networks, social networks, recommender systems, and more. Researchers have proposed various centrality measures, such as degree, clustering coefficient, betweenness, and closeness centralities, to compute similarity scores for predicting links in these networks. These centrality measures leverage both the local and global information of nodes within the network. In this study, we present a novel approach to link prediction using similarity score by utilizing average centrality measures based on local and global centralities, namely Similarity based on Average Degree (SACD), Similarity based on Average Betweenness (SACB), Similarity based on Average Closeness (SACC), and Similarity based on Average Clustering Coefficient (SACCC). Our approach involved determining centrality scores for each node, calculating the average centrality for the entire graph, and deriving similarity scores through common neighbors. We then applied centrality scores to these common neighbors and identified nodes with above average centrality. To evaluate our approach, we compared proposed measures with existing local similarity-based link prediction measures, including common neighbors, the Jaccard coefficient, Adamic-Adar, resource allocation, preferential attachment, as well as recent measures like common neighbor and the Centrality-based Parameterized Algorithm (CCPA), and keyword network link prediction (KNLP). We conducted experiments on four real-world datasets. The proposed similarity scores based on average centralities demonstrate significant improvements. We observed an average enhancement of 24% in terms of Area Under the Receiver Operating Characteristic (AUROC) compared to existing local similarity measures, and a 31% improvement over recent measures. Furthermore, we witnessed an average improvement of 49% and 51% in the Area Under Precision-Recall (AUPR) compared to existing and recent measures. Our comprehensive experiments highlight the superior performance of the proposed method.

REEV: review, evaluate and explain variants.

In the era of high throughput sequencing, special software is required for the clinical evaluation of genetic variants. We developed REEV (Review, Evaluateand Explain Variants), a user-friendly platform for clinicians and researchers in the field of rare disease genetics. Supporting data was aggregated from public data sources. We compared REEV with seven other tools for clinical variant evaluation. REEV (semi-)automatically fills individual ACMG criteria facilitating variant interpretation. REEV can store disease and phenotype data related to a case to use these for phenotype similarity measures. Users can create public permanent links for individual variants that can be saved as browser bookmarks and shared. REEV may help in the fast diagnostic assessment of genetic variants in a clinical as well as in a research context. REEV (https://reev.bihealth.org/) is free and open to all users and there is no login requirement.

Knowledge reduction by combining interval Type-2 Fuzzy similarity measures and interval Type-2 Fuzzy formal lattice

Knowledge reduction by combining interval Type-2 Fuzzy similarity measures and interval Type-2 Fuzzy formal lattice

A movement-aware measure for trajectory similarity and its application for ride-sharing path extraction in a road network

Recognizing common travel paths of crowds in a road network is valuable for understanding human mobility patterns and developing intelligent ride-sharing services. To achieve this, it is critical to measure the similarity of their trajectories. Although many measures have been proposed in the past decades, they often ignore movement consistency, exhibit one or more deficiencies in the face of noise and misaligned trajectories, or require extra parameters to tune predictions. In this paper, we propose an improved similarity measure called the directed segment path distance (DSPD), which considers the spatial proximity and movement consistency of trajectories. By integrating the spatial proximity distance and moving direction similarity between trajectories, the DSPD is a competitive parameter-free similarity measure that can effectively distinguish trajectories with different movement characteristics. To verify the effectiveness of the DSPD, we conducted a quantitative comparative study between the DSPD measure and 11 state-of-the-art trajectory similarity measures on six simulated trajectory datasets and applied the DSPD to two typical application scenarios: trajectory clustering for road network generation and retrieving common trajectories for ride-sharing path planning. The results demonstrate the effectiveness, robustness, and superiority of the DSPD and its great potential in trajectory search, clustering, and classification.

Automating modern code review processes with code similarity measurement

Context:Modern code review is a critical component in software development processes, as it ensures security, detects errors early and improves code quality. However, manual reviews can be time-consuming and unreliable. Automated code review can address these issues. Although deep-learning methods have been used to recommend code review comments, they are expensive to train and employ. Instead, information retrieval (IR)-based methods for automatic code review are showing promising results in efficiency, effectiveness, and flexibility. Objective:Our main objective is to determine the optimal combination of the vectorization method and similarity to measure what gives the best results in an automatic code review, thereby improving the performance of IR-based methods. Method:Specifically, we investigate different vectorization methods (Word2Vec, Doc2Vec, Code2Vec, and Transformer) that differ from previous research (TF-IDF and Bag-of-Words), and similarity measures (Cosine, Euclidean, and Manhattan) to capture the semantic similarities between code texts. We evaluate the performance of these methods using standard metrics, such as Blue, Meteor, and Rouge-L, and include the run-time of the models in our results. Results:Our results demonstrate that the Transformer model outperforms the state-of-the-art method in all standard metrics and similarity measurements, achieving a 19.1% improvement in providing exact matches and a 6.2% improvement in recommending reviews closer to human reviews. Conclusion:Our findings suggest that the Transformer model is a highly effective and efficient approach for recommending code review comments that closely resemble those written by humans, providing valuable insight for developing more efficient and effective automated code review systems.

An encompassed representation of timescale hierarchies in first-order reaction network

Complex networks are pervasive in various fields such as chemistry, biology, and sociology. In chemistry, first-order reaction networks are represented by a set of first-order differential equations, which can be constructed from the underlying energy landscape. However, as the number of nodes increases, it becomes more challenging to understand complex kinetics across different timescales. Hence, how to construct an interpretable, coarse-graining scheme that preserves the underlying timescales of overall reactions is of crucial importance. Here, we develop a scheme to capture the underlying hierarchical subsets of nodes, and a series of coarse-grained (reduced-dimensional) rate equations between the subsets as a function of time resolution from the original reaction network. Each of the coarse-grained representations guarantees to preserve the underlying slow characteristic timescales in the original network. The crux is the construction of a lumping scheme incorporating a similarity measure in deciphering the underlying timescale hierarchy, which does not rely on the assumption of equilibrium. As an illustrative example, we apply the scheme to four-state Markovian models and Claisen rearrangement of allyl vinyl ether (AVE), and demonstrate that the reduced-dimensional representation accurately reproduces not only the slowest but also the faster timescales of overall reactions although other reduction schemes based on equilibrium assumption well reproduce the slowest timescale but fail to reproduce the second-to-fourth slowest timescales with the same accuracy. Our scheme can be applied not only to the reaction networks but also to networks in other fields, which helps us encompass their hierarchical structures of the complex kinetics over timescales.