Text Similarity Measures Research Articles

The European Union in the annual United Nations General Assembly Debates

ABSTRACT We examine foreign policy preference convergence and cohesion among EU states, as revealed in the annual United Nations General Debate (UNGD). Of interest is whether the institutional adaptations following the Lisbon Treaty led to convergence towards the official EU position and more cohesion in speeches delivered by EU states. We argue that the UNGD is a distinct context where EU states feel compelled to differentiate themselves compared to other areas where EU coordination efforts are extensive. We explore EU convergence and cohesion based on all EUGD UNGD speeches between 1990 and 2019. We primarily apply text-as-data techniques to extract states’ positions on well-established foreign policy dimensions. We use wordscores and text similarity measures, comparing our analysis to ideal point estimates based on UNGA voting and co-sponsorship data. We find that EU states use the UNGD as a venue to articulate their own, differentiated positions as individual UN member states.

Meningkatkan Deduplikasi Data melalui Kesamaan Teks dalam Pembelajaran Mesin: Pendekatan Komprehensif

The issue of dirty data, particularly duplicate data, is a common problem in data management that can affect data quality, operational efficiency, and decision-making. This study highlights the importance of implementing sustainable deduplication strategies as a key step in managing dirty data. We explore solutions for detecting duplicate data by measuring text similarity indices. In this study, the authors utilize a literature review research method. Through this method, we collected various journals on data deduplication and text similarity techniques, comparing several methods to identify the most effective approach. The data deduplication process in this study consists of two stages: 1) Matching - calculating the similarity value of a record with previous records, and 2) Clustering - grouping all records deemed duplicates of a single entity. Furthermore, this study extends to the development of a Python application capable of identifying and grouping similar customer data based on text similarity values. The Text Similarity measurement method uses the Jaro-Winkler Similarity technique. Experimental results and evaluations show that the Text Similarity approach is effective in identifying duplicate data with a high degree of accuracy. This study emphasizes the importance of sustainable deduplication, where the deduplication process is conducted periodically and continuously to ensure optimal data quality.

Local large language models for privacy-preserving accelerated review of historic echocardiogram reports.

The study developed framework that leverages an open-source Large Language Model (LLM) to enable clinicians to ask plain-language questions about a patient's entire echocardiogram report history. This approach is intended to streamline the extraction of clinical insights from multiple echocardiogram reports, particularly in patients with complex cardiac diseases, thereby enhancing both patient care and research efficiency. Data from over 10 years were collected, comprising echocardiogram reports from patients with more than 10 echocardiograms on file at the Mount Sinai Health System. These reports were converted into a single document per patient for analysis, broken down into snippets and relevant snippets were retrieved using text similarity measures. The LLaMA-2 70B model was employed for analyzing the text using a specially crafted prompt. The model's performance was evaluated against ground-truth answers created by faculty cardiologists. The study analyzed 432 reports from 37 patients for a total of 100 question-answer pairs. The LLM correctly answered 90% questions, with accuracies of 83% for temporality, 93% for severity assessment, 84% for intervention identification, and 100% for diagnosis retrieval. Errors mainly stemmed from the LLM's inherent limitations, such as misinterpreting numbers or hallucinations. The study demonstrates the feasibility and effectiveness of using a local, open-source LLM for querying and interpreting echocardiogram report data. This approach offers a significant improvement over traditional keyword-based searches, enabling more contextually relevant and semantically accurate responses; in turn showing promise in enhancing clinical decision-making and research by facilitating more efficient access to complex patient data.

The adaptive community-response (ACR) method for collecting misinformation on social media

Social media can be a major accelerator of the spread of misinformation, thereby potentially compromising both individual well-being and social cohesion. Despite significant recent advances, the study of online misinformation is a relatively young field facing several (methodological) challenges. In this regard, the detection of online misinformation has proven difficult, as online large-scale data streams require (semi-)automated, highly specific and therefore sophisticated methods to separate posts containing misinformation from irrelevant posts. In the present paper, we introduce the adaptive community-response (ACR) method, an unsupervised technique for the large-scale collection of misinformation on Twitter (now known as ’X’). The ACR method is based on previous findings showing that Twitter users occasionally reply to misinformation with fact-checking by referring to specific fact-checking sites (crowdsourced fact-checking). In a first step, we captured such misinforming but fact-checked tweets. These tweets were used in a second step to extract specific linguistic features (keywords), enabling us to collect also those misinforming tweets that were not fact-checked at all as a third step. We initially present a mathematical framework of our method, followed by an explicit algorithmic implementation. We then evaluate ACR on the basis of a comprehensive dataset consisting of >25\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$>25$$\\end{document} million tweets, belonging to >300\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$>300$$\\end{document} misinforming stories. Our evaluation shows that ACR is a useful extension to the methods pool of the field, enabling researchers to collect online misinformation more comprehensively. Text similarity measures clearly indicated correspondence between the claims of false stories and the ACR tweets, even though ACR performance was heterogeneously distributed across the stories. A baseline comparison to the fact-checked tweets showed that the ACR method can detect story-related tweets to a comparable degree, while being sensitive to different types of tweets: Fact-checked tweets tend to be driven by high outreach (as indicated by a high number of retweets), whereas the sensitivity of the ACR method extends to tweets exhibiting lower outreach. Taken together, ACR’s capacity as a valuable methodological contribution to the field is based on (i) the adoption of prior, pioneering research in the field, (ii) a well-formalized mathematical framework and (iii) an empirical foundation via a comprehensive set of indicators.

Unveiling the impact of machine learning algorithms on the quality of online geocoding services: a case study using COVID-19 data

AbstractIn today's era, the address plays a crucial role as one of the key components that enable mobility in daily life. Address data are used by global map platforms and location-based services to pinpoint a geographically referenced location. Geocoding provided by online platforms is useful in the spatial tracking of reported cases and controls in the spatial analysis of infectious illnesses such as COVID-19. The first and most critical phase in the geocoding process is address matching. However, due to typographical errors, variations in abbreviations used, and incomplete or malformed addresses, the matching can seldom be performed with 100% accuracy. The purpose of this research is to examine the capabilities of machine learning classifiers that can be used to measure the consistency of address matching results produced by online geocoding services and to identify the best performing classifier. The performance of the seven machine learning classifiers was compared using several text similarity measures, which assess the match scores between the input address data and the services' output. The data utilized in the testing came from four distinct online geocoding services applied to 925 addresses in Türkiye. The findings from this study revealed that the Random Forest machine learning classifier was the most accurate in the address matching procedure. While the results of this study hold true for similar datasets in Türkiye, additional research is required to determine whether they apply to data in other countries.

Back to where it started?—Do expanded auditor's reports become sticky, generic and boilerplate over time?

Although auditing standards instruct auditors to avoid issuing standardised key audit matters (KAMs), standard setters warn and financial statement users worry about expanded auditor's reports becoming sticky, generic and boilerplate over time. Based on textual analysis of expanded auditor's reports from the UK, I provide longitudinal evidence of an increase in various textual similarity measures including year‐over‐year similarity (stickiness), within‐peer group similarity (generality) and boilerplate language over time. Next, I identify a number of crucial drivers of convergence and find that disclosure similarity diminishes the positive capital market effect of precise KAM reporting documented in prior studies. Together, these findings suggest that audit firms have successively developed standard text modules for KAM reporting that undermine the aim of expanded auditor's reports to provide informative value. This paper extends the literature on the relationship between institutional environments and unintended effects of auditor disclosure regulation.

Controllable forward secure identity-based encryption with equality test in privacy-preserving text similarity analysis

Text similarity measures have become increasingly pivotal in text-related research and applications, encompassing tasks such as information retrieval, text classification, document clustering. Unveiling the equality relationship between encrypted words is fundamental for ensuring privacy-preserving text similarity. Identity-based encryption with equality test (IBEET) emerges as a promising solution for computing similarity over encrypted data in cloud environment. However, prevalent IBEET schemes often lack control over the lifespan of trapdoors, potentially leading to misuse and unauthorized disclosure of users' privacy. In this paper, we introduce a novel primitive known as controllable forward secure identity-based encryption with equality test (CFS-IBEET), designed to support both permanent trapdoor and temporary trapdoor. We present a concrete CFS-IBEET scheme based on bilinear pairing and conduct a comprehensive security analysis against two types of adversaries in random oracle model. The comprehensive performance evaluation shows that our CFS-IBEET scheme offers significant advantages of high efficiency of encryption, decryption, trapdoor generation and test algorithms as well as the guarantee of controllable forward security, showcasing its applicability for collaborative filtering recommendation system in E-commerce age.

Automated Knowledge Integration from Heterogeneous Data Sources Using Text Analytics: A Case Study of COVID-19

Gathering information from multiple data sources takes a long time to collect, analyze, and classify. Furthermore, if the data sources have different data structures, the merged data structure must support such heterogeneity. In addition, semantic of data must also be considered. This paper proposes automated knowledge integration from heterogeneous data sources using ontology engineering combined with text analytics. Text stemming is used to preprocess data. Part-of-speech (POS) tagging, Universal Dependencies (UD), and text similarity measurement called cosine similarity are used to analyze and integrate data. Our work focuses on five COVID- 19 knowledge scopes: COVID-19, coronaviruses, diseases, pandemics, and vaccines. For evaluation, six ontologies were constructed with six different cosine similarity values ranging from 0.5 to 1.0. Each constructed ontology has COVID-19 related and non-COVID-19 data in a ratio of 70 to 30. The six constructed ontologies were evaluated for consistency with the original data. Using cosine similarity with 0.6, precision, recall, and F1-score are 0.82, 0.71, and 0.76, respectively, and the constructed ontology is optimal, containing the highest amount of relevant COVID-19 information for this case study.

Learning Discriminative Text Representation for Streaming Social Event Detection

Event detection on social platforms can help people perceive essential events and make actionable decisions. Existing document-pivot streaming social event detection methods generally embed documents and perform text clustering. They face the challenges of constantly changing context and unknown event categories and struggle by designing compound text representation methods and various similarity measures. However, phased, well-designed methods are excessively fragile and unable to utilize the potential of text representations fully. Meanwhile, their complex threshold settings result in clustering-based event detection suffering the pain of ever-changing environments. We propose a text representation learning method namely Text Similarity Contrastive Learning Neural Network (Text-SimCLNN) to tackle these challenges. Text-SimCLNN uses smaller parts to learn the similarity probability of text pairs from semantic and structural perspectives, effectively bridging the gap between text representation learning and similarity measure in streaming event detection. Event discovery and merging in streams can be easily performed based on the learned representations, and we use various techniques to speed up such processes. Furthermore, we introduce an online update mechanism that uses heterogeneous graphs to generate high-quality samples to enable stable and reliable inductive learning. Extensive experiments on two real-world datasets demonstrate that our method far exceeds state-of-the-art (SOTA).

A survey on short text similarity measurement methods

A survey on short text similarity measurement methods

An Approximate Algorithm for Maximum Inner Product Search over Streaming Sparse Vectors

Maximum Inner Product Search or top- k retrieval on sparse vectors is well understood in information retrieval, with a number of mature algorithms that solve it exactly. However, all existing algorithms are tailored to text and frequency-based similarity measures. To achieve optimal memory footprint and query latency, they rely on the near stationarity of documents and on laws governing natural languages. We consider, instead, a setup in which collections are streaming—necessitating dynamic indexing—and where indexing and retrieval must work with arbitrarily distributed real-valued vectors. As we show, existing algorithms are no longer competitive in this setup, even against naïve solutions. We investigate this gap and present a novel approximate solution, called Sinnamon , that can efficiently retrieve the top- k results for sparse real valued vectors drawn from arbitrary distributions. Notably, Sinnamon offers levers to trade off memory consumption, latency, and accuracy, making the algorithm suitable for constrained applications and systems. We give theoretical results on the error introduced by the approximate nature of the algorithm and present an empirical evaluation of its performance on two hardware platforms and synthetic and real-valued datasets. We conclude by laying out concrete directions for future research on this general top- k retrieval problem over sparse vectors.

Combined E-Learning Course Recommender System

In this paper, we propose learners course recommender system of the E-Dirassa platform. This system, which distinguishes between new and active learners, adopts three complementary recommendation approaches: content-based recommendation as well as course-based collaborative filtering for active learners, and static profile-based collaborative filtering for new learners. Let us also note the use of multi-criteria decision aid (MDA) techniques for the choice of the textual similarity measure to be used in the first approach of content-based recommendation.

Fine-grained semantic textual similarity measurement via a feature separation network

Fine-grained semantic textual similarity measurement via a feature separation network

Improving mathematics assessment readability: Do large language models help?

AbstractBackgroundReadability metrics provide us with an objective and efficient way to assess the quality of educational texts. We can use the readability measures for finding assessment items that are difficult to read for a given grade level. Hard‐to‐read math word problems can put some students at a disadvantage if they are behind in their literacy learning. Despite their math abilities, these students can perform poorly on difficult‐to‐read word problems because of their poor reading skills. Less readable math tests can create equity issues for students who are relatively new to the language of assessment. Less readable test items can also affect the assessment's construct validity by partially measuring reading comprehension.ObjectivesThis study shows how large language models help us improve the readability of math assessment items.MethodsWe analysed 250 test items from grades 3 to 5 of EngageNY, an open‐source curriculum. We used the GPT‐3 AI system to simplify the text of these math word problems. We used text prompts and the few‐shot learning method for the simplification task.Results and ConclusionsOn average, GPT‐3 AI produced output passages that showed improvements in readability metrics, but the outputs had a large amount of noise and were often unrelated to the input. We used thresholds over text similarity metrics and changes in readability measures to filter out the noise. We found meaningful simplifications that can be given to item authors as suggestions for improvement.TakeawaysGPT‐3 AI is capable of simplifying hard‐to‐read math word problems. The model generates noisy simplifications using text prompts or few‐shot learning methods. The noise can be filtered using text similarity and readability measures. The meaningful simplifications AI produces are sound but not ready to be used as a direct replacement for the original items. To improve test quality, simplifications can be suggested to item authors at the time of digital question authoring.

An unsupervised semantic text similarity measurement model in resource-limited scenes

As the basis of many artificial intelligence tasks, text similarity measurement has received extensive attention in current studies. However, few of them focus on the resource-limited scenes (i.e., limited computational resources and few training datasets), which are becoming increasingly popular and challenging with the development of the Internet of Things. Worse still, popular methods such as the deep-neural-network-based methods may lose their power in such scenes, since they typically require considerable computational resources. As for most current traditional methods, they also have issues of not effectively exploiting the semantic information in the sentences. As an alternative, this paper proposes a lightweight and semantically rich text similarity measurement model named the TES-TK model. In this model, a sentence is first transformed into a tree structure called TES-Tree with the integration of syntactic information, semantic knowledge, and topic distribution, aiming to comprehensively represent the multidimensional semantics of sentences. Afterward, a modified tree kernel model is designed to calculate the similarity between each pair of TES-Trees. In this way, the similarity score between the two related sentences can be retrieved. Experiments on 19 public benchmark datasets (STS2012–2015) demonstrate that the proposed approach exhibits significantly better performance than the compared eight peer methods on most datasets. Especially in resource-limited scenes, our approach achieved highly competitive results compared with the latest methods, such as BERT.

Experimental Comparison of Pre-Trained Word Embedding Vectors of Word2Vec, Glove, FastText for Word Level Semantic Text Similarity Measurement in Turkish

Experimental Comparison of Pre-Trained Word Embedding Vectors of Word2Vec, Glove, FastText for Word Level Semantic Text Similarity Measurement in Turkish

Identifying Museum Visitors via Social Network Analysis of Instagram

As social networking services (SNSs) have become increasingly influential, they are now a vital element in art museums because communication with visitors is crucial. However, conventional methods of visitor studies do not consider the characteristics of SNSs. Even when a museum uses a SNS as a marketing tool, it cannot sufficiently analyze the data to understand the visitor. Additionally, linking the SNS analysis content with the actual visitor in the museum requires an application to identify visitor types. Therefore, we extracted communities through a social network analysis of museum followers, analyzed the characteristics for specific groups, and designed a web-based, visitor-type analysis application for art museums based on text-similarity measurements. The experimental results demonstrated that followers of each art museum on a SNS form a community with similar characteristics or interests. Furthermore, our developed application provides a new way to analyze the type of visitors in an art museum. Consequently, because it can automatically identify visitor types, it improves the exhibition experience of visitors by linking them with the exhibition contents or guide services and is a new attempt to connect a SNS with the museum visitor.

An Ensemble Semantic Textual Similarity Measure Based on Multiple Evidences for Biomedical Documents.

With the increasing volume of the published biomedical literature, the fast and effective retrieval of the literature on the sequence, structure, and function of biological entities is an essential task for the rapid development of biology and medicine. To capture the semantic information in biomedical literature more effectively when biomedical documents are clustered, we propose a new multi-evidence-based semantic text similarity calculation method. Two semantic similarities and one content similarity are used, in which two semantic similarities include MeSH-based semantic similarity and word embedding-based semantic similarity. To fuse three different similarities more effectively, after, respectively, calculating two semantic and one content similarities between biomedical documents, feedforward neural network is applied to integrate the two semantic similarities. Finally, weighted linear combination method is used to integrate the semantic and content similarities. To evaluate the effectiveness, the proposed method is compared with the existing basic methods, and the proposed method outperforms the existing related methods. Based on the proven results of this study, this method can be used not only in actual biological or medical experiments such as protein sequence or function analysis but also in biological and medical research fields, which will help to provide, use, and understand thematically consistent documents.

A novel locality-sensitive hashing relational graph matching network for semantic textual similarity measurement

Recent efforts adopt interaction-based models to construct the interaction of words between sentences, which aim to predict whether two sentences are semantically equivalent or not in semantic textual similarity (STS) task. However, these methods lack the global semantic awareness, which make it difficult to distinguish syntactic differences and also suffer from the inference time cost, primarily due to the calculation of the pair-interactions of words. A novel model called Locality-Sensitive Hashing Relational Graph Matching Network (LSHRGMN) is therefore proposed, which tackles these problems by syntactic dependency graph and locality-sensitive hashing (LSH). Specifically, syntactic dependency graph is aware of the global semantic information via rooting in each word to construct several trees and merging all the trees into one graph. LSH mechanism is introduced into pair-interactions of words for the inference efficiency problem. Extensive experiments are conducted on three real-world datasets, and the result shows that the proposed approach acquires higher accuracy and intriguing inference speed.

Hybrid Methods of Bibliographic Coupling and Text Similarity Measurement for Biomedical Paper Recommendation.

The amount of available scientific literature is increasing, and studies have proposed various methods for evaluating document-document similarity in order to cluster or classify documents for science mapping and knowledge discovery. In this paper, we propose hybrid methods for bibliographic coupling (BC) and linear evaluation of text or content similarity: We combined BC with BM25, Cosine, and PMRA to compare their performances with single methods in paper recommendation tasks using TREC Genomics Track 2005datasets. For paper recommendation, BC and text-based methods complement each other, and hybrid methods were better than single methods. The combinations of BC with BM25 and BC with Cosine performed better than BC with PMRA. The performances were best when the weights of BM25, Cosine, and PMRA were 0.025, 0.2, and 0.2, respectively, in hybrid methods. For paper recommendation, the combinations of BC with text-based methods were better than BC or text-based methods used alone. The choice of method should depend on the actual data and research needs. In the future, the underlying reasons for the differences in performance and the specific part or type of information they complement in text clustering or recommendation need to be examined.