Text Similarity Research Articles

A BERT-GRU Model for Measuring the Similarity of Arabic Text

Semantic Textual Similarity (STS) aims to assess the semantic similarity between two pieces of text. As a challenging task in natural language processing, various approaches for STS in high-resource languages, such as English, have been proposed. In this paper, we are concerned with STS in low resource languages such as Arabic. A baseline approach for STS is based on vector embedding of the input text and application of similarity metric on the embedding space. In this contribution, we propose a cross-encoder neural network (Cross-BERT-GRU) to handle semantic similarity of Arabic sentences that benefits from both the strong contextual understanding of BERT and the sequential modeling capabilities of GRU. The architecture begins by inputting the BERT word embeddings for each word into a GRU cell to model long-term dependencies. Then, max pooling and average pooling are applied to the hidden outputs of the GRU cell, serving as the sentence -pair encoder. Finally, a softmax layer is utilized to predict the degree of similarity. The experiment results show a Spearman correlation coefficient of around 0.9 and that Cross-BERT-GRU outperforms the other BERT models in predicting the semantic textual similarity of Arabic sentences. The experimentation results also indicate that the performance improves by integrating data augmentation techniques.

ИНТЕРПРЕТАЦИЯ ДРЕВНЕГО ФОЛЬКЛОРНОГО ТЕКСТА ЯКУТОВ

The author deals with the text of an archival document discovered in the materials of the Second Kamchatka expedition led by Vitus Bering, in the collection of field notes of the expedition member, the German doctor and translator G. W. Steller. This fragment in Latin is a recording of a Yakut folk song of the first half of the 18th century. The purpose of this article is to introduce a previously unpublished archival text into scientific circulation by translating the text and describing its artistic features. As part of the research, the following tasks were set: to study the history of the text, give a description of the manuscript, transcribe and translate it into Russian and highlight the artistic means used. According to the fixation of Yakut folklore in the specified period is known to science the ethnographic work of J. J. Lindenau “Description of the nations of Siberia,” which contains the text of the algys ritual on the Ysekh holiday. There are also recordings of folk songs of northern nations performed by G. W. Steller, but this text has not yet been examined by folklorists. The work uses theoretical and practical research methods as: study of scientific works, the main research method is textual analysis, which uses the method of review, description, interpretation, connection, translation, decoding and analysis of the composition, artistic and visual means of the text. The author concludes that the ancient folk song has a developed syllable. Rhythmic alliterative verse, consonance of line endings close to varieties of rhyme, parallel constructions, constant epithets and comparisons as the main means of depiction demonstrate a fairly high level of development of Yakut folk poetry. The author sees the prospect of research in the discovery of similar texts of the specified period, which would allow more in-depth research in this area.

Analyzing user reactions using relevance between location information of tweets and news articles

In this study, we analyze the extent of user reactions based on user’s tweets to news articles, demonstrating the potential for home location prediction. To achieve this, we quantify users’ reactions to specific news articles based on the textual similarity between tweets and news articles, showcasing that users’ reactions to news articles about their cities are significantly higher than those about other cities. To maximize the difference in reactions, we introduce the concept of News Distinctness, which highlights the news articles that affect a specific location. By incorporating News Distinctness with users’ reactions to the news, we magnify its effects. Through experiments conducted with tweets collected from users whose home locations are in five representative cities within the United States and news articles describing events occurring in those cities, we observed a 6.75% to 40% improvement in the reaction score when compared to the average reactions towards news for outside of home location, clearly predicting the home location. Furthermore, News Distinctness increases the difference in reaction score between news in the home location and the average of the news outside of the home location by 12% to 194%. These results demonstrate that our proposed idea can be utilized to predict the users’ location, potentially recommending meaningful information based on the users’ areas of interest.

Coreference Resolution Based on High-Dimensional Multi-Scale Information.

Coreference resolution is a key task in Natural Language Processing. It is difficult to evaluate the similarity of long-span texts, which makes text-level encoding somewhat challenging. This paper first compares the impact of commonly used methods to improve the global information collection ability of the model on the BERT encoding performance. Based on this, a multi-scale context information module is designed to improve the applicability of the BERT encoding model under different text spans. In addition, improving linear separability through dimension expansion. Finally, cross-entropy loss is used as the loss function. After adding BERT and span BERT to the module designed in this article, F1 increased by 0.5% and 0.2%, respectively.

SUMEX: A hybrid framework for Semantic textUal siMilarity and EXplanation generation

Measuring semantic similarity between two pieces of text is a widely known problem in Natural language processing(NLP). It has many applications, such as finding similar medical notes of patients to accelerate the diagnosis process, plagiarism detection, and document clustering. Most state-of-the-art models are based on machine/deep learning and lack sufficient explanations for their results, limiting their adoption in critical domains like healthcare. This paper presents a hybrid framework SUMEX (Semantic textUal siMilarity and EXplanation generation) that uniquely combines ontology with a state-of-the-art embedding-based model for semantic textual similarity. The primary strength of the framework is that it explains its results in human-understandable natural language, which is vital in critical domains such as healthcare. Experiments have been conducted on two datasets of clinical notes using four embeddings: ScispaCy, BioWord2Vec, ClinicalBERT, and a customized Word2Vec trained on clinical notes. The SUMEX framework outperforms the embedding-based model on the benchmark datasets of ClinicalSTS by improving average precision scores by 7 % and reducing the false-positives-rate by 23 %. On the Patients Similarity Dataset, the average top-five and top-three precision scores were improved by 14% and 10%, respectively, using SUMEX. The SUMEX also generates explanations for its results in natural language. The domain experts evaluated the quality of the explanations. The results show that the generated explanations are of significantly good quality, with a score of 90 % and 93 % for measures of Completeness and Correctness, respectively. In addition, ChatGPT was also used for similarity score and generating explanations. The experiments show that the SUMEX framework performed better than the ChatGPT.

CLSESSP: Contrastive learning of sentence embedding with strong semantic prototypes

By drawing inspiration from the computer vision domain, contrastive learning has emerged as another paradigm for unsupervised sentence embedding. By using contrastive learning, most recent sentence embedding methods have achieved promising results. However, previous methods focused on constructing positive and negative pairs through traditional data augmentation, which was unable to balance the loss of semantics and the additional features brought by aggressive data augmentation. In the present study, we propose a prototype contrastive learning framework based on a distribution divergence minimization loss function and introduce the concept of strong semantic prototypes. Specifically, with the aid of prompts, our method constructs three semantic prototypes for each instance: a basic semantic prototype, a strong semantic prototype and a negative prototype. By combining InfoNCE loss with distribution divergence minimization loss, Contrastive Learning of Sentence Embedding with Strong Semantic Prototypes (CLSESSP) creates another optimization objective that integrates strong augmentation. The experiments conducted on the semantic text similarity (STS) datasets demonstrate that the proposed CLSESSP surpasses the strong baseline by an average of 2.9 points in the BERT-base model. Extensive experimental results on transfer and clustering tasks demonstrate the effectiveness of our proposed model compared to strong baselines. The code is available at https://github.com/KCshen1125/CLSESSP.

Ship Anomalous Behavior Detection in Port Waterways Based on Text Similarity and Kernel Density Estimation

The navigational safety of ships on waterways plays a crucial role in ensuring the operational efficiency of ports. Ship anomalous behavior detection is an important method of water traffic surveillance that can effectively identify abnormal ship behavior, such as sudden acceleration or deceleration. In order to detect potential anomalous ship behavior in real time, a method for ship anomalous behavior detection in waterways is proposed based on text similarity and kernel density estimation. Under the assumption of known traffic patterns entering and leaving the port, this method can identify ship behaviors that violate traffic patterns in real time. Firstly, kernel density estimation is applied to construct a traffic pattern density model for ship trajectories entering and leaving the port, used to estimate the density values of ship motion states. Simultaneously, a semantic transformation method is used to convert traffic pattern trajectory into pattern trajectory text, which is used to identify the ship’s traffic pattern. Subsequently, the historical trajectory data of the target ship are transformed into textual trajectories, and text similarity is used to identify ship inbound and outbound traffic patterns. Furthermore, the constructed traffic pattern density model is used to estimate real-time density values of the state of ship motion, and the trajectory points that exceed the threshold of the anomaly factor are marked as anomalies. Finally, the effectiveness of the proposed method is validated using simulation data, and the results indicate an accuracy of more than 90% for the comprehensive detection of anomalous behavior. This study, approaching the detection of potential ship anomalous behavior from the perspective of port traffic patterns, enriches the methods of ship anomalous behavior detection in port waterways.

Large language models for precision oncology: Clinical decision support through expert-guided learning.

e13609 Background: Precision oncology revolutionized cancer treatment by identifying molecular biomarkers to guide personalized care. The ever-growing body of medical literature presents a challenge for oncologists researching targeted therapies. While recent studies investigated large language models (LLMs) to streamline this process, LLM reliance on general rather than medical knowledge limits clinical relevance and trustworthiness. To address these limitations, we developed a retrieval augmented generation (RAG) system that integrates PubMed clinical studies, trial databases and oncological guidelines with LLMs to support targeted treatment recommendations. The Molecular Tumor Board (MTB) at the Center of Personalized Medicine (ZPMTUM) guided and evaluated treatment options proposed by the LLM to assess their applicability for clinical decision support. Methods: We used 10 publicly accessible fictional patient cases with 7 tumor types and 59 distinct molecular alterations. Our LLM system MEREDITH (Medical Evidence Retrieval and Data Integration for Tailored Healthcare) consists of Google's Gemini Pro, enhanced with RAG and Chain-of-Thought (CoT) prompting. To establish a benchmark, clinical experts at ZPMTUM manually annotated the cases. Informed by MTB expert feedback, we iteratively improved our LLM system from a draft system relying on PubMed-indexed data to an enhanced system, which replicated expert annotation processes by incorporating oncology guidelines, drug availability and trial databases (ClinicalTrials.gov, QuickQueck.de). ZPMTUM assessed credibility and clinical relevance of manually annotated and LLM-generated recommendations. Patient-level data on (likely) pathogenic molecular alterations and recommended treatment options were summarized using median and interquartile range (IQR). Semantic similarity between LLM and clinician responses was assessed using cosine similarity of text vector embeddings; paired t-test evaluated significance. Results: The median of (likely) pathogenic molecular alterations per patient was 2.5 (IQR: 2-3). ZPMTUM identified a median of 2 treatment options per patient (IQR: 1-3), while the enhanced LLM identified a median of 4 (IQR: 3-5). MEREDITH proposed multiple relevant treatment suggestions, including therapies based on preclinical studies, and molecular interactions, for further assessment by the MTB. ZPMTUM prioritized the most suitable clinical option. The mean semantic textual similarity of LLM responses increased significantly from 0.69 in the draft system to 0.76 in the enhanced system (p <0.001). Thus, feedback from ZPMTUM enhanced the model's ability to align its responses with clinician thought processes. Conclusions: Leveraging expert thought processes to instruct LLMs holds promise as a novel decision support tool for precision oncology.

Patient-patient interactions visualization for drug side effects in patients’ reviews

This paper describes the patient-patient interactions (PPIs) graph extraction framework from patient’s review transcripts. The concept is to visualise patients as nodes and interactions representing links. Links are made based on review text similarity. Nodes are categorized as positive or negative according to the patient’s attitude toward a given drug. Attitudes are then utilized to categorize the links as supporting or opposing the use of a certain drug. If both patients share the same attitude: negative (severe side effect) or positive (moderate side effect), the relationship is considered supportive; if not, the link is considered opposed. Resulting graph represent a drug as a dispute between two factions arguing on related drug. The framework is explained and evaluated using a dataset included 3,763 patients’ reviews linked to 255 different drugs, -predictive-value (0.37). We argue that, this is caused by derogatory jargon that is an expected feature of patient’s review. The true-negative-recognition-rate is 0.70 and true-positive-recognition-rate is 0.54. Total-average-accuracy, which is independent of class priors, is 0.66. Results show that, it is possible to use text proximity measures and sentiment analysis to capture PPIs structure.

A multifaceted architecture to Automate Essay Scoring for assessing english article writing: Integrating semantic, thematic, and linguistic representations

We propose a multifaceted architecture for automated essay scoring integrating semantic, thematic, and linguistic representations. Specifically, for the deep semantic content of the essay, we employ Convolutional Neural Network (ConvNet) and attention mechanism module to extract local semantic representation, and use ConvNet, Long Short-Term Memory network (LSTM), and attention mechanism module to capture the global semantic representation, providing the comprehensive deep semantic-facet representations. To answer the presence of prompt in the test essay, we utilize the Doc2Vec model to represent the target essay and the essay prompt as vectors and calculate their textual similarity as thematic-facet representations. In addition, we construct six categories of nine manual features, including grammatical errors, essay length, textual complexity, word complexity, sentence complexity, and the number of clauses, to capture the shallow linguistic representation that deep learning models may struggle to uncover. Finally, we integrate the multifaceted representation through the fully connected layer to automatically score essays. Experimental results on the standard public dataset ASAP and the more complex dataset CLC-FCE, which varies in standards, lengths, and topics, demonstrate that the proposed method effectively captures diverse aspects of textual representations. For the ASAP dataset, our method reduced the training time to 1862 s per essay and the inference time to 0.9778 s, achieving a 1.13 improvement in the Quadratic Weighted Kappa score over the previous best method. On the CLC-FCE dataset, our approach enhanced the Pearson's product-moment correlation coefficient and Spearman's rank correlation coefficient by 0.005 and 0.0337 respectively, surpassing prior benchmarks.

Few-shot intent detection with self-supervised pretraining and prototype-aware attention

Few-shot intent detection is a more challenging application. However, traditional prototypical networks based on averaging often suffer from issues such as missing key information, poor generalization capabilities. In previous work, using three-dimensional convolutional neural networks (3DCNN) to generate prototype representations faces challenges with long-distance dependencies. Furthermore, a pretrained encoder’s performance in a specific domain is often suboptimal because its knowledge of the specific domain is fragmented. Therefore, in this paper, we propose a simple yet effective two-stage learning strategy to address these issues. In the first stage, we propose a self-supervised multi-task pretraining (SMTP) strategy. SMTP utilizes unlabeled data from the current domain to help the pretrained encoder learn the semantic information of the text and implicitly distinguish semantically similar text representations without using any labels. SMTP aims to enhance the representation capability of the pretrained encoder in a specific domain. In the second stage, we propose a prototype-aware attention (PaAT) model to generate prototype representations of the same class. PaAT generates prototype representations by calculating the attention between class texts, which can effectively solve the long-distance dependence problem of 3DCNN. PaAT is a siamese architecture that can simultaneously generate prototype representations and sentence-level representations of unseen data. In addition, to prevent overfitting in few-shot learning, we introduce an unsupervised contrastive regularization term to constrain PaAT. Our method achieves state-of-the-art performance on four public datasets.11Our code is available: https://github.com/YS19999/SMTP-PaAT. .

Towards Building a Chatbot-Based First Aid Service in Arabic Language

In this paper, we introduce a modern standard Arabic (MSA) conversational AI in the field of first aid to assist anyone facing an emergency case. The proposed chatbot -based model not only provide adequate medical assistance, but also engage in empathetic interactions with users. Due to the lack of a reliable dataset that fits our purpose, we collect and construct a dataset of 374 records covering 55 first aid topic. Our chatbot architecture relies fine tuning pretrained language models in its intent classification and employs textual similarity to match user queries to records from the dataset. We evaluate the proposed model based using both a quantitative approach, through performance metrics of different modules, and a qualitative approach using a questionnaire.

MOVIE SIMILARITY FROM PLOT SUMMARIES

This project focuses on developing a Python application to analyze and measure the similarity between movie plot summaries. The goal is to provide a tool that can assist in identifying similarities between movies based on their storyline, enabling users to discover related movies or recommend similar ones.The project utilizes natural language processing (NLP) techniques, particularly text preprocessing, vectorization, and similarity metrics, to achieve its objectives. First, it preprocesses the plot summaries by removing stop words, punctuation, and performing stemming or lemmatization to normalize the text data. Next, it employs vectorization methods such as TF-IDF (Term Frequency-Inverse Document Frequency) or word embeddings like Word2Vec or GloVe to convert the text into numerical representations suitable for similarity calculations.

Textual similarity for legal precedents discovery: Assessing the performance of machine learning techniques in an administrative court

The importance of legal precedents in ensuring consistent jurisprudence is undisputed. Particularly in jurisdictions following the Common law, but even in Civil law systems, uniformity in case law requires adherence to precedents. However, with the growing volume of cases, manual identification becomes a bottleneck, prompting the need for automation. Leveraging the capabilities of natural language processing (NLP) and machine learning (ML), our study delves into the potential of automation in identifying similar cases indicative of precedents. Drawing from a unique, substantial dataset of legal cases from an administrative court in Brazil, we extensively evaluated over one hundred combinations of document representations and text vectorizations. Contrary to earlier studies that relied on minimal validation samples, ours employed a statistically significant sample vetted by legal experts. Our findings reveal that models focusing on granular text representations perform optimally, especially when extracting concepts and relations. Notably, while intricate models may not always guarantee superior outcomes, the importance of refining textual features cannot be understated. These findings pave the way for creating efficient decision support systems in judicial contexts and set a direction for future research aiming to integrate technology in legal decision-making.

SOFTWARE CAPABILITIES FOR TEXT IDENTIFICATION: COMPARISON FOR SIMILARITY, IDENTITY ESTABLISHMENT, UNIQUENESS CHECK

The study was carried out within the framework of identification linguistics and translational linguistics. The article describes some methods for determining the degree of similarity of texts: shingle algorithm, Levenshtein distance, systems for detecting plagiarism. The purpose of the work is to test the software capabilities of comparing texts for similarity, establishing their identity, and checking uniqueness. In a broad sense, these tasks fall within the area of text identification. In a qualitative (manual) assessment of the similarity of texts, identifying parameters are selected and selected specifically for the text under study. The use of electronic resources is determined by the desire for objectivity of the methods used to establish the identity of texts and the objectivity of the results obtained. Software products also make it possible to establish another, quantitative, characteristic — the degree of similarity of texts to each other or the degree of originality of the text. The work used services whose tasks include 1) comparing the similarity of two texts; 2) calculation of the Levenshtein distance; 3) detection of borrowing. The research material was an excerpt from an interview with Foreign Minister Sergei Lavrov. Reverse machine translation texts served as options for comparison with the source text. Reverse machine translation as a translation product is part of artificial intelligence and a model of the process of understanding and interpreting natural language. The results of using the proposed services made it possible to arrange five reverse machine translation options from the most unique text to the text that is most identical to the invariant. The study showed that the programs generally produce similar results, which can be applicable to solving research and applied problems related to establishing the identity and difference of texts. The prospect of the study is to identify lexical parameters that make it possible to classify reverse machine translation texts as the most or least identical with respect to the invariant.

Meningkatkan Deduplikasi Data melalui Kesamaan Teks dalam Pembelajaran Mesin: Pendekatan Komprehensif

The issue of dirty data, particularly duplicate data, is a common problem in data management that can affect data quality, operational efficiency, and decision-making. This study highlights the importance of implementing sustainable deduplication strategies as a key step in managing dirty data. We explore solutions for detecting duplicate data by measuring text similarity indices. In this study, the authors utilize a literature review research method. Through this method, we collected various journals on data deduplication and text similarity techniques, comparing several methods to identify the most effective approach. The data deduplication process in this study consists of two stages: 1) Matching - calculating the similarity value of a record with previous records, and 2) Clustering - grouping all records deemed duplicates of a single entity. Furthermore, this study extends to the development of a Python application capable of identifying and grouping similar customer data based on text similarity values. The Text Similarity measurement method uses the Jaro-Winkler Similarity technique. Experimental results and evaluations show that the Text Similarity approach is effective in identifying duplicate data with a high degree of accuracy. This study emphasizes the importance of sustainable deduplication, where the deduplication process is conducted periodically and continuously to ensure optimal data quality.

Local large language models for privacy-preserving accelerated review of historic echocardiogram reports.

The study developed framework that leverages an open-source Large Language Model (LLM) to enable clinicians to ask plain-language questions about a patient's entire echocardiogram report history. This approach is intended to streamline the extraction of clinical insights from multiple echocardiogram reports, particularly in patients with complex cardiac diseases, thereby enhancing both patient care and research efficiency. Data from over 10 years were collected, comprising echocardiogram reports from patients with more than 10 echocardiograms on file at the Mount Sinai Health System. These reports were converted into a single document per patient for analysis, broken down into snippets and relevant snippets were retrieved using text similarity measures. The LLaMA-2 70B model was employed for analyzing the text using a specially crafted prompt. The model's performance was evaluated against ground-truth answers created by faculty cardiologists. The study analyzed 432 reports from 37 patients for a total of 100 question-answer pairs. The LLM correctly answered 90% questions, with accuracies of 83% for temporality, 93% for severity assessment, 84% for intervention identification, and 100% for diagnosis retrieval. Errors mainly stemmed from the LLM's inherent limitations, such as misinterpreting numbers or hallucinations. The study demonstrates the feasibility and effectiveness of using a local, open-source LLM for querying and interpreting echocardiogram report data. This approach offers a significant improvement over traditional keyword-based searches, enabling more contextually relevant and semantically accurate responses; in turn showing promise in enhancing clinical decision-making and research by facilitating more efficient access to complex patient data.

Research on Key Technologies for Text Similarity Calculation Based on Small Datasets

Research on Key Technologies for Text Similarity Calculation Based on Small Datasets

A Data-Driven Approach to Discovering Process Choreography

Implementing approaches based on process mining in inter-organizational collaboration environments presents challenges related to the granularity of event logs, the privacy and autonomy of business processes, and the alignment of event data generated in inter-organizational business process (IOBP) execution. Therefore, this paper proposes a complete and modular data-driven approach that implements natural language processing techniques, text similarity, and process mining techniques (discovery and conformance checking) through a set of methods and formal rules that enable analysis of the data contained in the event logs and the intra-organizational process models of the participants in the collaboration, to identify patterns that allow the discovery of the process choreography. The approach enables merging the event logs of the inter-organizational collaboration participants from the identified message interactions, enabling the automatic construction of an IOBP model. The proposed approach was evaluated using four real-life and two artificial event logs. In discovering the choreography process, average values of 0.86, 0.89, and 0.86 were obtained for relationship precision, relation recall, and relationship F-score metrics. In evaluating the quality of the built IOBP models, values of 0.95 and 1.00 were achieved for the precision and recall metrics, respectively. The performance obtained in the different scenarios is encouraging, demonstrating the ability of the approach to discover the process choreography and the construction of business process models in inter-organizational environments.

Development of English Composition Correction and Scoring System Based on Text Similarity Algorithm

This study focuses on creating an innovative system that leverages a text similarity algorithm to provide accurate correction and scoring of English compositions. By harnessing advanced computational techniques, the system aims to streamline the grading process and offer more consistent feedback to students. The research methodology involves the design and implementation of the text similarity algorithm, which analyzes various linguistic aspects such as vocabulary usage, grammar, coherence, and organization. This algorithm is then integrated into a comprehensive scoring system that evaluates student compositions against reference texts and provides detailed feedback on areas for improvement. The effectiveness of the system is validated through rigorous testing using a diverse set of English compositions and comparison with human grading. The results of this investigation show how well the system that was created can automatically correct and score English papers with a high degree of correctness and dependability. By reducing the burden of manual grading and providing immediate feedback to students, the system has the potential to enhance the teaching and learning process in English language education. Overall, by developing a scalable and effective method for compositional assessment based on text similarity algorithms, this research advances educational technology.