Semantic Similarity Research Articles

Improving Paragraph Similarity by Sentence Interaction With BERT

ABSTRACTResearch on semantic similarity between relatively short texts, for example, at word‐ and sentence‐level, has progressed significantly in recent years. However, paragraph‐level similarity has not been researched in as much detail owing to the challenges associated with embedding representations, despite its utility in numerous applications. A rudimentary approach to paragraph‐level similarity involves treating each paragraph as an elongated sentence, thereby encoding the entire paragraph into a single vector. However, this results in the loss of long‐distance dependency information, ignoring interactions between sentences belonging to different paragraphs. In this paper, we propose a simple yet efficient method for estimating paragraph similarity. Given two paragraphs, it first obtains a vector for each sentence by leveraging advanced sentence‐embedding techniques. Next, the similarity between each sentence in the first paragraph and the second paragraph is estimated as the maximum cosine similarity value between the sentence and each sentence in the second paragraph. This process is repeated for all sentences in the first paragraph to determine the maximum similarity of each sentence with the second paragraph. Finally, overall paragraph similarity is computed by averaging the maximum cosine similarity values. This method alleviates long‐range dependency by embedding sentences individually. In addition, it accounts for sentence‐level interactions between the two paragraphs. Experiments conducted on two benchmark data sets demonstrate that the proposed method outperforms the baseline approach that encodes entire paragraphs into single vectors.

ECP-IEM: Enhancing seasonal crop productivity with deep integrated models.

Accurate crop yield forecasting is vital for ensuring food security and making informed decisions. With the increasing population and global warming, addressing food security has become a priority, so accurate yield forecasting is very important. Artificial Intelligence (AI) has increased the yield accuracy significantly. The existing Machine Learning (ML) methods are using statistical measures as regression, correlation and chi square test for predicting crop yield, all such model's leads to low accuracy when the number of factors (variables) such as the weather and soil conditions, the wind, fertilizer quantity, and the seed quality and climate are increased. The proposed methodology consists of different stages, like Data Collection, Preprocessing, Feature Extraction with Support Vector Machine (SVM), correlation with Normalized Google Distance (NGD), feature ranking with rising star. This study combines Bidirectional Gated Recurrent Unit (Bi-GRU) and Time Series CNN to predict crop yield and then recommendation for further improvement. The proposed model showed very good results in all datasets and showed significant improvement compared to baseline models. The ECP-IEM achieved an accuracy 96.34%, precision 94.56% and recall 95.23% on different datasets. Moreover, the proposed model was also evaluated based on MAE, MSE, and RMSE, which produced values of 0.191, 0.0674, and 0.238, respectively. This will help in improving production of crops by giving an early look about the yield of crops which will than help the farmer in improving the crops yield.

Patterns of co-occurrent skills in UK job adverts

A job usually involves the application of complementary or synergistic skills to perform the required tasks. Such relationships are implicitly recognised by employers in the skills they demand when recruiting new employees. Here we construct a skills network based on their co-occurrence in a national level data set of 65 million job postings from the UK spanning 2016 to 2022. We then apply multiscale graph-based community detection to obtain data-driven clusters of skills at different levels of resolution that reveal modular groupings of skills across scales. The obtained skill clusters occupy different roles within the skills network: some have broad reach across the network (high closeness centrality) while others have higher levels of within-cluster containment. Yet there is high interconnection across clusters and no skill silos. Furthermore, the skill clusters display varying levels of within-cluster semantic similarity, highlighting the difference between co-occurrence in adverts and intrinsic thematic consistency. The skill clusters are characterised by diverse levels of demand, with clear geographic variation across the UK, broadly reflecting the industrial characteristics of each region, e.g., London is an outlier as an international hub for finance, education and business. Comparison of data from 2016 and 2022 reveals increasing employer demand for a broader range of skills over time, with more adverts featuring skills spanning different clusters. Our analysis also shows that data-driven clusters differ from expert-authored categorisations, suggesting they may capture relationships between skills not immediately apparent in expert assessments.

Computational Sociolinguistics: An Emerging Multidisciplinary Research Area

ABSTRACTThis article explores the multidisciplinary field of computational sociolinguistics (CSLX). A social network analysis through a bibliometric approach is employed to investigate the landscape of scholarly publications through a series of keyword queries related to ‘Computational Sociolinguistics’. The study analyses publications contributing to this research area, focusing on semantic content analysis. The methodology involves the extraction and processing of data from the database Web of Science followed by semantic similarity assessments and classifications of abstracts using advanced natural language processing techniques to map the thematic clusters within the field as well as co‐authorship analysis. Key findings reveal significant multidisciplinarity in CSLX characterised by the integration of computational methods into sociolinguistic inquiries. The study identifies central topics and collaborative patterns, highlighting the influence of the computational profile over the sociolinguistic one. Conclusively, CSLX is positioned as an emerging field, with its development influenced by technological advancements and the increasing complexity of social interactions in computer‐mediated communication.

On Layered Area-Proportional Rectangle Contact Representations

Semantic word clouds visualize the semantic relatedness between the words of a text by placing pairs of related words close to each other. Formally, the problem of drawing semantic word clouds corresponds to drawing a rectangle contact representation of a graph whose vertices correlate to the words to be displayed and whose edges indicate that two words are semantically related. The goal is to maximize the number of realized contacts while avoiding any false adjacencies. We consider a variant of this problem that restricts input graphs to be layered and all rectangles to be of equal height, called Maximum Layered Contact Representation Of Word Networks or Max-LayeredCrown, as well as the variant Max-IntLayeredCrown, which restricts the problem to only rectangles of integer width and the placement of those rectangles to integer coordinates.We classify the corresponding decision problem k-IntLayeredCrown as NP-complete even for internally triangulated planar graphs and k-LayeredCrown as NP-complete for planar graphs. We introduce three algorithms: a 1/2-approximation for Max-LayeredCrown of internally triangulated planar graphs, and a PTAS and an XP algorithm for Max-IntLayeredCrown with rectangle width polynomial in n.

Do syntactic and semantic similarity lead to interference effects? Evidence from self-paced reading and event-related potentials using German

Do syntactic and semantic similarity lead to interference effects? Evidence from self-paced reading and event-related potentials using German

Sentiment Classification from Social Media Data with Gaussian Conditional Rule-Based Deep Convolutional Neural Network

Social media platforms share several emotions in the form of text, audio, and images, where the text-shared data are less convincing to understand the emotions. The classification of the sentiments should be accurate to achieve a better understanding of the values in the commercial industry. Twitter has proven to be an effective instrument for text-based sentiment analysis when compared to other traditional assessments like surveys and stock market performance records. Though the existing research on sentiment analysis achieved good results of performance, the techniques had drawbacks such as long-term sentence prediction, less robustness, high computational resources, lack of complementary feature extraction, and so on. In this research, the Gaussian conditional rule-based deep convolutional neural network (DCNN) (Gaussian conditional rule-based DCNN) is proposed to tackle the limitation of the existing techniques, where the Gaussian distribution is applied for the selection of the kernel filters in the formation of the feature maps, and the classifier parameters are adaptively trained using the proposed adaptive nature optimizer (ANO). Through the integration of the conditionality of the random forest algorithm into the developed architecture, the model attains the potential to adapt the processing concerning the supplementary contextual information or features. Utilizing the prominent word embedding approach maximizes the input quality, converts words into continuous vector representations in a high-dimensional space, and captures both syntactic and semantic associations between words. In addition, the Graph embedding preserves some of the graph’s structural characteristics and effectively captures the syntactic dependencies, co-occurrences, semantic similarities, as well as pertinent linguistic correlations among the texts. Specifically, the proposed ANO algorithm and the conditional rule exploited in the model provide stable as well as robust performance in the classification of sentiments. The experimental results demonstrate that the Gaussian conditional rule-based DCNN model achieves 98.36 %, 98.80%, and 97.44% for accuracy, specificity, and sensitivity, respectively.

Knowledge management for product innovation design based on dual-layer concept network

Creative design has become the core of competitiveness for enterprises, and the design process is an activity based on multi-domain knowledge. Designers need to have a lot of interdisciplinary knowledge in this activity, and there may be a burden on understanding. To alleviate understanding burden and promote knowledge integration, a dual-layer concept network is established based on complex networks. Knowledge association in the network originates from the perspective of innovation and product design, facilitating in providing designers with creative and content-related knowledge. This paper takes function, invention principle and scientific principle as the label to build a creative attribute network, and uses the semantic similarity between knowledge texts to build a semantic association network. Combining creative attributes and content semantics of knowledge, a novel knowledge management and browsing method based on a dual-layer concept network is proposed. Meanwhile, due to the ambiguity and uncertainty of design activity and knowledge demand, designers repeatedly query knowledge based on a dual-layer concept network. During this process, designers’ think of diverges and associates, which inspire them to integrate knowledge and generate creative ideas. Finally, this paper takes the problem of vibrating screens as an example to verify the practicability and effectiveness of this method. The methodology is expected to be served as an essential supporting tool for sparking innovation.

Spatiotemporal Interactive Modeling of Event-Based Dynamic Networks

Event-based dynamic networks exist in a wide range of areas, including traffic, biological, and social applications. Such a network consists of interaction event sequences over different locations, where each event may trigger or influence a series of subsequent events under certain intrinsic spatial structure because of their geographical and semantic proximities. Such influence patterns and triggering motivations reflect the nature and semantics of human/object behaviors in the network. Thus, modeling event-based dynamic networks properly is critically important. This article proposes a spatiotemporal interactive Hawkes process (SIHP) that describes how a series of events occurs and models the rate of interaction events between any pair of nodes on the network explicitly with the information from related historical events as well as geographical and semantic neighboring nodes. The proposed SIHP can not only learn the patterns of influence from historical interaction events on later ones, but can also understand the network dynamics by fully considering spatial structure knowledge. Specifically, we incorporate prior knowledge of spatial structure as a graph and design graph regularization in the SIHP, where model parameters are estimated by designing an alternating direction method of multiplier (ADMM) framework. Numerical experiments and a real case study on New York yellow taxi data validate the effectiveness of the proposed method.

Cross-Lingual English–Urdu Semantic Word Similarity Using Sentence Transformers

Semantic word similarity is a quantitative method of determining how much two terms are contextually identical, which is a considerable challenge for computational linguistics. The research community has examined a range of approaches to address this issue. However, most of these approaches are for a comparatively limited set of languages, especially English. Research on semantic word similarity for South Asian languages, particularly Urdu, is immature. In recent years, transformer-based approaches have proved extremely successful for a range of language processing tasks. The primary aim of this study is to develop and compare a variety of transformer-based approaches to the cross-lingual English–Urdu semantic word similarity task. This study evaluated a publicly available benchmark USWS-19 corpus that comprises 518 word pairs. This study mainly explored four types of transformer-based approaches: (a) cross-lingual sentence transformer-based approaches using the original dataset, (b) cross-lingual sentence transformer-based approaches using the translated dataset (translation plus monolingual analysis [T+MA] approach), (c) the feature fusion approach (mixture of features), and large language models. In addition, this study also explores the word embedding-based approach using the translated dataset (T+MA approach). In total, this study developed 29 transformer-based models, with the highest results (Pearson correlation = 0.788) achieved using a feature fusion approach, that is, Best-Two-SBERT (where SBERT stands for sentence-bidirectional encoder representations from transformers; using T+MA) + BEST Baseline (with Bing translator) + Best cross-lingual SBERT. This approach improved by 7% over previously reported results on the same corpus.

Semantic similarity is not emotional: No effect of similarity defined by valence, arousal, and dominance on short-term ordered recall.

In short-term ordered recall tasks, phonological similarity impedes item and order recall, while semantic similarity benefits item recall with a weak or null effect on order recall. Ishiguro and Saito recently suggested that these contradictory findings were due to an inadequate assessment of semantic similarity. They proposed a novel measure of semantic similarity based on the distance between items in a three-dimensional space composed of the semantic dimensions of valence, arousal, and dominance. We conducted an experimental examination of their proposal. In four experiments, participants performed an immediate serial recall or an immediate order reconstruction task. Performance of dissimilar lists was contrasted with performance for semantically similar lists defined by valence, arousal, and dominance or by the typical latent semantic analysis. Two sets of words were used to assess the reproducibility of the findings and similar results were observed with both sets. As expected, when similarity was defined with latent semantic analysis, items were better recalled without noticeable impact on their order. However, contrary to Ishiguro and Saito's predictions, when similarity was defined with valence, arousal and dominance, no effect emerged.

Sexual Orientation Disclosure and Strategic Navigation of Interpersonal Invisibility.

People with concealable stigmatized identities may strategically share or hide cues to their identity. They may likewise seek or avoid interpersonal invisibility (i.e., being ignored). Despite semantic similarities, identity concealment and invisibility are conceptually distinct, but existing empirical work has not explored whether they are independent-that is, whether people may sometimes reveal to be invisible, or conceal to be visible. To evaluate this distinction, we collected lesbian, gay, and bisexual (LGB) participants' free-response descriptions of situations where they revealed or concealed their sexual orientation to be visible or invisible (Study 1). Other LGB participants imagined social interactions in which they revealed or concealed their sexual orientation to be visible or invisible and rated their expected emotions in this situation (Study 2). We find that concealment and invisibility can occur independently of one another with separate effects on emotions, and that LGB people use identity disclosure and concealment to strategically navigate interpersonal invisibility.

Semantic memory space becomes denser with age.

Semantic memory, a repository for concepts and factual information, plays a vital role in acquiring and retrieving knowledge. This study explores the impact of age-related knowledge accumulation on semantic cognition, investigating whether a denser representational space affects retrieval processes. Using a semantic feature verification task, we employ both behavioral (reaction time; RT) and neurophysiological (event-related potential; ERP) measures to explore these dynamics across young and older adults. Findings revealed an age-related RT difference in retrieval of semantically incongruent features, indicative of increased semantic search demands with age. ERP results show attenuated N400 responses in older adults for congruent features, possibly reflecting increased semantic relatedness. The late frontal effect (LFE) shows sustained modulation in older adults, indicative of enhanced post-retrieval monitoring. We propose that this extended search through semantic memory reflects an increase in the number of features to evaluate. These results support the idea that aging leads to a more densely packed semantic space, impacting the speed and dynamics of semantic retrieval.

Augmenting LLMs to Securely Retrieve Information for Construction and Facility Management

In the past few years, generative AI has seen remarkable progress. The emergence of the transformer architecture has facilitated the creation of highly advanced language models that generate text, summarize content, and translate languages with impressive accuracy. Our study introduces a retrieval-augmented generation system tailored to the dynamic needs of facility management. The proposed system aims to provide instant, accurate access to essential information by integrating advanced techniques from natural language processing and information retrieval paradigms. The implementation leverages the Mixtral 8x7B model for multilingual text processing and the Milvus vector database for efficient document storage and retrieval. The dataset used includes documents such as images, operation manuals, inspection results, blueprints, and technical drawings, in various file formats. This diverse dataset reflects the variety of information encountered in construction and facility management. The evaluation involved generating question–answer pairs pertinent to facility management tasks and assessing the system’s performance using metrics such as ROUGE, BLEU, and semantic similarity. The findings suggest that retrieval-augmented generation systems can significantly enhance operational efficiency by reducing the time and effort required to access information while maintaining high security and data privacy standards.

An Empirical Study of the Non-Determinism of ChatGPT in Code Generation

There has been a recent explosion of research on Large Language Models (LLMs) for software engineering tasks, in particular code generation. However, results from LLMs can be highly unstable; non-deterministically returning very different code for the same prompt. Such non-determinism affects the correctness and consistency of the generated code, undermines developers’ trust in LLMs, and yields low reproducibility in LLM-based papers. Nevertheless, there is no work investigating how serious this non-determinism threat is. To fill this gap, this article conducts an empirical study on the non-determinism of ChatGPT in code generation. We chose to study ChatGPT because it is already highly prevalent in the code generation research literature. We report results from a study of 829 code generation problems across three code generation benchmarks (i.e., CodeContests, APPS and HumanEval) with three aspects of code similarities: semantic similarity, syntactic similarity, and structural similarity. Our results reveal that ChatGPT exhibits a high degree of non-determinism under the default setting: the ratio of coding tasks with zero equal test output across different requests is 75.76%, 51.00% and 47.56% for three different code generation datasets (i.e., CodeContests, APPS and HumanEval), respectively. In addition, we find that setting the temperature to 0 does not guarantee determinism in code generation, although it indeed brings less non-determinism than the default configuration ( temperature \(=\) 1). In order to put LLM-based research on firmer scientific foundations, researchers need to take into account non-determinism in drawing their conclusions.

Polycomb-associated and Trithorax-associated developmental conditions-phenotypic convergence and heterogeneity.

Polycomb group (PcG) and Trithorax group (TrxG) complexes represent two major components of the epigenetic machinery. This study aimed to delineate phenotypic similarities and differences across developmental conditions arising from rare variants in PcG and TrxG genes, using data-driven approaches. 462 patients with a PcG or TrxG-associated condition were identified in the DECIPHER dataset. We analysed Human Phenotype Ontology (HPO) data to identify phenotypes enriched in this group, in comparison to other monogenic conditions within DECIPHER. We then assessed phenotypic relationships between single gene diagnoses within the PcG and TrxG group, by applying semantic similarity analysis and hierarchical clustering. Finally, we analysed patient-level phenotypic heterogeneity in this group, irrespective of specific genetic diagnosis, by applying the same clustering approach. Collectively, PcG/TrxG diagnoses were associated with increased reporting of HPO terms relating to integument, growth, head and neck, limb and digestive abnormalities. Gene group analysis identified three multi-gene clusters differentiated by microcephaly, limb/digit dysmorphologies, growth abnormalities and atypical behavioural phenotypes. Patient-level analysis identified two large clusters differentiated by neurodevelopmental abnormalities and facial dysmorphologies respectively, as well as smaller clusters associated with more specific phenotypes including behavioural characteristics, eye abnormalities, growth abnormalities and skull dysmorphologies. Importantly, patient-level phenotypic clusters did not align with genetic diagnoses. Data-driven approaches can highlight pathway-level and gene-level phenotypic convergences, and individual-level phenotypic heterogeneities. Future studies are needed to understand the multi-level mechanisms contributing to both convergence and variability within this population, and to extend data collection and analyses to later-emerging health characteristics.

Generative Diffusion-Based Task Incremental Learning Method for Decoding Motor Imagery EEG

Background/Objectives: Motor neurorehabilitation can be realized by gradually learning diverse motor imagery (MI) tasks. EEG-based brain-computer interfaces (BCIs) provide an effective solution. Nevertheless, existing MI decoding methods cannot balance plasticity for unseen tasks and stability for old tasks. This paper proposes a generative diffusion-based task Incremental Learning (IL) method called GD-TIL. Methods: First, data augmentation is employed to increase data diversity by segmenting and recombining EEG signals. Second, to capture temporal-spatial features (TSFs) from different temporal resolutions, a multi-scale temporal-spatial feature extractor (MTSFE) is developed via integrating multiscale temporal-spatial convolutions, a dual-branch pooling operation, multiple multi-head self-attention mechanisms, and a dynamic convolutional encoder. The proposed self-supervised task generalization (SSTG) mechanism introduces a regularization constraint to guide MTSFE and unified classifier updating, which combines labels and semantic similarity between the augmentation with original views to enhance model generalizability for unseen tasks. In the IL phase, a prototype-guided generative replay module (PGGR) is used to generate old tasks’ TSFs by training a lightweight diffusion model based on the prototype and label of each task. Furthermore, the generated TSF is merged with a new TSF to fine-tune the convolutional encoder and update the classifier and PGGR. Finally, GD-TIL is evaluated on a self-collected ADL-MI dataset with two MI pairs and a public dataset with four MI tasks. Results: The continuous decoding accuracy reaches 80.20% and 81.32%, respectively. The experimental results exhibit the excellent plasticity and stability of GD-TIL, even beating the state-of-the-art IL methods. Conclusions: Our work illustrates the potential of MI-based BCI and generative AI for continuous neurorehabilitation.

Semantic search helper: A tool based on the use of embeddings in multi-item questionnaires as a harmonization opportunity for merging large datasets - A feasibility study.

Recent advances in natural language processing (NLP), particularly in language processing methods, have opened new avenues in semantic data analysis. A promising application of NLP is data harmonization in questionnaire-based cohort studies, where it can be used as an additional method, specifically when only different instruments are available for one construct as well as for the evaluation of potentially new construct-constellations. The present article therefore explores embedding models' potential to detect opportunities for semantic harmonization. Using models like SBERT and OpenAI's ADA, we developed a prototype application ("Semantic Search Helper") to facilitate the harmonization process of detecting semantically similar items within extensive health-related datasets. The approach's feasibility and applicability were evaluated through a use case analysis involving data from four large cohort studies with heterogeneous data obtained with a different set of instruments for common constructs. With the prototype, we effectively identified potential harmonization pairs, which significantly reduced manual evaluation efforts. Expert ratings of semantic similarity candidates showed high agreement with model-generated pairs, confirming the validity of our approach. This study demonstrates the potential of embeddings in matching semantic similarity as a promising add-on tool to assist harmonization processes of multiplex data sets and instruments but with similar content, within and across studies.

Evaluation of natural language processing models to measure similarity between scenarios written in Spanish

Requirements engineering is a critical phase in software development; it seeks to understand and document system requirements from early stages. Typically, requirements specification involves close collaboration between customers and development teams. Customers contribute their expertise in the domain language, while developers use more technical, computational terms. Despite these differences, achieving mutual understanding is crucial.One of the most widely used artifacts for this purpose is scenarios. In environments where multiple actors write scenarios, duplication is common. Thus, there is a need for mechanisms to detect similar scenarios and prevent redundancy. Inthis paper we empirically evaluate several pre-trained Natural Language Processing models to analyze the semantic similarity between scenarios in Spanish, identifying words or phrases with equivalent meanings. It is important to note that the analysis is performed in this language to contribute to the region.Finally, we present a tool that facilitates the creation of new scenarios by identifying potential similarities with existing ones. The tool supports multiple models, allowing users to select the most appropriate one to detect similar scenarios accurately during the definition process.

Task-Oriented Adversarial Attacks for Aspect-Based Sentiment Analysis Models

Adversarial attacks deliberately modify deep learning inputs, mislead models, and cause incorrect results. Previous adversarial attacks on sentiment analysis models have demonstrated success in misleading these models. However, most existing attacks in sentiment analysis have applied a generalized approach to input modifications, without considering the characteristics and objectives of the different analysis levels. Specifically, for aspect-based sentiment analysis, there is a lack of attack methods that modify inputs in accordance with the evaluated aspects. Consequently, unnecessary modifications are made, compromising the input semantics, making the changes more detectable, and avoiding the identification of new vulnerabilities. In previous work, we proposed a model to generate adversarial examples in particular for aspect-based sentiment analysis. In this paper, we assess the effectiveness of our adversarial example model in negatively impacting aspect-based model results while maintaining high levels of semantic inputs. To conduct this evaluation, we propose diverse adversarial attacks across different dataset domains, target architectures, and consider distinct levels of victim model knowledge, thus obtaining a comprehensive evaluation. The obtained results demonstrate that our approach outperforms existing attack methods in terms of accuracy reduction and semantic similarity, achieving a 65.30% reduction in model accuracy with a low perturbation ratio of 7.79%. These findings highlight the importance of considering task-specific characteristics when designing adversarial examples, as even simple modifications to elements that support task classification can successfully mislead models.