Semantic Similarity Research Articles

Disentangling inhibition and prediction in negation processing

Negation as a universal feature of human language is used effortlessly in everyday communication. However, experimental research has shown that the comprehension of negated sentences seems to require additional cognitive resources compared to affirmative sentences. Many studies investigating the processing of negation report longer reading and reaction times for negative compared to affirmative sentences and many studies report a Polarity by Truth interaction: false affirmative sentences lead to longer response times and larger N400 event-related potentials (ERPs) than true affirmative sentences, whereas the pattern is reversed for negative sentences where it is the true sentence that elicits longer reaction times and higher N400 ERPs compared to false negative sentences. These interactions have been discussed in the light of lexical associations, predictability, and the need to construct two subsequent mental representations. Furthermore, recent studies have shown that the comprehension of negated sentences seems to make use of neural resources that are typically involved in cognitive control and inhibitory mechanisms. As both processes have been associated with two different and temporally overlapping ERP components (the N400 and the P300), we focus on studies with high temporal resolution. We discuss linguistic aspects of negation, such as semantic similarity and contextual invariance of negation. We furthermore discuss the role of the verb as well as the position of the negative marker with respect to the verb, and their respective relevance for predictive and inhibitory mechanisms in negated sentences.

An interpretable metric of visual aesthetics for GUI design

ABSTRACT Computation-based aesthetics metrics have been developed to help designers predict visual aesthetics scores for GUI design. However, designers find these evaluative scores difficult to understand. This paper proposed an interpretable aesthetics metric for GUI design that integrates visual aesthetics (visual similarity and spatial proximity) and GUI structure (semantic similarity and white space) to model visual grouping distribution. Two experiments were conducted to validate the metric’s ability to predict aesthetics and interpret outputs. Experiment 1 showed that our metric had a stronger correlation with users’ impressions of GUI visual aesthetics than past metrics. Experiment 2 suggested that our metric was easier to interpret and appeared more useful to Visual/Graphic/GUI designers than a conventional score-based alternative, by visualising the metric outputs as an experimental tool. Furthermore, this paper provided five potential insights to further advance computational aesthetics research.

In search of an optimal method for analyzing deep structures: frame semantics and classification of argumentative structures

The subject of the research is the search for the optimal method of analyzing deep structures using frame semantics. The study of semantic roles, similarities and differences in the approaches of both C. Fillmore and B. Levin – M. R. Hovav make it possible to analyze the structure of a sentence in more detail and accurately, identify deep cases and determine semantic relations between words. The study of these aspects is key to understanding language constructs and their interpretation. The study of various approaches makes it possible to identify both common features and unique features, which is key for a complete understanding of language constructions. An interest in text analysis in the field of artificial intelligence, machine learning and computational linguistics, and an understanding of the semantic relationships between words will help create more accurate and efficient text processing algorithms. One of the research methods is the semantic analysis of sentences based on corpus data. This method includes the study of various linguistic constructions in the context of their use in real texts, which allows us to identify common patterns and rules for the use of these linguistic units in different situations. The scientific novelty of the study lies in the fact that the authors have determined the similarity of the approaches on how to understand the surface and deep structures of language of Ch. Fillmore and B. Levin and M. Rappaport. Their work, despite differences in methodology and terminology, together allow for in-depth investigation of the relationship between the meanings of verbs and the structure of arguments. As a result of the study, the natural relationships between deep cases and semantic roles in sentences of various types are revealed, and key points that need to be taken into account when analyzing deep structures for a more accurate definition of the semantic roles of arguments are highlighted: frame semantics and thematic grids. Disagreements and alternative points of view contribute to the constant development and improvement of linguistic theories. Such debates eventually lead to a deeper understanding of the implementation of the arguments and open up opportunities for further research in this area. Both C. Fillmore and B. Levin and M. Rappaport have made significant contributions to understanding the surface and deep structures of language, although their approaches and terminology may differ.

Multi-scale persistent spatiotemporal transformer for long-term urban traffic flow prediction

Long-term urban traffic flow prediction is an important task in the field of intelligent transportation, as it can help optimize traffic management and improve travel efficiency. To improve prediction accuracy, a crucial issue is how to model spatiotemporal dependency in urban traffic data. In recent years, many studies have adopted spatiotemporal neural networks to extract key information from traffic data. However, most models ignore the semantic spatial similarity between long-distance areas when mining spatial dependency. They also ignore the impact of predicted time steps on the next unpredicted time step for making long-term predictions. Moreover, these models lack a comprehensive data embedding process to represent complex spatiotemporal dependency. This paper proposes a multi-scale persistent spatiotemporal transformer (MSPSTT) model to perform accurate long-term traffic flow prediction in cities. MSPSTT adopts an encoder-decoder structure and incorporates temporal, periodic, and spatial features to fully embed urban traffic data to address these issues. The model consists of a spatiotemporal encoder and a spatiotemporal decoder, which rely on temporal, geospatial, and semantic space multi-head attention modules to dynamically extract temporal, geospatial, and semantic characteristics. The spatiotemporal decoder combines the context information provided by the encoder, integrates the predicted time step information, and is iteratively updated to learn the correlation between different time steps in the broader time range to improve the model's accuracy for long-term prediction. Experiments on four public transportation datasets demonstrate that MSPSTT outperforms the existing models by up to 9.5% on three common metrics.

Looking Beyond Single Images for Weakly Supervised Semantic Segmentation Learning.

This article studies the problem of learning weakly supervised semantic segmentation (WSSS) from image-level supervision only. Current popular solutions leverage object localization maps from classifiers as supervision for semantic segmentation learning, and struggle to make the localization maps capture more complete object content. Rather than previous efforts that primarily focus on intra-image information, we address the value of cross-image semantic relations for comprehensive object pattern mining. To achieve this, two neural co-attentions are incorporated into the classifier to complementarily capture cross-image semantic similarities and differences. In particular, given a pair of training images, one co-attention enforces the classifier to recognize the common semantics from co-attentive objects, while the other one, called contrastive co-attention, drives the classifier to identify the unique semantics from the rest, unshared objects. This helps the classifier discover more object patterns and better ground semantics in image regions. In addition to boosting object pattern learning, the co-attention can leverage context from other related images to improve localization map inference, hence eventually benefiting semantic segmentation learning. More importantly, our algorithm provides a unified framework that handles well different WSSS settings, i.e., learning WSSS with 1) precise image-level supervision only, 2) extra simple single-label data, and 3) extra noisy web data. Without bells and whistles, it sets new state-of-the-arts on all these settings. Moreover, our approach ranked 1st place in the Weakly-Supervised Semantic Segmentation Track of CVPR2020 Learning from Imperfect Data Challenge. The extensive experimental results demonstrate well the efficacy and high utility of our method.

LMGATCDA: Graph Neural Network With Labeling Trick for Predicting circRNA-Disease Associations.

Previous studies have proven that circular RNAs (circRNAs) are inextricably connected to the etiology and pathophysiology of complicated diseases. Since conventional biological research are frequently small-scale, expensive, and time-consuming, it is essential to establish an efficient and reasonable computation-based method to identify disease-related circRNAs. In this article, we proposed a novel ensemble model for predicting probable circRNA-disease associations based on multi-source similarity information(LMGATCDA). In particular, LMGATCDA first incorporates information on circRNA functional similarity, disease semantic similarity, and the Gaussian interaction profile (GIP) kernel similarity as explicit features, along with node-labeling of the three-hop subgraphs extracted from each linked target node as graph structural features. After that, the fused features are used as input, and further implied features are extracted by graph sampling aggregation (GraphSAGE) and multi-hop attention graph neural network (MAGNA). Finally, the prediction scores are obtained through a fully connected layer. With five-fold cross-validation, LMGATCDA demonstrated excellent competitiveness against gold standard data, reaching 95.37% accuracy and 91.31% recall with an AUC of 94.25% on the circR2Disease benchmark dataset. Collectively, the noteworthy findings from these case studies support our conclusion that the LMGATCDA model can provide reliable circRNA-disease associations for clinical research while helping to mitigate experimental uncertainties in wet-lab investigations.

PhraseMap: Attention-Based Keyphrases Recommendation for Information Seeking.

Many Information Retrieval (IR) approaches have been proposed to extract relevant information from a large corpus. Among these methods, phrase-based retrieval methods have been proven to capture more concrete and concise information than word-based and paragraph-based methods. However, due to the complex relationship among phrases and a lack of proper visual guidance, achieving user-driven interactive information-seeking and retrieval remains challenging. In this study, we present a visual analytic approach for users to seek information from an extensive collection of documents efficiently. The main component of our approach is a PhraseMap, where nodes and edges represent the extracted keyphrases and their relationships, respectively, from a large corpus. To build the PhraseMap, we extract keyphrases from each document and link the phrases according to word attention determined using modern language models, i.e., BERT. As can be imagined, the graph is complex due to the extensive volume of information and the massive amount of relationships. Therefore, we develop a navigation algorithm to facilitate information seeking. It includes (1) a question-answering (QA) model to identify phrases related to users' queries and (2) updating relevant phrases based on users' feedback. To better present the PhraseMap, we introduce a resource-controlled self-organizing map (RC-SOM) to evenly and regularly display phrases on grid cells while expecting phrases with similar semantics to stay close in the visualization. To evaluate our approach, we conducted case studies with three domain experts in diverse literature. The results and feedback demonstrate its effectiveness, usability, and intelligence.

Models of Polysemy in Two English Dictionaries

Abstract In this paper we argue in favor of the radial semantic structure of polysemous entries from the New Oxford Dictionary of English (NODE) and the Merriam-Webster dictionary. We formalized four polysemy theories as algorithms linking word senses into polysemy networks, considering the semantic similarity of dictionary definitions calculated using large language models. Chaining algorithms maximized similarity within polysemy nets and adequately distinguished senses that in the two dictionaries are directly related from those more distant.

BioLORD-2023: semantic textual representations fusing large language models and clinical knowledge graph insights.

In this study, we investigate the potential of large language models (LLMs) to complement biomedical knowledge graphs in the training of semantic models for the biomedical and clinical domains. Drawing on the wealth of the Unified Medical Language System knowledge graph and harnessing cutting-edge LLMs, we propose a new state-of-the-art approach for obtaining high-fidelity representations of biomedical concepts and sentences, consisting of 3 steps: an improved contrastive learning phase, a novel self-distillation phase, and a weight averaging phase. Through rigorous evaluations of diverse downstream tasks, we demonstrate consistent and substantial improvements over the previous state of the art for semantic textual similarity (STS), biomedical concept representation (BCR), and clinically named entity linking, across 15+ datasets. Besides our new state-of-the-art biomedical model for English, we also distill and release a multilingual model compatible with 50+ languages and finetuned on 7 European languages. Many clinical pipelines can benefit from our latest models. Our new multilingual model enables a range of languages to benefit from our advancements in biomedical semantic representation learning, opening a new avenue for bioinformatics researchers around the world. As a result, we hope to see BioLORD-2023 becoming a precious tool for future biomedical applications. In this article, we introduced BioLORD-2023, a state-of-the-art model for STS and BCR designed for the clinical domain.

Natural language processing for defining linguistic features in schizophrenia: A sample from Turkish speakers

Natural language processing (NLP) provides fast and accurate extraction of features related to the language of schizophrenia. We utilized NLP methods to test the hypothesis that schizophrenia is associated with altered linguistic features in Turkish, a non-Indo-European language, compared to controls. We also explored whether these possible altered linguistic features were language-dependent or -independent. We extracted and compared speech in schizophrenia (SZ, N = 38) and healthy well-matched control (HC, N = 38) participants using NLP. The analysis was conducted in two parts. In the first one, mean sentence length, total completed words, moving average type-token ratio to measure the lexical diversity, and first-person singular pronoun usage were calculated. In the second one, we used parts-of-speech tagging (POS) and Word2Vec in schizophrenia and control. We found that SZ had lower mean sentence length and moving average type-token ratio but higher use of first-person singular pronoun. All these significant results were correlated with the Thought and Language Disorder Scale score. The POS approach demonstrated that SZ used fewer coordinating conjunctions. Our methodology using Word2Vec detected that SZ had higher semantic similarity than HC and K-Means could differentiate between SZ and HC into two distinct groups with high accuracy, 86.84 %. Our findings showed that altered linguistic features in SZ are mostly language-independent. They are promising to describe language patterns in schizophrenia which proposes that NLP measurements may allow for rapid and objective measurements of linguistic features.

Semantic Web and Distance Learning: Euphoria, Frustration and Restart

In the nineties, two different trends on distance learning research arose. The first one aimed to create systems able to adapt themselves to students’ characteristics, based on instructional and learning theories, while the second aimed to facilitate the creation and management of distance courses, by reusing material. Technologies available so far had hindered the creation of systems able to conjoin the benefits achieved by both groups because adaptive systems were limited to a specific knowledge domain, and almost everything had to be recreated to be used in a different subject. On the other hand, systems able to deal with different subjects could not automatically adapt their content to students’ needs. The emergence of the Semantic Web, as well as Learning Objects, led researchers to develop projects based on them to solve that problem. At last, adaptive and reusable courses based on learning theories could be developed. Learning Object Metadata and other standards for e-learning, as well as ontologies about different knowledge domains, students’ characteristics and learning theories were the tools that researchers were missing to solve such problems. Although much research and publications have been done in that period, few practical results were noticed, frustrating not only the students, but also the teachers and other education-related workers, as they realized that the level of system autonomy they had aimed for had not been reached. This paper describes the euphoria caused by the Semantic Web, problems related to the engineering of ontologies that led to a frustration period, as well as the current period, in which research was resumed with less ambitious objectives. Thus, some considerations about the future of the Semantic Web Based Learning are presented.

Technical language processing for Prognostics and Health Management: applying text similarity and topic modeling to maintenance work orders

AbstractModern manufacturing paradigms have incorporated Prognostics and Health Management (PHM) to implement data-driven methods for fault detection, failure prediction, and assessment of system health. The maintenance operation has similarly benefitted from these advancements, and predictive maintenance is now being used across the industry. Despite these developments, most of the approaches in maintenance rely on numerical data from sensors and field devices for any sort of analysis. Text data from Maintenance Work Orders (MWOs) contain some of the most crucial information pertaining to the functioning of systems and components, but are still regarded as ‘black holes’, i.e., they store valuable data without being used in decision-making. The analysis of this data can help save time and costs in maintenance. While Natural Language Processing (NLP) methods have been very successful in understanding and examining text data from non-technical sources, progress in the analysis of technical text data has been limited. Non-technical text data are usually structured and consist of standardized vocabularies allowing the use of out-of-the-box language processing methods in their analysis. On the other hand, records from MWOs are often semi-structured or unstructured; and consist of complicated terminologies, technical jargon, and industry-specific abbreviations. Deploying traditional NLP to such data can result in an imprecise and flawed analysis which can be very costly. Owing to these challenges, we propose a Technical Language Processing (TLP) framework for PHM. To illustrate its capabilities, we use text data from MWOs of aircraft to address two scenarios. First, we predict corrective actions for new maintenance problems by comparing them with existing problems using syntactic and semantic textual similarity matching and evaluate the results with cosine similarity scores. In the second scenario, we identify and extract the most dominant topics and salient terms from the data using Latent Dirichlet Allocation (LDA). Using the results, we are able to successfully link maintenance problems to standardized maintenance codes used in the aviation industry.

The contribution of semantic distance knowledge to size constancy in perception and grasping when visual cues are limited

To achieve a stable perception of object size in spite of variations in viewing distance, our visual system needs to combine retinal image information and distance cues. Previous research has shown that, not only retinal cues, but also extraretinal sensory signals can provide reliable information about depth and that different neural networks (perception versus action) can exhibit preferences in the use of these different sources of information during size-distance computations. Semantic knowledge of distance, a purely cognitive signal, can also provide distance information. Do the perception and action systems show differences in their ability to use this information in calculating object size and distance? To address this question, we presented ‘glow-in-the-dark’ objects of different physical sizes at different real distances in a completely dark room. Participants viewed the objects monocularly through a 1-mm pinhole. They either estimated the size and distance of the objects or attempted to grasp them. Semantic knowledge was manipulated by providing an auditory cue about the actual distance of the object: “20 cm”, “30 cm”, and “40 cm”. We found that semantic knowledge of distance contributed to some extent to size constancy operations during perceptual estimation and grasping, but size constancy was never fully restored. Importantly, the contribution of knowledge about distance to size constancy was equivalent between perception and action. Overall, our study reveals similarities and differences between the perception and action systems in the use of semantic distance knowledge and suggests that this cognitive signal is useful but not a reliable depth cue for size constancy under restricted viewing conditions.

Intent Identification by Semantically Analyzing the Search Query

Understanding and analyzing the search intent of a user semantically based on their input query has emerged as an intriguing challenge in recent years. It suffers from small-scale human-labeled training data that produce a very poor hypothesis of rare words. The majority of data portals employ keyword-driven search functionality to explore content within their repositories. However, the keyword-based search cannot identify the users’ search intent accurately. Integrating a query-understandable framework into keyword search engines has the potential to enhance their performance, bridging the gap in interpreting the user’s search intent more effectively. In this study, we have proposed a novel approach that focuses on spatial and temporal information, phrase detection, and semantic similarity recognition to detect the user’s intent from the search query. We have used the n-gram probabilistic language model for phrase detection. Furthermore, we propose a probability-aware gated mechanism for RoBERTa (Robustly Optimized Bidirectional Encoder Representations from Transformers Approach) embeddings to semantically detect the user’s intent. We analyze and compare the performance of the proposed scheme with the existing state-of-the-art schemes. Furthermore, a detailed case study has been conducted to validate the model’s proficiency in semantic analysis, emphasizing its adaptability and potential for real-world applications where nuanced intent understanding is crucial. The experimental result demonstrates that our proposed system can significantly improve the accuracy for detecting the users’ search intent as well as the quality of classification during search.

Semantic and style based multiple reference learning for artistic and general image aesthetic assessment

Artistic Image Aesthetic Assessment (AIAA) is an emerging paradigm that predicts the aesthetic score as the popular aesthetic taste for an artistic image. Previous AIAA takes a single image as input to predict the aesthetic score of the image. However, most existing AIAA methods fail dramatically to predict the artistic images with a large variance of artistic subjective voting with only a single image. People are good at employing multiple similar references for making relative comparisons. Motivated by the practice that people considers similar semantics and specific artistic style to keep the consistency of the voting result, we present a novel Semantic and Style based Multiple Reference learning (SSMR) to mimic this natural process. Our novelty is mainly two-fold: (a) Similar Reference Index Generation (SRIG) module that considers artistic attribution of semantics and style to generate the index of reference images; (b) Multiple Reference Graph Reasoning (MRGR) module that employs graph convolutional network (GCN) to initialize and reason by adjusting the weight of edges with intrinsic relationships among multiple images. Our evaluation with the benchmark BAID, VAPS and TAD66K artistic aesthetic datasets demonstrates that the proposed SSMR outperforms state-of-the-art AIAA methods, and verifies the comparable to the SOTA IAA methods on the AVA general aesthetic dataset.

Exploratory study on the syntactic and semantic consistency of terms in project management glossaries to provide recommendations for a project management ontology

This paper shows an exploratory study of the syntactic and semantic similarities and discrepancies of the terms of four selected project management glossaries. The main purpose of this study is to provide recommendations on adoptions and adaptations of labels and/or definitions of project management glossary terms to be included in a new or existing project management ontology. As a result, a list of recommended terms for a project management ontology to be built is analyzed. The recommendation of terms will be limited to generic terms that can be located at the core level instead of the domain level in the context of an ontological architecture. In particular, the list of terms will be discussed in light of a previously developed project management ontology that will be updated in future work. Another goal of this work is to evaluate the level of syntactic and semantic consistency and harmonization that currently exists in these glossaries. As a result, it becomes apparent from this early research that many opportunities exist to improve these terminologies for greater consistency, harmonization, and standardization in the field.

Simcryocluster: a semantic similarity clustering method of cryo-EM images by adopting contrastive learning

BackgroundCryo-electron microscopy (Cryo-EM) plays an increasingly important role in the determination of the three-dimensional (3D) structure of macromolecules. In order to achieve 3D reconstruction results close to atomic resolution, 2D single-particle image classification is not only conducive to single-particle selection, but also a key step that affects 3D reconstruction. The main task is to cluster and align 2D single-grain images into non-heterogeneous groups to obtain sharper single-grain images by averaging calculations. The main difficulties are that the cryo-EM single-particle image has a low signal-to-noise ratio (SNR), cannot manually label the data, and the projection direction is random and the distribution is unknown. Therefore, in the low SNR scenario, how to obtain the characteristic information of the effective particles, improve the clustering accuracy, and thus improve the reconstruction accuracy, is a key problem in the 2D image analysis of single particles of cryo-EM.ResultsAiming at the above problems, we propose a learnable deep clustering method and a fast alignment weighted averaging method based on frequency domain space to effectively improve the class averaging results and improve the reconstruction accuracy. In particular, it is very prominent in the feature extraction and dimensionality reduction module. Compared with the classification method based on Bayesian and great likelihood, a large amount of single particle data is required to estimate the relative angle orientation of macromolecular single particles in the 3D structure, and we propose that the clustering method shows good results.ConclusionsSimcryoCluster can use the contrastive learning method to perform well in the unlabeled high-noise cryo-EM single particle image classification task, making it an important tool for cryo-EM protein structure determination

Why is a flamingo named as pelican and asparagus as celery? Understanding the relationship between targets and errors in a speeded picture naming task

ABSTRACT Speakers sometimes make word production errors, such as mistakenly saying pelican instead of flamingo. This study explored which properties of an error influence the likelihood of its selection over the target word. Analysing real-word errors in speeded picture naming, we investigated whether, relative to the target, naming errors were more typical representatives of the semantic category, were associated with more semantic features, and/or were semantically more closely related to the target than its near semantic neighbours were on average. Results indicated that naming errors tended to be more typical category representatives and possess more semantic features than the targets. Moreover, while not being the closest semantic neighbours, errors were largely near semantic neighbours of the targets. These findings suggest that typicality, number of semantic features, and semantic similarity govern activation levels in the production system, and we discuss possible mechanisms underlying these effects in the context of word production theories.

Intelligent System for Cognitive Study on Student Work Network Based on the Information of Public Opinion with the Background of IOT

Abstract Neurological, behavioral, and cognitive problems in teenage students are rising day by day. It is very important to handle these issues by gathering information from public opinion. The distribution of public opinion information in student work is unsystematic, and it is difficult to extract the needed information intelligently. An intelligent extraction algorithm of public opinion data from student work is proposed in this research and is based on human–computer interaction, machine learning, and computational techniques. In the student work, a fuzzy semantic autocorrelation mapping feature set of public opinion information is developed together with a spatial structural model of data semantic distribution features. The statistical feature quantity of semantic similarity of the public opinion data is retrieved from the student work, and the semantic ambiguity is decreased. Analyses using adaptive learning and machine understanding and human–computer interaction are used to process it. In the human–computer interaction machine understanding center, the processor adjusts the grid partition of public opinion information of the student work according to the difference of statistical features, then constructs the feature decomposition model of student behavior, and then performs context mapping. Finally, the semantic analysis is carried out to analyze the student behavior based on the cognitive study. The simulation outcomes demonstrate that the proposed method is computationally inexpensive, has low time complexity, and has real-time monitoring ability of public opinion information on student behavior.

Beyond images: an integrative multi-modal approach to chest x-ray report generation.

Image-to-text radiology report generation aims to automatically produce radiology reports that describe the findings in medical images. Most existing methods focus solely on the image data, disregarding the other patient information accessible to radiologists. In this paper, we present a novel multi-modal deep neural network framework for generating chest x-rays reports by integrating structured patient data, such as vital signs and symptoms, alongside unstructured clinical notes. We introduce a conditioned cross-multi-head attention module to fuse these heterogeneous data modalities, bridging the semantic gap between visual and textual data. Experiments demonstrate substantial improvements from using additional modalities compared to relying on images alone. Notably, our model achieves the highest reported performance on the ROUGE-L metric compared to relevant state-of-the-art models in the literature. Furthermore, we employed both human evaluation and clinical semantic similarity measurement alongside word-overlap metrics to improve the depth of quantitative analysis. A human evaluation, conducted by a board-certified radiologist, confirms the model's accuracy in identifying high-level findings, however, it also highlights that more improvement is needed to capture nuanced details and clinical context.