Word Semantic Similarity Research Articles

Global Ontology Entities Embeddings

Ontologies are among the most widely used types of knowledge representation formalisms. The application of deep learning techniques in the field of ontology engineering has reinforced the need to learn and generate representations of ontological data. This allows ontologies to be exploited by such models, and thus automate various ontology engineering tasks, where most of the existing tools and machine learning approaches require a numerical feature vectors associated with each concept. This paper outlines a novel approach for learning global ontology entities embeddings by exploiting the structure and the various taxonomic and semantic relationships present in ontologies, taking into account all the information present in the ontological graph and carried by the OWL/RDF triples. Thus, producing global ontology entities embeddings capturing the global ontological graph semantics and similarities enclosed in the source ontology. Three different neural network models have been proposed based on two architectures: multi-input and multi-output, trained using the contrastive estimation technique. The evaluation on OWL/RDF ontologies and word semantic similarity tasks using various graph and WordNet based similarity measures, show that our approach yields competitive results outperforming the state-of-the-art ontology and word embedding models.

EXSCLAIM!: Harnessing materials science literature for self-labeled microscopy datasets

This work introduces the EXSCLAIM! toolkit for the automatic extraction, separation, and caption-based natural language annotation of images from scientific literature. EXSCLAIM! is used to show how rule-based natural language processing and image recognition can be leveraged to construct an electron microscopy dataset containing thousands of keyword-annotated nanostructure images. Moreover, it is demonstrated how a combination of statistical topic modeling and semantic word similarity comparisons can be used to increase the number and variety of keyword annotations on top of the standard annotations from EXSCLAIM! With large-scale imaging datasets constructed from scientific literature, users are well positioned to train neural networks for classification and recognition tasks specific to microscopy-tasks often otherwise inhibited by a lack of sufficient annotated training data.

Defining Semantically Close Words of Kazakh Language with Distributed System Apache Spark

This work focuses on determining semantically close words and using semantic similarity in general in order to improve performance in information retrieval tasks. The semantic similarity of words is an important task with many applications from information retrieval to spell checking or even document clustering and classification. Although, in languages with rich linguistic resources, the methods and tools for this task are well established, some languages do not have such tools. The first step in our experiment is to represent the words in a collection in a vector form and then define the semantic similarity of the terms using a vector similarity method. In order to tame the complexity of the task, which relies on the number of word (and, consequently, of the vector) pairs that have to be combined in order to define the semantically closest word pairs, A distributed method that runs on Apache Spark is designed to reduce the calculation time by running comparison tasks in parallel. Three alternative implementations are proposed and tested using a list of target words and seeking the most semantically similar words from a lexicon for each one of them. In a second step, we employ pre-trained multilingual sentence transformers to capture the content semantics at a sentence level and a vector-based semantic index to accelerate the searches. The code is written in MapReduce, and the experiments and results show that the proposed methods can provide an interesting solution for finding similar words or texts in the Kazakh language.

Improving Efficiency and Accuracy in English Translation Learning: Investigating a Semantic Analysis Correction Algorithm

ABSTRACT Translation serves as a vital link in connecting individuals from diverse cultural backgrounds, assuming greater significance in the context of globalization. With the continued growth of international communication, the importance of effective translation in fostering mutual understanding, exploring new horizons, and building relationships cannot be overstated. However, the persistent language barrier necessitates improvements in translation accuracy and efficiency. In recent years, information technology has facilitated the development of innovative algorithms for English translation. This study aims to investigate correction algorithms for English translation learning, proposing a method that encompasses semantic word similarity calculations, a log-linear model construction, and the selection of appropriate translations using a dependency tree-to-string approach. Our research examines the performance of this system through comprehensive testing, showcasing noteworthy enhancements in the efficacy and precision of the translations in English. By addressing the users’ needs for English translation correction, this work contributes to bridging language gaps and fostering effective cross-cultural communication.

Text Vectorization Techniques Based on Wordnet

Abstract The utilization of text vectorization techniques has become essential for numerous classification tasks in present-day natural language processing. Word embedding methods commonly used today, such as Word2Vec, GloVe, etc., are based on the semantic similarity of words. WordNet, as a lexical database of words, provides a rich source of semantic information. In our article, we propose a text vectorization technique using extended text data with the data augmentation method, specifically by replacing words with their synonyms obtained from WordNet. The results obtained from text classification tasks using multiple classifiers demonstrate that expanding the corpus with this method leads to improved vector representations of words.

Automatic Keyword Extraction Algorithm for Chinese Text based on Word Clustering

There are some problems in automatic keyword extraction of Chinese text, such as large feature extraction error, low precision of extracted keywords, and poor real-time performance. Therefore, an automatic keyword extraction algorithm for Chinese text based on word clustering is designed. Calculate keyword frequency, document frequency and inverse document frequency features through statistical algorithm, measure the degree of interdependence between keywords with the help of point mutual information, and construct keyword feature item quantification matrix with the help of vector space model corresponding to keywords and feature items to complete keyword feature quantification and realize keyword feature extraction of Chinese text. Calculate the average semantic similarity of keyword words, determine the similarity of keyword features, and eliminate the keyword features with high similarity; Set the comprehensive feature value of the importance of single word words in Chinese text, determine the importance of single word words in the text, remove the single word words with low importance, and use Bayesian framework to reduce the dimension of high-dimensional keyword feature data to realize preprocessing research. The mapping results of keyword vector space model are determined by word clustering algorithm, the text clusters of keyword space clustering results are calculated by clustering algorithm, and the keywords are classified by DBN method. On this basis, the automatic keyword extraction model of Chinese text is designed to realize the automatic keyword extraction of Chinese text. The experimental results show that the design algorithm can effectively reduce the feature extraction error and improve the extraction efficiency.

Bidirectional cross-linguistic influence with different-script languages: Evidence from eye tracking

AbstractThis study compared patterns of nonselective cross-language activation in L1 and L2 visual word recognition with different-script bilinguals. The aim was to determine (1) whether lexical processing is nonselective in the L1 (as in L2), and (2) if the same cross-linguistic factors affected processing similarly in each language. To examine the time course of activation, eye movements were tracked during lexical decision. Thirty-two Japanese–English bilinguals responded to 250 target words in Japanese and in English. The same participants and items (i.e., cognate translation equivalents) were used to directly compare L1 and L2 processing. Response latencies as well as eye movements representing early and late processing were analyzed using mixed-effects regression modeling. Similar cross-linguistic effects, namely cognate word frequency, phonological similarity, and semantic similarity, were found in both languages. These factors affected processing to different degrees in each language, however. While cognate frequency was significant as early as the first fixation, effects of cross-linguistic phonological and semantic similarity arose later in time. Increased phonological similarity slowed responses in L2 but speeded them in L1, while greater semantic overlap was facilitatory in both languages. Results are discussed from the perspective of the BIA+ model of visual word recognition.

The "Small World of Words" free association norms for Rioplatense Spanish.

Large-scale word association datasets are both important tools used in psycholinguistics and used as models that capture meaning when considered as semantic networks. Here, we present word association norms for Rioplatense Spanish, a variant spoken in Argentina and Uruguay. The norms were derived through a large-scale crowd-sourced continued word association task in which participants give three associations to a list of cue words. Covering over 13,000 words and +3.6 M responses, it is currently the most extensive dataset available for Spanish. We compare the obtained dataset with previous studies in Dutch and English to investigate the role of grammatical gender and studies that used Iberian Spanish to test generalizability to other Spanish variants. Finally, we evaluated the validity of our data in word processing (lexical decision reaction times) and semantic (similarity judgment) tasks. Our results demonstrate that network measures such as in-degree provide a good prediction of lexical decision response times. Analyzing semantic similarity judgments showed that results replicate and extend previous findings demonstrating that semantic similarity derived using spreading activation or spectral methods outperform word embeddings trained on text corpora.

Developing a Large Benchmark Corpus for Urdu Semantic Word Similarity

The semantic word similarity task aims to quantify the degree of similarity between a pair of words. In literature, efforts have been made to create standard evaluation resources to develop, evaluate, and compare various methods for semantic word similarity. The majority of these efforts focused on English and some other languages. However, the problem of semantic word similarity has not been thoroughly explored for South Asian languages, particularly Urdu. To fill this gap, this study presents a large benchmark corpus of 518 word pairs for the Urdu semantic word similarity task, which were manually annotated by 12 annotators. To demonstrate how our proposed corpus can be used for the development and evaluation of Urdu semantic word similarity systems, we applied two state-of-the-art methods: (1) a word embedding–based method and (2) a Sentence Transformer–based method. As another major contribution, we proposed a feature fusion method based on Sentence Transformers and word embedding methods. The best results were obtained using our proposed feature fusion method (the combination of best features of both methods) with a Pearson correlation score of 0.67. To foster research in Urdu (an under-resourced language), our proposed corpus will be free and publicly available for research purposes.

Automated Interpretation of Place Descriptions: Determining Entity Types for Querying OSM

This paper is concerned with interpretation of natural language place descriptions, as they are a rich source of geographic information. A place description is interpreted by matching geographic entities occurring in the text against the OpenStreetMap (OSM) database. This paper is mainly concerned with interpretation of paraphrased places, i.e., entities for which no name is given and which may only by described. Our objective is to determine suitable entity types that allow querying the OpenStreetMap database for the respective place. For example, if we wish to identify a place to eat, we have to check for entities of an a-priori unknown type (cafe, restaurant, etc.). Challenges arise from the open-endedness of language, its ambiguity, and context-sensitivity as well as from mismatches between human conceptualization of place and database ontologies. The contributions of this paper are, first, to present a hard problem that is key to geo-information retrieval beyond named entities. Second, we propose context-sensitive methods for identifying place types based on semantic word similarity. We evaluate the methods on text extracted from Wikipedia and travel blogs, revealing their contribution to advancing automated interpretation of place descriptions to paraphrased places.

Saussure on synonymy

Saussure’s observations on synonymie in De la double essence du langage (Saussure 2011) cast an interesting light on his general account of linguistic meaning. Synonymie covers three distinct concepts for Saussure: 1) semantic similarity of words in a particular language, based on the contrasts between (and mutual demarcation of) meanings in the lexicon, 2) semantic identity of different words in a particular language, by definition impossible in natural language, and 3) multifunctionality of a word in a particular language, to be understood in terms of a hierarchical relationship between a word’s encoded language-particular meaning and its instances of use (including figurative uses). As well as tracing the connections between these three concepts, this article calls attention to the internal consistency of Saussure’s explanations and underscores their importance for a coherent and unified approach to the linguistic sign in accordance with Saussure’s basic theoretical assumptions.

Using big data to understand bilingual performance in semantic fluency: Findings from the Canadian Longitudinal Study on Aging.

This study aimed to characterize verbal fluency performance in monolinguals and bilinguals using data from the Canadian Longitudinal Study on Aging (CLSA). A large sample of adults aged 45-85 (n = 12,875) completed a one-minute animal fluency task in English. Participants were English-speaking monolinguals (n = 9,759), bilinguals who spoke English as their first language (L1 bilinguals, n = 1,836), and bilinguals who spoke English as their second language (L2 bilinguals, n = 1,280). Using a distributional modeling approach to quantify the semantic similarity of words, we examined the impact of word frequency and pairwise semantic similarity on performance on this task. Overall, L1 bilinguals outperformed monolinguals on the verbal fluency task: they produced more items, and these items were of lower average frequency and semantic similarity. Monolinguals in turn outperformed L2 bilinguals on these measures. The results held across different age groups, educational, and income levels. These results demonstrate an advantage for bilinguals compared to monolinguals on a category fluency task, when performed in the first language, indicating that, at least in the CLSA sample, bilinguals have superior semantic search capabilities in their first language compared to monolingual speakers of that language.

Semantic and Acoustic Markers in Schizophrenia-Spectrum Disorders: A Combinatory Machine Learning Approach.

Speech is a promising marker to aid diagnosis of schizophrenia-spectrum disorders, as it reflects symptoms like thought disorder and negative symptoms. Previous approaches made use of different domains of speech for diagnostic classification, including features like coherence (semantic) and form (acoustic). However, an examination of the added value of each domain when combined is lacking as of yet. Here, we investigate the acoustic and semantic domains separately and combined. Using semi-structured interviews, speech of 94 subjects with schizophrenia-spectrum disorders (SSD) and 73 healthy controls (HC) was recorded. Acoustic features were extracted using a standardized feature-set, and transcribed interviews were used to calculate semantic word similarity using word2vec. Random forest classifiers were trained for each domain. A third classifier was used to combine features from both domains; 10-fold cross-validation was used for each model. The acoustic random forest classifier achieved 81% accuracy classifying SSD and HC, while the semantic domain classifier reached an accuracy of 80%. Joining features from the two domains, the combined classifier reached 85% accuracy, significantly improving on separate domain classifiers. For the combined classifier, top features were fragmented speech from the acoustic domain and variance of similarity from the semantic domain. Both semantic and acoustic analyses of speech achieved ~80% accuracy in classifying SSD from HC. We replicate earlier findings per domain, additionally showing that combining these features significantly improves classification performance. Feature importance and accuracy in combined classification indicate that the domains measure different, complementing aspects of speech.

Zero-shot language extension for dialogue state tracking via pre-trained models and multi-auxiliary-tasks fine-tuning

Dialogue state tracking (DST), a crucial component of the task-oriented dialogue system (TOD), is designed to track the user’s goal. Existing DST models mainly focus on monolingual dialogue input, failing to meet the growing needs of a TOD to provide multilingual services. Therefore, this paper proposes a novel Zero-shot Language Extension scenario for DST, extending the monolingual DST to multilingual DST without extra high-cost dialogue data annotation. In this scenario, the multilingual DST only needs a single shared model to handle multilingual input and generate a unified dialogue state. This setting makes deploying a complete multilingual TOD easy since it could be reused by the downstream components from existing monolingual TOD. Specifically, we achieve the language extension by multi-auxiliary-tasks fine-tuning of multilingual pre-trained models, where five relevant auxiliary tasks are jointly designed, including monolingual DST, cross-lingual DST, forward word translation, utterance recovery, and semantic similarity. The extended multilingual DST model can be enhanced through joint optimization with all the auxiliary tasks by capturing multilingual context understanding and cross-lingual alignment characteristics. Comprehensive experiments on Multilingual WOZ dataset (English → German and English → Italian) and cross-lingual MultiWOZ dataset (English → Chinese and Chinese → English) demonstrate the effectiveness and superiority of the proposed method.

A New Feature-Based Method for Similarity Measurement under the Linux Operating System

This paper presents a new algorithm in an important research field which is the semantic word similarity estimation. A new feature-based algorithm is proposed for measuring the word semantic similarity for the Arabic language. It is a highly systematic language where its words exhibit elegant and rigorous logic. The score of sematic similarity between two Arabic words is calculated as a function of their common and total taxonomical features. An Arabic knowledge source is employed for extracting the taxonomical features as a set of all concepts that subsumed the concepts containing the compared words. The previously developed Arabic word benchmark datasets are used for optimizing and evaluating the proposed algorithm. In this paper, the performance of the new feature-based algorithm is compared against the performance of seven ontology-based algorithms adapted to Arabic. The results of the evaluation and comparison experiments show that the new proposed algorithm outperforms the adapted word similarity algorithms on the Arabic word benchmark dataset. The proposed algorithm will be included in the AWN-similarity which is free open-source software for Arabic.

COVID‑19 and a Greek Catholic Hierarch’s Experience of Virtual Communication with Parishioners

In this article, the author carries out a communicative and pragmatic analysis of texts and a stylistic analysis of the keywords of religious discourse. The object of the research is the discourse of the Greek Catholic Church in Slovakia. The subject of the research is the keywords and main topics in the corpus of Archbishop Milan Chautur’s publications on social networks. The research methodology is based on the principles of discourse analysis, computational linguistics, linguistics 2.0, and quantitative linguistics. To realise the purpose of the research, the author uses the linguistic tool Sketch Engine, a fourth-generation concordancer, which provides quick search and statistical processing of data. For the corpus, the author refers to material from Chautur’s Internet messages in the profile named “Vladyka Milan odpovedá” (Vladyka Chautur responds) over one year of the pandemic, i. e., between 2020 and 2021. Using the Simple Maths method, the author extracts keywords based on two corpora – the main one (created by the user) and the background one (built into the system). Vladyka’s web-communication fits the modern trend of active communication through social networks, where it is not only used by representatives of the secular world, but also by the clergy. Considering the fact that the background corpus of Slovak Web 2011 (skTenTen11) is a language model, the author singles out the following keywords: isus (Jesus), Christ (Christ), pandémia (pandemic), kovid (Covid), bohorodička (Virgin Mary), cerkov (Church), vakcinácium (vaccinacium), and eparchia (diocese). These words have the highest coefficient of semantic similarity (Score) of words from the studied and background corpora. Using the linguistic criterion for isolating keywords, it is possible to single out the cognitive components of the archbishop’s discursive field, such as discursive dominants: human life, the Church on earth and in heaven, family.

Content Classification based-on Latent Semantic Analysis and Support Vector Machine (LSA-SVM)

The diversity of the content of a web page can have a negative impact if used by the wrong user. Almost a half of internet users are children. Therefore, it is important to classify web pages to find out which pages are worthy of being seen by children and that are not feasible. One method that can be used is the Support Vector Machine (SVM) algorithm. SVM is a binary classification whose working principle is to find the best hyperplane to separate the two classes. To obtain better classification accuracy, the SVM is combined with the Latent Semantic Analysis (LSA) algorithm. The data used in this study were taken from the DMOZ web data which has been classified into two categories. The data is then entered into the pre-processing stage for further feature extraction using LSA. The LSA algorithm is used to find out the semantic similarities of words and text contained in web pages. The results of feature extraction are then classified using SVM with RBF kernel. Based on the testing result, we obtain a classification accuracy of 64%.

A Simple Yet Robust Algorithm for Automatic Extraction of Parallel Sentences: A Case Study on Arabic-English Wikipedia Articles

Parallel corpora are vital components in several applications of Natural Language Processing (NLP), particularly in machine translation. In this paper, we present a novel method for automatically creating parallel sentences from comparable corpora. The method requires a bilingual dictionary as well as an adequate word-vectorisation method. We use Arabic and English Wikipedia as a comparable corpus to apply our proposed method and construct a parallel corpus between Arabic and English. The created Arabic-English corpus consists of 105,010 parallel sentences with a total number of 4.6M words. During our study, we compared two methods of word vectorisation, word embedding and term frequency-inverse document frequency, in terms of their usefulness in computing similarities between well-formed and syntactically ill-formed sentences. We also quantitatively and qualitatively examined the parallel corpus produced by our proposed method and compared it with other available Arabic-English parallel corpora counterparts: GlobalVoices, TED, and Wiki-OPUS. We explored the main advantages and shortcomings of these corpora when used for NLP applications, such as word semantic similarity identification and Neural Machine Translation (NMT). The word semantic similarity models trained on our parallel corpus outperformed models trained on other corpora in the task of English non-similar word identification. Our parallel corpus also proved competitive when building Arabic-English NMT systems, yielding results comparable to those of the automatically created Wiki-OPUS corpus and of the manually created TED corpus, while achieving results superior to the smaller GlobalVoices corpus.

Phonological and semantic similarity of misperceived words in babble: Effects of sentence context, age, and hearing loss.

This study investigated how age and hearing loss influence the misperceptions made when listening to sentences in babble. Open-set responses to final words in sentences with low and high context were analyzed for younger adults with normal hearing and older adults with normal or impaired hearing. All groups performed similarly in overall accuracy but differed in error type. Misperceptions for all groups were analyzed according to phonological and semantic properties. Comparisons between groups indicated that misperceptions for older adults were more influenced by phonological factors. Furthermore, older adults with hearing loss omitted more responses. Overall, across all groups, results suggest that phonological confusions most explain misperceptions in low context sentences. In high context sentences, the meaningful sentence context appears to provide predictive cues that reduce misperceptions. When misperceptions do occur, responses tend to have greater semantic similarity and lesser phonological similarity to the target, compared to low context sentences. In this way, semantic similarity may index a postdictive process by which ambiguities due to phonological confusions are resolved to conform to the semantic context of the sentence. These patterns demonstrate that context, age, and hearing loss affect the misperceptions, and potential sentence interpretation, made when listening to sentences in babble.

Developing a Cross-lingual Semantic Word Similarity Corpus for English–Urdu Language Pair

Semantic word similarity is a quantitative measure of how much two words are contextually similar. Evaluation of semantic word similarity models requires a benchmark corpus. However, despite the millions of speakers and the large digital text of the Urdu language on the Internet, there is a lack of benchmark corpus for the Cross-lingual Semantic Word Similarity task for the Urdu language. This article reports our efforts in developing such a corpus. The newly developed corpus is based on the SemEval-2017 task 2 English dataset, and it contains 1,945 cross-lingual English–Urdu word pairs. For each of these pairs of words, semantic similarity scores were assigned by 11 native Urdu speakers. In addition to corpus generation, this article also reports the evaluation results of a baseline approach, namely “Translation Plus Monolingual Analysis” for automated identification of semantic similarity between English–Urdu word pairs. The results showed that the path length similarity measure performs better for the Google and Bing translated words. The newly created corpus and evaluation results are freely available online for further research and development.