Word Sense Disambiguation Model Research Articles

Application of the transformer model algorithm in chinese word sense disambiguation: a case study in chinese language

This study aims to explore the research methodology of applying the Transformer model algorithm to Chinese word sense disambiguation, seeking to resolve word sense ambiguity in the Chinese language. The study introduces deep learning and designs a Chinese word sense disambiguation model based on the fusion of the Transformer with the Bi-directional Long Short-Term Memory (BiLSTM) algorithm. By utilizing the self-attention mechanism of Transformer and the sequence modeling capability of BiLSTM, this model efficiently captures semantic information and context relationships in Chinese sentences, leading to accurate word sense disambiguation. The model’s evaluation is conducted using the PKU Paraphrase Bank, a Chinese text paraphrase dataset. The results demonstrate that the model achieves a precision rate of 83.71% in Chinese word sense disambiguation, significantly outperforming the Long Short-Term Memory algorithm. Additionally, the root mean squared error of this algorithm is less than 17, with a loss function value remaining around 0.14. Thus, this study validates that the constructed Transformer-fused BiLSTM-based Chinese word sense disambiguation model algorithm exhibits both high accuracy and robustness in identifying word senses in the Chinese language. The findings of this study provide valuable insights for advancing the intelligent development of word senses in Chinese language applications.

Word Sense Disambiguation applied to Assamese-Hindi Bilingual Statistical Machine Translation

Word Sense Disambiguation (WSD) is concerned with automatically assigning the appropriate sense to an ambiguous word. WSD is an important task and plays a crucial role in many Natural Language Processing (NLP) applications. A Statistical Machine Translation (SMT) system translates a source into a target language based on phrase-based statistical translation. MT plays a crucial role in a WSD system, as a source language word may be associated with multiple translations in the target language. This study aims to apply WSD to the input of the MT system to enhance the disambiguation output. Hindi WordNet was used by selecting the most frequent synonym to obtain the most accurate translation. This study also compared Naïve Bayes (NB) and Decision Tree (DT) to test and build a WSD model. NB was more appropriate for the WSD task than DT when evaluated in the Weka machine learning toolkit. To the best of our knowledge, no such work has been carried out yet for the Assamese Indo-Aryan language. The applied WSD achieved better results than the baseline MT system without embedding the WSD module. The results were analyzed by linguist scholars. Furthermore, the Assamese-Hindi transliteration system was merged with the baseline MT system for the translation of proper nouns. This study marks a remarkable contribution to Assamese NLP, which is a low computationally aware Indian language.

APPROACHES TO AUTOMATIC WORD SENSE DISAMBIGUATION BASED ON UNEVEN DISTRIBUTION OF WORD SENSES IN CORPUS

Word Sense Disambiguation (WSD) is a key task of automatic semantic analysis that affects other upstream tasks. Nevertheless, the selection of appropriate sense of ambiguous word in context is a complicated task even for human native speakers. It is even more relevant for automatic disambiguation models. That is why we need any observations and heuristics able to make the WSD task simpler or performance higher. Researchers have noticed that the distribution of ambiguous word senses follow certain laws. In the paper we discuss three hypotheses about word senses distribution in corpus: 1) Most Frequent Sense, MFS; 2) One Sense per Discourse and 3) One Sense per Collocation. The following results were obtained on the material of a corpus of Russian texts. Most Frequent Sense based algorithm demonstrates relatively high precision on both training and test set (85.7% and 71.1% respectively). The One Sense per Discourse hypothesis has been confirmed in 93% of texts. The One Sense per Collocation hypothesis has been confirmed in 84.46% word pairs from texts. The exceptions are related to difficulties and errors by manual word sense labeling. Heuristics based on uneven distribution of ambiguous words in corpus allow to make WSD task simpler and can be used by collecting training data sets for WSD models.

Comparison of Pre-trained vs Custom-trained Word Embedding Models for Word Sense Disambiguation

The prime objective of word sense disambiguation (WSD) is to develop such machines that can automatically recognize the actual meaning (sense) of ambiguous words in a sentence. WSD can improve various NLP and HCI challenges. Researchers explored a wide variety of methods to resolve this issue of sense ambiguity. However, majorly, their focus was on English and some other well-reputed languages. Urdu with more than 300 million users and a large amount of electronic text available on the web is still unexplored. In recent years, for a variety of Natural Language Processing tasks, word embedding methods have proven extremely successful. This study evaluates, compares, and applies a variety of word embedding approaches to Urdu Word embedding (both Lexical Sample and All-Words), including pre-trained (Word2Vec, Glove, and FastText) as well as custom-trained (Word2Vec, Glove, and FastText trained on the Ur-Mono corpus). Two benchmark corpora are used for the evaluation in this study: (1) the UAW-WSD-18 corpus and (2) the ULS-WSD-18 corpus. For Urdu All-Words WSD tasks, top results have been achieved (Accuracy=60.07 and F1=0.45) using pre-trained FastText. For the Lexical Sample, WSD has been achieved (Accuracy=70.93 and F1=0.60) using custom-trained GloVe word embedding method.

Findings on Ad Hoc Contractions

Abbreviations are often overlooked, since their frequency and acceptance are almost second nature in everyday communication. Business names, handwritten notes, online messaging, professional domains, and different languages all have their own set of abbreviations. The abundance and frequent introduction of new abbreviations cause multiple areas of overlaps and ambiguity, which mean documents often lose their clarity. We reverse engineered the process of creating these ad hoc abbreviations and revealed some preliminary statistics on what makes them easier or harder to define. In addition, we generated candidate definitions for which it proved difficult for a word sense disambiguation model to select the correct definition.

Attention-based Stacked Bidirectional Long Short-term Memory Model for Word Sense Disambiguation

Word sense disambiguation is a basic task in Natural Language Processing which aims to identify the most appropriate sense of ambiguous words in different contexts by applying algorithm models. In this work, we propose a model that uses a stacked bidirectional Long Short-Term Memory neural network and attention mechanism to determine the sense of ambiguous words. First, the stacked bidirectional Long Short-Term Memory is employed for deep embedding-based representation of sentences containing ambiguous words. Then, we utilize the self-attention mechanism to highlight the contextual features of ambiguous words, and then construct the overall semantic representation of sentences. Finally, the sentence semantic representation is applied to the multilayer perception classifier to generate the appropriate category of the ambiguous word sense items. This model is tested on the Semeval-2007 task-17: English lexical samples dataset and using examples of ambiguous words sourced from Oxford, Cambridge, and Collins dictionaries as extra test datasets. The effectiveness of the proposed approach is demonstrated via comparison with existing word sense disambiguation approaches. Our experimental results show that the proposed model outperforms other word sense disambiguation methods in terms of the evaluation metrics (Average Accuracy, Micro F1-Score, Kappa, and Matthews Correlation Coefficient), and exhibits strong interpretability.

Multi-Head Self-Attention Gated-Dilated Convolutional Neural Network for Word Sense Disambiguation

Word sense disambiguation (WSD) is to determine correct sense of ambiguous word based on its context. WSD is widely used in text classification, machine translation and information retrieval and so on. In order to improve accuracy of simplified Chinese WSD, a WSD model based on multi-head self-attention and gated-dilated convolutional neural network(AGDCNN) is proposed. Ambiguous word is viewed as the center and 4 adjacent lexical units are extracted successively toward the left and right side. Words, parts of speech, and semantic categories in 4 adjacent lexical units are vectorized and the vectorized results are input into gated-dilated convolutional neural network to get discriminative features. Then, multi-head self-attention is adopted to learn the difference and connection among discriminative features fully. Finally, classification weights are output from adaptive average pooling layer. Experiments are conducted on SemEval-2007: Task#5 and SemEval-2021: Task#2. Experimental results show that AGDCNN model has higher accuracy compared with other methods. Our goal is to improve the quality of simplified Chinese WSD as much as possible based on current linguistic resources and machine learning methods. The challenge we face is to extract effective discriminative features and design disambiguation model in high quality. Our novelty lies in that gated-dilated convolution is combined with multi-head self-attention to extract effective discriminative features, and learn their difference and connection from word form, parts of speech, and semantic categories.

Sentence Level Amharic Word Sense Disambiguation

Lexical ambiguity, phonological ambiguity, structural ambiguity, referential ambiguity, semantic ambiguity, and orthographic ambiguity were all types of Amharic ambiguity. The other ambiguities were out of this research because the study focuses on lexical-semantic, orthographic, and semantic ambiguities. Until now, some experts have been researching the Amharic word sense disambiguation system. Recent research, on the other hand, did not take into account antonym, troponymy, holonomy, and homonym relationships in the WordNet; this problem was overcome by this study. Using a Deep Learning method, we are developing an Amharic word sense disambiguation model. We use a design science research strategy to close the gap, starting with problem identification and concluding with final communication. 159 ambiguous words, 1214 synsets, and 2164 sentence datasets were used to create three distinct Deep Learning algorithms in three separate experiments. Using the given dataset, the overall performance of the model is measured using performance metrics in precision, F-measure, and confusion matrix. In this study, LSTM, CNN, and Bi-LSTM obtained 94 percent, 95 percent, and 96 percent accuracy respectively in the third experiment, based on performance measurement.

Word Sense Disambiguation Using Clustered Sense Labels

Sequence labeling models for word sense disambiguation have proven highly effective when the sense vocabulary is compressed based on the thesaurus hierarchy. In this paper, we propose a method for compressing the sense vocabulary without using a thesaurus. For this, sense definitions in a dictionary are converted into sentence vectors and clustered into the compressed senses. First, the very large set of sense vectors is partitioned for less computational complexity, and then it is clustered hierarchically with awareness of homographs. The experiment was done on the English Senseval and Semeval datasets and the Korean Sejong sense annotated corpus. This process demonstrated that the performance greatly increased compared to that of the uncompressed sense model and is comparable to that of the thesaurus-based model.

Word Sense Disambiguation Models Emerging Trends: A Comparative Analysis

Word Sense Disambiguation (WSD) arises due to the presence of ambiguity in the text during the semantic analysis of natural languages. It is a major unsolved problem in the area of Natural Language Processing (NLP) and its applications. This paper explores and reviews WSD algorithms that have contributed to, or created state-of-art solutions in recent years. Moreover, this paper also aims to analyze the recent technological trends in the domain of WSD which can give us leverage to identify the possible future trajectory of the search for better WSD solutions.

A REVIEW OF WORD SENSE DISAMBIGUATION METHOD

Background: Word Sense Disambiguation (WSD) is known to have a detrimental effect on the precision of information retrieval systems, where WSD is the ability to identify the meanings of words in context. There is a challenge in inference-correct-sensing on ambiguous words. Through many years of research, there have been various solutions to WSD that have been proposed; they have been divided into supervised and knowledge-based unsupervised. Objective: The first objective of this study was to explore the state-of-art of the WSD method with a hybrid method using ontology concepts. Then, with the findings, we may understand which tools are available to build WSD components. The second objective was to determine which method would be the best in giving good performance results of WSD, by analysing how the methods were used to answer specific WSD questions, their production, and how their performance was analysed. Methods: A review of the literature was conducted relating to the performance of WSD research, which used a comparison method of information retrieval analysis. The study compared the types of methods used in case, and examined methods for tools production, tools training, and analysis of performance. Results: In total 12 papers were found that satisfied all 3 inclusion criteria, and there was an anchor paper assigned to be referred. We chose the knowledge-based unsupervised approach because it has fewer word sets constraints than the supervised approaches which require training data. Concept-based ontology will help WSD in finding the semantic words concept with respect to another concept around it. Conclusion: Many methods was explored and compared to determine the most suitable way to build a WSD model based on semantics between words in query texts that can be related to the knowledge concept by using ontological knowledge presentation.

Word sense disambiguation based on context selection using knowledge-based word similarity

In this paper, we introduce a novel knowledge-based word-sense disambiguation (WSD) system. In particular, the main goal of our research is to find an effective way to filter out unnecessary information by using word similarity. For this, we adopt two methods in our WSD system. First, we propose a novel encoding method for word vector representation by considering the graphical semantic relationships from the lexical knowledge bases, and the word vector representation is utilized to determine the word similarity in our WSD system. Second, we present an effective method for extracting the contextual words from a text for analyzing an ambiguous word based on word similarity. The results demonstrate that the suggested methods significantly enhance the baseline WSD performance in all corpora. In particular, the performance on nouns is similar to those of the state-of-the-art knowledge-based WSD models, and the performance on verbs surpasses that of the existing knowledge-based WSD models.

Weakly Supervised Word Sense Disambiguation Using Automatically Labelled Collections

State-of-the-art supervised word sense disambiguation models require large sense-tagged training sets. However, many low-resource languages, including Russian, lack such a large amount of data. To cope with the knowledge acquisition bottleneck in Russian, we first utilized the method based on the concept of monosemous relatives to automatically generate a labelled training collection. We then introduce three weakly supervised models trained on this synthetic data. Our work builds upon the bootstrapping approach: relying on this seed of tagged instances, the ensemble of the classifiers is used to label samples from unannotated corpora. Along with this method, different techniques were exploited to augment the new training examples. We show the simple bootstrapping approach based on the ensemble of weakly supervised models can already produce an improvement over the initial word sense disambiguation models.

CSI: A Coarse Sense Inventory for 85% Word Sense Disambiguation

Word Sense Disambiguation (WSD) is the task of associating a word in context with one of its meanings. While many works in the past have focused on raising the state of the art, none has even come close to achieving an F-score in the 80% ballpark when using WordNet as its sense inventory. We contend that one of the main reasons for this failure is the excessively fine granularity of this inventory, resulting in senses that are hard to differentiate between, even for an experienced human annotator. In this paper we cope with this long-standing problem by introducing Coarse Sense Inventory (CSI), obtained by linking WordNet concepts to a new set of 45 labels. The results show that the coarse granularity of CSI leads a WSD model to achieve 85.9% F1, while maintaining a high expressive power. Our set of labels also exhibits ease of use in tagging and a descriptiveness that other coarse inventories lack, as demonstrated in two annotation tasks which we performed. Moreover, a few-shot evaluation proves that the class-based nature of CSI allows the model to generalise over unseen or under-represented words.

Hybrid Sense Classification Method for Large-Scale Word Sense Disambiguation

Word sense disambiguation (WSD) is a task of determining a reasonable sense of a word in a particular context. Although recent studies have demonstrated some progress in the advancement of neural language models, the scope of research is still such that the senses of several words can only be determined in a few domains. Therefore, it is necessary to move toward developing a highly scalable process that can address a lot of senses occurring in various domains. This paper introduces a new large WSD dataset that is automatically constructed from the Oxford Dictionary, which is widely used as a standard source for the meaning of words. We propose a new WSD model that individually determines the sense of the word in accordance with its part of speech in the context. In addition, we introduce a hybrid sense prediction method that separately classifies the less frequently used senses for achieving a reasonable performance. We have conducted comparative experiments to demonstrate that the proposed method is more reliable compared with the baseline approaches. Also, we investigated the adaptation of the method to a realistic environment with the use of news articles.

A Sense Annotated Corpus for All-Words Urdu Word Sense Disambiguation

Word Sense Disambiguation (WSD) aims to automatically predict the correct sense of a word used in a given context. All human languages exhibit word sense ambiguity, and resolving this ambiguity can be difficult. Standard benchmark resources are required to develop, compare, and evaluate WSD techniques. These are available for many languages, but not for Urdu, despite this being a language with more than 300 million speakers and large volumes of text available digitally. To fill this gap, this study proposes a novel benchmark corpus for the Urdu All-Words WSD task. The corpus contains 5,042 words of Urdu running text in which all ambiguous words (856 instances) are manually tagged with senses from the Urdu Lughat dictionary. A range of baseline WSD models based on n -gram are applied to the corpus, and the best performance (accuracy of 57.71%) is achieved using word 4-gram. The corpus is freely available to the research community to encourage further WSD research in Urdu.

Deep Learning Models For Word Sense Disambiguation: A Comparative Study

A specific word can have different meanings depending on the context in which it appears. Identifying the proper sense of the word is crucial in many tasks such as Machine Translation, Anaphora Resolution, Search Engine Recommendation. A word with wrong sense can make the whole sentence meaningless. The task of Word Sense Disambiguation (WSD), is to assign a sense to the ambiguous word based on the context. This is a basic problem which needs to be resolved in the field of NLP and has a variety of solutions. Usual approaches involve supervised machine learning techniques which uses bag-of-words approach. But to achieve better results, WSD should move to sequence modeling rather than the traditional bag of words approach for better performance. Deep neural networks have been used for variety of NLP tasks and a Bi-Directional Long Short-Term Memory (BiLSTM) or a Bi-Directional Gated Recurrent Unit (BiGRU) are possible candidate solutions. The proposed model uses one word per BiGRU or BiLSTM approach, where each word has a model trained for disambiguation. This helps in easy modification of existing model whenever needed. A comparative study on various deep learning models is also performed. The evaluation study of the models shows Bi-Directional models outperforming the other deep learning models.

Interactive medical word sense disambiguation through informed learning.

Medical word sense disambiguation (WSD) is challenging and often requires significant training with data labeled by domain experts. This work aims to develop an interactive learning algorithm that makes efficient use of expert's domain knowledge in building high-quality medical WSD models with minimal human effort. We developed an interactive learning algorithm with expert labeling instances and features. An expert can provide supervision in 3 ways: labeling instances, specifying indicative words of a sense, and highlighting supporting evidence in a labeled instance. The algorithm learns from these labels and iteratively selects the most informative instances to ask for future labels. Our evaluation used 3 WSD corpora: 198 ambiguous terms from Medical Subject Headings (MSH) as MEDLINE indexing terms, 74 ambiguous abbreviations in clinical notes from the University of Minnesota (UMN), and 24 ambiguous abbreviations in clinical notes from Vanderbilt University Hospital (VUH). For each ambiguous term and each learning algorithm, a learning curve that plots the accuracy on the test set against the number of labeled instances was generated. The area under the learning curve was used as the primary evaluation metric. Our interactive learning algorithm significantly outperformed active learning, the previous fastest learning algorithm for medical WSD. Compared to active learning, it achieved 90% accuracy for the MSH corpus with 42% less labeling effort, 35% less labeling effort for the UMN corpus, and 16% less labeling effort for the VUH corpus. High-quality WSD models can be efficiently trained with minimal supervision by inviting experts to label informative instances and provide domain knowledge through labeling/highlighting contextual features.

Neural Network Models for Word Sense Disambiguation: An Overview

Abstract The following article presents an overview of the use of artificial neural networks for the task of Word Sense Disambiguation (WSD). More specifically, it surveys the advances in neural language models in recent years that have resulted in methods for the effective distributed representation of linguistic units. Such representations – word embeddings, context embeddings, sense embeddings – can be effectively applied for WSD purposes, as they encode rich semantic information, especially in conjunction with recurrent neural networks, which are able to capture long-distance relations encoded in word order, syntax, information structuring.

A Game-Theoretic Approach to Word Sense Disambiguation

This article presents a new model for word sense disambiguation formulated in terms of evolutionary game theory, where each word to be disambiguated is represented as a node on a graph whose edges represent word relations and senses are represented as classes. The words simultaneously update their class membership preferences according to the senses that neighboring words are likely to choose. We use distributional information to weigh the influence that each word has on the decisions of the others and semantic similarity information to measure the strength of compatibility among the choices. With this information we can formulate the word sense disambiguation problem as a constraint satisfaction problem and solve it using tools derived from game theory, maintaining the textual coherence. The model is based on two ideas: Similar words should be assigned to similar classes and the meaning of a word does not depend on all the words in a text but just on some of them. The article provides an in-depth motivation of the idea of modeling the word sense disambiguation problem in terms of game theory, which is illustrated by an example. The conclusion presents an extensive analysis on the combination of similarity measures to use in the framework and a comparison with state-of-the-art systems. The results show that our model outperforms state-of-the-art algorithms and can be applied to different tasks and in different scenarios.