Word Sense Disambiguation Approach Research Articles

A Comprehensive Review of Word Sense Disambiguation Research in few Indian Languages: Implications for Educational Tools

Natural Languages are inherently ambiguous. Word Sense Disambiguation is one such problem where one word has multiple meaning depending upon the context in which it appears in the text. It can be considered as an intermediate step for many NLP applications like Machine Translation, Summarization, Query Processing, etc. Developing a Word Sense Disambiguation (WSD) system for the Gujarati language can have significant applications in the education sector. There are different approaches available for WSD which includes Supervised, Unsupervised and knowledge-based approaches. The work done is English language is extensive for this problem but for other regional languages more research needs to be done. This paper presents various works, mentioning their proposed method, datasets used, limitations and performance for some Indian languages like Hindi, Malayalam, Bengali, etc. The paper also enlists some general observations about existing approaches for word sense disambiguation. In the last section, the paper proposes a method to resolve WSD problem for Gujarati Language using Genetic Algorithm.

EnhancedBERT: A feature-rich ensemble model for Arabic word sense disambiguation with statistical analysis and optimized data collection

Accurate assignment of meaning to a word based on its context, known as Word Sense Disambiguation (WSD), remains challenging across languages. Extensive research aims to develop automated methods for determining word senses in different contexts. However, the literature lacks the presence of datasets generated for the Arabic language WSD. This paper presents a dataset comprising a hundred polysemous Arabic words. Each word in the dataset encompasses 3–8 distinct senses, with ten example sentences per sense. Some statistical operations are conducted to gain insights into the dataset, enlightening its characteristics and properties. Subsequently, a novel WSD approach is proposed to utilize similarity measures and find the overlap between contextual information and dictionary definitions. The proposed method uses the power of BERT, a pre-trained language model, to enable effective Arabic word disambiguation. In training, new features are integrated to improve the model's ability to differentiate between various senses of words. The proposed BERT models are combined to compose an ensemble model architecture to improve the classification performances. The performance of the WSD system outperforms state-of-the-art systems, achieving an approximate F1-score of 96 %. Statistical analyses are performed to evaluate the overall performance of the WSD approach by providing additional information on model predictions. A case study was implemented to test the effectiveness of WSD in sentiment analysis, a downstream task.

Stacking of BERT and CNN Models for Arabic Word Sense Disambiguation

We propose a new approach for Arabic Word Sense Disambiguation (AWSD) by hybridization of single-layer Convolutional Neural Network (CNN) with contextual representation (BERT). WSD is the task of automatically detecting the correct meaning of a word used in a given context. WSD can be performed as a classification task, and the context is generally a short sentence. Kim [ 26 ] proved that combining a CNN with an RNN (recurrent neural network) provides a good result for text classification. Here, we use a concatenation of BERT models as a word embedding to get simultaneously the target and context representation. Our approach improves the performance of WSD in Arabic languages. The experimental results show that our model outperforms the state-of-the-art approaches and improves the accuracy of 96.42% on the Arabic WordNet dataset.

Sentence Semantic Similarity based Complex Network approach for Word Sense Disambiguation

Word Sense Disambiguation is a branch of Natural Language Processing(NLP) that deals with multi-sense words. The multi-sense words are referred to as the polysemous words. The term lexical ambiguity is introduced by the multi-sense words. The existing sense disambiguation module works effectively for single sentences with available context information. The word embedding plays a vital role in the process of disambiguation. The context-dependent word embedding model is used for disambiguation. The main goal of this research paper is to disambiguate the polysemous words by considering available context information. The main identified challenge of disambiguation is the ambiguous word without context information. The discussed complex network approach is disambiguating ambiguous sentences by considering the semantic similarities. The sentence semantic similarity-based network is constructed for disambiguating ambiguous sentences. The proposed methodology is trained with SemCor, Adaptive-Lex, and OMSTI standard lexical resources. The findings state that the discussed methodology is working fine for disambiguating large documents where the sense of ambiguous sentences is on the adjacent sentences.

Lexeme connexion measure of cohesive lexical ambiguity revealing factor: a robust approach for word sense disambiguation of Bengali text

Lexeme connexion measure of cohesive lexical ambiguity revealing factor: a robust approach for word sense disambiguation of Bengali text

Attention-based Stacked Bidirectional Long Short-term Memory Model for Word Sense Disambiguation

Word sense disambiguation is a basic task in Natural Language Processing which aims to identify the most appropriate sense of ambiguous words in different contexts by applying algorithm models. In this work, we propose a model that uses a stacked bidirectional Long Short-Term Memory neural network and attention mechanism to determine the sense of ambiguous words. First, the stacked bidirectional Long Short-Term Memory is employed for deep embedding-based representation of sentences containing ambiguous words. Then, we utilize the self-attention mechanism to highlight the contextual features of ambiguous words, and then construct the overall semantic representation of sentences. Finally, the sentence semantic representation is applied to the multilayer perception classifier to generate the appropriate category of the ambiguous word sense items. This model is tested on the Semeval-2007 task-17: English lexical samples dataset and using examples of ambiguous words sourced from Oxford, Cambridge, and Collins dictionaries as extra test datasets. The effectiveness of the proposed approach is demonstrated via comparison with existing word sense disambiguation approaches. Our experimental results show that the proposed model outperforms other word sense disambiguation methods in terms of the evaluation metrics (Average Accuracy, Micro F1-Score, Kappa, and Matthews Correlation Coefficient), and exhibits strong interpretability.

A Thematic-Role-Based Approach for Word Sense Disambiguation

The present paper investigates the thematic roles that can be developed for the purpose of Word Sense Disambiguation. In MT systems and electronic word databases, thematic role relations are not clearly included among other semantic relations. Identifying thematic roles of predicates helps in disambiguating word senses and hence producing more accurate translation. For instance, the meaning of the verb ‘eat’ differs depending on the thematic roles it assigns for its Subject and Object. When it assigns an animate Agent for its Subject and food Patient for its Object, it means ‘take in solid food’. However, when it assigns Force for its Subject and metal Theme for its Object, it means ‘cause to deteriorate due to the action of water, air or an acid’. Accordingly, different translations are produced in each context. Selectional restrictions are also tackled in the analysis of the sample verbs. The implementation is made on three MT systems: Al Wafi, Sakhr and Google. They all produce incorrect translations of the sample verbs. A suggested translation is proposed for each verb after analyzing its thematic roles and selectional restrictions. In this way, the present paper is significant since it helps in improving the performance of MT systems. The present paper will focus only on a group of English verbs that convey a variety of meanings. It will show a number of problematic cases in translation that occur due to the lack of thematic roles in the core of the system. After developing thematic roles, it is expected that such cases will be disambiguated.

A Word Sense Disambiguatin Approach For Romanian Language

The automatic word sense disambiguation has been a topic of interest since the 1950s. Meaning disambiguation is not an end in itself, it is an intermediate process, necessary at a certain level to use in natural language processing. It is obviously useful for applications that require language interpretation (message communication, human-machine interaction), but it is also used in fields whose main purpose is not to understand natural language. In this paper we propose an approach for word sense disambiguation for Romanian language, based on a supervised learning, a large number of features, and information gain to reduce the dimensions. The evaluation for the Romanian words subtask is produced by the extensive feature set in conjunction with a Maximum Entropy classifier and outcomes that are better than other similar researches.

ULMFiT Embedding(s) for Context and Extended Gloss Intersection for Marathi Word Sense Disambiguation

Ambiguities in the word meanings makes all the natural language processing (NLP) tasks very difficult, word sense disambiguation (WSD) is used to resolve these ambiguities. Now a day’s NLP-based human assistive systems are in demand, in which machines are expected to resolve word sense ambiguities. Today, due to the availability of machine readable dictionaries knowledge-based WSD approaches have become popular; it explores semantic relations between the contextual features and possible glosses of the given ambiguous word. Inductive transfer learning-based language models have great potential to represent the different semantic features of the word, which can be used in various NLP tasks. Universal language model fine-tuning for text classification (ULMFiT) is a popular transfer learning model used to embed various semantic features in digitally resource scare and morphologically rich language like marathi. In this reported work, the ambiguous words from the Marathi input sentence is extracted and have obtained its possible synset and glosses from IndoWordNet, these glosses are then extended using hypernym and hyponym relations. We have obtained the word embedding of marathi context and extended glosses using ULMFiT model. For the test run, we have crafted the test-bed of 6000 marathi sentences of 280 moderately ambiguous words harvested from marathi websites, which caters for 1200 senses. The winner sense is declared based on the maximum intersection score between the pair of context and gloss embedding. We have obtained the average accuracy up to 57.10% for our dataset.

Supervised, Unsupervised and Semi-Supervised Word Sense Disambiguation Approaches

Word Sense Disambiguation (WSD) aims to help humans figure out what a word means when used in a certain setting. According to the Neuro Linguistic Programming (NLP) community, WSD is an AI-complete issue with no human solution in sight. WSD has found widespread usage in a wide variety of applications, including but not limited to: Machine translation (MT), Information Retrieval (IR), Data Mining (DM), Information Extraction (IE), and Lexicology (Lex). It is discovered that WSD may be learned effectively using a variety of different methodologies, including supervised, semi-supervised, and unsupervised methods. These methodologies are sorted into groups according to the kind and quantity of annotated (identified) corpora (data) they need as the primary source of information utilized to distinguish between senses. The unsupervised method employs unannotated (unidentifiable) corpora for training, whereas the semi-supervised method requires a less number of annotated corpora than supervised methods. All these three strategies will critically be discussed in this study.

A metaheuristic with a neural surrogate function for Word Sense Disambiguation

Word Sense Disambiguation (WSD) is one of the earliest problems in natural language processing which aims to determine the correct sense of words in context. The semantic information provided by WSD systems is highly beneficial to many tasks such as machine translation, information extraction, and semantic parsing. In this work, a new approach for WSD is proposed which uses a neural network as a surrogate fitness function in a metaheuristic algorithm. Also, a new method for simultaneous training of word and sense embeddings is proposed in this work. Accordingly, the node2vec algorithm is employed on the WordNet graph to generate sequences containing both words and senses. These sequences are then used along with paragraphs from Wikipedia in the word2vec algorithm to generate embeddings for words and senses at the same time. In order to address data imbalance in this task, sense probability distribution data extracted from the training corpus is used in the search process of the proposed simulated annealing algorithm. Furthermore, we introduce a new approach for clustering and mapping senses in the WordNet graph, which considerably improves the accuracy of the proposed method. In this approach, nodes in the WordNet graph are clustered on the condition that no two senses of the same word be present in one cluster. Then, repeatedly, all nodes in each cluster are mapped to a randomly selected node from that cluster, meaning that the representative node can take advantage of the training instances of all the other nodes in the cluster. Training the proposed method in this work is done using the SemCor dataset and the SemEval-2015 dataset has been used as the validation set. The final evaluation of the system is performed on SensEval-2, SensEval-3, SemEval-2007, SemEval-2013, SemEval-2015, and the concatenation of all five mentioned datasets. The performance of the system is also evaluated on the four content word categories, namely, nouns, verbs, adjectives, and adverbs. Experimental results show that the proposed method achieves accuracies in the range of 74.8 to 84.6 percent in the ten aforementioned evaluation categories which are close to and in some cases better than the state of the art in this task.

ADCSA-WSD: Adapted Discrete Crow Search Algorithm for Word Sense Disambiguation

In the field of natural language processing, the semantic disambiguation of words is beneficial to several applications, which helps us to identify the correct meaning of a word or a sequence of words according to the given context. It can be formulated as a combinatorial optimization problem where the goal is to find the set of meanings that contribute to improving the semantic relationship between target words. The Crow Search Algorithm (CSA) is a nature-inspired algorithm. It mimics the food foraging behavior of crow birds and their social interaction. CSA can deal with both continuous and discrete optimization problems. In this paper, the Word Sense Disambiguation (WSD) is modelled as a combinatorial optimization problem that is by nature a discrete problem. For this propose the discrete version of CSA has been adapted for solving the WSD problem and a DCSA-based WSD approach is proposed and called ADCSA-WSD. The proposed approach has been evaluated and compared with state-of-the-art approaches using three well-known benchmark datasets (SemCor 3.0, SensEval-02, SensEval-03). Experimental results show that ADCSA-WSD approach is performing better than other approaches.

A novel word sense disambiguation approach using WordNet knowledge graph

Various applications in computational linguistics and artificial intelligence rely on high-performing word sense disambiguation techniques to solve challenging tasks such as information retrieval, machine translation, question answering, and document clustering. While text comprehension is intuitive for humans, machines face tremendous challenges in processing and interpreting a human’s natural language. This paper presents a novel knowledge-based word sense disambiguation algorithm, namely Sequential Contextual Similarity Matrix Multiplication (SCSMM). The SCSMM algorithm combines semantic similarity, heuristic knowledge, and document context to respectively exploit the merits of local sense-based context between consecutive terms, human knowledge about terms, and a document’s main topic in disambiguating terms. Unlike other algorithms, the SCSMM algorithm guarantees the capture of the maximum sentence context while maintaining the terms’ order within the sentence. The proposed algorithm outperformed all other algorithms when disambiguating nouns on the combined gold standard datasets, while demonstrating comparable results to current state-of-the-art word sense disambiguation systems when dealing with each dataset separately. Furthermore, the paper discusses the impact of granularity level, ambiguity rate, sentence size, and part of speech distribution on the performance of the proposed algorithm.

МОДИФИЦИРОВАННЫЙ МЕТОД УСТРАНЕНИЯ НЕОДНОЗНАЧНОСТИ СМЫСЛА СЛОВ, ОСНОВАННЫЙ НА МЕТОДАХ РАСПРЕДЕЛЕННОГО ПРЕДСТАВЛЕНИЯ

In the text mining tasks, textual representation should be not only efficient but also interpretable,as this enables an understanding of the operational logic underlying the data miningmodels. This paper describes a modified Word Sense Disambiguation (WSD) method which extendstwo well-known variations of the Lesk WSD approach. Given a word and its context, Leskbases its calculations on the overlap between the context of a word and each definition of its senses(gloss) in order to select the proper meaning. The main contribution of the proposed method isthe adoption of the concept of “similarity” between definition and context instead of "overlap", inaddition to expanding the definition with examples provided by WordNet for each sense of thetarget word. The proposed method is also characterized by the use of text similarity measurementfunctions defined in a distributed semantic space. The proposed method has been tested on fivedifferent benchmark datasets for words sense disambiguation tasks and compared with severalbasic methods, including simple Lesk, extended Lesk, WordNet 1st sense, Babelfy and UKB. Theresults show that proposed method outperforms most basic methods with the exception of Babelfyand the WN 1st sense methods.

A REVIEW ON WORD SENSE DISAMBIGUATION EMPHASIZING THE DATA RESOURCES ON WORDNET AND CORPUS

Word Sense Disambiguation is a disambiguating technique of finding the most relevant sense of an ambiguous word with the aid of its surrounding words. In this paper, we pointed out the various Word Sense Disambiguation approaches along with its different techniques, state of the art, comparative studies of the existing system highlighting its benefits and limitations across all the widely well known Indian and foreign languages. In this paper, we converse our study by emphasizing to all the studies that uses WordNet, IndoWordNet or corpus as the main data resources of the referred languages.

An adaptive approach for word sense disambiguation for Hindi language

Word sense disambiguation has been a classical problem and it is still considered as a big challenge for NLP researchers. Word vectors have shown remarkable improvements from baseline performances for various NLP challenges. An adaptive approach is developed to resolve the ambiguity at word level using word vectors and presented in this paper. CBOW and Skip-gram models of Word2Vec are used to generate the word embeddings. An input sentence is split into two sets: The first set contains all possible ambiguous words and the second set is constituted from all the unambiguous words. The words from the first set are processed through IndoWordNet to collect all the senses of respective words, and such senses are stored in sensedictionary. An adaptive approach is applied to find out the exact interpretation of the ambiguous word from the sense dictionary for each word from the set of ambiguous words with the context words from the set of unambiguous words. The proposed approach is evaluated on a large scale Hindi corpus and claimed better results than previous attempts.

Hypernymy in WordNet, Its Role in WSD and Its Limitations

This paper closely analysed and depicted the role of the hyponymy that is used in the lexical database called the Word Net to relate the words semantically with a relation ... Is a kind of.... The relation has very important role for disambiguating the senses of a polysemy word in any natural language and is, therefore, massively used in knowledge-based Word Sense Disambiguation (WSD) approaches. However, the limitation of this relation is not so cared about. In many cases, the use of the hypernymy does not only mean nothing for disambiguation but also it increases only the computational effort for the system. The improper use of hypernymy is very costly. In some cases, which are described later, the use of this relation even decreases the accuracy of the WSD approaches.

Context expansion approach for graph-based word sense disambiguation

Word sense disambiguation is a process to correctly identify the meanings of words in a given context. Being important in many natural language processing applications, this process is crucial in automatically understanding natural language expressions. Herein, we propose a variation of a well-known unsupervised graph-based word sense disambiguation method that utilizes all possible semantic information from a used lexical resource to increase graph-semantic connectivity for identifying the intended meanings of words in a given context. If the words have multiple potential meanings (senses) based on context, the proposed method builds an expanded graph representing most relevant semantic information of the words to be disambiguated. Nodes in the graph correspond to the context expansion set, which contains all associated information of each possible meaning of the word (word sense), and edges represent the semantic similarity between the expanded sets (nodes). Simultaneously, actual meaning is assigned to each target word using a locate graph centrality measure, which provides the degree of importance between graph nodes. Unlike most existing graph-based word sense disambiguation methods, wherein semantic relations (edges) between nodes are measured at the word level, the proposed method measures graph node semantic relations at the sentence level by expanding the words’ context, which contains all associated information for each possible word sense. Consequently, the proposed method can capture a higher degree of semantic information than existing approaches, thereby increasing semantic connectivity through a graph’s edges. Empirical results on benchmark datasets demonstrate that the proposed method outperforms all compared state-of-the-art graph-based word sense disambiguation approaches reported herein. We also report results obtained by applying the proposed method to a sentiment analysis task. These results demonstrate that the proposed method can determine the overall sentiment orientation of a given textual context.

Explicitly Modeling Word Translations in Neural Machine Translation

In this article, we show that word translations can be explicitly incorporated into NMT effectively to avoid wrong translations. Specifically, we propose three cross-lingual encoders to explicitly incorporate word translations into NMT: (1) Factored encoder, which encodes a word and its translation in a vertical way; (2) Gated encoder, which uses a gated mechanism to selectively control the amount of word translations moving forward; and (3) Mixed encoder, which stitchingly learns a word and its translation annotations over sequences where words and their translations are alternatively mixed. Besides, we first use a simple word dictionary approach and then a word sense disambiguation (WSD) approach to effectively model the word context for better word translation. Experimentation on Chinese-to-English translation demonstrates that all proposed encoders are able to improve the translation accuracy for both traditional RNN-based NMT and recent self-attention-based NMT (hereafter referred to as Transformer ). Specifically, Mixed encoder yields the most significant improvement of 2.0 in BLEU on the RNN-based NMT, while Gated encoder improves 1.2 in BLEU on Transformer . This indicates the usefulness of an WSD approach in modeling word context for better word translation. This also indicates the effectiveness of our proposed cross-lingual encoders in explicitly modeling word translations to avoid wrong translations in NMT. Finally, we discuss in depth how word translations benefit different NMT frameworks from several perspectives.

A Semantic Framework for Extracting Taxonomic Relations from Text Corpus

Nowadays, ontologies have been exploited in many current applications due to the abilities in representing knowledge and inferring new knowledge. However, the manual construction of ontologies is tedious and time-consuming. Therefore, the automated ontology construction from text has been investigated. The extraction of taxonomic relations between concepts is a crucial step in constructing domain ontologies. To obtain taxonomic relations from a text corpus, especially when the data is deficient, the approach of using the web as a source of collective knowledge (a.k.a web-based approach) is usually applied. The important challenge of this approach is how to collect relevant knowledge from a large amount of web pages. To overcome this issue, we propose a framework that combines Word Sense Disambiguation (WSD) and web approach to extract taxonomic relations from a domain-text corpus. This framework consists of two main stages: concept extraction and taxonomic-relation extraction. Concepts acquired from the concept-extraction stage are disambiguated through WSD module and passed to stage of extraction taxonomic relations afterward. To evaluate the efficiency of the proposed framework, we conduct experiments on datasets about two domains of tourism and sport. The obtained results show that the proposed method is efficient in corpora which are insufficient or have no training data. Besides, the proposed method outperforms the state of the art method in corpora having high WSD results.