Word Semantic Similarity Research Articles

Word Sense Disambiguation Using Cosine Similarity Collaborates with Word2vec and WordNet

Words have different meanings (i.e., senses) depending on the context. Disambiguating the correct sense is important and a challenging task for natural language processing. An intuitive way is to select the highest similarity between the context and sense definitions provided by a large lexical database of English, WordNet. In this database, nouns, verbs, adjectives, and adverbs are grouped into sets of cognitive synonyms interlinked through conceptual semantics and lexicon relations. Traditional unsupervised approaches compute similarity by counting overlapping words between the context and sense definitions which must match exactly. Similarity should compute based on how words are related rather than overlapping by representing the context and sense definitions on a vector space model and analyzing distributional semantic relationships among them using latent semantic analysis (LSA). When a corpus of text becomes more massive, LSA consumes much more memory and is not flexible to train a huge corpus of text. A word-embedding approach has an advantage in this issue. Word2vec is a popular word-embedding approach that represents words on a fix-sized vector space model through either the skip-gram or continuous bag-of-words (CBOW) model. Word2vec is also effectively capturing semantic and syntactic word similarities from a huge corpus of text better than LSA. Our method used Word2vec to construct a context sentence vector, and sense definition vectors then give each word sense a score using cosine similarity to compute the similarity between those sentence vectors. The sense definition also expanded with sense relations retrieved from WordNet. If the score is not higher than a specific threshold, the score will be combined with the probability of that sense distribution learned from a large sense-tagged corpus, SEMCOR. The possible answer senses can be obtained from high scores. Our method shows that the result (50.9% or 48.7% without the probability of sense distribution) is higher than the baselines (i.e., original, simplified, adapted and LSA Lesk) and outperforms many unsupervised systems participating in the SENSEVAL-3 English lexical sample task.

A hybrid approach for paraphrase identification based on knowledge-enriched semantic heuristics

In this paper, we propose a hybrid approach for sentence paraphrase identification. The proposal addresses the problem of evaluating sentence-to-sentence semantic similarity when the sentences contain a set of named-entities. The essence of the proposal is to distinguish the computation of the semantic similarity of named-entity tokens from the rest of the sentence text. More specifically, this is based on the integration of word semantic similarity derived from WordNet taxonomic relations, and named-entity semantic relatedness inferred from Wikipedia entity co-occurrences and underpinned by Normalized Google Distance. In addition, the WordNet similarity measure is enriched with word part-of-speech (PoS) conversion aided with a Categorial Variation database (CatVar), which enhances the lexico-semantics of words. We validated our hybrid approach using two different datasets; Microsoft Research Paraphrase Corpus (MSRPC) and TREC-9 Question Variants. In our empirical evaluation, we showed that our system outperforms baselines and most of the related state-of-the-art systems for paraphrase detection. We also conducted a misidentification analysis to disclose the primary sources of our system errors.

Albanian Text Classification: Bag of Words Model and Word Analogies

Abstract Background: Text classification is a very important task in information retrieval. Its objective is to classify new text documents in a set of predefined classes, using different supervised algorithms. Objectives: We focus on the text classification for Albanian news articles using two approaches. Methods/Approach: In the first approach, the words in a collection are considered as independent components, allocating to each of them a conforming vector in the vector’s space. Here we utilized nine classifiers from the scikit-learn package, training the classifiers with part of news articles (80%) and testing the accuracy with the remaining part of these articles. In the second approach, the text classification treats words based on their semantic and syntactic word similarities, supposing a word is formed by n-grams of characters. In this case, we have used the fastText, a hierarchical classifier, that considers local word order, as well as sub-word information. We have measured the accuracy for each classifier separately. We have also analyzed the training and testing time. Results: Our results show that the bag of words model does better than fastText when testing the classification process for not a large dataset of text. FastText shows better performance when classifying multi-label text. Conclusions: News articles can serve to create a benchmark for testing classification algorithms of Albanian texts. The best results are achieved with a bag of words model, with an accuracy of 94%.

A Comprehensive Study of the Parameters in the Creation and Comparison of Feature Vectors in Distributional Semantic Models

ABSTRACT Measuring the semantic similarity and relatedness of words can play a vital role in many natural language processing tasks. Distributional semantic models computing these measures can have many different parameters, such as different weighting schemes, vector similarity measures, feature transformation functions and dimensionality reduction techniques. Despite their importance there is no truly comprehensive study simultaneously evaluating the numerous parameters of such models, while also considering the interaction of these parameters with each other. We would like to address this gap with our systematic study. Taking the necessary distributional information extracted from the chosen dataset as already granted, we evaluate all important aspects of the creation and comparison of feature vectors in distributional semantic models. Testing altogether 10 parameters simultaneously, we try to find the best combination of parameter settings, with a large number of settings examined in case of some of the parameters. Beside evaluating the conventionally used settings for the parameters, we also propose numerous novel variants, as well as novel combinations of parameter settings, some of which significantly outperform the combinations of settings in general use, thus achieving state-of-the-art results.

A framework for intelligent question answering system using semantic context-specific document clustering and Wordnet

The question answering system plays an important role in information retrieval field, where the user is in need of getting a precise answer instead of large collections of documents. The aim of this paper is to investigate techniques for improving sentence-based question answering system. To achieve this, a POS-Tagger-based question pattern analysis model is proposed to identify question type based on pattern template for the user-submitted query. Next, the knowledge base is created from a large corpus by clustering the documents by grouping on domain context. The proposed semantic-word-based answer generator model deals with the user query mapping with an appropriate sentence in the knowledge base. By the proposed models, the system reduces the search gap among user queries and answer sentences using Wordnet. It considers word order, overlap, sentence similarity, string distance, unambiguous words and semantic similarity of words. The proposed algorithm evaluates with benchmark datasets such as 20Newsgroup and TREC-9 QA, and proves its efficiency by statistical test for significance.

A survey on word embedding techniques and semantic similarity for paraphrase identification

In natural language processing (NLP), paraphrase identification (PI) determines the relatedness between the pair of sentences having fewer or negligible lexical overlap but still pointing towards the same meaning. The major challenge faced while attempting to solve this problem is the many possible linguistic variations conveying the same purpose. This paper aims to provide a detailed survey of traditional similarity measures, statistical machine translation metrics, machine learning and deep learning techniques and a well-defined flow between them. This article encompasses various word embedding methods and step-wise derivation of its learning module. This survey paper also provides a definite flow pointing towards the evolution of deep learning in an unambiguous manner. A comparative analysis of various techniques to solve PI is presented and it will provide research directions to work in the similar domain.

A survey on word embedding techniques and semantic similarity for paraphrase identification

A survey on word embedding techniques and semantic similarity for paraphrase identification

Document-based topic coherence measures for news media text

There is a rising need for automated analysis of news text, and topic models have proven to be useful tools for this task. However, as the quality of the topics induced by topic models greatly varies, much research effort has been devoted to their automated evaluation. Recent research has focused on topic coherence as a measure of a topic’s quality. Existing topic coherence measures work by considering the semantic similarity of topic words. This makes them unfit to detect the coherence of transient topics with semantically unrelated topic words, which abound in news media texts. In this paper, we introduce the notion of document-based topic coherence and propose novel topic coherence measures that estimate topic coherence based on topic documents rather than topic words. We evaluate the proposed measures on two datasets containing topics manually labeled for document-based coherence, on which the proposed measures outperform a strong baseline as well as word-based coherence measures. We also demonstrate the usefulness of document-based coherence measures for automated topic discovery from news media texts.

Weighting-based semantic similarity measure based on topological parameters in semantic taxonomy

AbstractSemantic measures are used in handling different issues in several research areas, such as artificial intelligence, natural language processing, knowledge engineering, bioinformatics, and information retrieval. Hierarchical feature-based semantic measures have been proposed to estimate the semantic similarity between two concepts/words depending on the features extracted from a semantic taxonomy (hierarchy) of a given lexical source. The central issue in these measures is the constant weighting assumption that all elements in the semantic representation of the concept possess the same relevance. In this paper, a new weighting-based semantic similarity measure is proposed to address the issues in hierarchical feature-based measures. Four mechanisms are introduced to weigh the degree of relevance of features in the semantic representation of a concept by using topological parameters (edge, depth, descendants, and density) in a semantic taxonomy. With the semantic taxonomy of WordNet, the proposed semantic measure is evaluated for word semantic similarity in four gold-standard datasets. Experimental results show that the proposed measure outperforms hierarchical feature-based semantic measures in all the datasets. Comparison results also imply that the proposed measure is more effective than information-content measures in measuring semantic similarity.

A Semantic Similarity Algorithm Based on the Nearest Common Ancestor Node

The word semantic similarity calculation is the basis of keyword search in the field of semantic web information retrieval. In view of the deficiency of semantic similarity computation in the existing algorithms, this paper discussed the influence of node depth on the semantic similarity, and presented an improved similarity calculation method based on the nearest common ancestor node, which can represent the depth of the two sememe nodes that need to compute the similarity. The simulation was carried out by using simple biological ontology and HowNet system architecture data, and the comparison with the original algorithm was done. The results prove the correctness and rationality of the improved method.

Hierarchical Rhetorical Sentence Categorization for Scientific Papers

Important information in scientific papers can be composed of rhetorical sentences that is structured from certain categories. To get this information, text categorization should be conducted. Actually, some works in this task have been completed by employing word frequency, semantic similarity words, hierarchical classification, and the others. Therefore, this paper aims to present the rhetorical sentence categorization from scientific paper by employing TF-IDF and Word2Vec to capture word frequency and semantic similarity words and employing hierarchical classification. Every experiment is tested in two classifiers, namely Naïve Bayes and SVM Linear. This paper shows that hierarchical classifier is better than flat classifier employing either TF-IDF or Word2Vec, although it increases only almost 2% from 27.82% when using flat classifier until 29.61% when using hierarchical classifier. It shows also different learning model for child-category can be built by hierarchical classifier.

Computational linguistic analysis applied to a semantic fluency task to measure derailment and tangentiality in schizophrenia

Although rating scales to assess formal thought disorder exist, there are no objective, high-reliability instruments that can quantify and track it. This proof-of-concept study shows that CoVec, a new automated tool, is able to differentiate between controls and patients with schizophrenia with derailment and tangentiality. According to ratings from the derailment and tangentiality items of the Scale for the Assessment of Positive Symptoms, we divided the sample into three groups: controls, patients without formal thought disorder, and patients with derailment/tangentiality. Their lists of animals produced during a one-minute semantic fluency task were processed using CoVec, a newly developed software that measures the semantic similarity of words based on vector semantic analysis. CoVec outputs were Mean Similarity, Coherence, Coherence-5, and Coherence-10. Patients with schizophrenia produced fewer words than controls. Patients with derailment had a significantly lower mean number of words and lower Coherence-5 than controls and patients without derailment. Patients with tangentiality had significantly lower Coherence-5 and Coherence-10 than controls and patients without tangentiality. Despite the small samples of patients with clinically apparent thought disorder, CoVec was able to detect subtle differences between controls and patients with either or both of the two forms of disorganization.

On the Suitability of Applying WordNet to Privacy Measurement

Privacy protection is a fundamental issue in the era of big data. For personalized privacy protection, it is necessary to measure the amount or the degree of privacy leakage. To facilitate such measurement, semantic similarities and relationships of words should be determined since the words may come from multiple sources and present themselves in as many different ways as one can imagine, an intrinsic nature of big data. WordNet has been widely used for measuring the semantic similarity of words. This paper aims at analyzing the suitability of applying WordNet to measuring the semantic similarity or relatedness of words in the field of privacy. The analysis includes an experiment designed to obtain human rating scores as the benchmark dataset and a comprehensive comparison between results from four WordNet based measures and the human rating scores. The conclusion of the analysis is that current WordNet based measures are not very suitable for privacy measurement. Therefore, this paper also provides some suggestions on possible ways of enhancing WordNet to improve the effectiveness of semantic similarity or relatedness measurement in the field of privacy to support more effective and personalized privacy protection.

Word Sense Disambiguation for Arabic Exploiting Arabic WordNet and Word Embedding

Word Sense Disambiguation (WSD) is a task which aims to identify the meaning of a word given its context. This problem has been investigated and analyzed in depth in English. However, work in Arabic has been limited despite the fact that there are half a billion native Arabic speakers. In this work, we present multiple approaches for the problem of WSD in Arabic utilizing recent developments and successes in learning word embeddings with approaches such as GloVe, and Word2vec. The primary shortcoming of word embeddings is the single vector representation of a word’s meaning, although many words are polysemous. Our main contribution in this work is to computationally obtain an embedding for each sense, using an Arabic WordNet (AWN) to overcome the problem of WSD. We also compute word semantic similarity giving thought to multiple Arabic stemming algorithms. Finally, we make available a large pre-processed corpus that is ready to be used for further experiments and a WSD test data based on AWN,1 seeking to fill gaps in Arabic NLP (ANLP) compared to English.

Word predictability and semantic similarity show distinct patterns of brain activity during language comprehension

ABSTRACTWe investigate the effects of two types of relationship between the words of a sentence or text – predictability and semantic similarity – by reanalysing electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) data from studies in which participants comprehend naturalistic stimuli. Each content word's predictability given previous words is quantified by a probabilistic language model, and semantic similarity to previous words is quantified by a distributional semantics model. Brain activity time-locked to each word is regressed on the two model-derived measures. Results show that predictability and semantic similarity have near identical N400 effects but are dissociated in the fMRI data, with word predictability related to activity in, among others, the visual word-form area, and semantic similarity related to activity in areas associated with the semantic network. This indicates that both predictability and similarity play a role during natural language comprehension and modulate distinct cortical regions.

Multilingual semantic dictionaries for natural language processing: The case of BabelNet

Accurate semantic modeling lies at the very core of today’s Natural Language Processing (NLP). Getting a handle on the various phenomena that regulate the meaning of linguistic utterances can pave the way for solving many compelling and ambitious tasks in the field, from Machine Translation to Question Answering and Information Retrieval. A complete semantic model of language, however, needs first of all reliable building blocks. In the last two decades, research in lexical semantics (which focuses on the meaning of individual linguistic elements, i.e., words and expressions), has produced increasingly comprehensive and effective machine-readable dictionaries in multiple languages: like humans, NLP systems can now leverage these sources of lexical knowledge to discriminate among various senses of a given lexeme, thereby improving their performances on downstream tasks and applications. In this paper, we focus on the case study of BabelNet, a large multilingual encyclopedic dictionary and semantic network, to describe in detail how such knowledge resources are built, improved and exploited for crucial NLP tasks such as Word Sense Disambiguation, Entity Linking and Semantic Similarity.

Extracting Personae Interactive Relation in Chinese Microblog Based on an Improved Dependency Trigram Kernel

As the rapid development of social media, microblog has arrested a lot of attention. However, the automatic information extraction task of the microblog is relatively rare because the microblog text is quite complex and irregular. In this paper, we utilized dependency trigram kernel to construct persons relations extraction model. Firstly, we described the dependency trigram kernel (DTK) for relations extraction. Secondly, we used words semantic similarity tool HowNet to improve semantic similarity of dependency trigram's words. And then we proposed "(POS,GR) (Part of Speech,Grammatical role)" pair to improve dependency trigram's words syntax similarity. Finally, we evaluated the validity of the relation extraction model by experimenting, and the results of experiment show that the F-value of our improved DTK is higher than original DTK for microblog persons relation extraction.

The Research of Chinese Words Semantic Similarity Calculation with Multi-Information

Text similarity has a relatively wide range of applications in many fields, such as intelligent information retrieval, question answering system, text rechecking, machine translation, and so on. The text similarity computing based on the meaning has been used more widely in the similarity computing of the words and phrase. Using the knowledge structure of the and its method of knowledge description, taking into account the other factor and weight that influenced similarity, making full use of depth and density of the Concept-Sememe tree, an improved method of Chinese word similarity calculation based on semantic distance was provided in this paper. Finally the effectiveness of this method was verified by the simulation results.

Document Classification Using N-gram and Word Semantic Similarity

Document Classification Using N-gram and Word Semantic Similarity

On retrieving intelligently plagiarized documents using semantic similarity

Plagiarism in text documents can be done in many ways. The most common form of plagiarizing a text document is to copy a chunk of text and alter it intelligently, thereby making it look original. Such cases are hard to detect since they require semantic analysis of the document. External sources of knowledge such as WordNet have been employed to help detect such cases. However, such an approach might often miss the contextual significance of the employed words, as well as suffer from the issue of synonymy and polysemy. We propose an architecture that uses a semantic similarity measure that exploits the semantic similarity of words, as mined from within the data corpus, thereby using localized contextual information. In this work, an approach for detecting plagiarism in text document has been proposed using a semantic similarity measure with a Nearest Neighbor (NN) search, and using a kernel in multiclass support vector machine. We test our approach on a plagiarism dataset specially developed to test the efficacy of the solution with varying level of plagiarism. The results have been compared with that of well-known commercial software, Turnitin®, having access to a large database. Our experiments suggest that using semantic kernels can help detect plagiarism, which can outsmart available techniques.