Semantic Relatedness Of Words Research Articles

Harmonic memory signals in the human cerebral cortex induced by semantic relatedness of words

When we memorize multiple words simultaneously, semantic relatedness among those words assists memory. For example, the information about “apple”, “banana,” and “orange” will be connected via a common concept of “fruits” and become easy to retain and recall. Neural mechanisms underlying this semantic integration in verbal working memory remain unclear. Here I used electroencephalography (EEG) and investigated neural signals when healthy human participants memorized five nouns semantically related (Sem trial) or not (NonSem trial). The regularity of oscillatory signals (8–30 Hz) during the retention period was found to be lower in NonSem than Sem trials, indicating that memorizing words unrelated to each other induced a non-harmonic (irregular) waveform in the temporal cortex. These results suggest that (i) semantic features of a word are retained as a set of neural oscillations at specific frequencies and (ii) memorizing words sharing a common semantic feature produces harmonic brain responses through a resonance or integration (sharing) of the oscillatory signals.

SEWAR: A corpus-based N-gram approach for extracting semantically-related words from Arabic medical corpus

Automatic aggregation of similar words into semantically related groups (or clusters) is of interest to many natural language processing (NLP) applications. Extracting semantically related words and quasi-synonyms from text is a relatively new research area for the under-resourced Arabic language. Previous attempts addressed the problem of single-word term extraction. However, the absence of multiword terms (MWTs) dictionary, ontology, and semantic network makes extracting and identifying high-quality MWTs for the Arabic language a challenging problem. The main goal of this study is to extract corpus-based and coherent MWTs in the form of bigram and trigram sequences as an adequate representation of syntactic and semantic clusters. Therefore, this study contributes to this problem by implementing an algorithm named SEWAR, which uses the FastText algorithm to extract high-quality MWTs from a medical health corpus in Arabic. If put into practice, SEWAR can provide a suitable and helpful solution to classify and direct medical health questions to proper medical practitioners without human intervention. In addition, SEWAR can be applied to plenty of NLP tasks, such as information retrieval, question answering, and text summarization. Three metrics were used to assess the extracted MWTs; the pointwise mutual information (PMI), the cosine similarity, and the clustering purity measure. The results were promising and encouraging to generalize and apply SEWAR to extract MWTs from any Arabic corpus.

한글 단어의 관련성 유형에 따른 성인의 의미 규준 조사

Objectives The purpose of this study were to explore the semantic norm of Hangul words based on type of semantic relationship. We investigated various words that exhibited diverse types of semantic relationships with a single word. Methods We conducted a free association task and controlled association task on 180 Hangul words with 150 adults over the age of 20 who were native Korean speakers. In the free association task, participants were asked to freely report words that came to mind when presented with a given word, while in the controlled association task, they were asked to report words related to the given word semantically, categorically, or functionally. Results The words reported in the free association task were found to overlap with those reported in the controlled association task for the same given word in approximately 74% of cases. Specifically, categorical related words were the most common, accounting for 52%. Subsequently, functional-related words, semantic-related words, and pure associative words constituted the majority of the sample, in descending order. In the free association task, it was found that higher word frequency was associated with a greater number of words generated. However, in the controlled association task, such a correlation was not observed. Conclusions While some cases existed where words related by association differed in their semantic, categorical, or functional relationships, the majority of these words were common. Thus, when designing learning materials, it is crucial to take these factors into account, as words with semantic relationships are likely to be retrieved together.

Word embedding dimensionality reduction using dynamic variance thresholding (DyVaT)

Natural language processing (NLP) provides a framework for large-scale text analysis. One common processing method uses vector space models (VSMs) which embed word attributes, called features, into highly dimensional vectors. Comprehensive VSMs are generated on sources such as the GoogleNews archive. A thesaurus, a collection of semantically-related words, can be created for a particular root word using cosine similarity with a given VSM. Many methods have been developed to reduce the complexity of these models by maintaining useful semantic information while discarding non-informative features. One such method, variance thresholding, retains high-variance features above a manually-determined threshold, providing higher differentiation between words for classification purposes. Our research developed a dimension-reducing methodology called dynamic variance thresholding (DyVaT). DyVaT reduces the specificity of word embeddings by maintaining low-variance features, allowing for a broader thesaurus preserving semantic similarity. A dynamic variance threshold, determining which low-variance features are retained, is selected using the kneedle algorithm, improving the current results. Our test case for examining the efficiency of DyVat in creating a contextual thesaurus is the visual, auditory and kinesthetic learning style context. We conclude that DyVaT is a valid method for generating loosely-connected word collections with potential uses in NLP classification or clustering tasks.

Constricted semantic relations in acute depression

BackgroundIt has been suggested that mood influences the breadth of associated information available for retrieval, with positive mood broadening and negative mood constricting the scope of associations. In this study, we asked whether this mood-associations connection is related to controlled processes which were linked to clinical symptoms in depression. MethodsWe used the semantic priming paradigm, which allows the dissociation of automatic and controlled processes by using short and long intervals between prime and target words. We further examined whether the strength of semantic relations (weak or strong) influence the priming effects in both neurotypical and depressed individuals. ResultsExperiment 1, testing neurotypical individuals, showed priming effects for strong semantically-related words regardless of interval length, but priming effects for weak semantically-related words were smaller in short intervals than in long intervals. Experiment 2, testing depressed individuals in long intervals, showed smaller priming effects for weak semantically-related words than shown by neurotypicals, but priming effects for strong semantically-related words which were comparable between the groups. LimitationsThis study cannot determine the source for the differences in priming effects between depressed individuals and neurotypicals, and further studies are needed. ConclusionsThis is the first study to show priming impairments in depressed individuals. We discuss our results in light of leading theories concerning cognitive impairment in depression, as well as the newly emerged field of digital psychiatry.

Prediction in challenging situations: Most bilinguals can predict upcoming semantically-related words in their L1 source language when interpreting

AbstractPrediction is an important part of language processing. An open question is to what extent people predict language in challenging circumstances. Here we tested the limits of prediction by asking bilingual Dutch native speakers to interpret Dutch sentences into their English counterparts. In two visual world experiments, we recorded participants’ eye movements to co-present visual objects while they engaged in interpreting tasks (consecutive and simultaneous interpreting). Most participants showed anticipatory eye movements to semantically-related upcoming target words in their L1 source language during both consecutive and simultaneous interpretation. A quarter of participants during simultaneous interpretation however did not move their eyes, an extremely unusual participant behaviour in visual world studies. Overall, the findings suggest that most people predict in the source language under challenging interpreting situations. Further work is required to understand the causes of the absence of (anticipatory) eye movements during simultaneous interpretation in a substantial subset of individuals.

Beyond the Failure of Direct-Matching in Keyword Evaluation: A Sketch of a Graph Based Solution.

The starting point of this paper is the observation that methods based on the direct match of keywords are inadequate because they do not consider the cognitive ability of concept formation and abstraction. We argue that keyword evaluation needs to be based on a semantic model of language capturing the semantic relatedness of words to satisfy the claim of the human-like ability of concept formation and abstraction and achieve better evaluation results. Evaluation of keywords is difficult since semantic informedness is required for this purpose. This model must be capable of identifying semantic relationships such as synonymy, hypernymy, hyponymy, and location-based abstraction. For example, when gathering texts from online sources, one usually finds a few keywords with each text. Still, these keyword sets are neither complete for the text nor are they in themselves closed, i.e., in most cases, the keywords are a random subset of all possible keywords and not that informative w.r.t. the complete keyword set. Therefore all algorithms based on this cannot achieve good evaluation results and provide good/better keywords or even a complete keyword set for a text. As a solution, we propose a word graph that captures all these semantic relationships for a given language. The problem with the hyponym/hyperonym relationship is that, unlike synonyms, it is not bidirectional. Thus the space of keyword sets requires a metric that is non-symmetric, in other words, a quasi-metric. We sketch such a metric that works on our graph. Since it is nearly impossible to obtain such a complete word graph for a language, we propose for the keyword task a simpler graph based on the base text upon which the keyword sets should be evaluated. This reduction is usually sufficient for evaluating keyword sets.

Language dialect based speech emotion recognition through deep learning techniques

The primordial way of communication is through vocal signals, which pave the way for support between individuals in a social structure. Computer applications provide a way to create Automatic Speech Recognition (ASR) with a combination of Speech Emotion Recognition (SER) to detect and identify emotions in the speech signals. The semantic relatedness of words with abstract concepts proves to be complicated than concrete ideas. An ensemble of different clustering techniques is utilized to automatically segregate sense distinctions in the various dialects of sentences spoken to tackle this issue. The interpretation of word sense of a word may change with time and group of people. The proposed model maps characters to word sense with weights provided by Senticnet with trial-and-error methods and tuning. The proposed model utilizes stop words to distinguish word senses with 72.78% accuracy for regional dialects.

An ERP Study on the Role of Phonological Processing in Reading Two-Character Compound Chinese Words of High and Low Frequency.

Unlike in English, the role of phonology in word recognition in Chinese is unclear. In this event-related potential experiment, we investigated the role of phonology in reading both high- and low-frequency two-character compound Chinese words. Participants executed semantic and homophone judgment tasks of the same precede-target pairs. Each pair of either high- or low-frequency words were either unrelated (control condition) or related semantically or phonologically (homophones). The induced P200 component was greater for low- than for high-frequency word-pairs both in semantic and phonological tasks. Homophones in the semantic judgment task and semantically-related words in the phonology task both elicited a smaller N400 than the control condition, word frequency-independently. However, for low-frequency words in the phonological judgment task, it was found that the semantically related pairs released a significantly larger P200 than the control condition. Thus, the semantic activation of both high- and low-frequency words may be no later than phonological activation.

Topic discovery by spectral decomposition and clustering with coordinated global and local contexts

Topic modeling is an active research field due to its broad applications such as information retrieval, opinion extraction and authorship identification. It aims to discover topic structures from a collection of documents. Significant progress have been made by the latent dirichlet allocation (LDA) and its variants. However, the “bag-of-words” assumption is usually made for the whole document by conventional methods, which ignores the semantics of local context that play crucial roles in topic modeling and document understanding. In this paper, we propose a novel coordinated embedding topic model (CETM), which incorporates spectral decomposition and clustering technique by leveraging both global and local context information to discover topics. In particular, CETM learns coordinated embeddings by using spectral decomposition, capturing the word semantic relations effectively. To infer the topic distribution, we employ a clustering algorithm to capture semantic centroids of coordinated embeddings and derive a fast algorithm to obtain the topic structures. We conduct extensive experiments on three real-world datasets to evaluate the effectiveness of CETM. Quantitatively, compared to state-of-the-art topic modeling approaches, CETM achieves significantly better performance in terms of topic coherence and text classification. Qualitatively, CETM is able to learn more coherent topics and more accurate word distributions for each topic.

The language of well-being: Tracking fluctuations in emotion experience through everyday speech.

The words that people use have been found to reflect stable psychological traits, but less is known about the extent to which everyday fluctuations in spoken language reflect transient psychological states. We explored within-person associations between spoken words and self-reported state emotion among 185 participants who wore the Electronically Activated Recorder (EAR; an unobtrusive audio recording device) and completed experience sampling reports of their positive and negative emotions 4 times per day for 7 days (1,579 observations). We examined language using the Linguistic Inquiry and Word Count program (LIWC; theoretically created dictionaries) and open-vocabulary themes (clusters of data-driven semantically-related words). Although some studies give the impression that LIWC's positive and negative emotion dictionaries can be used as indicators of emotion experience, we found that when computed on spoken language, LIWC emotion scores were not significantly associated with self-reports of state emotion experience. Exploration of other categories of language variables suggests a number of hypotheses about substantive everyday correlates of momentary positive and negative emotion that can be tested in future studies. These findings (a) suggest that LIWC positive and negative emotion dictionaries may not capture self-reported subjective emotion experience when applied to everyday speech, (b) emphasize the importance of establishing the validity of language-based measures within one's target domain, (c) demonstrate the potential for developing new hypotheses about personality processes from the open-ended words that are used in everyday speech, and (d) extend perspectives on intraindividual variability to the domain of spoken language. (PsycINFO Database Record (c) 2020 APA, all rights reserved).

Wasf-Vec

Word clustering is a serious challenge in low-resource languages. Since words that share semantics are expected to be clustered together, it is common to use a feature vector representation generated from a distributional theory-based word embedding method. The goal of this work is to utilize Modern Standard Arabic (MSA) for better clustering performance of the low-resource Iraqi vocabulary. We began with a new Dialect Fast Stemming Algorithm (DFSA) that utilizes the MSA data. The proposed algorithm achieved 0.85 accuracy measured by the F1 score. Then, the distributional theory-based word embedding method and a new simple, yet effective, feature vector named Wasf-Vec word embedding are tested. Wasf-Vec word representation utilizes a word’s topology features. The difference between Wasf-Vec and distributional theory-based word embedding is that Wasf-Vec captures relations that are not contextually based. The embedding is followed by an analysis of how the dialect words are clustered within other MSA words. The analysis is based on the word semantic relations that are well supported by solid linguistic theories to shed light on the strong and weak word relation representations identified by each embedding method. The analysis is handled by visualizing the feature vector in two-dimensional (2D) space. The feature vectors of the distributional theory-based word embedding method are plotted in 2D space using the t-sne algorithm, while the Wasf-Vec feature vectors are plotted directly in 2D space. A word’s nearest neighbors and the distance-histograms of the plotted words are examined. For validation purpose of the word classification used in this article, the produced classes are employed in Class-based Language Modeling (CBLM). Wasf-Vec CBLM achieved a 7% lower perplexity (pp) than the distributional theory-based word embedding method CBLM. This result is significant when working with low-resource languages.

God-like robots: the semantic overlap between representation of divine and artificial entities

Artificial intelligence and robots may progressively take a more and more prominent place in our daily environment. Interestingly, in the study of how humans perceive these artificial entities, science has mainly taken an anthropocentric perspective (i.e., how distant from humans are these agents). Considering people’s fears and expectations from robots and artificial intelligence, they tend to be simultaneously afraid and allured to them, much as they would be to the conceptualisations related to the divine entities (e.g., gods). In two experiments, we investigated the proximity of representation between artificial entities (i.e., artificial intelligence and robots), divine entities and natural entities (i.e., humans and other animals) at both an explicit (Study 1) and an implicit level (Study 2). In the first study, participants evaluated these entities explicitly on positive and negative attitudes. Hierarchical clustering analysis showed that participants’ representation of artificial intelligence, robots and divine entities were similar, while the representation of humans tended to be associated with that of animals. In the second study, participants carried out a word/non-word decision task including religious semantic-related words and neutral words after the presentation of a masked prime referring to divine entities, artificial entities and natural entities (or a control prime). Results showed that after divine and artificial entity primes, participants were faster to identify religious words as words compared to neutral words arguing for a semantic activation. We conclude that people make sense of the new entities by relying on already familiar entities and in the case of artificial intelligence and robots, people appear to draw parallels to divine entities.

Word Formation Strategies and Semantic Relations in Football Discourse

This paper explores the word formation processes and semantic relations in football discourse. It aims to investigate the type of word formation and semantic relations of words that are associated with the written football discourse. Knowledge of word formation with semantic relations of words is very relevant in enriching or expanding vocabulary power in English. It facilitates the learning of English words and aids effective communication. Application of word formation knowledge can solve the problem of inadequate or limited vocabulary power among the learners and users of English language in a second language situation. English as Second Language (ESL) learners cannot acquire vocabulary and affixes knowledge in the classroom alone, adequate attention need to be paid to using materials from the media, that is both print and electronic media. This will assist in improving their vocabulary development. There are different types of word formation strategies in English namely: blending, clipping, back formation, coinage, reduplicatives, borrowing, compounding, acronyms and abbronyms. At the lexico-semantic level, we have synonyms, antonyms and lexical repetition. Keywords : word formation, vocabulary, media, affixation, abbronyms, synonyms, discourse DOI : 10.7176/RHSS/9-8-07 Publication date : April 30 th 2019

On the articulation between acoustic and semantic uncertainty in speech perception: Investigating the interaction between sources of information in perceptual classification.

Listeners processing speech signals have to deal with two main classes of uncertainty occurring in the vicinity of a given speech segment: both acoustic properties of the contextual environment (Ladefoged and Broadbent, 1957; Sjerps and McQueen, 2013) and lexical hypotheses based on word co-occurence probabilities or semantic relations (e.g., Connine, 1987; Gow and Olson, 2015) may affect the interpretation of a given sound. We investigate this issue by independently manipulating (1) semantic relationships between words using word embeddings estimations and (2) acoustic relations between a contextual part and the final word in the sentence. Based on word pairs that contrast on their vowel target only (e.g., french “balle” versus “belle”, pronounced /bal/ vs. /bEl/ -- eng. “ball” versus “beauty”), 3 types of sentences are generated: (1) a sentence that would semantically “prime” the word /bal/ (“Le joueur a dévié la”, eng. “The player deflected the”), (2) a sentence that would favour the word /bEl/ “Le prince a charmé la”, eng. “The prince charmed the”), and (3) a semantically incongruous sentence in both cases “Le journaliste a découvert la”, eng. “The journalist discovered the.” Listeners are presented with fully ambiguous final words (acoustically located between, e.g., /bal/ and /bEl/) in contexts where semantic influence varies (sentence-types 1/2/3) and is balanced with acoustic manipulations of formant frequencies favouring one word or the other. This will provide cues to modelling how both sources of entropy alter speech perception.

Using multiple Web resources and inference rules to classify Chinese word semantic relation

PurposeThe purpose of this paper is to classify Chinese word semantic relations, which are synonyms, antonyms, hyponyms and meronymys.Design/methodology/approachBasically, four simple methods are applied, ontology-based, dictionary-based, pattern-based and morpho-syntactic method. The authors make good use of search engine to build lexical and semantic resources for dictionary-based and pattern-based methods. To improve classification performance with more external resources, they also classify the given word pairs in Chinese and in English at the same time by using machine translation.FindingsExperimental results show that the approach achieved an average F1 score of 50.87 per cent, an average accuracy of 70.36 per cent and an average recall of 40.05 per cent over all classification tasks. Synonym and antonym classification achieved high accuracy, i.e. above 90 per cent. Moreover, dictionary-based and pattern-based approaches work effectively on final data set.Originality/valueFor many natural language processing (NLP) tasks, the step of distinguishing word semantic relation can help to improve system performance, such as information extraction and knowledge graph generation. Currently, common methods for this task rely on large corpora for training or dictionaries and thesauri for inference, where limitation lies in freely data access and keeping built lexical resources up-date. This paper builds a primary system for classifying Chinese word semantic relations by seeking new ways to obtain the external resources efficiently.

Enhancing Topic Modeling for Short Texts with Auxiliary Word Embeddings

Many applications require semantic understanding of short texts, and inferring discriminative and coherent latent topics is a critical and fundamental task in these applications. Conventional topic models largely rely on word co-occurrences to derive topics from a collection of documents. However, due to the length of each document, short texts are much more sparse in terms of word co-occurrences. Recent studies show that the Dirichlet Multinomial Mixture (DMM) model is effective for topic inference over short texts by assuming that each piece of short text is generated by a single topic. However, DMM has two main limitations. First, even though it seems reasonable to assume that each short text has only one topic because of its shortness, the definition of “shortness” is subjective and the length of the short texts is dataset dependent. That is, the single-topic assumption may be too strong for some datasets. To address this limitation, we propose to model the topic number as a Poisson distribution, allowing each short text to be associated with a small number of topics (e.g., one to three topics). This model is named PDMM. Second, DMM (and also PDMM) does not have access to background knowledge (e.g., semantic relations between words) when modeling short texts. When a human being interprets a piece of short text, the understanding is not solely based on its content words, but also their semantic relations. Recent advances in word embeddings offer effective learning of word semantic relations from a large corpus. Such auxiliary word embeddings enable us to address the second limitation. To this end, we propose to promote the semantically related words under the same topic during the sampling process, by using the generalized Pólya urn (GPU) model. Through the GPU model, background knowledge about word semantic relations learned from millions of external documents can be easily exploited to improve topic modeling for short texts. By directly extending the PDMM model with the GPU model, we propose two more effective topic models for short texts, named GPU-DMM and GPU-PDMM. Through extensive experiments on two real-world short text collections in two languages, we demonstrate that PDMM achieves better topic representations than state-of-the-art models, measured by topic coherence. The learned topic representation leads to better accuracy in a text classification task, as an indirect evaluation. Both GPU-DMM and GPU-PDMM further improve topic coherence and text classification accuracy. GPU-PDMM outperforms GPU-DMM at the price of higher computational costs.

Topic segmentation using word-level semantic relatedness functions

Semantic relatedness deals with the problem of measuring how much two words are related to each other. While there is a large body of research for developing new measures, the use of semantic relatedness (SR) measures in topic segmentation has not been explored. In this research the performance of different SR measures is evaluated in the topic segmentation problem. To this end, two topic segmentation algorithms that use the difference in SR of words are introduced. Our results indicate that using an SR measure trained with a general domain corpora achieves better results than topic segmentation algorithms using Wordnet or simple word repetition. Furthermore, when compared with computationally more complex algorithms performing global analysis, our local analysis, enhanced with general domain lexical semantic information, achieves comparable results.

Using Hashtag Graph-Based Topic Model to Connect Semantically-Related Words Without Co-Occurrence in Microblogs

In this paper, we introduce a new topic model to understand the chaotic microblogging environment by using hashtag graphs. Inferring topics on Twitter becomes a vital but challenging task in many important applications. The shortness and informality of tweets leads to extreme sparse vector representations with a large vocabulary. This makes the conventional topic models (e.g., latent Dirichlet allocation [1] and latent semantic analysis [2] ) fail to learn high quality topic structures. Tweets are always showing up with rich user-generated hashtags. The hashtags make tweets semi-structured inside and semantically related to each other. Since hashtags are utilized as keywords in tweets to mark messages or to form conversations, they provide an additional path to connect semantically related words. In this paper, treating tweets as semi-structured texts, we propose a novel topic model, denoted as Hashtag Graph-based Topic Model ( HGTM ) to discover topics of tweets. By utilizing hashtag relation information in hashtag graphs, HGTM is able to discover word semantic relations even if words are not co-occurred within a specific tweet. With this method, HGTM successfully alleviates the sparsity problem. Our investigation illustrates that the user-contributed hashtags could serve as weakly-supervised information for topic modeling, and the relation between hashtags could reveal latent semantic relation between words. We evaluate the effectiveness of HGTM on tweet (hashtag) clustering and hashtag classification problems. Experiments on two real-world tweet data sets show that HGTM has strong capability to handle sparseness and noise problem in tweets. Furthermore, HGTM can discover more distinct and coherent topics than the state-of-the-art baselines.

Effects of the Relative Word Frequency and Spelling-to-Sound Correspondency of the Heterographic Homophone Priming on Semantically Related Word Recognition

The aim of this study was to examine the effects of relative word frequency and the spelling-to-sound correspondence of heregoraphic homophone priming on the recognition of semanticallyrelated words. A cross-modal semantic paradigm has been applied, where homophone primes were presented auditorily and targets were presented visually. The difference in lexical decision latencies of the visual targets was calculated between semantically-related prime and semantically-unrelated prime to measure the size of facilitation. A facilitatory effect was shown only when the spelling corresponds to sound, and no such effect was observed for the relative word frequency condition. These findings indicated that an auditorily presented homophone activates all of its corresponding orthographic words and that the spelling-to-sound correspondence is more influential than the relative word frequency for homophone recognition.