Hyperspace Analogue To Language Research Articles

A quantum-like semantic model for text retrieval in Arabic

The subject of study. The paper focuses on the extraction of semantics from texts in Arabic. In particular, the applicability of the Bell test to word pairs is investigated as a measure of the semantic words relatedness in a context. The study applies the quantum formalism to the task of information retrieval in Arabic texts and presents the results of this work. The authors also examine the influence of the context width on the effectiveness of information retrieval. Method. The research is based on the vector representation of the context. It uses the well-known approach based on the HAL (Hyperspace Analogue to Language) matrix and Bell test. The HAL matrix allows taking into account both the frequency of the words occurrence in the context and the distance to the target word. Quantum theory operates with probability density matrices. Quantum theory allows describing probabilities in the vector space in a more natural way i.e., words can be represented as vectors. Main results. The results demonstrate that using the Bell’s test for texts in Arabic provides a better ranking of search results compared to the results of search services. Practical significance. The research results can be used in the development of the information retrieval systems, as well as for the further development of methods based on the distributive hypothesis.

Applying the Bell's Test to Chinese Texts.

Search engines are able to find documents containing patterns from a query. This approach can be used for alphabetic languages such as English. However, Chinese is highly dependent on context. The significant problem of Chinese text processing is the missing blanks between words, so it is necessary to segment the text to words before any other action. Algorithms for Chinese text segmentation should consider context; that is, the word segmentation process depends on other ideograms. As the existing segmentation algorithms are imperfect, we have considered an approach to build the context from all possible n-grams surrounding the query words. This paper proposes a quantum-inspired approach to rank Chinese text documents by their relevancy to the query. Particularly, this approach uses Bell’s test, which measures the quantum entanglement of two words within the context. The contexts of words are built using the hyperspace analogue to language (HAL) algorithm. Experiments fulfilled in three domains demonstrated that the proposed approach provides acceptable results.

Vector-Space Models of Semantic Representation From a Cognitive Perspective: A Discussion of Common Misconceptions

Models that represent meaning as high-dimensional numerical vectors—such as latent semantic analysis (LSA), hyperspace analogue to language (HAL), bound encoding of the aggregate language environment (BEAGLE), topic models, global vectors (GloVe), and word2vec—have been introduced as extremely powerful machine-learning proxies for human semantic representations and have seen an explosive rise in popularity over the past 2 decades. However, despite their considerable advancements and spread in the cognitive sciences, one can observe problems associated with the adequate presentation and understanding of some of their features. Indeed, when these models are examined from a cognitive perspective, a number of unfounded arguments tend to appear in the psychological literature. In this article, we review the most common of these arguments and discuss (a) what exactly these models represent at the implementational level and their plausibility as a cognitive theory, (b) how they deal with various aspects of meaning such as polysemy or compositionality, and (c) how they relate to the debate on embodied and grounded cognition. We identify common misconceptions that arise as a result of incomplete descriptions, outdated arguments, and unclear distinctions between theory and implementation of the models. We clarify and amend these points to provide a theoretical basis for future research and discussions on vector models of semantic representation.

Using an analogical reasoning framework to infer language patterns for negative life events

BackgroundFeelings of depression can be caused by negative life events (NLE) such as the death of a family member, a quarrel with one’s spouse, job loss, or strong criticism from an authority figure. The automatic and accurate identification of negative life event language patterns (NLE-LP) can help identify individuals potentially in need of psychiatric services. An NLE-LP combines a person (subject) and a reasonable negative life event (action), e.g. <parent:divorce> or < boyfriend:break_up>.MethodsThis paper proposes an analogical reasoning framework which combines a word representation approach and a pattern inference method to mine/extract NLE-LPs from psychiatric consultation documents. Word representation approaches such as skip-gram (SG) and continuous bag-of-words (CBOW) are used to generate word embeddings. Pattern inference methods such as cosine similarity (COSINE) and cosine multiplication similarity (COSMUL) are used to infer patterns.ResultsExperimental results show our proposed analogical reasoning framework outperforms the traditional methods such as positive pairwise mutual information (PPMI) and hyperspace analog to language (HAL), and can effectively mine highly precise NLE-LPs based on word embeddings. CBOW with COSINE of analogical reasoning is the best word representation and inference engine. In addition, both word embeddings and the inference engine provided by the analogical reasoning framework can further be used to improve the HAL model.ConclusionsOur proposed framework is a very simple matching function based on these word representation approaches and is applied to significantly improve HAL model mining performance.

Incorporating noun compounds in distributional-based semantic representation approaches for measuring semantic relatedness

Identifying noun compounds in natural language documents is very important for handling their various linguistic features, such as semantic, syntactic, and pragmatic features. In this study, we introduce a knowledge-based method for incorporating noun compounds in distributional-based semantic representation approaches. Wikipedia is exploited as a knowledge resource for extracting noun compounds based on its structural features. The categories are then used to classify the extracted noun compounds as linguistic terms and named entities. Next, the look-up list technique is employed to identify the noun compounds when extracting the semantics of the terms using the corpus-based approach for semantic representation. To obtain the semantic representation, we use five well-known distributional-based approaches: latent semantic analysis (LSA), hyperspace analogue to language (HAL), correlated occurrence analogue to lexical semantic (COALS), bound encoding of the aggregate language environment (BEAGLE), and explicit semantic analysis (ESA). The proposed method was evaluated by measuring the semantic relatedness using five benchmark datasets employed in previous studies. The experimental results demonstrate that incorporating noun compounds in the distributional-based semantic representation helps to improve the semantic evidence for the relationships among words.

Named Entity System for Tweets in Hindi Language

Due to the growing need of smart-health applications in Hindi language, there is a rapid demand for health-related Named Entity Recognition (NER) system for Hindi. For the purpose of the same, this research considers Twitter social network to extract tweets dated 1st October 2016 to 15th October 2017 from Patanjali, Dabur and other Hindi language-oriented Twitter based health sites; while considering four NE types- Person, Disease, Consumable and Organization. To the best of its knowledge, the considered Twitter dataset and NE types for Hindi language is one of the first resources that is being taken care. This article introduces three stage NER system for Tweets in Hindi language (HinTwtNER system)- pre-processing stage; machine Learning stage (Hyperspace Analogue to Language (HAL) and Conditional Random Field (CRF)); and post-processing stage. HinTwtNER looks into binary features and achieves an overall F-score of 49.87% which is comparable to the Twitter based NER systems for English and other languages.

Predicting Lexical Priming Effects from Distributional Semantic Similarities: A Replication with Extension.

In two experiments, we attempted to replicate and extend findings by Günther et al. (2016) that word similarity measures obtained from distributional semantics models—Latent Semantic Analysis (LSA) and Hyperspace Analog to Language (HAL)—predict lexical priming effects. To this end, we used the pseudo-random method to generate item material while systematically controlling for word similarities introduced by Günther et al. (2016) which was based on LSA cosine similarities (Experiment 1) and HAL cosine similarities (Experiment 2). Extending the original study, we used semantic spaces created from far larger corpora, and implemented several additional methodological improvements. In Experiment 1, we only found a significant effect of HAL cosines on lexical decision times, while we found significant effects for both LSA and HAL cosines in Experiment 2. As further supported by an analysis of the pooled data from both experiments, this indicates that HAL cosines are a better predictor of priming effects than LSA cosines. Taken together, the results replicate the finding that priming effects can be predicted from distributional semantic similarity measures.

How useful are corpus-based methods for extrapolating psycholinguistic variables?

Subjective ratings for age of acquisition, concreteness, affective valence, and many other variables are an important element of psycholinguistic research. However, even for well-studied languages, ratings usually cover just a small part of the vocabulary. A possible solution involves using corpora to build a semantic similarity space and to apply machine learning techniques to extrapolate existing ratings to previously unrated words. We conduct a systematic comparison of two extrapolation techniques: k-nearest neighbours, and random forest, in combination with semantic spaces built using latent semantic analysis, topic model, a hyperspace analogue to language (HAL)-like model, and a skip-gram model. A variant of the k-nearest neighbours method used with skip-gram word vectors gives the most accurate predictions but the random forest method has an advantage of being able to easily incorporate additional predictors. We evaluate the usefulness of the methods by exploring how much of the human performance in a lexical decision task can be explained by extrapolated ratings for age of acquisition and how precisely we can assign words to discrete categories based on extrapolated ratings. We find that at least some of the extrapolation methods may introduce artefacts to the data and produce results that could lead to different conclusions that would be reached based on the human ratings. From a practical point of view, the usefulness of ratings extrapolated with the described methods may be limited.

English word frequency and recognition in bilinguals: Inter-corpus comparison and error analysis

Objective: This study is the second of a two-part investigation on lexical effects on bilinguals’ performance on a clinical English word recognition test. Focus is on word-frequency effects using counts provided by four corpora. Design: Frequency of occurrence was obtained for 200 NU-6 words from the Hoosier mental lexicon (HML) and three contemporary corpora, American National Corpora, Hyperspace analogue to language (HAL), and SUBTLEXUS. Correlation analysis was performed between word frequency and error rate. Study sample: Ten monolinguals and 30 bilinguals participated. Bilinguals were further grouped according to their age of English acquisition and length of schooling/working in English. Results: Word frequency significantly affected word recognition in bilinguals who acquired English late and had limited schooling/working in English. When making errors, bilinguals tended to replace the target word with a word of a higher frequency. Overall, the newer corpora outperformed the HML in predicting error rate. Conclusions: Frequency counts provided by contemporary corpora predict bilinguals’ recognition of English monosyllabic words. Word frequency also helps explain top replacement words for misrecognized targets. Word-frequency effects are especially prominent for bilinguals foreign born and educated.

Higher-order semantic structures in an African Grey parrot’s vocalizations: evidence from the hyperspace analog to language (HAL) model

Previous research has described the significant role that social interaction plays in both the acquisition and use of speech by parrots. The current study analyzed the speech of one home-raised African Grey parrot (Psittacus erithacus erithacus) across three different social contexts: owner interacting with parrot in the same room, owner and parrot interacting out of view in adjacent rooms, and parrot home alone. The purpose was to determine the extent to which the subject's speech reflected an understanding of the contextual substitutability (e.g., the word street can be substituted in context for the word road) of the vocalizations that comprised the units in her repertoire (i.e., global co-occurrence of repertoire units; Burgess in Behav Res Methods Instrum Comput 30:188-198, 1998; Lund and Burgess in Behav Res Methods Instrum Comput 28:203-208, 1996). This was accomplished via the human language model hyperspace analog to language (HAL). HAL is contextually driven and bootstraps language "rules" from input without human intervention. Because HAL does not require human tutelage, it provided an objective measure to empirically examine the parrot's vocalizations. Results indicated that the subject's vocalization patterns did contain global co-occurrence. The presence of this quality in this nonhuman's speech may be strongly indicative of higher-order cognitive skills.

Extended information inference model for unsupervised categorization of web short texts

Traditional text-processing methods encounter significant performance degradation when they are applied to web short texts, with their inherent characteristics including feature sparseness, lack of sufficient hand-labelled training examples, domain dependence, and asyntactic expression. In this paper we propose a modified information inference model that can mimic human cognitive behaviour to categorize various web short texts in an unsupervised manner. The model is based on the conceptual space theory and hyperspace analogue to language (HAL) model, and it is a novel development in that it combines domain-specific knowledge and universal knowledge via a fusion mechanism for multiple HAL spaces. Moreover, in the realization of conceptual space, a concept is represented geometrically by a two-tuple of property sets, which can effectively improve the representation accuracy of the information contained in combined concepts. Two measurements of the relationship between concepts are used to implement the information inference for web short texts. The experimental evaluation of our model is conducted via three different tasks on web short text categorization, and the results indicate the applicability and usefulness of the proposed method.

Using a Self-Organizing Map (SOM) and the Hyperspace Analog to Language (HAL) Model to Identify Patterns of Syntax and Structure in the Songs of Humpback Whales

Two different fully automated models were used to examine syntax and structure in humpback whalesong. Songs were initially classified via a Self-Organizing Map (SOM), and then examined, via the Hyperspace Analog to Language (HAL) model, for evidence of a type of higher level organization -global co-occurrence - found in human language. HAL was able to identify particular “classes” ofsong units which were used interchangeably to form patterns in the song, not unlike the use of noun verb-direct object in human language, where the noun, verb, or direct object can be any one of many possibilities from that particular class. Further, HAL identified specific patterns unique to the songsand their respective geographical areas. These patterns provide support for the idea that humpback whale songs are unique to specific region and may be transmitted culturally.

Semantic Space models for classification of consumer webpages on metadata attributes

To deal with the quantity and quality issues with online healthcare resources, creating web portals centred on particular health topics and/or communities of users is a strategy to provide access to a reduced corpus of information resources that meet quality and relevance criteria. In this paper we use hyperspace analogue to language (HAL) to model the language use patterns of webpages as Semantic Spaces. We have applied machine learning methods, including support vector machine (SVM), decision forest, and a novel summed similarity measure (SSM) to automatically classify online webpages on their Semantic Space models. We find classification accuracy on metadata attributes to be over 93% for ‘medical’ versus ‘supportive’ perspective, over 92% for disease stage of ‘early’ versus ‘advanced’, and over 90% for author credentials of ‘lay’ versus ‘clinician’ based on webpages of the Breast Cancer Knowledge Online portal. These results indicate that language use patterns can be used to automate such classification with useful levels of accuracy.

Exploring lexical co-occurrence space using HiDEx

Hyperspace analog to language (HAL) is a high-dimensional model of semantic space that uses the global co-occurrence frequency of words in a large corpus of text as the basis for a representation of semantic memory. In the original HAL model, many parameters were set without any a priori rationale. We have created and publicly released a computer application, the High Dimensional Explorer (HiDEx), that makes it possible to systematically alter the values of these parameters to examine their effect on the co-occurrence matrix that instantiates the model. We took an empirical approach to understanding the influence of the parameters on the measures produced by the models, looking at how well matrices derived with different parameters could predict human reaction times in lexical decision and semantic decision tasks. New parameter sets give us measures of semantic density that improve the model's ability to predict behavioral measures. Implications for such models are discussed.

HAL-Based Evolutionary Inference for Pattern Induction From Psychiatry Web Resources

Negative and stressful life events play a significant role in triggering depressive episodes. Psychiatric services that can identify such events efficiently are vital for mental health care and prevention. Meaningful patterns, e.g., <lost, parents>, must be extracted from psychiatric texts before these services can be provided. This study presents an evolutionary text-mining framework capable of inducing variable-length patterns from unannotated psychiatry Web resources. The proposed framework can be divided into two parts: 1) a cognitive motivated model such as hyperspace analog to language (HAL) and 2) an evolutionary inference algorithm (EIA). The HAL model constructs a high-dimensional context space to represent words as well as combinations of words. Based on the HAL model, the EIA bootstraps with a small set of seed patterns, and then iteratively induces additional relevant patterns. To avoid moving in the wrong direction, the EIA further incorporates relevance feedback to guide the induction process. Experimental results indicate that combining the HAL model and relevance feedback enables the EIA to not only induce patterns from the unannotated Web corpora, but also achieve useful results in a reasonable amount of time. The proposed framework thus significantly reduces reliance on annotated corpora.

Generation of Phonetic Units for Mixed-Language Speech Recognition Based on Acoustic and Contextual Analysis

This work presents a novel approach to generating phonetic units in order to recognize mixed-language or multilingual speech. Acoustic and contextual analysis is performed to characterize multilingual phonetic units for phone set creation. Acoustic likelihood is utilized for similarity estimation of phone models. The hyperspace analog to language (HAL) model is adopted for contextual modeling and contextual similarity estimation. A confusion matrix combining acoustic and contextual similarities between every two phonetic units is built for phonetic unit clustering. Multidimensional scaling (MDS) method is applied to the confusion matrix for reducing dimensionality. Experimental results indicate that the created phonetic set provides a compact and robust set that considers acoustic and contextual information for mixed-language or multilingual speech recognition.

Word frequency effects in high-dimensional co-occurrence models: A new approach

The HAL (hyperspace analog to language) model of lexical semantics uses'global word co-occurrence from a large corpus of text to calculate the distance between words in co-occurrence space. We have implemented a system called HiDEx (High Dimensional Explorer) that extends HAL in two ways: It removes unwanted influence of orthographic frequency from the measures of distance, and it finds the number of words within a certain distance of the word of interest (NCount, the number of neighbors). These two changes to the HAL model produce measures of word neighborhood density that are reliably predictive of human lexical decision reaction times.

Effects of chronic non-clinical depression on the use of positive and negative words in language contexts

The purpose of this study was to investigate the effects of chronic non-clinical depression (ChD) on the use of ambiguous and unambiguous emotional words in language contexts. Participants with ChD and normal non-depressed (ND) participants’ responses to positive and negative stimulus words on a free association word test were compared using the Hyperspace Analogue to Language (HAL) model of memory’s context density metric, a measure of the range of contexts in which a word is used in language. ChD participants generated negative words that were used in more restricted contexts than ND participants. However, ND and ChD participants used positive words in a similar range of contexts. The results suggest that people with ChD tend to use negative words that are probably more negative in connotation, yet they are not impaired with respect to using words in contexts where positive words are required.

A computational approach to modeling population differences.

Four experiments were conducted to determine whether the Hyperspace Analogue to Language (HAL) model of semantic memory could differentiate between two different populations. An analysis of the differences in densities (or average distances between word neighbors in semantic space) in HAL matrices--generated from text corpora derived from younger and older adults--confirmed that HAL was able to distinguish between the two age groups. This difference was again detected when structured interview data were used to build the corpora. A third experiment, designed to test the specificity of HAL in detecting differences between groups, did not detect any difference in the densities of the memory representations when older adults generated both the test corpora. The final experiment, conducted on the language of adults with Alzheimer's and normal adults, again demonstrated that HAL could discriminate between the two populations. These results suggest that HAL is capable of modeling, on the basis of changes in mean density, some of the differences between populations without modifying the model itself but, rather, by changing the text corpus from which the model creates its representations in semantic space.

From simple associations to the building blocks of language: Modeling meaning in memory with the HAL model

This paper presents a theoretical approach of how simple, episodic associations are transduced into semantic and grammatical categorical knowledge. The approach is implemented in the hyperspace analogue to language (HAL) model of memory, which uses a simple global co-occurrence learning algorithm to encode the context in which words occur. This encoding is the basis for the formation of meaning representations in a high-dimensional context space. Results are presented, and the argument is made that this simple process can ultimately provide the language-comprehension system with semantic and grammatical information required in the comprehension process.