Hyperspace Analogue To Language Research Articles

From simple associations to the building blocks of language: Modeling meaning in memory with the HAL model

This paper presents a theoretical approach of how simple, episodic associations are transduced into semantic and grammatical categorical knowledge. The approach is implemented in the hyperspace analogue to language (HAL) model of memory, which uses a simple global co-occurrence learning algorithm to encode the context in which words occur. This encoding is the basis for the formation of meaning representations in a high-dimensional context space. Results are presented, and the argument is made that this simple process can ultimately provide the language-comprehension system with semantic and grammatical information required in the comprehension process.

Explorations in context space: Words, sentences, discourse

Deriving representations of meaning, whether at the word, the sentence, or the discourse level, is a problem with a long history in cognitive psychology and psycholinguistics. In this article, we describe a computational model of high‐dimensional context space, the Hyperspace Analog to Language (HAL), and present simulation evidence that HAL's vector representations can provide sufficient information to make semantic, grammatical, and abstract distinctions. Human participants were able to use the context neighborhoods that HAL generates to match words with similar items and to derive the word (or a similar word) from the neighborhood, thus demonstrating the cognitive compatibility of the representations with human processing. An experiment exploring the meaning of agent‐ and patient‐oriented verbs provided the context for discussing how the connotative aspects of word neighborhoods could provide cues in establishing a discourse model. Using the vector representations to build a sentence‐level representation was attempted, but the results were not promising. This limitation of the representations is presented in the context of other similar models that have been more effective. A final set of experiments explored the nature of verbs and verb instruments, and we introduce a new methodology that extracts conceptual intersections from the high‐dimensional context space.

Producing high-dimensional semantic spaces from lexical co-occurrence

A procedure that processes a corpus of text and produces numeric vectors containing information about its meanings for each word is presented. This procedure is applied to a large corpus of natural language text taken from Usenet, and the resulting vectors are examined to determine what information is contained within them. These vectors provide the coordinates in a high-dimensional space in which word relationships can be analyzed. Analyses of both vector similarity and multidimensional scaling demonstrate that there is significant semantic information carried in the vectors. A comparison of vector similarity with human reaction times in a single-word priming experiment is presented. These vectors provide the basis for a representational model of semantic memory, hyperspace analogue to language (HAL).