Representation Of Word Meaning Research Articles

The hippocampus supports the representation of abstract concepts: Implications for the study of recognition memory

Words, unlike images, are symbolic representations. The associative details inherent within a word's meaning and the visual imagery it generates, are inextricably connected to the way words are processed and represented. It is well recognised that the hippocampus associatively binds components of a memory to form a lasting representation, and here we show that the hippocampus is especially sensitive to abstract word processing. Using fMRI during recognition, we found that the increased abstractness of words produced increased hippocampal activation regardless of memory outcome. Interestingly, word recollection produced hippocampal activation regardless of word content, while the parahippocampal cortex was sensitive to concreteness of word representations, regardless of memory outcome. We reason that the hippocampus has assumed a critical role in the representation of uncontextualized abstract word meaning, as its information-binding ability allows the retrieval of the semantic and visual associates that, when bound together, generate the abstract concept represented by word symbols. These insights have implications for research on word representation, memory, and hippocampal function, perhaps shedding light on how the human brain has adapted to encode and represent abstract concepts.

Mapping semantic space: Exploring the higher-order structure of word meaning

Multiple representation theories posit that concepts are represented via a combination of properties derived from sensorimotor, affective, and linguistic experiences. Recently, it has been proposed that information derived from social experience, or socialness, represents another key aspect of conceptual representation. How these various dimensions interact to form a coherent conceptual space has yet to be fully explored. To address this, we capitalized on openly available word property norms for 6339 words and conducted a large-scale investigation into the relationships between 18 dimensions. An exploratory factor analysis reduced the dimensions to six higher-order factors: sub-lexical, distributional, visuotactile, body action, affective and social interaction. All these factors explained unique variance in performance on lexical and semantic tasks, demonstrating that they make important contributions to the representation of word meaning. An important and novel finding was that the socialness dimension clustered with the auditory modality and with mouth and head actions. We suggest this reflects experiential learning from verbal interpersonal interactions. Moreover, formally modelling the network structure of semantic space revealed pairwise partial correlations between most dimensions and highlighted the centrality of the interoception dimension. Altogether, these findings provide new insights into the architecture of conceptual space, including the importance of inner and social experience, and highlight promising avenues for future research.

Spelling Facilitates Reading: A Tutorial on the Spell-to-Read Approach.

According to the American Speech-Language-Hearing Association (ASHA), prevention, identification, assessment, and intervention of children who are learning to read and write are within the scope of practice for school-based speech-language pathologists (SLPs). Furthermore, for SLPs who work in the school setting, it is not uncommon to have struggling readers and poor spellers on their caseloads. Importantly, for students who have difficulty in spelling, their spelling errors are among the early indicators of dyslexia and can provide a means for identifying readers who may benefit from early intervention. SLPs can both assess spelling skills and implement evidence-based spelling and literacy diagnosis and instruction. Spelling instruction in kindergarten through the high school grades that is heavily grounded in metalinguistic activities can provide access to mental representations of word spellings, pronunciations, and meanings; links between whole-word and phonics approaches to reading instruction; and a foundation for reading fluency and comprehension. Learning to spell is essential for learning to read. Accordingly, this tutorial aims to elucidate how to (a) assess and identify phases of spelling development and (b) teach spelling to facilitate reading through a listening-first approach.

Determining the Relativity of Word Meanings Through the Construction of Individualized Models of Semantic Memory.

Distributional models of lexical semantics are capable of acquiring sophisticated representations of word meanings. The main theoretical insight provided by these models is that they demonstrate the systematic connection between the knowledge that people acquire and the experience that they have with the natural language environment. However, linguistic experience is inherently variable and differs radically across people due to demographic and cultural variables. Recently, distributional models have been used to examine how word meanings vary across languages and it was found that there is considerable variability in the meanings of words across languages for most semantic categories. The goal of this article is to examine how variable word meanings are across individual language users within a single language. This was accomplished by assembling 500 individual user corpora attained from the online forum Reddit. Each user corpus ranged between 3.8 and 32.3 million words each, and a count-based distributional framework was used to extract word meanings for each user. These representations were then used to estimate the semantic alignment of word meanings across individual language users. It was found that there are significant levels of relativity in word meanings across individuals, and these differences are partially explained by other psycholinguistic factors, such as concreteness, semantic diversity, and social aspects of language usage. These results point to word meanings being fundamentally relative and contextually fluid, with this relativeness being related to the individualized nature of linguistic experience.

The Iconicity Ring Hypothesis Bridges the Gap Between Symbol Grounding and Linguistic Relativity.

Kemmerer captured the drastic change in theories of word meaning representations, contrasting the view that word meaning representations are amodal and universal, with the view that they are grounded and language-specific. However, he does not address how language can be simultaneously grounded and language-specific. Here, we approach this question from the perspective of language acquisition and evolution. We argue that adding a new element-iconicity-is critically beneficial and offer the iconicity ring hypothesis, which explains how language-specific, secondary iconicity might emerge from biologically grounded and universally shared iconicity in the course of language acquisition and evolution.

Deep learning models to study sentence comprehension in the human brain

ABSTRACT Recent artificial neural networks that process natural language achieve unprecedented performance in tasks requiring sentence-level understanding. As such, they could be interesting models of the integration of linguistic information in the human brain. We review works that compare these artificial language models with human brain activity and we assess the extent to which this approach has improved our understanding of the neural processes involved in natural language comprehension. Two main results emerge. First, the neural representation of word meaning aligns with the context-dependent, dense word vectors used by the artificial neural networks. Second, the processing hierarchy that emerges within artificial neural networks broadly matches the brain, but is surprisingly inconsistent across studies. We discuss current challenges in establishing artificial neural networks as process models of natural language comprehension. We suggest exploiting the highly structured representational geometry of artificial neural networks when mapping representations to brain data.

From Word Types to Tokens and Back: A Survey of Approaches to Word Meaning Representation and Interpretation

Abstract Vector-based word representation paradigms situate lexical meaning at different levels of abstraction. Distributional and static embedding models generate a single vector per word type, which is an aggregate across the instances of the word in a corpus. Contextual language models, on the contrary, directly capture the meaning of individual word instances. The goal of this survey is to provide an overview of word meaning representation methods, and of the strategies that have been proposed for improving the quality of the generated vectors. These often involve injecting external knowledge about lexical semantic relationships, or refining the vectors to describe different senses. The survey also covers recent approaches for obtaining word type-level representations from token-level ones, and for combining static and contextualized representations. Special focus is given to probing and interpretation studies aimed at discovering the lexical semantic knowledge that is encoded in contextualized representations. The challenges posed by this exploration have motivated the interest towards static embedding derivation from contextualized embeddings, and for methods aimed at improving the similarity estimates that can be drawn from the space of contextual language models.

Color-concept associations reveal an abstract conceptual space

People have systematic associations between concepts and colors, and concepts can be represented as distributions of association strengths across color space (Schloss et al., 2018; Rathore et al., 2018). These patterns of color-concept associations are highly structured, which may be critical in explaining how people flexibly draw conceptual inferences about colors across contexts (Mukherjee et al., 2021). Here we consider the structure of the semantic space defined by color-concept associations, evaluating three hypotheses. First, the structure may be arbitrary, so that items evoking similar color associations are conceptually unrelated. Second, it may be redundant with the conceptual structure apparent in natural language, as estimated via word-embeddings computed from large corpora. Third, it may express a representational space important to human cognition but distinct from those latent in natural language. To adjudicate these possibilities, we selected 30 concepts spanning 6 semantic domains (clothes, emotions, scenes, fruits, directions, times-of-day) and collected association ratings between each concept and 58 colors sampled across CIELAB color space. Pairwise dissimilarities between concepts in this color-concept association space could be well captured by embedding them in a reduced 3-dimensional space, where concepts with similar color-concept associations were embedded near each other. A comparison of these embeddings to those extracted from natural-language corpora revealed highly distinct conceptual spaces. To determine whether the color-concept space is reflected in mental representations of word meanings, we used a triadic comparison task to embed the same items based on direct judgments of semantic similarity. These correlated more strongly with color-concept similarities, suggesting that the color-concept associations express aspects of semantic knowledge beyond color that elude word-embeddings derived from language corpora, supporting our third hypothesis. The results provide an initial basis for understanding abstract conceptual relations in human cognition through the lens of visual properties of concepts — in this case, color-concept associations.

A gradualist view of word meaning in language acquisition and language use

For both children and adults, communicating with each other effectively depends on having enough knowledge about particular entities, actions, or relations to understand and produce the words being used. Speakers draw on conventional meanings shared with their interlocutors, but do they share every detail of word meaning? They need not have identical, or fully specified, representations for the meanings of all the terms they make use of. Rather, they need only have represented enough about the meanings of the words used by another speaker to understand what is intended in context on a particular occasion. Reliance on partial meanings is common in both children and adults. More detailed, shared, representations of word meanings for a domain depend on acquiring additional knowledge about that domain and its contents.

From Semantic Vectors to Analogical Mapping

Human reasoning goes beyond knowledge about individual entities, extending to inferences based on relations between entities. Here we focus on the use of relations in verbal analogical mapping, sketching a general approach based on assessing similarity between patterns of semantic relations between words. This approach combines research in artificial intelligence with work in psychology and cognitive science, with the aim of minimizing hand coding of text inputs for reasoning tasks. The computational framework takes as inputs vector representations of individual word meanings, coupled with semantic representations of the relations between words, and uses these inputs to form semantic-relation networks for individual analogues. Analogical mapping is operationalized as graph matching under cognitive and computational constraints. The approach highlights the central role of semantics in analogical mapping.

Predicting patterns of similarity among abstract semantic relations.

Although models of word meanings based on distributional semantics have proved effective in predicting human judgments of similarity among individual concepts, it is less clear whether or how such models might be extended to account for judgments of similarity among relations between concepts. Here we combine an individual-differences approach with computational modeling to predict human judgments of similarity among word pairs instantiating a variety of abstract semantic relations (e.g., contrast, cause-effect, part-whole). A measure of cognitive capacity predicted individual differences in the ability to discriminate among distinct relations. The human pattern of relational similarity judgments, both at the group level and for individual participants, was best predicted by a model that takes representations of word meanings based on distributional semantics as its inputs and uses them to learn an explicit representation of relations. These findings indicate that although the meanings of abstract semantic relations are not directly coded in the meanings of individual words, important aspects of relational similarity can be derived from distributional semantics. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Лексико-конструкционный подход к синтаксической альтернации

Syntactic alternation cannot be treated as a lexical or as a constructional phenomenon (as is in lexical and constructional approaches, respectively). Realizing the significance of both factors, in the present paper I offer a lexical-constructional account of syntactic alternation which seems to prevail against previous ones by eliminating their shortcomings but exploiting their advantages. I demonstrate the explanatory power of my conception by thorough analysis of three Russian verb classes. They include the alternation of manner-of-motion and directional motion verbs as well as the locative alternation and instrument–subject alternation. The fundamental feature of my proposal is the underspecified character of lexical-semantic representations. Although underspecification may have several forms, the general idea consists in postulating optional components in lexical representations. It can be stated that verbs alternate syntactically if they own an underspecified but encyclopedically and pragmatically enriched representation, so being compatible with all meanings appearing in alternations. If a verb does not have a lexical-semantic representation that can result in different interpretations, it cannot participate in an alternation. Because of the possible lexical specification of either variant, an underspecified type of representation is inevitably required for alternating verbs. The lexical-constructional approach to syntactic alternation naturally extends to lexical pragmatics. This extension means that the hypotheses about the characteristics of word meaning representations can also be applied to cases where words reach their full meanings in corresponding contexts without the change of syntactic structure.

Orienting to different dimensions of word meaning alters the representation of word meaning in early processing regions.

Conscious processing of word meaning can be guided by attention. In this event-related functional magnetic resonance imaging study in 22 healthy young volunteers, we examined in which regions orienting attention to two fundamental and generic dimensions of word meaning, concreteness versus valence, alters the semantic representations coded in activity patterns. The stimuli consisted of 120 nouns in written or spoken modality which varied factorially along the concreteness and valence axis. Participants performed a forced-choice judgement of either concreteness or valence. Rostral and subgenual anterior cingulate were strongly activated during valence judgement, and precuneus and the dorsal attention network during concreteness judgement. Task and stimulus type interacted in right posterior fusiform gyrus, left lingual gyrus, precuneus, and insula. In the right posterior fusiform gyrus and the left lingual gyrus, the correlation between the pairwise similarity in activity patterns evoked by words and the pairwise distance in valence and concreteness was modulated by the direction of attention, word valence or concreteness. The data indicate that orienting attention to basic dimensions of word meaning exerts effects on the representation of word meaning in more peripheral nodes, such as the ventral occipital cortex, rather than the core perisylvian language regions.

Dual coding of knowledge in the human brain.

How does the human brain code knowledge about the world? While disciplines such as artificial intelligence represent world knowledge based on human language, neurocognitive models of knowledge have been dominated by sensory embodiment, in which knowledge is derived from sensory/motor experience and supported by high-level sensory/motor and association cortices. The neural correlates of an alternative disembodied symbolic system had previously been difficult to establish. A recent line of studies exploring knowledge about visual properties, such as color, in visually deprived individuals converge to provide positive, compelling evidence for non-sensory, language-derived, knowledge representation in dorsal anterior temporal lobe and extended language network, in addition to the sensory-derived representations, leading to a sketch of a dual-coding knowledge neural framework.

Idiosyncratic Tower of Babel: Individual Differences in Word-Meaning Representation Increase as Word Abstractness Increases

Humans primarily rely on language to communicate, on the basis of a shared understanding of the basic building blocks of communication: words. Do we mean the same things when we use the same words? Although cognitive neural research on semantics has revealed the common principles of word-meaning representation, the factors underlying the potential individual variations in word meanings are unknown. Here, we empirically characterized the intersubject consistency of 90 words across 20 adult subjects (10 female) using both behavioral measures (rating-based semantic-relationship patterns) and neuroimaging measures (word-evoked brain activity patterns). Across both the behavioral and neuroimaging experiments, we showed that the magnitude of individual disagreements on word meanings could be modeled on the basis of how much language or sensory experience is associated with a word and that this variation increases with word abstractness. Uncovering the cognitive and neural origins of word-meaning disagreements across individuals has implications for potential mechanisms to modulate such disagreements.

Using EEG to decode semantics during an artificial language learning task.

BackgroundAs we learn a new nonnative language (L2), we begin to build a new map of concepts onto orthographic representations. Eventually, L2 can conjure as rich a semantic representation as our native language (L1). However, the neural processes for mapping a new orthographic representation to a familiar meaning are not well understood or characterized.MethodsUsing electroencephalography and an artificial language that maps symbols to English words, we show that it is possible to use machine learning models to detect a newly formed semantic mapping as it is acquired.ResultsThrough a trial‐by‐trial analysis, we show that we can detect when a new semantic mapping is formed. Our results show that, like word meaning representations evoked by a L1, the localization of the newly formed neural representations is highly distributed, but the representation may emerge more slowly after the onset of the symbol. Furthermore, our mapping of word meanings to symbols removes the confound of the semantics to the visual characteristics of the stimulus, a confound that has been difficult to disentangle previously.ConclusionWe have shown that the L1 semantic representation conjured by a newly acquired L2 word can be detected using decoding techniques, and we give the first characterization of the emergence of that mapping. Our work opens up new possibilities for the study of semantic representations during L2 learning.

Representing word meaning in context via lexical substitutes

Representing the meaning of individual words is crucial for most natural language processing (NLP) tasks. This, however, is a challenge because word meaning often depends on the context. Recent approaches to representing word meaning in context rely on lexical substitution (LS), where a word is represented with a set of meaning-preserving substitutes. While face valid, it is not clear to what extent substitute-based representation corresponds to the more established sense-based representation required for many NLP tasks. We present an empirical study that addresses this question by quantifying the correspondence between substitute- and sense-based meaning representations. We compile a high-quality dataset annotated with lexical substitutes and sense labels from two well-established sense inventories, and conduct a correlation analysis using a number of substitute-based similarity measures. Furthermore, as recent work has demonstrated the efficacy of system-produced substitutes for word meaning representation, we compare human- and system-produced substitutes to determine the performance gap between the two. Lastly, we investigate to what extent the results translate to the fundamental semantic task of word sense induction (WSI). Our experiments show the validity of LS for word meaning in context representation and justify the use of system-produced substitutes for WSI.

Ways of Representation of Word Meanings in Human Lexicon

Ways of Representation of Word Meanings in Human Lexicon

Meta-Representational Skills in Bullying Roles: The Influence of Definitional Competence and Empathy.

This study investigated the influence of meta-representational aspects on bullying. Meta-representation was operationalized in terms of the metalinguistic skill to produce conventional definitions, reflecting culturally shared representations and of the meta-level capacity to represent others’ mental states underlying empathic disposition. One hundred and seventeen children, aged between 8;5 and 10;11 years, completed a definitional task and self-report questionnaires on bullying roles and empathic disposition. Descriptive, correlational, and regression analyses were performed. Results confirmed that hostile roles are negatively related to definitional competence and to empathic disposition. Lack of definitional competence was the main predictor (accounting for about 16% of variance), followed by empathy (explaining a further 6% of variance) of Primary School children’s disposition to assume aggressive behaviors. These findings suggest that a lack of general meta-representational abilities may hinder the development of abstract and other-centered perspective taking, and compromise (compromising) social adjustment. This implies the need to work, particularly in school, on enhancing meta-representational and metalinguistic skills, such as the ability to recognize mental states and verbally make explicit cultural-semantic word meaning representations.

Decoding and encoding models reveal the role of mental simulation in the brain representation of meaning.

How the brain representation of conceptual knowledge varies as a function of processing goals, strategies and task-factors remains a key unresolved question in cognitive neuroscience. In the present functional magnetic resonance imaging study, participants were presented with visual words during functional magnetic resonance imaging (fMRI). During shallow processing, participants had to read the items. During deep processing, they had to mentally simulate the features associated with the words. Multivariate classification, informational connectivity and encoding models were used to reveal how the depth of processing determines the brain representation of word meaning. Decoding accuracy in putative substrates of the semantic network was enhanced when the depth processing was high, and the brain representations were more generalizable in semantic space relative to shallow processing contexts. This pattern was observed even in association areas in inferior frontal and parietal cortex. Deep information processing during mental simulation also increased the informational connectivity within key substrates of the semantic network. To further examine the properties of the words encoded in brain activity, we compared computer vision models—associated with the image referents of the words—and word embedding. Computer vision models explained more variance of the brain responses across multiple areas of the semantic network. These results indicate that the brain representation of word meaning is highly malleable by the depth of processing imposed by the task, relies on access to visual representations and is highly distributed, including prefrontal areas previously implicated in semantic control.