Semantic Grounding of Novel Spoken Words in the Primary Visual Cortex

Max Garagnani,Friedemann Pulvermüller,Evgeniya Kirilina

doi:10.3389/fnhum.2021.581847

Abstract

Embodied theories of grounded semantics postulate that, when word meaning is first acquired, a link is established between symbol (word form) and corresponding semantic information present in modality-specific—including primary—sensorimotor cortices of the brain. Direct experimental evidence documenting the emergence of such a link (i.e., showing that presentation of a previously unknown, meaningless word sound induces, after learning, category-specific reactivation of relevant primary sensory or motor brain areas), however, is still missing. Here, we present new neuroimaging results that provide such evidence. We taught participants aspects of the referential meaning of previously unknown, senseless novel spoken words (such as “Shruba” or “Flipe”) by associating them with either a familiar action or a familiar object. After training, we used functional magnetic resonance imaging to analyze the participants’ brain responses to the new speech items. We found that hearing the newly learnt object-related word sounds selectively triggered activity in the primary visual cortex, as well as secondary and higher visual areas.These results for the first time directly document the formation of a link between the novel, previously meaningless spoken items and corresponding semantic information in primary sensory areas in a category-specific manner, providing experimental support for perceptual accounts of word-meaning acquisition in the brain.

Highlights

When a language is learnt, at least some of its novel symbols must be ‘‘grounded’’ in perceptions and actions; if not, the language learner might not know what linguistic symbols relate to in the physical world, i.e., what they are used to speak about, and, what they ‘‘mean’’ (Freud, 1891; Locke, 1909/1847; Searle, 1980; Harnad, 1990, 2012; Cangelosi et al, 2000)
As the reaction times (RT) alone cannot reveal whether participants have successfully learned the novel words, we looked at d’ values
Auditory presentation of newly learnt spoken words activated left-lateralized superior temporal cortex and, after they had co-occurred with different exemplars from the same conceptual category, the novel sounds sparked visual cortex, including left posterior fusiform and bilateral primary visual cortex (BA 17)

Summary

Introduction

When a language is learnt, at least some of its novel symbols must be ‘‘grounded’’ in perceptions and actions; if not, the language learner might not know what linguistic symbols relate to in the physical world, i.e., what they are used to speak about, and, (in one sense) what they ‘‘mean’’ (Freud, 1891; Locke, 1909/1847; Searle, 1980; Harnad, 1990, 2012; Cangelosi et al, 2000). Learning Novel Object and Action Words (Barsalou, 2008; Glenberg and Gallese, 2012; Pulvermüller, 2013) have long postulated that repeated co-occurrence of symbol and referent object (and/or action execution) leads to the emergence of associative links in the cortex, ‘‘cell assembly’’ circuits (Hebb, 1949) binding symbols (word-form representations emerging in perisylvian areas) with corresponding semantic information coming from the senses and the motor system (Pulvermüller and Preissl, 1991; Pulvermüller, 1999). It is impossible to match for all relevant psycholinguistic features when considering utterances from natural languages, and, any studies on real words suffer from this ‘‘confounded nuisance’’ problem (Cutler, 1981)

Methods

Results

Conclusion