The emergence of links between lexical acquisition and object categorization: a computational study

Abstract

Language is about symbols, and those symbols must be grounded in the physical world. Children learn to associate language with sensorimotor experiences during their development. In light of this, we first provide a computational account of how words are mapped to their perceptually grounded meanings. Moreover, the main part of this work proposes and implements a computational model of how word learning influences the formation of object categories to which those words refer. This model simulates the bi-directional relationship between word and object category learning: (1) object categorization provides mental representations of meanings that are mapped to words to form lexical items; (2) linguistic labels help object categorization by providing additional teaching signals; and (3) these two learning processes interplay with each other and form a developmental feedback loop. Compared with the method that performs these two tasks separately, our model shows promising improvements in both word-to-world mapping and perceptual categorization, suggesting a unified view of lexical and category learning in an integrative framework. Most importantly, this work provides a cognitively plausible explanation of the mechanistic nature of early word learning and object learning from co-occurring multisensory data.