Computational Models Of Language Acquisition Research Articles

Cross-linguistically consistent semantic and syntactic annotation of child-directed speech

AbstractCorpora of child speech and child-directed speech (CDS) have enabled major contributions to the study of child language acquisition, yet semantic annotation for such corpora is still scarce and lacks a uniform standard. Semantic annotation of CDS is particularly important for understanding the nature of the input children receive and developing computational models of child language acquisition. For example, under the assumption that children are able to infer meaning representations for (at least some of) the utterances they hear, the acquisition task is to learn a grammar that can map novel adult utterances onto their corresponding meaning representations, in the face of noise and distraction by other contextually possible meanings. To study this problem and to develop computational models of it, we need corpora that provide both adult utterances and their meaning representations, ideally using annotation that is consistent across a range of languages in order to facilitate cross-linguistic comparative studies. This paper proposes a methodology for constructing such corpora of CDS paired with sentential logical forms, and uses this method to create two such corpora, in English and Hebrew. The approach enforces a cross-linguistically consistent representation, building on recent advances in dependency representation and semantic parsing. Specifically, the approach involves two steps. First, we annotate the corpora using the Universal Dependencies (UD) scheme for syntactic annotation, which has been developed to apply consistently to a wide variety of domains and typologically diverse languages. Next, we further annotate these data by applying an automatic method for transducing sentential logical forms (LFs) from UD structures. The UD and LF representations have complementary strengths: UD structures are language-neutral and support consistent and reliable annotation by multiple annotators, whereas LFs are neutral as to their syntactic derivation and transparently encode semantic relations. Using this approach, we provide syntactic and semantic annotation for two corpora from CHILDES: Brown’s Adam corpus (English; we annotate $$\approx$$ ≈ 80% of its child-directed utterances), all child-directed utterances from Berman’s Hagar corpus (Hebrew). We verify the quality of the UD annotation using an inter-annotator agreement study, and manually evaluate the transduced meaning representations. We then demonstrate the utility of the compiled corpora through (1) a longitudinal corpus study of the prevalence of different syntactic and semantic phenomena in the CDS, and (2) applying an existing computational model of language acquisition to the two corpora and briefly comparing the results across languages.

Acquiring a language vs. inducing a grammar

Standard computational models of language acquisition treat acquiring a language as a process of inducing a set of string-generating rules from a collection of linguistic data assumed to be generated by these very rules. In this paper I give theoretical and empirical arguments that such a model is radically unlike what a human language learner must do to acquire their native language. Most centrally, I argue that such models presuppose that linguistic data is directly a product of a grammar, ignoring the myriad non-grammatical systems involved in the use of language. The significance of these non-target systems in shaping the linguistic data children are exposed to undermines any simple reverse inference from linguistic data to grammatical competence.

SCALa: A blueprint for computational models of language acquisition in social context

Theories and data on language acquisition suggest a range of cues are used, ranging from information on structure found in the linguistic signal itself, to information gleaned from the environmental context or through social interaction. We propose a blueprint for computational models of the early language learner (SCALa, for Socio-Computational Architecture of Language Acquisition) that makes explicit the connection between the kinds of information available to the social learner and the computational mechanisms required to extract language-relevant information and learn from it. SCALa integrates a range of views on language acquisition, further allowing us to make precise recommendations for future large-scale empirical research.

Gradual syntactic triggering: The gradient parameter hypothesis

ABSTRACT In this article, we propose a reconceptualization of the principles and parameters (P&P) framework. We argue that in lieu of discrete parameter values, a parameter value exists on a gradient plane that encodes a learner’s confidence that a particular parametric structure licenses the utterances in the learner’s linguistic input. Crucially, this gradient parameter hypothesis obviates the need for default parameter values. Default parameter values can be put to use effectively from the perspective of linguistic learnability but are lacking in terms of empirical and theoretical consistency. We present findings from a computational implementation of a gradient P&P learner. The findings suggest that the gradient parameter hypothesis provides the basis for a viable alternative to existing computational models of language acquisition in the classic P&P paradigm. We close with a brief discussion of how a gradient parameter space offers a path to address shortcomings that have been attributed to the P&P framework.

Language learning as language use: A cross-linguistic model of child language development.

While usage-based approaches to language development enjoy considerable support from computational studies, there have been few attempts to answer a key computational challenge posed by usage-based theory: the successful modeling of language learning as language use. We present a usage-based computational model of language acquisition which learns in a purely incremental fashion, through online processing based on chunking, and which offers broad, cross-linguistic coverage while uniting key aspects of comprehension and production within a single framework. The model's design reflects memory constraints imposed by the real-time nature of language processing, and is inspired by psycholinguistic evidence for children's sensitivity to the distributional properties of multiword sequences and for shallow language comprehension based on local information. It learns from corpora of child-directed speech, chunking incoming words together to incrementally build an item-based "shallow parse." When the model encounters an utterance made by the target child, it attempts to generate an identical utterance using the same chunks and statistics involved during comprehension. High performance is achieved on both comprehension- and production-related tasks: the model's shallow parsing is evaluated across 79 single-child corpora spanning English, French, and German, while its production performance is evaluated across over 200 single-child corpora representing 29 languages from the CHILDES database. The model also succeeds in capturing findings from children's production of complex sentence types. Together, our modeling results suggest that much of children's early linguistic behavior may be supported by item-based learning through online processing of simple distributional cues, consistent with the notion that acquisition can be understood as learning to process language. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Constructing a Language From Scratch: Combining Bottom–Up and Top–Down Learning Processes in a Computational Model of Language Acquisition

We present a computational model that allows us to study the interplay of different processes involved in first language acquisition. We build on the assumption that language acquisition is usage-driven and assume that there are different processes in language acquisition operating at different levels. Bottom-up processing allows a learner to identify regularities in the linguistic input received, while top–down processing exploits prior experience and previous knowledge to guide choices made during bottom–up processing. To shed light on the interplay between top–down and bottom–up processing in language acquisition, we present a computational model of language acquisition that is based on bootstrapping mechanisms and is usage-based in that it relies on discovered regularities to segment speech into word-like units. Based on this initial segmentation, our model induces a construction grammar that in turn acts as a top–down prior that guides the segmentation of new sentences into words. We spell out in detail these processes and their interplay, showing that top–down processing increases both understanding performance and segmentation accuracy. Our model thus contributes to a better understanding of the interplay between bottom–up and top–down processes in first language acquisition and thus to a better understanding of the mechanisms and architecture involved in language acquisition.

The Utility of Cognitive Plausibility in Language Acquisition Modeling: Evidence From Word Segmentation.

The informativity of a computational model of language acquisition is directly related to how closely it approximates the actual acquisition task, sometimes referred to as the model's cognitive plausibility. We suggest that though every computational model necessarily idealizes the modeled task, an informative language acquisition model can aim to be cognitively plausible in multiple ways. We discuss these cognitive plausibility checkpoints generally and then apply them to a case study in word segmentation, investigating a promising Bayesian segmentation strategy. We incorporate cognitive plausibility by using an age-appropriate unit of perceptual representation, evaluating the model output in terms of its utility, and incorporating cognitive constraints into the inference process. Our more cognitively plausible model shows a beneficial effect of cognitive constraints on segmentation performance. One interpretation of this effect is as a synergy between the naive theories of language structure that infants may have and the cognitive constraints that limit the fidelity of their inference processes, where less accurate inference approximations are better when the underlying assumptions about how words are generated are less accurate. More generally, these results highlight the utility of incorporating cognitive plausibility more fully into computational models of language acquisition.

Acquiring formulaic language

In recent years, psycholinguistic studies have built support for the notion that formulaic language is more widespread and pervasive in adult sentence processing than previously assumed. These findings are mirrored in a number of developmental studies, suggesting that children’s item-based units do not diminish, but persist into adulthood, in keeping with a number of approaches emerging from cognitive linguistics. In the present paper, we describe a simple, psychologically motivated computational model of language acquisition in which the learning and use of formulaic expressions represents the foundation for comprehension and production processes. The model is shown to capture key psycholinguistic findings on children’s sensitivity to the properties of multiword strings and use of lexically specific multiword frames in morphological development. The results of these simulations, we argue, stress the importance of adopting a developmental perspective to better understand how formulaic expressions come to play an important role in adult language use.

What is grammar like? A usage-based constructionist perspective

This paper is intended to elucidate some implications of usage-based linguistic theory for statistical and computational models of language acquisition, focusing on morphology and morphophonology. I discuss the need for grammar (a.k.a. abstraction), the contents of individual grammars (a potentially infinite number of constructions, paradigmatic mappings and predictive relationships between phonological units), the computational characteristics of constructions (complex non-crossover interactions among partially redundant features), resolution of competition among constructions (probability matching), and the need for multimodel inference in modeling internal grammars underlying the linguistic performance of a community.

From Language to Motor Gavagai: Unified Imitation Learning of Multiple Linguistic and Nonlinguistic Sensorimotor Skills

We identify a strong structural similarity between the Gavagai problem in language acquisition and the problem of imitation learning of multiple context-dependent sensorimotor skills from human teachers. In both cases, a learner has to resolve concurrently multiple types of ambiguities while learning how to act in response to particular contexts through the observation of a teacher's demonstrations. We argue that computational models of language acquisition and models of motor skill learning by demonstration have so far only considered distinct subsets of these types of ambiguities, leading to the use of distinct families of techniques across two loosely connected research domains. We present a computational model, mixing concepts and techniques from these two domains, involving a simulated robot learner interacting with a human teacher. Proof-of-concept experiments show that: 1) it is possible to consider simultaneously a larger set of ambiguities than considered so far in either domain; and 2) this allows us to model important aspects of language acquisition and motor learning within a single process that does not initially separate what is “linguistic” from what is “nonlinguistic.” Rather, the model shows that a general form of imitation learning can allow a learner to discover channels of communication used by an ambiguous teacher, thus addressing a form of abstract Gavagai problem (ambiguity about which observed behavior is “linguistic”, and in that case which modality is communicative).

Toward a unified account of comprehension and production in language development

Although Pickering & Garrod (P&G) argue convincingly for a unified system for language comprehension and production, they fail to explain how such a system might develop. Using a recent computational model of language acquisition as an example, we sketch a developmental perspective on the integration of comprehension and production. We conclude that only through development can we fully understand the intertwined nature of comprehension and production in adult processing.

From symbol grounding to socially shared embodied language knowledge

Much language-related research in cognitive robotics appeals to usage-based models of language as proposed in cognitive linguistics and developmental psychology [1, 2] that emphasise the significance of learning, embodiment and general cognitive development for human language acquisition. Over and above these issues, however, what takes centre stage in these theories are social-cognitive skills of “intention-reading” that are seen as “primary in the language acquisition process” [1] – and also as difficult to incorporate into computational models of language acquisition. The present paper addresses these concerns: we describe work in progress on a series of experiments that take steps towards closing the gap between ‘solipsistic’ symbol grounding in individual robotic agents and socially framed embodied language acquisition in learners that attend to common ground [3] with changing interlocutors.

The proper treatment of language acquisition and change in a population setting

Language acquisition maps linguistic experience, primary linguistic data (PLD), onto linguistic knowledge, a grammar. Classically, computational models of language acquisition assume a single target grammar and one PLD source, the central question being whether the target grammar can be acquired from the PLD. However, real-world learners confront populations with variation, i.e., multiple target grammars and PLDs. Removing this idealization has inspired a new class of population-based language acquisition models. This paper contrasts 2 such models. In the first, iterated learning (IL), each learner receives PLD from one target grammar but different learners can have different targets. In the second, social learning (SL), each learner receives PLD from possibly multiple targets, e.g., from 2 parents. We demonstrate that these 2 models have radically different evolutionary consequences. The IL model is dynamically deficient in 2 key respects. First, the IL model admits only linear dynamics and so cannot describe phase transitions, attested rapid changes in languages over time. Second, the IL model cannot properly describe the stability of languages over time. In contrast, the SL model leads to nonlinear dynamics, bifurcations, and possibly multiple equilibria and so suffices to model both the case of stable language populations, mixtures of more than 1 language, as well as rapid language change. The 2 models also make distinct, empirically testable predictions about language change. Using historical data, we show that the SL model more faithfully replicates the dynamics of the evolution of Middle English.

A Computational Model of Language Acquisition: the Emergence of Words

In this paper, we discuss a computational model that is able to detect and build word-like representations on the basis of sensory input. The model is designed and tested with a further aim to investigate how infants may learn to communicate by means of spoken language. The computational model makes use of a memory, a perception module, and the concept of 'learning drive'. Learning takes place within a communicative loop between a 'caregiver' and the 'learner'. Experiments carried out on three European languages with different genetic background (Finnish, Swedish, and Dutch) show that a robust word representation can be learned in using less than 100 acoustic tokens (examples) of that word. The model is inspired by the memory structure that is assumed functional for human cognitive processing.

A Context-Based Computational Model of Language Acquisition by Infants and Children

This research attempts to understand howchildren learn to use language. Instead ofusing syntax-based grammar rules to model thedifferences between children's language andadult language, as has been done in the past, anew model is proposed. In the new researchmodel, children acquire language by listeningto the examples of speech that they hear intheir environment and subsequently use thespeech examples that have been previously heardin similar contextual situations. A computermodel is generated to simulate this new modelof language acquisition. The MALL computerprogram will ‘listen’ to examples of humanspeech, as would occur around a child, and thentry to use these examples in new situationsthat are similar to the contextual situationsin which the language examples were ‘heard’. This will provide a better understanding of howchildren learn to use language and howeducators can assist or improve the languagelearning process by providing required examplesof speech or by helping children to develop abetter understanding of similarities betweenvarious contexts.

Schemas, Computation, and Language Acquisition

The word schema is used in three overlapping, but somewhat different, senses: as a translation of Piaget’s term scheme; as the unit of process-and-representation in Arbib’s theory of perceptual structures and distributed motor control; and as a computational unit in the attempt to relate the mechanisms underlying language acquisition to other cognitive systems. In the Piagetian literature, the term schema may refer either to a set of regularities in observed behavior or to an internal mechanism which could produce such regularities. A major task, then, is to relate ‘private’ mechanisms to public behavior. The first section of this paper provides a general analysis of this problem. The second offers a more concrete example: a computational model of language acquisition. The third section then relates these considerations to Piaget’s notion of mutual verification and to some general philosophical questions about learning.

Learning to count: a computational model of language acquisition.

Effective procedures are described by which a learner can discover how to name the numbers 1, 2, . . . , 99, . . . in an unfamiliar language from a representative set of numerals presented in association with their numerical values. These procedures have been implemented as a com­puter program which will learn to count, and to interpret well formed numerals, in languages as diverse as English, French, Japanese, Mixtec, Suppire and Biblical Welsh; their effectiveness confirms the view that semantic information can be of considerable assistance in the acquisition of syntax.

Learning to count: a computational model of language acquisition

Effective procedures are described by which a learner can discover how to name the numbers 1, 2, 99, ... in an unfamiliar language from a representative set of numerals presented in association with their numerical values. These procedures have been implemented as a computer program which will learn to count, and to interpret well formed numerals, in languages as diverse as English, French, Japanese, Mixtec, Suppire and Biblical Welsh; their effectiveness confirms the view that semantic information can be of considerable assistance in the acquisition of syntax.