Domain-general Learning Mechanisms Research Articles

A cognitive modeling approach to learning and using reference biases in language.

During real-time language processing, people rely on linguistic and non-linguistic biases to anticipate upcoming linguistic input. One of these linguistic biases is known as the implicit causality bias, wherein language users anticipate that certain entities will be rementioned in the discourse based on the entity's particular role in an expressed causal event. For example, when language users encounter a sentence like "Elizabeth congratulated Tina…" during real-time language processing, they seemingly anticipate that the discourse will continue about Tina, the object referent, rather than Elizabeth, the subject referent. However, it is often unclear how these reference biases are acquired and how exactly they get used during real-time language processing. In order to investigate these questions, we developed a reference learning model within the PRIMs cognitive architecture that simulated the process of predicting upcoming discourse referents and their linguistic forms. Crucially, across the linguistic input the model was presented with, there were asymmetries with respect to how the discourse continued. By utilizing the learning mechanisms of the PRIMs architecture, the model was able to optimize its predictions, ultimately leading to biased model behavior. More specifically, following subject-biased implicit causality verbs the model was more likely to predict that the discourse would continue about the subject referent, whereas following object-biased implicit causality verbs the model was more likely to predict that the discourse would continue about the object referent. In a similar fashion, the model was more likely to predict that subject referent continuations would be in the form of a pronoun, whereas object referent continuations would be in the form of a proper name. These learned biases were also shown to generalize to novel contexts in which either the verb or the subject and object referents were new. The results of the present study demonstrate that seemingly complex linguistic behavior can be explained by cognitively plausible domain-general learning mechanisms. This study has implications for psycholinguistic accounts of predictive language processing and language learning, as well as for theories of implicit causality and reference processing.

Revisiting Subject-Object Asymmetry in the Production of Cantonese Relative Clauses: Evidence From Elicited Production in 3-Year-Olds.

Emergentist approaches to language acquisition identify a core role for language-specific experience and give primacy to other factors like function and domain-general learning mechanisms in syntactic development. This directly contrasts with a nativist structurally oriented approach, which predicts that grammatical development is guided by Universal Grammar and that structural factors constrain acquisition. Cantonese relative clauses (RCs) offer a good opportunity to test these perspectives because its typologically rare properties decouple the roles of frequency and complexity in subject- and object-RCs in a way not possible in European languages. Specifically, Cantonese object RCs of the classifier type are frequently attested in children’s linguistic experience and are isomorphic to frequent and early-acquired simple SVO transitive clauses, but according to formal grammatical analyses Cantonese subject RCs are computationally less demanding to process. Thus, the two opposing theories make different predictions: the emergentist approach predicts a specific preference for object RCs of the classifier type, whereas the structurally oriented approach predicts a subject advantage. In the current study we revisited this issue. Eighty-seven monolingual Cantonese children aged between 3;2 and 3;11 (Mage: 3;6) participated in an elicited production task designed to elicit production of subject- and object- RCs. The children were very young and most of them produced only noun phrases when RCs were elicited. Those (nine children) who did produce RCs produced overwhelmingly more object RCs than subject RCs, even when animacy cues were controlled. The majority of object RCs produced were the frequent classifier-type RCs. The findings concur with our hypothesis from the emergentist perspectives that input frequency and formal and functional similarity to known structures guide acquisition.

Aging effects and feasibility of statistical learning tasks across modalities

ABSTRACT Knowledge on statistical learning (SL) in healthy elderly is scarce. Theoretically, it is not clear whether aging affects modality-specific and/or domain-general learning mechanisms. Practically, there is a lack of research on simplified SL tasks, which would ease the burden of testing in clinical populations. Against this background, we conducted two experiments across three modalities (auditory, visual and visuomotor) in a total of 93 younger and older adults. In Experiment 1, SL was induced in all modalities. Aging effects appeared in the tasks relying on an explicit posttest to assess SL. We hypothesize that declines in domain-general processes that predominantly modulate explicit learning mechanisms underlie these aging effects. In Experiment 2, more feasible tasks were developed for which the level of SL was maintained in all modalities, except the auditory modality. These tasks are more likely to successfully measure SL in elderly (patient) populations in which task demands can be problematic.

Implicit sequence learning using auditory cues leads to modality-specific representations.

The implicit acquisition of statistical information from the environment is considered a fundamental type of human learning. Paradigms using visually cued sequences have been frequently used to study implicit learning. However, learning sequences of auditory cues is likely to be important in domains such as language or music. In three experiments, we established a novel auditorily cued implicit perceptual-motor sequence learning paradigm to compare to traditional visually cued sequence learning and identify whether this type of learning generalizes across cue modality. Participants exhibited reliable sequence-specific learning to auditory cues in all three experiments (Experiments 1-3), which was generally not influenced by explicit knowledge (Experiment 2). In Experiment 3, a large drop in knowledge expression in the novel cue modality was observed, suggesting that the acquired implicit sequence knowledge depended largely on sensory-specific representations. Overall, auditorily cued learning was similar to, though proceeded faster than, learning in comparable visually cued sequence learning paradigms. Similarity between learning from cues in different sensory modalities suggests that there may be a common process for the automatic extraction of sequential statistical structure. However, the lack of robust transfer sequence knowledge across modalities argues against a purely domain-general learning mechanism for all kinds of sequences. By expanding quantitative methodologies to characterize sequence learning in the auditory domain, these findings illustrate the possibility of bridging research in sequence and statistical learning domains to identify common mechanisms of complex cognitive skill and language learning.

"This is a Titanic song": the effect of familiarity on children's learning affective meaning in music.

Usage-based theories of language acquisition are thought to rely on domain-general learning mechanisms, such as mastering familiar routines by rote before generalizing to novel unfamiliar instances. If so, then the role of familiarity should extend to non-linguistic domains, like music. The purpose of the present study was to test the role of familiarity in children's learning of affective meaning of music. Music carries an affective meaning that is relayed through its elements, such as mode, rhythm, and tempo. The previous research has found differences between children and adults in their understanding of music's affective meaning, suggesting that this meaning is learned. We predicted that children would initially learn the affective meaning of familiar musical pieces before generalizing to unfamiliar pieces. Children between 3 and 5years of age heard 16 musical segments, one for each emotion (i.e., anger, sadness, and happiness), and their accuracy in pairing music-emotion was measured. For younger children, their familiarity with the piece was positively associated with their accuracy. These results suggest that familiarity plays a role in learning affective meaning in music, providing support for the claim that this learning mechanism is domain-general.

Numerosity discrimination in deep neural networks: Initial competence, developmental refinement and experience statistics

Both humans and non-human animals exhibit sensitivity to the approximate number of items in a visual array, as indexed by their performance in numerosity discrimination tasks, and even neonates can detect changes in numerosity. These findings are often interpreted as evidence for an innate 'number sense'. However, recent simulation work has challenged this view by showing that human-like sensitivity to numerosity can emerge in deep neural networks that build an internal model of the sensory data. This emergentist perspective posits a central role for experience in shaping our number sense and might explain why numerical acuity progressively increases over the course of development. Here we substantiate this hypothesis by introducing a progressive unsupervised deep learning algorithm, which allows us to model the development of numerical acuity through experience. We also investigate how the statistical distribution of numerical and non-numerical features in natural environments affects the emergence of numerosity representations in the computational model. Our simulations show that deep networks can exhibit numerosity sensitivity prior to any training, as well as a progressive developmental refinement that is modulated by the statistical structure of the learning environment. To validate our simulations, we offer a refinement to the quantitative characterization of the developmental patterns observed in human children. Overall, our findings suggest that it may not be necessary to assume that animals are endowed with a dedicated system for processing numerosity, since domain-general learning mechanisms can capture key characteristics others have attributed to an evolutionarily specialized number system.

How do bodies become special? Electrophysiological evidence for the emergence of body-related cortical processing in the first 14 months of life.

There is general consensus that the representation of the human face becomes functionally specialized within the first few months of an infant's life. The literature is divided, however, on the question whether the specialized representation of the remainder of the human body form follows a similarly rapid trajectory or emerges more slowly and in line with domain-general learning mechanisms. Our study investigates visual event-related potentials (ERPs) in adults (P1 and N170) and infants (P1, N290, P400, and Nc) of 3 age groups (3.5, 10, and 14 months) to compare the emergence of face- and body-structural encoding. Our findings show that visual ERPs were absent (P1, N290, P400) or smaller (Nc) for bodies than for faces at 3.5 months. At older ages, P400 was smaller (10 months) and peaked later (14 months) for bodies than for faces. Effects of stimulus orientation were not reliably found until 14 months, where they were more broadly distributed for faces than for bodies. Inverted faces, but not bodies, produced an adult-like pattern for P400 at 14 months, emphasizing the role of P400 as the precursor of the adult N170. Importantly, our findings argue that structural encoding of the human body form emerges later in infancy and is qualitatively different from the structural encoding for faces. This is commensurate with infant motor development and the experience of viewing complete body shapes later than faces. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

What Mechanisms Underlie Implicit Statistical Learning? Transitional Probabilities Versus Chunks in Language Learning

In a prior review, Perrruchet and Pacton (2006) noted that the literature on implicit learning and the more recent studies on statistical learning focused on the same phenomena, namely the domain-general learning mechanisms acting in incidental, unsupervised learning situations. However, they also noted that implicit learning and statistical learning research favored different interpretations, focusing on the selection of chunks and the computation of transitional probabilities aimed at discovering chunk boundaries, respectively. This paper examines the state of the debate 12years later. The link between contrasting theories and their historical roots has disappeared, but a number of studies were aimed at contrasting the predictions of these two approaches. Overall, these studies strongly question the still prevalent account based on the statistical computation of pairwise associations. Various chunk-based models provide much better predictions in a number of experimental situations. However, these models rely on very different conceptual frameworks, as illustrated by a comparison between Bayesian models of word segmentation, PARSER, and a connectionist model (TRACX).

An emergentist perspective on the origin of number sense.

The finding that human infants and many other animal species are sensitive to numerical quantity has been widely interpreted as evidence for evolved, biologically determined numerical capacities across unrelated species, thereby supporting a ‘nativist’ stance on the origin of number sense. Here, we tackle this issue within the ‘emergentist’ perspective provided by artificial neural network models, and we build on computer simulations to discuss two different approaches to think about the innateness of number sense. The first, illustrated by artificial life simulations, shows that numerical abilities can be supported by domain-specific representations emerging from evolutionary pressure. The second assumes that numerical representations need not be genetically pre-determined but can emerge from the interplay between innate architectural constraints and domain-general learning mechanisms, instantiated in deep learning simulations. We show that deep neural networks endowed with basic visuospatial processing exhibit a remarkable performance in numerosity discrimination before any experience-dependent learning, whereas unsupervised sensory experience with visual sets leads to subsequent improvement of number acuity and reduces the influence of continuous visual cues. The emergent neuronal code for numbers in the model includes both numerosity-sensitive (summation coding) and numerosity-selective response profiles, closely mirroring those found in monkey intraparietal neurons. We conclude that a form of innatism based on architectural and learning biases is a fruitful approach to understanding the origin and development of number sense.This article is part of a discussion meeting issue ‘The origins of numerical abilities'.

Learning Parametric Stress without Domain-Specific Mechanisms

State-of-the-art learning mechanisms for stress in Optimality Theory (see, e.g., Tesar and Smolensky 2000; Boersma and Pater 2016; Jarosz 2013) make use of probabilistic mechanisms that are domain-general in that they do not refer to the content of constraints and must not be in UG. By contrast, Pearl (2007, 2011) has argued that domain-general probabilistic learners of parametric grammars (Yang 2002) are insufficient for word stress, and, instead, domain-general learning mechanisms must be stipulated in UG alongside the parameters themselves. We propose a modification of Yang’s (2002) learner based on Jarosz’s (2015) learner for Optimality Theory: the Expectation Driven Parameter Learner, and show that this modification yields a dramatic improvement in accuracy (from 4.3% to 96%) on a representative typology generated by Dresher and Kaye’s (1990) parameter set. This suggests that domain-general learning mechanisms may be sufficient for learning stress after all, contra Pearl (2007, 2011), regardless of which grammatical representation (parameters or violable constraints) is a better reflection of the human language capacity.

TRACX2: a connectionist autoencoder using graded chunks to model infant visual statistical learning.

Even newborn infants are able to extract structure from a stream of sensory inputs; yet how this is achieved remains largely a mystery. We present a connectionist autoencoder model, TRACX2, that learns to extract sequence structure by gradually constructing chunks, storing these chunks in a distributed manner across its synaptic weights and recognizing these chunks when they re-occur in the input stream. Chunks are graded rather than all-or-nothing in nature. As chunks are learnt their component parts become more and more tightly bound together. TRACX2 successfully models the data from five experiments from the infant visual statistical learning literature, including tasks involving forward and backward transitional probabilities, low-salience embedded chunk items, part-sequences and illusory items. The model also captures performance differences across ages through the tuning of a single-learning rate parameter. These results suggest that infant statistical learning is underpinned by the same domain-general learning mechanism that operates in auditory statistical learning and, potentially, in adult artificial grammar learning.This article is part of the themed issue 'New frontiers for statistical learning in the cognitive sciences'.

Is statistical learning a mechanism?

Philosophers of science have offered several definitions of mechanism, most of which have biological or neuroscientific roots. In this paper, I consider whether these definitions apply equally well to cognitive science. I examine this question by looking at the case of statistical learning, which has been called a domain-general learning mechanism in the cognitive scientific literature. I argue that statistical learning does not constitute a mechanism in the philosophical sense of the term. This conclusion points to significant limitations in the scope of the mechanist philosophy when it comes to accounting for explanation in cognitive science.

Shifting the Prototype: Experience with Faces Influences Affective and Attractiveness Preferences

While some researchers have suggested that preferences for attractive faces are the result of a domain-specific beauty detection module, others argue these preferences develop based on averages of stimuli through a domain-general learning mechanism. We tested whether cognitive and perceptual mechanisms sensitive to experience underlie facial preferences by familiarizing participants with human, chimpanzee, or morphed faces (60%-chimp/40%-human). Results indicated that participants familiarized with human-chimp morphs showed greater zygomaticus major activity, a physiological correlate of positive affect (Study 1), and higher explicit attractiveness ratings (Study 2) to faces morphed to some degree with chimpanzees. These results demonstrate that experience shifts attractiveness preferences away from the normative average, and suggest that a domain-general cognitive mechanism better accounts for facial preferences than a domain-specific innate beauty-detector.

Integrating Physical Constraints in Statistical Inference by 11‐Month‐Old Infants

Much research on cognitive development focuses either on early-emerging domain-specific knowledge or domain-general learning mechanisms. However, little research examines how these sources of knowledge interact. Previous research suggests that young infants can make inferences from samples to populations (Xu & Garcia, 2008) and 11- to 12.5-month-old infants can integrate psychological and physical knowledge in probabilistic reasoning (Teglas, Girotto, Gonzalez, & Bonatti, 2007; Xu & Denison, 2009). Here, we ask whether infants can integrate a physical constraint of immobility into a statistical inference mechanism. Results from three experiments suggest that, first, infants were able to use domain-specific knowledge to override statistical information, reasoning that sometimes a physical constraint is more informative than probabilistic information. Second, we provide the first evidence that infants are capable of applying domain-specific knowledge in probabilistic reasoning by using a physical constraint to exclude one set of objects while computing probabilities over the remaining sets.

Maturation: For better or for worse?

Meisel has argued in favour of the Fundamental Difference Hypothesis (FDH), a version of the Critical Period Hypothesis, for L2 acquisition. Most particularly, he has proposed that maturational changes in domain-specific, language learning mechanisms are in evidence as young as four years of age. In a nutshell, the proposal is that for certain domains of morphosyntactic acquisition, if exposure to a language does not begin before 4;0, the developmental sequences will be different from (2)L1, and more like adult L2. Thus, even early child L2 (cL2) can be characterized as fundamentally different from (2)L1 because the configuration of internal learning mechanisms used to develop certain linguistic domains differs between these populations. The FDH, like all critical period hypotheses, casts maturation of the language making capacity (LMC to adopt the same terminology as Meisel) in a negative light. In other words, the system is not only changing, it is changing for the worse: what was once possible is no longer possible, such as mastery of a gender system or alleged error-free acquisition of gender. However, neuro-cognitive maturation need not be considered synonymous with a decline in the LMC in all respects. What I focus on in this commentary is the research evidence supporting a maturational perspective that points to how some domains of the system can change for the better. I would like to argue in agreement with Meisel that neuro-cognitive maturation could result in fundamental differences between cL2 and (2)L1 learners. But, at the same time, I would like to highlight reasons why “difference” should not be treated as a euphemism for “deficit”. Maturational proposals are, of course, not restricted to L2 acquisition, but have also been put forward for L1 acquisition (e.g. Rizzi 1993/1994, Wexler 1998, 2003). Given that in the first five years of life, infants/ children are undergoing extensive neuro-cognitive development, it seems only logical to consider that innate linguistic knowledge, together with domain-specific and domain-general learning mechanisms, might be un-

Neurocognitive basis of implicit learning of sequential structure and its relation to language processing.

The ability to learn and exploit environmental regularities is important for many aspects of skill learning, of which language may be a prime example. Much of such learning proceeds in an implicit fashion, that is, it occurs unintentionally and automatically and results in knowledge that is difficult to verbalize explicitly. An important research goal is to ascertain the underlying neurocognitive mechanisms of implicit learning abilities and understand its contribution to perception, language, and cognition more generally. In this article, we review recent work that investigates the extent to which implicit learning of sequential structure is mediated by stimulus-specific versus domain-general learning mechanisms. Although much of previous implicit learning research has emphasized its domain-general aspect, here we highlight behavioral work suggesting a modality-specific locus. Even so, our data also reveal that individual variability in implicit sequence learning skill correlates with performance on a task requiring sensitivity to the sequential context of spoken language, suggesting that implicit sequence learning to some extent is domain-general. Taking into consideration this behavioral work, in conjunction with recent imaging studies, we argue that implicit sequence learning and language processing are both complex, dynamic processes that partially share the same underlying neurocognitive mechanisms, specifically those that rely on the encoding and representation of phonological sequences.

Language Learning in Infancy: Does the Empirical Evidence Support a Domain Specific Language Acquisition Device?

Poverty of the Stimulus Arguments have convinced many linguists and philosophers of language that a domain specific language acquisition device (LAD) is necessary to account for language learning. Here we review empirical evidence that casts doubt on the necessity of this domain specific device. We suggest that more attention needs to be paid to the early stages of language acquisition. Many seemingly innate language-related abilities have to be learned over the course of several months. Further, the language input contains rich stochastic information that can be accessed by domain-general learning mechanisms. Computer simulation has shown how mechanisms that are not domain specific can exploit the information contained in language. We conclude that (i) Poverty of the Stimulus Arguments need to be conceptually clarified and (ii) more empirical research needs to be carried out before we can rule out that data driven general purpose mechanisms can account for language learning.

Nativism and Neurobiology: Representations, Representing, and the Continuum of Cognition

Whether or not the mind contains innately specified representations is highly contestable, especially in light of neurobiological evidence for the plasticity of the brain. In what follows, I provide an overview of the debate as it now stands and a discussion of the possibility, proposed by Clark (1998) and others, that representations need not be localized and are better understood as distributed systems. I then seek to tie the debate into a similar controversy surrounding the architecture of the mind. While advocates of modularity find arguments for innately specified and domain-specific representations palatable, as the thesis of innateness only strengthens their claims, favorers of a more domain-general learning mechanism are not convinced by arguments for innate specificity and instead insist that representations emerge or are learned. Rather than come down on one side of these issues, I propose, in the spirit of Cundall (2006), that cognition is more aptly conceived of as a continuum: the domains by which certain “representers” are constrained turn out to be innate, while many of the complex representations, in particular, higher-level social–cognitive representations, come from more general learning and development. Thus, the problem of reconciling nativism and neurobiology turns out to be a matter not of choosing one of two extremes, but instead, adopting an intermediary view.

U-Shaped Curves in Development: A PDP Approach

As the articles in this issue attest, U-shaped curves in development have stimulated a wide spectrum of research across disparate task domains and age groups and have provoked a variety of ideas about their origins and theoretical significance. In our view, the ubiquity of the general pattern suggests that U-shaped curves can arise from multiple factors, and that the various viewpoints represented herein may be useful for explaining some aspects of developmental change. In this spirit, we offer an additional way of thinking about such phenomena. Specifically, we suggest that U-shaped curves can arise within a domain-general learning mechanism as it slowly masters a domain characterized by statistical regularities and exceptions. This idea differs from those considered thus far, and may encompass many of the phenomena addressed by other views, three of which we outline briefly here. JOURNAL OF COGNITION AND DEVELOPMENT, 1(5), 137–145 Copyright © 2004, Lawrence Erlbaum Associates, Inc.

Emergentism, parsimony and the development of process models of language acquisition.

In their very balanced critique of emergentist approaches to language, Sabbagh & Gelman focus on two main issues. The first is the question of whether language acquisition can be explained by domain-general learning mechanisms. The second is the question of the degree of specification that emergentist models of language acquisition actually provide.