Non-adjacent Dependencies Research Articles

Impaired statistical learning of non-adjacent dependencies in adolescents with specific language impairment.

Being able to track dependencies between syntactic elements separated by other constituents is crucial for language acquisition and processing (e.g., in subject-noun/verb agreement). Although long assumed to require language-specific machinery, research on statistical learning has suggested that domain-general mechanisms may support the acquisition of non-adjacent dependencies. In this study, we investigated whether individuals with specific language impairment (SLI)—who have problems with long-distance dependencies in language—also have problems with statistical learning of non-adjacent relations. The results confirmed this hypothesis, indicating that statistical learning may subserve the acquisition and processing of long-distance dependencies in natural language.

Action at a distance: dependency sensitivity in a New World primate

Sensitivity to dependencies (correspondences between distant items) in sensory stimuli plays a crucial role in human music and language. Here, we show that squirrel monkeys (Saimiri sciureus) can detect abstract, non-adjacent dependencies in auditory stimuli. Monkeys discriminated between tone sequences containing a dependency and those lacking it, and generalized to previously unheard pitch classes and novel dependency distances. This constitutes the first pattern learning study where artificial stimuli were designed with the species' communication system in mind. These results suggest that the ability to recognize dependencies represents a capability that had already evolved in humans’ last common ancestor with squirrel monkeys, and perhaps before.

The development of nonadjacent dependency learning in natural and artificial languages.

Nonadjacent dependencies occur over one or more intervening units and require learners to track discontinuous sequential relationships. These discontinuous relationships are present at multiple levels in language (e.g., as seen in morphosyntactic dependencies and at the phonological level in vowel harmony). Experiments suggest that these dependencies are acquired using statistical learning mechanisms and that this learning is also affected by perceptual biases. Artificial and natural language studies have shown that infants are sensitive to these statistical regularities but there appear to be developmental constraints on learning. Developmental investigations have also examined how knowledge and processing of the intervening elements affect learning, and whether categories can be acquired using nonadjacent dependency information. WIREs Cogn Sci 2013, 4:511-522. doi: 10.1002/wcs.1244 CONFLICT OF INTEREST: The authors have declared no conflicts of interest for this article. For further resources related to this article, please visit the WIREs website.

Syntax processing properties of generic cortical circuits

Higher cognitive functioning is assumed to be largely symbolic/representational and compositional in nature. At various processing stages, from perceptual to motor, discrete structural elements with intricate temporal dependencies are combined into increasingly complex constructs [1]. Mapping such complex computational processes to the underlying neuronal infrastructure and assessing the properties of the neuronal system responsible for their implementation is not straightforward, but it is likely to yield important insights into the nature of neural computation. In order to address these issues, we adopt ideas and formalisms developed by theoretical linguistics to study the nature of rule-like or compositional behavior in the language domain, namely the acquisition of formal (artificial) grammars. The Artificial Grammar Learning (AGL) paradigm has a long tradition in psycholinguistic research (see, e.g. [2] for an overview), as a means to study the nature of syntactic processing and implicit sequence learning. With mere exposure and without performance feedback, human beings implicitly acquire knowledge about the structural regularities implemented by complex rule systems. In this work, we investigate to which extent generic cortical microcircuits can support formally explicit symbolic computations, instantiated by the same grammars used in the human AGL literature and implementing various types of local and non-adjacent dependencies between the sequence elements, thus requiring varying degrees of computational complexity and online processing memory to be adequately learned. We use concrete implementations of input-driven recurrent networks composed of noisy, spiking neurons, built according to the reservoir computing framework and dynamically shaped by a variety of synaptic and intrinsic plasticity mechanisms operating concomitantly [3]. Additionally, we compare supervised and unsupervised learning rules for the decoding algorithms, with varying degrees of biological plausibility. We show that, when shaped by plasticity, these models are capable of acquiring the structure of simple (regular) grammars. When asked to judge string legality (in a manner similar to human subjects), the networks perform at a qualitatively comparable level. We uncover which plasticity mechanisms are crucial for the task, with the aim of specifying a minimal model. Furthermore, the capability of the networks to process (bounded) recursive constructions including multiple patterns of non-adjacent dependencies accurately reflects recent results of human performance, highlighting inherent limitations imposed by the nature of neuronal processing.

All together now: concurrent learning of multiple structures in an artificial language.

Natural languages contain many layers of sequential structure, from the distribution of phonemes within words to the distribution of phrases within utterances. However, most research modeling language acquisition using artificial languages has focused on only one type of distributional structure at a time. In two experiments, we investigated adult learning of an artificial language that contains dependencies between both adjacent and non-adjacent words. We found that learners rapidly acquired both types of regularities and that the strength of the adjacent statistics influenced learning of both adjacent and non-adjacent dependencies. Additionally, though accuracy was similar for both types of structure, participants' knowledge of the deterministic non-adjacent dependencies was more explicit than their knowledge of the probabilistic adjacent dependencies. The results are discussed in the context of current theories of statistical learning and language acquisition.

Finding DNA Regulatory Motifs with Position-dependent Models

We consider the problem of de novo DNA motif discovery. The position weight matrix (PWM) model has been extensively used, yet this model makes the assumption that nucleotides at different positions are independent of each other. Recent results have shown that nucleotides bound by transcription factors often exhibit adjacent or nonadjacent dependencies. We address this problem by devising positional dependency models capable of capturing adjacent dependencies and non-adjacent dependencies (SPWDM). Our algorithms are based on Gibbs sampling to update the model parameter and dependencies structure. We compare two scoring functions:   -score and a conditional probability based score. We also improve several Gibbs sampling stages. Experiments are carried out on simulated and real data, showing that the SPWDM model makes improvement over pure PWM. The modifications to the Gibbs sampling algorithm are also shown to be effective. 

Effects of Prior Phonotactic Knowledge on Infant Word Segmentation: The Case of Nonadjacent Dependencies

In this study, the authors explored whether French-learning infants use nonadjacent phonotactic regularities in their native language, which they learn between the ages of 7 and 10 months, to segment words from fluent speech. Two groups of 20 French-learning infants were tested using the head-turn preference procedure at 10 and 13 months of age. In Experiment 1, infants were familiarized with 2 passages: 1 containing a target word with a frequent nonadjacent phonotactic structure and the other containing a target word with an infrequent nonadjacent phonotactic structure in French. During the test phase, infants were presented with 4 word lists: 2 containing the target words presented during familiarization and 2 other control words with the same phonotactic structure. In Experiment 2, the authors retested infants' ability to segment words with the infrequent phonotactic structure. Ten- and 13-month-olds were able to segment words with the frequent phonotactic structure, but it is only by 13 months, and only under the circumstances of Experiment 2, that infants could segment words with the infrequent phonotactic structure. These results provide new evidence showing that infant word segmentation is influenced by prior nonadjacent phonotactic knowledge.

Non-adjacent dependency learning in infants at familial risk of dyslexia

This study tests the hypothesis that developmental dyslexia is (partly) caused by a deficit in implicit sequential learning, by investigating whether infants at familial risk of dyslexia can track non-adjacent dependencies in an artificial language. An implicit learning deficit would hinder detection of such dependencies, which mark grammatical relations (e.g. between 'is' and '-ing' in 'she is happily singing'). In a head-turn experiment with infants aged 1;6, family risk and typically developing infants were exposed to one of two novel languages containing dependencies of the type a-X-c, b-X-d or a-X-d, b-X-c, with fixed first and third elements and twenty-four different X elements. During test, typically developing children listened longer to ungrammatical strings (i.e. that did not correspond to their training language). However, family-risk children did not discriminate between grammatical and ungrammatical strings, indicating deficient implicit learning. The implications of these findings in relation to dyslexia and other language-based disorders are discussed.

Statistical learning of probabilistic nonadjacent dependencies by multiple-cue integration

Previous studies have indicated that dependencies between nonadjacent elements can be acquired by statistical learning when each element predicts only one other element (deterministic dependencies). The present study investigates statistical learning of probabilistic nonadjacent dependencies, in which each element predicts several other elements with a certain probability, as is more common in natural language. Three artificial language learning experiments compared statistical learning of deterministic and probabilistic nonadjacent dependencies. In Experiment 1, participants listened to sequences of three non-words containing either deterministic or probabilistic dependencies between the first and the last non-words. Participants exposed to deterministic dependencies subsequently distinguished correct sequences from sequences that violated the nonadjacent dependencies; participants exposed to probabilistic dependencies did not. However, when visual (Experiment 2) and phonological cues (Experiment 3) were added participants learned both kinds of dependencies, demonstrating statistical learning of probabilistic nonadjacent dependencies when additional cues are present.

Processing multiple non-adjacent dependencies: evidence from sequence learning

Processing non-adjacent dependencies is considered to be one of the hallmarks of human language. Assuming that sequence-learning tasks provide a useful way to tap natural-language-processing mechanisms, we cross-modally combined serial reaction time and artificial-grammar learning paradigms to investigate the processing of multiple nested (A(1)A(2)A(3)B(3)B(2)B(1)) and crossed dependencies (A(1)A(2)A(3)B(1)B(2)B(3)), containing either three or two dependencies. Both reaction times and prediction errors highlighted problems with processing the middle dependency in nested structures (A(1)A(2)A(3)B(3)_B(1)), reminiscent of the 'missing-verb effect' observed in English and French, but not with crossed structures (A(1)A(2)A(3)B(1)_B(3)). Prior linguistic experience did not play a major role: native speakers of German and Dutch-which permit nested and crossed dependencies, respectively-showed a similar pattern of results for sequences with three dependencies. As for sequences with two dependencies, reaction times and prediction errors were similar for both nested and crossed dependencies. The results suggest that constraints on the processing of multiple non-adjacent dependencies are determined by the specific ordering of the non-adjacent dependencies (i.e. nested or crossed), as well as the number of non-adjacent dependencies to be resolved (i.e. two or three). Furthermore, these constraints may not be specific to language but instead derive from limitations on structured sequence learning.

A rostro-caudal gradient of structured sequence processing in the left inferior frontal gyrus

In this paper, we present two novel perspectives on the function of the left inferior frontal gyrus (LIFG). First, a structured sequence processing perspective facilitates the search for functional segregation within the LIFG and provides a way to express common aspects across cognitive domains including language, music and action. Converging evidence from functional magnetic resonance imaging and transcranial magnetic stimulation studies suggests that the LIFG is engaged in sequential processing in artificial grammar learning, independently of particular stimulus features of the elements (whether letters, syllables or shapes are used to build up sequences). The LIFG has been repeatedly linked to processing of artificial grammars across all different grammars tested, whether they include non-adjacent dependencies or mere adjacent dependencies. Second, we apply the sequence processing perspective to understand how the functional segregation of semantics, syntax and phonology in the LIFG can be integrated in the general organization of the lateral prefrontal cortex (PFC). Recently, it was proposed that the functional organization of the lateral PFC follows a rostro-caudal gradient, such that more abstract processing in cognitive control is subserved by more rostral regions of the lateral PFC. We explore the literature from the viewpoint that functional segregation within the LIFG can be embedded in a general rostro-caudal abstraction gradient in the lateral PFC. If the lateral PFC follows a rostro-caudal abstraction gradient, then this predicts that the LIFG follows the same principles, but this prediction has not yet been tested or explored in the LIFG literature. Integration might provide further insights into the functional architecture of the LIFG and the lateral PFC.

Implicit Acquisition of Grammars With Crossed and Nested Non‐Adjacent Dependencies: Investigating the Push‐Down Stack Model

A recent hypothesis in empirical brain research on language is that the fundamental difference between animal and human communication systems is captured by the distinction between finite-state and more complex phrase-structure grammars, such as context-free and context-sensitive grammars. However, the relevance of this distinction for the study of language as a neurobiological system has been questioned and it has been suggested that a more relevant and partly analogous distinction is that between non-adjacent and adjacent dependencies. Online memory resources are central to the processing of non-adjacent dependencies as information has to be maintained across intervening material. One proposal is that an external memory device in the form of a limited push-down stack is used to process non-adjacent dependencies. We tested this hypothesis in an artificial grammar learning paradigm where subjects acquired non-adjacent dependencies implicitly. Generally, we found no qualitative differences between the acquisition of non-adjacent dependencies and adjacent dependencies. This suggests that although the acquisition of non-adjacent dependencies requires more exposure to the acquisition material, it utilizes the same mechanisms used for acquiring adjacent dependencies. We challenge the push-down stack model further by testing its processing predictions for nested and crossed multiple non-adjacent dependencies. The push-down stack model is partly supported by the results, and we suggest that stack-like properties are some among many natural properties characterizing the underlying neurophysiological mechanisms that implement the online memory resources used in language and structured sequence processing.

Statistical segmentation of streams of syllables: a pilot EEG study

Statistical segmentation of streams of syllables: a pilot EEG study

Tier-adjacency is not a necessary condition for learning phonotactic dependencies

One hypothesis raised by Newport and Aslin to explain how speakers learn dependencies between nonadjacent phonemes is that speakers track bigram probabilities between two segments that are adjacent to each other within a tier of their own. The hypothesis predicts that a dependency between segments separated from each other at the tier level cannot be learned. Were this true, it could add psycholinguistic plausibility to phonological theories that limit possible nonadjacent dependencies to those between two tier-adjacent segments. Contrary to the prediction, the experiments in this paper show that adults can learn dependencies between two consonants that are nonadjacent to each other at the tier level as well as the segmental level. The results suggest that the range of learnable sound patterns may be wider than the range of attested sound patterns and that additional learning mechanisms should be considered to explain how speakers learn nonadjacent phonotactic dependencies.

Acquisition of Nonadjacent Phonological Dependencies in the Native Language During the First Year of Life.

Languages instantiate many different kinds of dependencies, some holding between adjacent elements and others holding between nonadjacent elements. In the domain of phonology-phonotactics, sensitivity to adjacent dependencies has been found to appear between 6 and 10 months. However, no study has directly established the emergence of sensitivity to nonadjacent phonological dependencies in the native language. The present study focuses on the emergence of a perceptual Labial-Coronal (LC) bias, a dependency involving two nonadjacent consonants. First, Experiment 1 shows that a preference for monosyllabic consonant-vowel-consonant LC words over CL (Coronal-Labial) words emerges between 7 and 10 months in French-learning infants. Second, two experiments, presenting only the first or last two phonemes of the original stimuli, establish that the LC bias at 10 months cannot be explained by adjacent dependencies or by a preference for more frequent coronal consonants (Experiment 2a & b). At 7 months, by contrast, infants appear to react to the higher frequency of coronal consonants (Experiment 3a & b). The present study thus demonstrates that infants become sensitive to nonadjacent phonological dependencies between 7 and 10 months. It further establishes a change between these two ages from sensitivity to local properties to nonadjacent dependencies in the phonological domain.

Comprehension of Linguistic Dependencies: Speed‐Accuracy Tradeoff Evidence for Direct‐Access Retrieval From Memory

Comprehenders can rapidly and efficiently interpret expressions with various types of non-adjacent dependencies. In the sentence The boy that the teacher warned fell, boy is readily interpreted as the subject of the verb fall despite the fact that a relative clause, that the teacher warned, intervenes between the two dependent elements. We review research investigating three memory operations proposed for resolving this and other types of non-adjacent dependencies: serial search retrieval, in which the dependent constituent is recovered by a search process through representations in memory, direct-access retrieval in which the dependent constituent is recovered directly by retrieval cue operations without search, and active maintenance of the dependent constituent in focal attention. Studies using speed-accuracy tradeoff methodology to examine the full timecourse of interpreting a wide range of non-adjacent dependencies indicate that comprehenders retrieve dependent constituents with a direct-access operation, consistent with the claim that representations formed during comprehension are accessed with a cue-driven, content-addressable retrieval process. The observed timecourse profiles are inconsistent with a broad class of models based on several search operations for retrieval. The profiles are also inconsistent with active maintenance of a constituent while concurrently processing subsequent material, and suggest that, with few exceptions, direct-access retrieval is required to process non-adjacent dependencies.

Can prosody be used to discover hierarchical structure in continuous speech?

We tested whether adult listeners can simultaneously keep track of variations in pitch and syllable duration in order to segment continuous speech into phrases and group these phrases into sentences. The speech stream was constructed so that prosodic cues signaled hierarchical structures (i.e., phrases embedded within sentences) and non-adjacent relations (i.e., AxB rules within phrases), while transitional probabilities between syllables favored adjacent dependencies that straddled phrase and sentence boundaries. In Experiments 1 and 2, participants hierarchically segmented the stream and learned the grammar used to generate the phrases when prosodic cues were consistent with their native language. In Experiment 3, participants segmented the stream based on transitional probabilities when no prosodic cues were present and all syllables had the same pitch and duration. In Experiment 4, participants were able to exploit non-native prosody in order to learn hierarchical relations and non-adjacent dependencies. These results suggest that prosodic cues such as pitch declination and final lengthening provide a stronger basis for learning than transitional probabilities even when both are unfamiliar and not wholly consistent with native language informational structure.

Neurophysiology of Hungarian subject–verb dependencies with varying intervening complexity

Non-adjacent dependencies are thought to be more costly to process than sentences wherein dependents immediately follow or precede what they depend on. In English locality effects have been revealed, while in languages with rich case marking (German and Hindi) sentence final structures show anti-locality-effects. The motivation of the current study is to test whether locality effects can be directly applied to a typologically different language than those investigated so far. Hungarian is a “topic prominent” language; it permits a variation of possible word sequencing for semantic reasons, including SVO word order. Hungarian also has a rich morphological system (e.g., rich case system) and postpositions to indicate grammatical functions. In the present ERP study, Hungarian subject–verb dependencies were compared by manipulating the mismatch of number agreement between the sentence's initial noun phrase and the sentence's final intransitive verb as well as the complexity of the intervening sentence material, interrupting the dependencies. Possible lexical class and frequency or cloze-probability effects for the first two words of the intervening sentence material were revealed when used separate baseline for each word, while at the third word of the intervening material as well as at the main verb ERPs were not modulated by complexity but at the verb ERPs were enhanced by grammaticality. Ungrammatical sentences enlarged the amplitude of both LAN and P600 components at the main verb. These results are in line with studies suggesting that the retrieval of the first element of a dependency is not influenced by distance from the second element, as the first element is directly accessible when needed for integration (e.g., McElree, 2000).

Learning Recursion: Multiple Nested and Crossed Dependencies

Language acquisition in both natural and artificial language learning settings crucially depends on extracting information from ordered sequences. A shared sequence learning mechanism is thus assumed to underlie both natural and artificial language learning. A growing body of empirical evidence is consistent with this hypothesis. By means of artificial language learning experiments, we may therefore gain more insight in this shared mechanism. In this paper, we review empirical evidence from artificial language learning and computational modeling studies, as well as natural language data, and suggest that there are two key factors that help deter-mine processing complexity in sequence learning, and thus in natural language processing. We propose that the specific ordering of non-adjacent dependencies (i.e. nested or crossed), as well as the number of non-adjacent dependencies to be resolved simultaneously (i.e. two or three) are important factors in gaining more insight into the boundaries of human sequence learning; and thus, also in natural language processing. The implications for theories of linguistic competence are discussed.

Tracking irregular morphophonological dependencies in natural language: Evidence from the acquisition of subject-verb agreement in French

This study examines French-learning infants’ sensitivity to grammatical non-adjacent dependencies involving subject-verb agreement (e.g., le/les garçons lit/lisent ‘the boy(s) read(s)’) where number is audible on both the determiner of the subject DP and the agreeing verb, and the dependency is spanning across two syntactic phrases. A further particularity of this subsystem of French subject-verb agreement is that number marking on the verb is phonologically highly irregular. Despite the challenge, the HPP results for 24- and 18-month-olds demonstrate knowledge of both number dependencies: between the singular determiner le and the non-adjacent singular verbal forms and between the plural determiner les and the non-adjacent plural verbal forms. A control experiment suggests that the infants are responding to known verb forms, not phonological regularities. Given the paucity of such forms in the adult input documented through a corpus study, these results are interpreted as evidence that 18-month-olds have the ability to extract complex patterns across a range of morphophonologically inconsistent and infrequent items in natural language.