Online Language Comprehension Research Articles

Five-year-olds’ sensitivity to knowledge discrepancies about object identity during online language comprehension

In everyday communication, children experience situations where their knowledge or perspectives differ from those of their communicative partner. The current study examined this issue in the context of real-time language comprehension, focusing on 5-year-old children’s ability to manage knowledge discrepancies about the identity of mutually visible objects. In Experiment 1, we examined 5-year-olds’ ability to manage privileged knowledge about an object’s identity. Using a referential communication task, we tested children (N = 60) in either a shared knowledge condition, where both the child and the speaker knew the identity of a visually misleading object (e.g., a candle that looks like an apple), or a privileged knowledge condition, where only the child knew the identity of the visually misleading object. Of interest was whether children could suppress private knowledge while processing a phonologically related word (e.g., “Look at the candy”). Results showed that children did not inhibit this knowledge during the early moments of referential interpretation. In Experiment 2 (N = 30), we contrasted the privileged knowledge condition in Experiment 1 with the more traditional scenario used to test common ground use, where the child knows the speaker cannot see certain display objects. Results confirmed a stronger ability to manage discrepancies in the latter case. Together, the findings demonstrate differences in children’s ability to manage distinct types of knowledge discrepancies during real-time language comprehension.

Preschoolers modulate contrastive inferences during online language comprehension.

This study examined 4- and 5-year-olds' incremental interpretation of size adjectives, focusing on whether contrastive inferences are modulated by speaker behavior. Children (N=120, 59 females, mostly White, tested between July, 2018 and August, 2019) encountered either a conventional or unconventional speaker who labeled objects in a correspondingly typical or atypical way. Critical utterances contained size adjectives (e.g., "Look at the big duck"). With conventional speakers, gaze measures indicated that children rapidly used the adjective to differentiate members of a contrasting pair, indicating that even 4-year-olds derive contrastive inferences. With unconventional speakers, contrastive inferences were delayed in processing. The findings demonstrate that preschoolers adjust their use of pragmatic cues when presented with evidence disconfirming their default assumptions about a speaker.

Effects of world knowledge on the prediction of upcoming verbs: an eye-tracking study

Some theories of sentence processing make a distinction between two kinds of meaning: a linguistic meaning encoded at the lexicon (i.e., selectional restrictions), and an extralinguistic knowledge derived from our everyday experiences (i.e., world knowledge). According to such theories, the former meaning is privileged over the latter in terms of the time-course of its access and influence during on-line language comprehension. The present study aims to examine whether world knowledge anomalies (that do not violate selectional restrictions) are rapidly detected during online sentence processing. In an eye-tracking experiment, we used materials in which the likelihood of a specific verb (entrevistar or secuestrar, the Spanish translations for to interview and to kidnap) depended on the agent of the event (periodista or terrorista, the Spanish translations for journalist and terrorist). The results showed an effect of typicality in regression path duration and total reading times at both the verb region and the spillover region, thus providing evidence that world knowledge is rapidly accessed and used during on-line sentence comprehension.

Referential Chains Reveal Predictive Processes and Form-to-Function Mapping: An Electroencephalographic Study Using Naturalistic Story Stimuli.

In discourse pragmatics, different referential forms are claimed to be indicative of the cognitive status of a referent in the current discourse. Referential expressions thereby possess a double function: They point back to an (existing) referent (form-to-function mapping), and they are used to derive predictions about a referent’s subsequent recurrence in discourse. Existing event-related potential (ERP) research has mainly focused on the form-to-function mapping of referential expression. In the present ERP study, we explore the relationship of form-to-function mapping and prediction derived from the antecedent of referential expressions in naturalistic auditory language comprehension. Specifically, the study investigates the relationship between the form of a referential expression (pronoun vs. noun) and the form of its antecedent (pronoun vs. noun); i.e., it examines the influence of the interplay of predictions derived from an antecedent (forward-looking function) and the form-to-function mapping of an anaphor (backward-looking function) on the ERPs time-locked to anaphoric expressions. The results in the time range of the P300 and N400 allow for a dissociation of these two functions during online language comprehension.

Online pragmatic interpretations of scalar adjectives are affected by perceived speaker reliability.

Linguistic communication requires understanding of words in relation to their context. Among various aspects of context, one that has received relatively little attention until recently is the speakers themselves. We asked whether comprehenders' online language comprehension is affected by the perceived reliability with which a speaker formulates pragmatically well-formed utterances. In two eye-tracking experiments, we conceptually replicated and extended a seminal work by Grodner and Sedivy (2011). A between-participant manipulation was used to control reliability with which a speaker follows implicit pragmatic conventions (e.g., using a scalar adjective in accordance with contextual contrast). Experiment 1 replicated Grodner and Sedivy's finding that contrastive inference in response to scalar adjectives was suspended when both the spoken input and the instructions provided evidence of the speaker's (un)reliability: For speech from the reliable speaker, comprehenders exhibited the early fixations attributable to a contextually-situated, contrastive interpretation of a scalar adjective. In contrast, for speech from the unreliable speaker, comprehenders did not exhibit such early fixations. Experiment 2 provided novel evidence of the reliability effect in the absence of explicit instructions. In both experiments, the effects emerged in the earliest expected time window given the stimuli sentence structure. The results suggest that real-time interpretations of spoken language are optimized in the context of a speaker identity, characteristics of which are extrapolated across utterances.

The activation of object-state representations during online language comprehension

Understanding the time-course of event knowledge activation is crucial for theories of language comprehension. We report two experiments using the ‘visual world paradigm’ (VWP) that investigated the dynamic mapping between object-state representations and real-time language processing. In Experiment 1, participants heard sentences that described events resulting in either a substantial change of state (e.g. The chef will chop the onion) or a minimal change of state (e.g. The chef will weigh the onion). Concurrently, they viewed pictures depicting two versions of the target object (e.g., an onion) corresponding to the intact and changed states, and two unrelated distractors. A second sentence referred to the object with either a backward or a forward shift in event time (e.g. But first/And then, he will smell the onion). In Experiment 2, Degree of Change was manipulated by using different nouns in the first sentence (e.g. The girl will stomp on thepenny/egg). The second sentence was similar to the ones used in Experiment 1 (e.g., But first/And then, she will look atthe penny/egg). The results from both experiments showed that participants looked more at the ‘appropriate’ state of the object that matched the language context, but the shift of visual attention emerged only when the object name was heard. Our findings suggest that situationally appropriate object representations do trigger eye movements to the corresponding states of the target object, but inappropriate representations are not necessarily eliminated from consideration until the language forces it.

Adult Age Differences in the Use of Conceptual Combination as an Associative Encoding Strategy.

It is well-established that aging impairs memory for associations more than it does memory for single items. Aging also impacts processes involved in online language comprehension, including the ability to form integrated, message-level representations. These changes in comprehension processes could impact older adults’ associative memory performance, perhaps by reducing or altering the effectiveness of encoding strategies that encourage semantic integration. The present study examined age differences in the use of a strategy termed conceptual combination, which involves integrating two words (e.g., “winter” and “salad”) into a single concept (“a salad for winter”). We recorded ERPs while participants studied unrelated noun pairs using a strategy that either did or did not encourage conceptual combination. We also varied the concreteness of the first noun in each pair in order to measure compositional concreteness effects, or ERP differences at the second noun due to the concreteness of the first noun. At the first nouns, older adults showed word-level concreteness effects that were similar to those of younger adults. However, compositional concreteness effects were diminished in older adults, consistent with reduced semantic integration. Older adults’ associative memory performance was better for word pairs studied during the conceptual combination task versus the non-combinatory encoding task; however, the magnitude of the age-related associative memory deficit did not differ between tasks. Finally, analyses of both memory accuracy and trial-by-trial ratings of perceived combination success suggested that older adults had disproportionate difficulty applying the conceptual combination strategy to word pairs that began with abstract nouns. Overall, these results indicate that changes to integrative language processing that occur with age are not independent of – and may sometimes exacerbate – age-related memory decline.

Word recall is affected by surrounding metrical context

ABSTRACTIt has been claimed that English has a metrical structure, or rhythm, in which stressed and unstressed syllables alternate. In previous research regular, alternating patterns have been shown to facilitate online language comprehension. Expanding these findings to downstream processing would lead to the prediction that metrical regularity enhances memory. Research from the memory literature, however, indicates that regular patterns are less salient and therefore less well remembered, and strings of similar sounds are harder to remember. This work suggests that, like lists of words with similar sounds, lists of words with similar metrical patterns would be less accurately remembered than comparable metrically irregular patterns. This study investigates these conflicting predictions by examining metrical regularity in a recall task. We find that words are better recalled when they do not match their metrical context, suggesting a regular metrical structure may not be beneficial in all contexts.

Information Integration in Modulation of Pragmatic Inferences During Online Language Comprehension

Upon hearing a scalar adjective in a definite referring expression such as "the big…," listeners typically make anticipatory eye movements to an item in a contrast set, such as a big glass in the context of a smaller glass. Recent studies have suggested that this rapid, contrastive interpretation of scalar adjectives is malleable and calibrated to the speaker's pragmatic competence. In a series of eye-tracking experiments, we explore the nature of the evidence necessary for the modulation of pragmatic inferences in language comprehension, focusing on the complementary roles of top-down information-(knowledge about the particular speaker's pragmatic competence)and bottom-up cues(distributional information about the use of scalar adjectives in the environment). We find that bottom-up evidence alone (e.g., the speaker says "the big dog" in a context with one dog), in large quantities, can be sufficient to trigger modulation of the listener's contrastive inferences, with or without top-down cues to support this adaptation. Further, these findings suggest that listeners track and flexibly combine multiple sources of information in service of efficient pragmatic communication.

Counting ‘uhm’s: How tracking the distribution of native and non-native disfluencies influences online language comprehension

Abstract Disfluencies, like uh, have been shown to help listeners anticipate reference to low-frequency words. The associative account of this ‘disfluency bias’ proposes that listeners learn to associate disfluency with low-frequency referents based on prior exposure to non-arbitrary disfluency distributions (i.e., greater probability of low-frequency words after disfluencies). However, there is limited evidence for listeners actually tracking disfluency distributions online. The present experiments are the first to show that adult listeners, exposed to a typical or more atypical disfluency distribution (i.e., hearing a talker unexpectedly say uh before high-frequency words), flexibly adjust their predictive strategies to the disfluency distribution at hand (e.g., learn to predict high-frequency referents after disfluency). However, when listeners were presented with the same atypical disfluency distribution but produced by a non-native speaker, no adjustment was observed. This suggests pragmatic inferences can modulate distributional learning, revealing the flexibility of, and constraints on, distributional learning in incremental language comprehension.

Tracking Affective Language Comprehension: Simulating and Evaluating Character Affect in Morally Loaded Narratives.

Facial electromyography research shows that corrugator supercilii (“frowning muscle”) activity tracks the emotional valence of linguistic stimuli. Grounded or embodied accounts of language processing take such activity to reflect the simulation or “re-enactment” of emotion, as part of the retrieval of word meaning (e.g., of “furious”) and/or of building a situation model (e.g., for “Mark is furious”). However, the same muscle also expresses our primary emotional evaluation of things we encounter. Language-driven affective simulation can easily be at odds with the reader’s affective evaluation of what language describes (e.g., when we like Mark being furious). In a previous experiment (‘t Hart et al., 2018) we demonstrated that neither language-driven simulation nor affective evaluation alone seem sufficient to explain the corrugator patterns that emerge during online language comprehension in these complex cases. Those results showed support for a multiple-drivers account of corrugator activity, where both simulation and evaluation processes contribute to the activation patterns observed in the corrugator. The study at hand replicates and extends these findings. With more refined control over when precisely affective information became available in a narrative, we again find results that speak against an interpretation of corrugator activity in terms of simulation or evaluation alone, and as such support the multiple-drivers account. Additional evidence suggests that the simulation driver involved reflects simulation at the level of situation model construction, rather than at the level of retrieving concepts from long-term memory. In all, by giving insights into how language-driven simulation meshes with the reader’s evaluative responses during an unfolding narrative, this study contributes to the understanding of affective language comprehension.

The life and times of focus alternatives: Tracing the activation of alternatives to a focused constituent in language comprehension

AbstractThe function of focus is to activate a set of alternatives, providing the locus for focus‐sensitive particles like only. In the past decade, psycholinguistic research has shown that listeners entertain a set of alternatives in online language comprehension, similar to the algorithm stipulated by Alternative Semantics. The purpose of the present review is to gain a comprehensive picture of the role of focus alternatives in utterance comprehension and interpretation. Specifically, we focus on how the processing of focus particles interacts with alternatives activated by focus. We show that focus marking activates a network of related concepts, but over time, only those that can be considered as focus alternatives in the relevant context of the utterance are retained in the mental representation of the discourse. Focus particles, in turn, increase the competition between the focused element and its alternatives during the initial stages of comprehension, helping contextual restriction of the set of alternatives. Overall, the studies presented in this review show that focus does not only guide a listener in determining which alternatives are relevant for the purpose of conversation, it also plays a fundamental role in the memory representation of a discourse.

The aging factor in presupposition processing

This work takes a first step towards a psycholinguistic investigation on the role of aging in processing presuppositions. Experimental data collected support two main results. First, in online language comprehension older adults exhibit higher processing costs when presuppositions involve demanding mental representations. Second, the ability to recover information introduced in the discourse as taken for granted is affected by the age factor. Overall, beyond other more explored levels of pragmatic processing, the decline with age of processing speed and working memory ability seems to affect presuppositions processing.

Why wait for the verb? Turkish speaking children use case markers for incremental language comprehension

During language comprehension we must rapidly determine the thematic roles of arguments (who did what to whom) in order to semantically integrate the players into a single event and predict upcoming structure. While some languages signal these relations mostly with reliable word order, others rely more on case markers. The present study explores whether Turkish-speaking children use case marking predictively during online language comprehension. Specifically, we use the visual-world paradigm to test whether 4-year-olds (and adults) can use a contrast between nominative and accusative case on the first noun to predict the referent of the second noun in verb-medial and verb-final spoken sentences. In verb-medial sentences, both children and adults used case to predict the upcoming noun, but children did so only after hearing the verb. In verb-final structures, however, both children and adults made predictive looks to the correct referent prior to the second noun (and the verb). Thus, Turkish-speaking preschoolers interpret case marking incrementally, independent of the verb, and use it to anticipate the upcoming argument. These findings are inconsistent with the hypothesis that the online interpretation of case marking depends on a late maturing neural circuit. The predictive use of case at four provides strong evidence that children's comprehension relies on broad, abstract mappings between syntax and semantics, which allow children to determine the event role of a case marked argument prior to identifying the verb.

What do you know? ERP evidence for immediate use of common ground during online reference resolution

Recent evidence on the time-course of conversational perspective taking is mixed. Some results suggest that listeners rapidly incorporate an interlocutor’s knowledge during comprehension, while other findings suggest that listeners initially interpret language egocentrically. A key finding in support of the egocentric view comes from visual-world eye-tracking studies — listeners systematically look at potential referents that are known to them but unknown to the speaker. An alternative explanation is that eye movements might be driven by attentional processes that are unrelated to referent identification. To address this question, we assessed the time-course of perspective taking using event-related potentials (ERP). Participants were instructed to select a referent from a display of four animals (e.g., “Click on the brontosaurus with the boots”) by a speaker who could only see three of the animals. A competitor (e.g., a brontosaurus with a purse) was either mutually visible, visible only to the listener, or absent from the display. Results showed that only the mutually visible competitor elicited an ERP signature of referential ambiguity. Critically, ERPs exhibited no evidence of referential confusion when the listener had privileged access to the competitor. Contra the egocentric hypothesis, this pattern of results indicates that listeners did not consider privileged competitors to be candidates for reference. These findings are consistent with theories of language processing that allow socio-pragmatic information to rapidly influence online language comprehension. The results also suggest that eye-tracking evidence in studies of online reference resolution may include distraction effects driven by privileged competitors and highlight the importance of using multiple measures to investigate perspective use.

The Temporal Prediction of Stress in Speech and Its Relation to Musical Beat Perception.

While rhythmic expectancies are thought to be at the base of beat perception in music, the extent to which stress patterns in speech are similarly represented and predicted during on-line language comprehension is debated. The temporal prediction of stress may be advantageous to speech processing, as stress patterns aid segmentation and mark new information in utterances. However, while linguistic stress patterns may be organized into hierarchical metrical structures similarly to musical meter, they do not typically present the same degree of periodicity. We review the theoretical background for the idea that stress patterns are predicted and address the following questions: First, what is the evidence that listeners can predict the temporal location of stress based on preceding rhythm? If they can, is it thanks to neural entrainment mechanisms similar to those utilized for musical beat perception? And lastly, what linguistic factors other than rhythm may account for the prediction of stress in natural speech? We conclude that while expectancies based on the periodic presentation of stresses are at play in some of the current literature, other processes are likely to affect the prediction of stress in more naturalistic, less isochronous speech. Specifically, aspects of prosody other than amplitude changes (e.g., intonation) as well as lexical, syntactic and information structural constraints on the realization of stress may all contribute to the probabilistic expectation of stress in speech.

Hearing and seeing meaning in noise: Alpha, beta, and gamma oscillations predict gestural enhancement of degraded speech comprehension.

During face‐to‐face communication, listeners integrate speech with gestures. The semantic information conveyed by iconic gestures (e.g., a drinking gesture) can aid speech comprehension in adverse listening conditions. In this magnetoencephalography (MEG) study, we investigated the spatiotemporal neural oscillatory activity associated with gestural enhancement of degraded speech comprehension. Participants watched videos of an actress uttering clear or degraded speech, accompanied by a gesture or not and completed a cued‐recall task after watching every video. When gestures semantically disambiguated degraded speech comprehension, an alpha and beta power suppression and a gamma power increase revealed engagement and active processing in the hand‐area of the motor cortex, the extended language network (LIFG/pSTS/STG/MTG), medial temporal lobe, and occipital regions. These observed low‐ and high‐frequency oscillatory modulations in these areas support general unification, integration and lexical access processes during online language comprehension, and simulation of and increased visual attention to manual gestures over time. All individual oscillatory power modulations associated with gestural enhancement of degraded speech comprehension predicted a listener's correct disambiguation of the degraded verb after watching the videos. Our results thus go beyond the previously proposed role of oscillatory dynamics in unimodal degraded speech comprehension and provide first evidence for the role of low‐ and high‐frequency oscillations in predicting the integration of auditory and visual information at a semantic level.

The Aging Factor in Presupposition Processing

The Aging Factor in Presupposition Processing

“I know something you don’t know”: Discourse and social context effects on the N400 in adolescents

Adolescence is a time of great cognitive and social development. Despite this, relatively few studies to date have investigated how perspective taking affects on-line language comprehension in adolescents. In the current study, we addressed this gap in the literature, making use of a Joint Comprehension Task in which two individuals with differing background knowledge jointly attend to linguistic stimuli. Using event-related potentials, we investigated adolescents’ electrophysiological responses to (a) semantically anomalous sentence stimuli in discourse context and (b) semantically plausible sentence stimuli that the participants believe another individual finds semantically implausible. Our results demonstrate that a robust “N400 effect” (i.e., a well-established event-related potential, known to be sensitive to lexical-semantic integration difficulties) is elicited by semantically anomalous sentences; this N400 effect is subsequently attenuated by discourse context. Lastly, a “social N400 effect” is elicited by sentences that are semantically plausible for the participants if they believe that another individual finds the sentences implausible. The results suggest that adolescents integrate the perspective of others during on-line language comprehension via simulation; that is, adolescents use their own language processing system to interpret language input from the perspective of other jointly attending individuals.

When You're Happy and I Know It: Four-Year-Olds' Emotional Perspective Taking During Online Language Comprehension.

Using a novel emotional perspective-taking task, this study investigated 4-year-olds' (n=97) use of a speaker's emotional prosody to make inferences about the speaker's emotional state and, correspondingly, their communicative intent. Eye gaze measures indicated preschoolers used emotional perspective inferences to guide their real-time interpretation of ambiguous statements. However, these sensitivities were less apparent in overt responses, suggesting preschoolers' ability to integrate emotional perspective cues is at an emergent state. Perspective taking during online language processing was positively correlated with receptive vocabulary and an offline measure of emotional perspective taking, but not with cognitive perspective taking, conflict or delay inhibitory control, or working memory. Together, the results underscore how children's emerging communicative competence involves different kinds of perspective inferences with distinct cognitive underpinnings.