Effects of Case-marking on the Anticipatory Processing of Korean Sentences

The data presented here was collected as part of the ENRICH project for the study described in: Kaplan, E.C., Wagner, A.E., Toffanin, P. and Baskent, D. (2021). Do Musicians and Non-musicians Differ in Speech-on-Speech Processing? Front. Psychol. 12:623787. doi: 10.3389/fpsyg.2021.623787 Files contained in this data set: demographics.csv: CSV-file that contains participant demographics and musical background information. datSRT.csv: CSV-file that contains the correctly recalled word scores from the sentence recall task per trial and participant. datAcc.csv: CSV-file that contains the accuracy scores for the eye-tracking experiment - visual world paradigm. datGaze.csv: CSV-file that contains the gaze-fixation data for each trial and participant. datPupil.csv: CSV-file that contains pupil data for each trial and participant. README.txt

Healthy adults ▷ 11 participants (1 male, age: mean=58.9 years, range=48–75 years) ID Age Sex Years post-onset Etiology, location Aphasic syndrome, severity P01 42 F 15 Ischemic, A. cerebri left Anomic, mild P02 51 M 14 Ischemic, A. cerebri left Anomic, mild P03 61 M 12 Ischemic, A. cerebri left Anomic, mild P04 46 F 2 Ischemic, A. cerebri left Broca, moderate P05 74 M 18 Ischemic, A. cerebri left Broca, moderate Experimental design

Eye movements have become a commonly used response measure in studies of spoken language processing. These studies are included in the so-called ‘visual world paradigm' in which participants' eye movements are monitored during scene viewing in language comprehension and production activities. In this chapter the most important aspects for running eye-tracking studies in children are revised. Developmental studies using eye movements have increased in the last ten years from babies to adolescents. However, there are only a handful of papers based on the ‘visual world paradigm' that analyze the spoken language in children with language disorders. These studies using eye movements have explored spoken word recognition; verb argument and thematic relations; and narrative comprehension and production. Results has proven eye tracker to be an effective tool for understanding language representation and processing in children with language disorders.

Speech in everyday conversations is riddled with discourse markers (DMs), such as well, you know, and like. However, in many lab-based studies of speech comprehension, such DMs are typically absent from the carefully articulated and highly controlled speech stimuli. As such, little is known about how these DMs influence online word recognition. The present study specifically investigated the online processing of DM like and how it influences the activation of words in the mental lexicon. We specifically targeted the cohort competitor (CC) effect in the Visual World Paradigm: Upon hearing spoken instructions to “pick up the beaker,” human listeners also typically fixate—next to the target object—referents that overlap phonologically with the target word (cohort competitors such as beetle; CCs). However, several studies have argued that CC effects are constrained by syntactic, semantic, pragmatic, and discourse constraints. Therefore, the present study investigated whether DM like influences online word recognition by activating its cohort competitors (e.g., lightbulb). In an eye-tracking experiment using the Visual World Paradigm, we demonstrate that when participants heard spoken instructions such as “Now press the button for the, like … unicycle,” they showed anticipatory looks to the CC referent (lightbulb)well before hearing the target. This CC effect was sustained for a relatively long period of time, even despite hearing disambiguating information (i.e., the /k/ in like). Analysis of the reaction times also showed that participants were significantly faster to select CC targets (lightbulb) when preceded by DM like. These findings suggest that seemingly trivial DMs, such as like, activate their CCs, impacting online word recognition. Thus, we advocate a more holistic perspective on spoken language comprehension in naturalistic communication, including the processing of DMs.

This study aimed to examine age-related differences in the comprehension of Korean comparative sentences with varying word orders by employing both offline and online measures, and to investigate how variations in word order affect sentence processing across different age groups. A total of 52 monolingual native Korean speakers, 26 young adults, and 26 older adults, completed a sentence-picture-matching task under two word order conditions: comparative-first and nominative-first. Offline measures included accuracy and response time, while an online method involved eye-tracking within the Visual World Paradigm. Data analyses were performed using linear and generalized linear mixed-effects models. Older adults demonstrated lower accuracy and longer response times compared to younger individuals. Distinctive fixation patterns were observed, particularly in the sentential-final phrase, across different age groups. Specifically, nominative-first sentences elicited greater target advantage scores among younger adults, whereas older adults showed higher scores in comparative-first sentences. The study highlights the potential of comparative sentences in elucidating age-related changes in sentence comprehension. These differences were evident not only in offline tasks but also in real-time processing, as evidenced by eye-tracking data. The findings suggest distinct processing strategies employed by young and older adults and underscore the importance of considering both syntactic and semantic cues in sentence comprehension.

The Visual World Paradigm (VWP) is one of the most widely used experimental paradigms in psycholinguistic research for studying online word and sentence processing. Over the past three decades, a wealth of research has shown that eye movements can reveal much about the systematic and temporal relationship between fixations on visual information and online language processing. The present systematic review takes stock of the current state of the VWP as an experimental paradigm, outlining critical issues that should guide future research. In this paper, we provide a systematic overview of the role of language functions using the VWP in speech comprehension from a cross-linguistic perspective. Our findings contribute to the VWP literature by evaluating existing methodological approaches and outlining directions for future research. The review reveals that although methodological sophistication has increased, especially through the adoption of methods such as LMMs and GAMMs, a significant gap remains in linguistic diversity.

This study investigated the relative role of sub-syllabic components (initial consonant, rime, and tone) in spoken word recognition of Mandarin Chinese using an eye-tracking experiment with a visual world paradigm. Native Mandarin speakers (all born and grew up in Beijing) were presented with four pictures and an auditory stimulus. They were required to click the picture according to the sound stimulus they heard, and their eye movements were tracked during this process. For a target word (e.g., tang2 “candy”), nine conditions of competitors were constructed in terms of the amount of their phonological overlap with the target: consonant competitor (e.g., ti1 “ladder”), rime competitor (e.g., lang4 “wave”), tone competitor (e.g., niu2 “cow”), consonant plus rime competitor (e.g., tang1”soup”), consonant plus tone competitor (e.g., tou2 “head”), rime plus tone competitor (e.g., yang2 “sheep”), cohort competitor (e.g., ta3 “tower”), cohort plus tone competitor (e.g., tao2 “peach”), and baseline competitor (e.g., xue3 “snow”). A growth curve analysis was conducted with the fixation to competitors, targets, and distractors, and the results showed that (1) competitors with consonant or rime overlap can be adequately activated, while tone overlap plays a weaker role since additional tonal information can strengthen the competitive effect only when it was added to a candidate that already bears much phonological similarity with the target. (2) Mandarin words are processed in an incremental way in the time course of word recognition since different partially overlapping competitors could be activated immediately; (3) like the pattern found in English, both cohort and rime competitors were activated to compete for lexical activation, but these two competitors were not temporally distinctive and mainly differed in the size of their competitive effects. Generally, the gradation of activation based on the phonological similarity between target and candidates found in this study was in line with the continuous mapping models and may reflect a strategy of native speakers shaped by the informative characteristics of the interaction among different sub-syllabic components.

Sentence comprehension relies on the ability to rapidly integrate different types of linguistic and non-linguistic information. However, there is currently a paucity of research exploring how preschool children with autism understand sentences using different types of cues. The mechanisms underlying sentence comprehension remains largely unclear. The present study presents a protocol to examine the sentence comprehension abilities of preschool children with autism. More specifically, a visual world paradigm of eye-tracking is used to explore the moment-to-moment sentence comprehension in the children. The paradigm has multiple advantages. First, it is sensitive to the time course of sentence comprehension and thus can provide rich information about how sentence comprehension unfolds over time. Second, it requires minimal task and communication demands, so it is ideal for testing children with autism. To further minimize the computational burden of children, the present study measures eye movements that arise as automatic responses to linguistic input rather than measuring eye movements that accompany conscious responses to spoken instructions.

An eye-tracking experiment in the Visual World Paradigm was conducted to examine the effects of language history on the predictive parsing of sentences containing relative clauses in the first-learned language of fluent bilingual adults. We compared heritage speakers of Spanish (HSs)—who had spent most of their lives immersed in an English-dominant society—to Spanish–English late bilinguals (LBs), who did not begin immersion in an English-dominant society until adulthood. Consistent with studies of monolinguals, the LBs demonstrated a subject/object relative clause processing asymmetry, i.e. a processing advantage during subject relative clauses and a processing disadvantage during object relative clauses. This suggests that the LBs actively predicted the syntactic structure of subject relative clauses, consistent with the active filler hypothesis. The HSs, on the other hand, did not exhibit this processing asymmetry, suggesting less active prediction. We conclude, therefore, that decreased exposure to the first-learned language causes less active prediction in first-language processing, which causes both disadvantages, and interestingly, advantages, in processing speed.

Accounts based on the pragmatic maxim of quantity make different predictions about the computation of scalar versus ignorance inferences. These different predictions are evaluated in two eye-tracking experiments using a visual world paradigm to assess the on-line computation of inferences. The test sentences contained disjunction phrases, which engender both kinds of inferences. The first experiment documented that both inferences are computed immediately upon encountering the disjunctive connective, at nearly identical temporal locations. The second experiment was designed to determine whether or not there exists an intermediate stage at which the truth of the corresponding conjunction phrase is ignored. No such stage was found.

ABSTRACTPrediction during sign language comprehension may enable signers to integrate linguistic and non-linguistic information within the visual modality. In two eye-tracking experiments, we investigated American Sign language (ASL) semantic prediction in deaf adults and children (aged 4–8 years). Participants viewed ASL sentences in a visual world paradigm in which the sentence-initial verb was either neutral or constrained relative to the sentence-final target noun. Adults and children made anticipatory looks to the target picture before the onset of the target noun in the constrained condition only, showing evidence for semantic prediction. Crucially, signers alternated gaze between the stimulus sign and the target picture only when the sentential object could be predicted from the verb. Signers therefore engage in prediction by optimising visual attention between divided linguistic and referential signals. These patterns suggest that prediction is a modality-independent process, and theoretical implications are discussed.

During spoken language comprehension, listeners make use of both knowledge-based and signal-based sources of information, but little is known about how cues from these distinct levels of representational hierarchy are weighted and integrated online. In an eye-tracking experiment using the visual world paradigm, we investigated the flexible weighting and integration of morphosyntactic gender marking (a knowledge-based cue) and contextual speech rate (a signal-based cue). We observed that participants used the morphosyntactic cue immediately to make predictions about upcoming referents, even in the presence of uncertainty about the cue's reliability. Moreover, we found speech rate normalization effects in participants' gaze patterns even in the presence of preceding morphosyntactic information. These results demonstrate that cues are weighted and integrated flexibly online, rather than adhering to a strict hierarchy. We further found rate normalization effects in the looking behavior of participants who showed a strong behavioral preference for the morphosyntactic gender cue. This indicates that rate normalization effects are robust and potentially automatic. We discuss these results in light of theories of cue integration and the two-stage model of acoustic context effects. (PsycINFO Database Record (c) 2020 APA, all rights reserved).

We investigated whether native Finnish speakers can grasp the meaning of Russian onomatopoeic words without any prior knowledge of the Russian language. In Experiment 1, elicitation test, naïve listeners generated associations for the acoustic events depicted by onomatopoeic words they heard. A cluster analysis suggested presence of different types of cues that affect the elicitation of associations. In the Facilitating cluster, associations were mostly correct; in the Counteracting cluster, they were predominantly incorrect. Worthy of note, many of the incorrect associations were systematic. In the Mixed cluster, there was a combination of cues; and in the Undefined cluster, no discrete cues affecting the formation of common associations were found. In Experiment 2, the same stimulus words were used in an eye-tracking experiment using visual world paradigm. It was shown that the participants have even better chances to map the onomatopoeic words to the correct semantic domain when extralinguistic information is available, in this case target images presented on the experimental display. The availability of both audio and visual inputs substantially boosted this process in all four clusters. Our findings support the view that imitative sound symbolism offers a scaffolding material for connecting onomatopoeias to their referents when words are pronounced in isolation. Cross-linguistic sound symbolism offers a good explanation to the presence of different cues that affect semantic recognition of unknown onomatopoeic words. On a larger scale, cross-linguistic similarities in onomatopoeias may be part of a broader phenomenon, universal sound symbolism, form-meaning mapping shared by a wide array of languages. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Psycholinguistic research spanning a number of decades has produced diverging results with regard to the nature of constraint integration in online sentence processing. For example, evidence that language users anticipatorily fixate likely upcoming referents in advance of evidence in the speech signal supports rapid context integration. By contrast, evidence that language users activate representations that conflict with contextual constraints, or only indirectly satisfy them, supports nonintegration or late integration. Here we report on a self-organizing neural network framework that addresses 1 aspect of constraint integration: the integration of incoming lexical information (i.e., an incoming word) with sentence context information (i.e., from preceding words in an unfolding utterance). In 2 simulations, we show that the framework predicts both classic results concerned with lexical ambiguity resolution (Swinney, 1979; Tanenhaus, Leiman, & Seidenberg, 1979), which suggest late context integration, and results demonstrating anticipatory eye movements (e.g., Altmann & Kamide, 1999), which support rapid context integration. We also report 2 experiments using the visual world paradigm that confirm a new prediction of the framework. Listeners heard sentences like "The boy will eat the white …" while viewing visual displays with objects like a white cake (i.e., a predictable direct object of "eat"), white car (i.e., an object not predicted by "eat," but consistent with "white"), and distractors. In line with our simulation predictions, we found that while listeners fixated white cake most, they also fixated white car more than unrelated distractors in this highly constraining sentence (and visual) context.

Anticipatory eye movements are reported in studies on motor control as well as on language comprehension, implying that this major orienting system is involved in generating goal-directed behaviour within the action and the language domain. The cognitive contribution of these anticipatory eye movements to language and motor control, however, is still not well understood. This study investigated whether anticipatory eye movements reflect the working of a predictive mechanism that is shared between action and language and if so, whether the predictions are based primarily on an anticipation of the next discrete event (movement or word), or rather represent a semantic understanding of the end goal of the whole event (action or sentence). To this end, we designed two highly comparable paradigms with complex action sequences – one relying more strongly on the action and the other on the language system. The data demonstrated a pattern of predictive looks in our action observation paradigm that was similar to that observed in the visual world paradigm. These findings provide empirical evidence for the idea of a shared predictive mechanism that allows for fluent behaviour in action and language. Moreover, the pattern in both paradigms was such that it demonstrated an increase in predictive looks in the final action step. This finding implies that the predictive mechanism accumulates semantic information relevant for our overall (motor or linguistic) behavioural goals, rather than just predicting discrete events when making decisions about complex action sequences. Such a predictive mechanism facilitates understanding of complex situations, allowing for efficient and adaptive interaction with our environment.