Sentence-picture Matching Task Research Articles

Saccades and Blinks Index Cognitive Demand During Auditory Noncanonical Sentence Comprehension.

Noncanonical sentence structures pose comprehension challenges because they require increased cognitive demand. Prosody may partially alleviate this cognitive load. These findings largely stem from behavioral studies, yet physiological measures may reveal additional insights into how cognition is deployed to parse sentences. Pupillometry has been at the forefront of investigations into physiological measures of cognitive demand during auditory sentence comprehension. This study offers an alternative approach by examining whether eye-tracking measures, including blinks and saccades, index cognitive demand during auditory noncanonical sentence comprehension and whether these metrics are sensitive to reductions in cognitive load associated with typical prosodic cues. We further investigated how eye-tracking patterns differ across correct and incorrect responses, as a function of time, and how each related to behavioral measures of cognition. Canonical and noncanonical sentence comprehension was measured in 30 younger adults using an auditory sentence-picture matching task. We also assessed participants' attention and working memory. Blinking and saccades both differentiate noncanonical sentences from canonical sentences. Saccades further distinguish noncanonical structures from each other. Participants made more saccades on incorrect than correct trials. The number of saccades also related to working memory, regardless of syntax. However, neither eye-tracking metric was sensitive to the changes in cognitive demand that was behaviorally observed in response to typical prosodic cues. Overall, these findings suggest that eye-tracking indices, particularly saccades, reflect cognitive demand during auditory noncanonical sentence comprehension when visual information is present, offering greater insights into the strategies and neural resources participants use to parse auditory sentences.

Interplay of Semantic Plausibility and Word Order Canonicity in Sentence Processing of People With Aphasia Using a Verb-Final Language.

The Western Aphasia Battery is widely used to assess people with aphasia (PWA). Sequential Commands (SC) is one of the most challenging subtests for PWA. However, test items confound linguistic factors that make sentences difficult for PWA. The current study systematically manipulated semantic plausibility and word order in sentences like those in SC to examine how these factors affect comprehension deficits in aphasia. Fifty Korean speakers (25 PWA and 25 controls) completed a sentence-picture matching task that manipulated word order (canonical vs. noncanonical) and semantic plausibility (plausible vs. less plausible). Analyses focused on accuracy and aimed to identify sentence types that best discriminate the groups. Additionally, we explored which sentence type serves as the best predictor of aphasia severity. PWA demonstrated greater difficulties in processing less plausible sentences than plausible ones compared to the controls. Across the groups, noncanonical and less plausible sentences elicited lower accuracy than canonical and plausible sentences. Notably, the accuracy of PWA and control groups differed in noncanonical and less plausible sentences. Additionally, aphasia severity significantly correlated with less plausible sentences. Even in languages with flexible word order, PWA find it challenging to process sentences with noncanonical syntactic structures and less plausible semantic roles.

Online Eye Tracking for Aphasia: A Feasibility Study Comparing Web and Lab Tracking and Implications for Clinical Use.

Studies using eye-tracking methodology have made important contributions to the study of language disorders such as aphasia. Nevertheless, in clinical groups especially, eye-tracking studies often include small sample sizes, limiting the generalizability of reported findings. Online, webcam-based tracking offers a potential solution to this issue, but web-based tracking has not been compared with in-lab tracking in past studies and has never been attempted in groups with language impairments. Patients with post-stroke aphasia (n=16) and age-matched controls (n=16) completed identical sentence-picture matching tasks in the lab (using an EyeLink system) and on the web (using WebGazer.js), with the order of sessions counterbalanced. We examined whether web-based eye tracking is as sensitive as in-lab eye tracking in detecting group differences in sentence processing. Patients were less accurate and slower to respond to all sentence types than controls. Proportions of gazes to the target and foil picture were computed in 100ms increments, which showed that the two modes of tracking were comparably sensitive to overall group differences across different sentence types. Web tracking showed comparable fluctuations in gaze proportions to target pictures to lab tracking in most analyses, whereas a delay of approximately 500-800ms appeared in web compared to lab data. Web-based eye tracking is feasible to study impaired language processing in aphasia and is sensitive enough to detect most group differences between controls and patients. Given that validations of webcam-based tracking are in their infancy and how transformative this method could be to several disciplines, much more testing is warranted.

Sentence comprehension in Malay-speaking adults with aphasia: the role of affix integration

ABSTRACT Background One of the impairments in comprehension mostly reported for aphasia is that in reversible passive sentences, where a systematic asymmetry with reversible active sentences has been attested across languages. In several accounts, this selective impairment has been proposed to be the result of the specific syntax of passive sentences. However, some results particularly from flexible word order languages suggest that a pattern where both active and passive sentences are impaired also exists. Aims In this study, we examined the actives and passives in the comprehension of a flexible word order language, namely standard Malay, in people with aphasia (PWA), aiming to confirm that the predominant pattern in this language is one of generalised impairment across reversible sentences (active and passive), and not an asymmetrical one. The role of fluency in determining the pattern of impairment was also explored. Methods & procedures Fourteen healthy adults and 20 PWA, 14 with fluent aphasia and 6 with non-fluent aphasia, took part in a comprehension study in standard Malay. Standard Malay is a flexible word order language that relies on the parsing of the voice affix on the verb to correctly interpret both active and passive sentences. Participants were tested on a sentence-picture matching task on comprehension of active and passive reversible clauses. Outcomes and results The study revealed that PWA were overall less accurate than healthy speakers in comprehension of reversible active and passive sentences, and non-fluent PWA were less accurate than fluent PWA with no effect of sentence type (active/passive). While not predominant, an asymmetric pattern was present in some participants. We propose that an impairment in thematic role assignment leads to a generalised impairment in comprehension of reversible sentences, while preserved thematic role mapping and impaired syntax lead to an asymmetric impairment. Conclusions The study confirmed that speakers of Malay mostly show a generalised impairment in both active and passive sentences. Given the nature of the language, we propose that this suggests that impairment in Malay is predominantly shown on affix integration, first in mapping of thematic roles, and then in the full syntactic processing.

Online Metaphor Comprehension in Adults with Autism Spectrum Disorders: An Eye Tracking Study

AbstractThe aim of this study was to investigate novel metaphor comprehension in adults with autism spectrum disorder (ASD). Previous literature is conflicting about whether individuals with ASD have impairment in this particular type of figurative language. Participants in the study completed a visual world paradigm eye-tracking task, which involved selecting an interpretation of an auditorily presented sentence (i.e. a picture-sentence matching task), where images corresponded to literal and metaphorical interpretations. Thus, the study also investigated online processing, via reaction times and eye movements. Forty adults participated in the study (18 with ASD and 22 typically-developing controls). Each participant completed the AQ questionnaire and had their vocabulary assessed. Results showed that participants with ASD comprehended metaphorical utterances with the same accuracy as controls. However, they had significantly slower reaction times, and specifically, were approximately 800 ms slower. Analysis of eye movements revealed that participants with ASD showed significantly longer fixation times on both the target and distractor image, the latter of which suggests difficulty overcoming the literal interpretation. Consistent with some prior studies, we showed that adults with ASD are not impaired in novel metaphor comprehension, but they were clearly less efficient. Verbal abilities did not significantly relate to performance. Finally, our online processing measure (eye tracking) provided us with insights into the nature of the ASD inefficiency (i.e. a literality bias).

Metaphor comprehension and production in verbally able children with Autism Spectrum Disorder.

Research in the field of figurative language processing in Autism Spectrum Disorders (ASD) has demonstrated that autistic individuals experience systematic difficulties in the comprehension of different types of metaphors. However, there is scarce evidence regarding metaphor production skills in ASD. Importantly, the exact source of metaphor processing difficulties in ASD remains largely controversial. The debate has mainly focused on the mediating role of structural language skills (i.e., lexical knowledge) and cognitive abilities (i.e., Theory of Mind and executive functions) in ASD individuals' ability to comprehend and generate metaphors. The present study examines metaphor comprehension and production in 18 Greek-speaking verbally able children with ASD and 31 typically-developing (TD) controls. Participants completed two tasks, namely, a low-verbal multiple-choice sentence-picture matching task that tested their ability to comprehend conventional predicate metaphors, and a sentence continuation task that assessed their ability to generate metaphors. The study also included measures of fluid intelligence, expressive vocabulary, and working memory within the sample. The results show that the ASD group had significantly lower performance than the TD group in both metaphor comprehension and production. The findings also reveal that expressive vocabulary skills were a key factor in the metaphor comprehension and production performance of the children with ASD. Working memory capacity was also found to correlate significantly with metaphor comprehension performance in the ASD group. Conversely, no correlations were found in the TD group with neither of the above factors. Of note, children with ASD generated significantly more inappropriate responses and no-responses to the metaphor production task compared with the control group. The overall results reveal that children with ASD had difficulty with both comprehending and using metaphorical language. The findings also indicate that TD children may employ diverse cognitive strategies or rely on different underlying skills when processing metaphors compared with children with ASD.

The impact of cognate vocabulary on explicit L2 rule learning

According to Hopp’s Lexical Bottleneck Hypothesis, difficulties in second language (L2) lexical processing may lead to non-target syntactic computations. In line with this hypothesis, cognates – which are processed faster than non-cognates, as defined by the cognate facilitation effect – can ease L2 syntactic processing. In order to investigate whether cognates additionally facilitate L2 syntax learning, we had two groups of Spanish natives learn an artificial language drawing on Basque. Each group explicitly learnt a set of either Spanish–Basque cognates or non-cognates and a word order grammatical rule. Then, two sentence–picture matching tasks tested their ability to apply the rule (1) with cognates vs. non-cognates and (2) with novel cognate words. Results showed that, in both cases, cognate learners were better at applying the rule than non-cognate learners. This finding suggests that the cognate facilitation effect and its role in the Lexical Bottleneck Hypothesis can be extended from L2 processing to L2 learning. We end by mentioning possible implications of our results for second language teaching in adulthood.

Enhanced prosody adds to morpho-syntactic cues in the interpretation of structural ambiguities in German

ABSTRACT This study investigated the effects of syntactically marked and enhanced prosody on local ambiguity resolution in German SVO and OVS sentences. In a visual-world experiment, thirty younger and thirty elderly healthy participants performed a sentence-picture matching task. Response accuracy, reaction times and fixations proportions to the target picture were analysed using linear mixed models. We found no support for beneficial effects of syntactically marked prosody, however, results suggested a facilitative role of enhanced prosodic cues (i.e. increased f0 maximum) prior to the point of disambiguation in SVO structures, as well as the beneficial effects of enhanced prosody adding to morpho-syntactic cues in OVS structures. Both age groups showed comparable cue use but inter-individual variability in prosodic cue processing. Overall, our study replicates and extends previous findings demonstrating the importance of examining variability in prosodic cue processing in future research.

Subject–verb agreement: Three experiments on Catalan

This study delves into the syntactic parsing abilities of children and infants exposed to Catalan as their first language. Focusing first on ages 3 to 6, we conducted two sentence-picture matching tasks. In experiment 1, 3 to 4-year-old children failed in identifying singular third-person subjects within null-subject sentences, although they performed above chance in all other scenarios, including plural third-person subjects and sentences with overt full DP subjects. This is reminiscent of the results of Pérez-Leroux for Spanish. In experiment 2, with the same design but involving numeral distractors, children’s performance was above chance level across all conditions from age 3 to 4. Then, in experiment 3, we moved to a younger age range with the help of eye-tracking techniques. The findings revealed that infants at 22 months had the ability to parse subject–verb agreement in sentences with third-person null subjects, and at 19 months there was evidence of parsing for third-person plural null subjects. These findings are inconsistent with the perception of children grappling with syntactic agreement computation. We argue that instances of underperformance in subject–verb agreement parsing identified in the literature often stem from task-related and pragmatic issues rather than core syntactic delay. If so, the putative asymmetry between early production of verbal inflection and late comprehension disappears; rather, the results suggest early establishment of matching operations and mastery of language-specific agreement properties before production starts.

When morphology is not enough: The acquisition of voice in monolingual Greek children and bilingual children with Greek as a heritage language

ABSTRACT The acquisition of voice in Greek remains understudied, especially in heritage populations. Voice in Greek poses a challenging acquisition task for children due to its syncretism, marking various verb classes as well as passives. The present study explores the acquisition of anticausatives, reflexives, and passives in 6-to-8-year-old monolingual Greek and Greek-German bilingual children with Greek as their heritage language. This age coincides with the immersion of children in the education system. A Sentence Picture Matching task tested participants’ interpretation preferences when ambiguity arises in reflexives and (optionally) (un)marked anticausatives (reflexive/anticausative vs. passive interpretation); it also tested accuracy in passives compared to actives-transitives (passive vs. active interpretation). All children exhibited adult-like performance in reflexives. In anticausatives, despite their lower performance, they were qualitatively similar to adults, exhibiting knowledge of verb classes. Children’s preferences were predicted by age (active-morphology/ACT) and vocabulary (non-active/NACT), confirming the role of the mental lexicon in the acquisition of NACT anticausatives. The only difference between bilingual and monolingual children was in optionally (un)marked anticausatives, both in ACT and NACT, that was marginally predicted by age and grammar production respectively, suggesting that optionality affects bilingual more than monolingual children. In passives, children were less accurate than adults despite ceiling performance in actives-transitives. Bilingual children performed less well than monolingual children and performance was predicted by grammar and exposure. These findings reflect the difficulties of monolingual children with passives and their late acquisition which further affects bilingual development due to reduced input in heritage Greek.

The acquisition of the verbal passive: The role of verb type

ABSTRACT This study examines the comprehension of verbal passives by children acquiring European Portuguese, in particular with respect to the predictions of the Universal Phase Requirement (UPR) and the Universal Freezing Hypothesis (UFH) regarding children’s performance with different types of predicates. Both hypotheses entail the prediction that children perform better with passives of predicates that encode a result state, either because they tend to make better adjectival passives or because they have a complex event structure. Moreover, the UPR predicts poorer performance with long passives (i.e., those with a by-phrase) than with short passives, to the extent that construing an adjectival interpretation is more difficult when a by-phrase is present; the UFH predicts no difference between the two. Portuguese-speaking children between the ages of 3 and 8 years were tested on their comprehension of verbal passives of non-actional predicates and actional predicates with or without a result state, using two sentence-picture matching tasks. The results do not entirely fulfill the predictions of the UPR and the UFH: we replicate the delay seen in English with non-actional predicates, but no effect of the result state is seen in the case of actional predicates. We propose that the relevant aspectual property of the passivized predicate may be affectedness rather than the availability of a result state.

Comprehension of Ditransitive Constructions in Mandarin-Speaking Children with Developmental Language Disorder and Children with Autism Spectrum Disorder Plus Language Impairment.

This study examined and compared the comprehension of Mandarin ditransitive constructions in children with developmental language disorder (DLD) and children with autism spectrum disorder plus language impairment (ALI). Eighteen children with DLD, 17 children with ALI, and 27 age-matched typically developing (TDA) children, participated in a sentence-picture matching task on four patterns of Mandarin ditransitive constructions. Both children with DLD and children with ALI received significantly lower accuracy than TDA children in general and their most common errors were thematic role reversals. However, while children with ALI evinced a generalized deficit in all four patterns, only the comprehension of S1 (Subj. + Vgei + IO + DO) and S3 (Subj. + gei + IO + V + DO) was affected in children with DLD, with that of S2 (Subj. + V + DO + gei + IO) and S4 (Subj. + V + IO + DO) preserved in this population. Additionally, thematic role reversal errors were more dominant in children with DLD than in children with ALI who also committed a relatively higher proportion of Wrong Theme and No Recipient errors. It is concluded thatthe primary deficit of children with DLD lies in representing dependent relationships between the arguments and the verb as involved in thematic role assignment, but this is less critical in children with ALI, with their performance on the comprehension task possibly also related to other factors associated with the condition. To enhance the development of ditransitive constructions, intervention efforts for children with DLD and children with ALI could focus on strengthening the connection between each argument and its thematic role.

The comprehension of passives in Mandarin children with and without DLD: from the perspective of Edge Feature Underspecification Hypothesis

AbstractThis paper investigates the comprehension of long and short passives in 15 Mandarin preschool children with Developmental Language Disorder (DLD) (aged 4;2–5;11 years), 15 Typically Developing Age-matched (TDA) (aged 4;3–5;8 years) children, and 15 Typically Developing Younger (TDY) (aged 3;2–4;3 years) children by using the picture-sentence matching task. The results reveal that children with DLD encounter more difficulty comprehending long passives compared with short passive, that they perform worse on the comprehension task than TDA children and TDY children, and that this population is more likely to commit thematic role reversal errors and point to pictures with the incorrect agent (patient) than typically developing children. Given that Mandarin passives are Topic Structures, we maintain that children with DLD are insensitive to the edge feature of the moved element in long passives, leading to Relativized Minimality effect and causing the asymmetry between the comprehension of long and short passives. These results align well with the Edge Feature Underspecification Hypothesis. Errors found in the children with DLD in the comprehension task point toward impaired syntactic knowledge and the lexical semantic deficit.

The Acquisition of Relative Clauses in Autism: The Role of Executive Functions and Language

PurposeRelative clauses present a well-known processing asymmetry between object-extracted and subject-extracted dependencies across both typical and atypical populations. The present study aimed at exploring the comprehension of object and subject relative clauses as conceptualized by the Relativized Minimality framework in autistic children and in a group of age- and IQ-matched typically-developing children. The study also explored the way performance in relative clauses would be affected by the children’s language and executive function skills.MethodRelative clause comprehension was tested through a sentence-picture matching task and language was tested with a receptive vocabulary task. Executive functions were assessed through backward digit recall and a Flanker test.ResultsObject relative clauses were harder to parse for both groups than subject relatives, while number mismatch between the moved object Noun Phrase and the intervening subject Noun Phrase in object relatives boosted both groups’ performances. Typically-developing children’s performance in object relatives was predicted by both language and executive functions, while autistic children failed to use language and did not systematically draw on their executive functions in object relative clause comprehension.ConclusionThe findings suggest that relative clause processing in autism follows a normal developmental trajectory, and that difficulty with parsing object relative clauses stems from reduced language and executive functions rather than deficits in the children’s morphosyntactic skills.

Diffuse glioma-induced structural reorganization in close association with preexisting syntax-related networks

Glioma in the left frontal cortex has been reported to cause agrammatic comprehension and induce global functional connectivity alterations within the syntax-related networks. However, it remains unclear to what extent the structural reorganization is affected by preexisting syntax-related networks. We examined 28 patients with a diffuse glioma in the left hemisphere and 23 healthy participants. Syntactic abilities were assessed by a picture-sentence matching task with various sentence types. The lesion responsible for agrammatic comprehension was identified by region-of-interest-based lesion-symptom mapping (RLSM). Cortical structural alterations were examined by surface-based morphometry (SBM), in which the cortical thickness and fractal dimension were measured with three-dimensional magnetic resonance imaging (MRI). Fiber tracking on the human population-averaged diffusion MRI template was performed to examine whether the cortical structural alterations were associated with the syntax-related networks. The RLSM revealed associations between agrammatic comprehension and a glioma in the posterior limb of the left internal capsule. The SBM demonstrated that decreased cortical thickness and/or increased complexity of the right posterior insula were associated not only with agrammatic comprehension of the patients but also with the syntactic abilities of healthy participants. The fiber tracking revealed that the route between these two regions was anatomically integrated into the preexisting syntax-related networks previously identified. These results suggest a potential association between agrammatic comprehension in patients with diffuse glioma and structural variations in specific tracts and cortical regions, which may be closely related to the syntax-related networks.

Acquisition of Mandarin long passives by children with developmental language disorder

ABSTRACT This study investigated the comprehension and production of long passives (i.e. bei-constructions with an overt agent) in Mandarin-speaking children with developmental language disorder (DLD). Seventeen preschool children with DLD (1 female; mean age: 61 months old) and 23 typically developing (TD) children (6 females; mean age: 62 months old) participated in a sentence-picture matching task (for comprehension) and an elicited production task. Their nonverbal working memory (NVWM) was measured with the fourth edition of the Wechsler Preschool and Primary Scale of Intelligence. Results showed that children with DLD were less accurate and more likely to choose the picture with reversed thematic roles than their TD peers on passives in the sentence-picture matching task; in the elicited production task, they produced fewer target responses than TD children in passives. For NVWM, although that of the DLD group was lower than that of TD children, most children in the DLD group were within the average range. Furthermore, their performance on passives in the comprehension and production tasks was significantly correlated with their NVWM, which adds to the body of work suggesting links between complex syntax and working memory. However, the fact that NVWM could be preserved in the face of difficulties with passives suggests that this link may be due to NVWM enhancing performance during tasks with a high visual component, while it may not be underlyingly responsible for syntactic impairments in children with DLD.

Similarity-Based Interference in Sentence Comprehension in Aphasia: a Computational Evaluation of Two Models of Cue-Based Retrieval

Sentence comprehension requires the listener to link incoming words with short-term memory representations in order to build linguistic dependencies. The cue-based retrieval theory of sentence processing predicts that the retrieval of these memory representations is affected by similarity-based interference. We present the first large-scale computational evaluation of interference effects in two models of sentence processing — the activation-based model and a modification of the direct-access model — in individuals with aphasia (IWA) and control participants in German. The parameters of the models are linked to prominent theories of processing deficits in aphasia, and the models are tested against two linguistic constructions in German: pronoun resolution and relative clauses. The data come from a visual-world eye-tracking experiment combined with a sentence-picture matching task. The results show that both control participants and IWA are susceptible to retrieval interference, and that a combination of theoretical explanations (intermittent deficiencies, slow syntax, and resource reduction) can explain IWA’s deficits in sentence processing. Model comparisons reveal that both models have a similar predictive performance in pronoun resolution, but the activation-based model outperforms the direct-access model in relative clauses.

Optionality in semantic-pragmatic interface of bilingualism? Bare numeral constructions in Tibetan and Chinese bilinguals’ grammar

Bare numeral constructions (BNCs) can be anaphoric in Chinese but not Tibetan. Since the interpretation of BNCs requires a specific context, we consider it to involve a semantic-pragmatic interface, which has been argued to be vulnerable to crosslinguistic transfer for bilinguals (e.g., Sorace 2005, 2011). This study conducted a controlled sentence-picture matching truth value judgment task to examine whether Tibetan-Chinese bilinguals show crosslinguistic influence when interpreting BNCs in both languages. The data suggests that crosslinguistic effects did occur among some bilinguals. However, there were more bilinguals who successfully differentiated the two languages regarding the interpretation of BNCs. Our findings imply that early bilinguals may not necessarily show optionality in interface phenomena.

Intervention effects in Catalan agrammatism

The goal of the present study is to test the agrammatic comprehension of clitic left dislocation and contrastive focus in Catalan, in order to evaluate the Generalised Minimality hypothesis of Grillo (2009). According to this hypothesis, the comprehension deficit observed in agrammatism is the result of the underspecification of scope-discourse features giving rise to generalised intervention effects. We conducted two sentence-picture matching tasks to assess the comprehension of clitic left dislocation and contrastive focus with nine and seven Broca’s aphasic subjects, respectively, as well as control participants. The results show that the comprehension of SVO sentences and object clitics was preserved, whereas the comprehension of object dislocations and object focalisations was compromised. These findings are consistent with the analysis of the deficit as an instance of generalised intervention effects. Yet, we also examined the prediction that a relevant syntactic feature mismatch between the subject and the object would suffice to block generalised minimality effects; in particular, the number features of subject and object were controlled for. The agrammatic subjects’ performance on mismatched sentences did not differ from their performance on sentences where the subject and the object were matched in number. These findings call the hypothesis into question and stress the need for future research.

Processing of Centre-embedded and Right-branching Relative Clauses in Slovenian

The current consensus in the literature of processing of relative clauses states that centre-embedded relative clauses introduce a heavy computational load. While this is well-established, most evidence for it comes from English, while the empirical evidence from many other languages is still lacking. Here, we try to fill this gap by researching the differences in the processing times of centre-embedded and right-branching relative clauses in Slovenian. We report results from a sentence-picture matching task, in which we observe longer reaction times and lower accuracy when the participants are dealing with centre-embedded relative clauses, compared to right-branching ones. This result provides important evidence in a so far largely under-investigated language, contributing to the theoretical claim that the difficulties observed in the processing of centre-embedded relative clauses are language-independent.