Naturalistic Language Research Articles

Robust assessment of the cortical encoding of word-level expectations using the temporal response function

Objective. Speech comprehension involves detecting words and interpreting their meaning according to the preceding semantic context. This process is thought to be underpinned by a predictive neural system that uses that context to anticipate upcoming words. However, previous studies relied on evaluation metrics designed for continuous univariate sound features, overlooking the discrete and sparse nature of word-level features. This mismatch has limited effect sizes and hampered progress in understanding lexical prediction mechanisms in ecologically-valid experiments.Approach. We investigate these limitations by analyzing both simulated and actual electroencephalography (EEG) signals recorded during a speech comprehension task. We then introduce two novel assessment metrics tailored to capture the neural encoding of lexical surprise, improving upon traditional evaluation approaches.Main results. The proposed metrics demonstrated effect-sizes over 140% larger than those achieved with the conventional temporal response function (TRF) evaluation. These improvements were consistent across both simulated and real EEG datasets.Significance. Our findings substantially advance methods for evaluating lexical prediction in neural data, enabling more precise measurements and deeper insights into how the brain builds predictive representations during speech comprehension. These contributions open new avenues for research into predictive coding mechanisms in naturalistic language processing.

It “bendy dis one”: recognising and building upon Australian Aboriginal students’ linguistic repertoires as educational resources

There is a considerable body of research documenting the languages First Nations children bring to school. However, despite language being a key factor impacting on educational outcomes, there is little research focussing on the students’ language use within schools outside specific learning contexts. Therefore, in this study we sought to explore the language resources used by Australian Aboriginal children at school, particularly those underexamined cohorts living in regional and remote locations. We examined how they used language in the classroom and in the playground, and with different audiences (e.g. peers vs teachers). Naturalistic language data were collected in seven schools located in different settings within Western Australia. 230 children and their teachers were recorded. The data show these students draw on their full linguistic repertoire when at school – moving between lighter or heavier varieties of Australian Aboriginal English (AAE), their traditional languages (e.g. Martu and Kija), SAE, as well as Kriol (an Australian creole) in some remote locations. We also found they adjusted their language choices according to the audience and content of their discussions and they utilised processes of translanguaging as part of these school interactions.

Scale matters: Large language models with billions (rather than millions) of parameters better match neural representations of natural language.

Recent research has used large language models (LLMs) to study the neural basis of naturalistic language processing in the human brain. LLMs have rapidly grown in complexity, leading to improved language processing capabilities. However, neuroscience researchers haven't kept up with the quick progress in LLM development. Here, we utilized several families of transformer-based LLMs to investigate the relationship between model size and their ability to capture linguistic information in the human brain. Crucially, a subset of LLMs were trained on a fixed training set, enabling us to dissociate model size from architecture and training set size. We used electrocorticography (ECoG) to measure neural activity in epilepsy patients while they listened to a 30-minute naturalistic audio story. We fit electrode-wise encoding models using contextual embeddings extracted from each hidden layer of the LLMs to predict word-level neural signals. In line with prior work, we found that larger LLMs better capture the structure of natural language and better predict neural activity. We also found a log-linear relationship where the encoding performance peaks in relatively earlier layers as model size increases. We also observed variations in the best-performing layer across different brain regions, corresponding to an organized language processing hierarchy.

The unique contribution of uncertainty reduction during naturalistic language comprehension

Language comprehension is an incremental process with prediction. Delineating various mental states during such a process is critical to understanding the relationship between human cognition and the properties of language. Entropy reduction, which indicates the dynamic decrease of uncertainty as language input unfolds, has been recognized as effective in predicting neural responses during comprehension. According to the entropy reduction hypothesis (Hale, 2006), entropy reduction is related to the processing difficulty of a word, the effect of which may overlap with other well-documented information-theoretical metrics such as surprisal or next-word entropy. However, the processing difficulty was often confused with the information conveyed by a word, especially lacking neural differentiation. We propose that entropy reduction represents the cognitive neural process of information gain that can be dissociated from processing difficulty. This study characterized various information-theoretical metrics using GPT-2 and identified the unique effects of entropy reduction in predicting fMRI time series acquired during language comprehension. In addition to the effects of surprisal and entropy, entropy reduction was associated with activations in the left inferior frontal gyrus, bilateral ventromedial prefrontal cortex, insula, thalamus, basal ganglia, and middle cingulate cortex. The reduction of uncertainty, rather than its fluctuation, proved to be an effective factor in modeling neural responses. The neural substrates underlying the reduction in uncertainty might imply the brain's desire for information regardless of processing difficulty.

Evaluating a Rapid Coaching Intervention Delivered Remotely to Families

Families are central to promoting communication in young children with disabilities with complex communication needs. Providing coaching on naturalistic language interventions (NLI) gives parents tools for independently intervening on communication across activities in the home. Both parents and professionals have limited resources, which requires maximizing parent and professional interactions when meeting. A rapid coaching intervention delivered via distance technology is one approach for addressing this issue. In this study, a multiple baseline across behaviors design replicated across participants was used to evaluate a rapid coaching intervention with parents. When the intervention was introduced, parents displayed improvements in naturalistic strategies and procedures. Child-level expressive communication was measured within the context of the designs, and data were variable. Improvements in the complexity of speech and responding to parents were observed. Implications are provided for future studies.

Neural correlates of depression-related smartphone language use in adolescents

Magnetic resonance imaging has provided pathophysiological insights into adolescent depression but is a relatively inaccessible technology. Generating scalable indicators of depression that are informed by neuroscience is therefore critical for providing solutions that allow us to detect and treat this devastating disorder. In this preregistered study, we investigated whether passively acquired smartphone-based language usage represents such an indicator of depression and explored whether the neural correlates of depression mediate or moderate this association. Forty adolescents (ages 14–18 years) with (n = 26) and without (n = 14) depression completed clinical assessments and a resting-state fMRI scan, prior to downloading a passive mobile sensing app to their smartphones. Linguistic features derived from over 1.2 million words (319,364 messages) across all smartphone apps were used to examine word usage patterns. Independent components analysis followed by dual regression was used to derive intrinsic networks commonly associated with depression: central executive network (CEN), default mode network (DMN), and salience network (SN). Depression was associated with more negative emotion word usage and fewer future-focus word usage on a daily basis (all ps < 0.05). Higher depressive symptoms and brain networks DMN and CEN were associated with greater first-person pronoun usage (all ps < 0.04). Accounting for CEN connectivity amplified the positive association between depressive symptoms and first-person pronoun usage. Lower SN–CEN connectivity moderated the association between depression and negative emotion word usage. Depression in adolescents is associated with naturalistic language usage during smartphone activities and may represent neurocognitive biases that are candidate treatment targets for interventions.

메타버스 플랫폼 로블록스를 통한 경험적 학습

Key technologies of the Fourth Industrial Revolution, such as Virtual Reality (VR) and the Metaverse, are transforming English education. This paper examines the effects of experiential learning in the Roblox metaverse on Korean EFL learners’ learning flow and academic achievement over a semester. This study involved 55 freshmen in a required university English course, divided into an experimental group (n = 28) and a control group (n = 27). The experimental group engaged in interactive activities within the Roblox platform, utilizing game scenarios to practice English in contextually rich environments. The control group received traditional classroom instruction covering the same themes and content. The results indicated that the experimental group showed statistically significant improvement in both learning flow and English academic achievement, suggesting that metaverse platforms can offer valuable educational experiences beyond conventional classroom settings. The use of Roblox for naturalistic language use in game scenarios provides learners with opportunities to practice language in contexts resembling real-life situations. This study suggests that experiential learning in the Roblox metaverse can extend beyond mere gameplay to provide educational value and potential as a learning tool, thereby enhancing the effectiveness of language learning for students.

Four dimensions of naturalistic language production in aphasia after stroke.

There is a rich tradition of research on the neuroanatomical correlates of spoken language production in aphasia using constrained tasks (e.g., picture naming), which offer controlled insights into the distinct processes that govern speech and language (i.e., lexical-semantic access, morphosyntactic construction, phonological encoding, speech motor programming/execution). Yet these tasks do not necessarily reflect everyday language use. In contrast, naturalistic language production (also referred to as connected speech or discourse) more closely approximates typical processing demands, requiring the dynamic integration of all aspects of speech and language. The brain bases of naturalistic language production remain relatively unknown, however, in part because of the difficulty in deriving features that are salient, quantifiable, and interpretable relative to both speech-language processes and the extant literature. The present cross-sectional observational study seeks to address these challenges by leveraging a validated and comprehensive auditory-perceptual measurement system that yields four explanatory dimensions of performance-Paraphasia (misselection of words and sounds), Logopenia (paucity of words), Agrammatism (grammatical omissions), and Motor speech (impaired speech motor programming/execution). We used this system to characterize naturalistic language production in a large and representative sample of individuals with acute post-stroke aphasia (n = 118). Scores on each of the four dimensions were correlated with lesion metrics, and multivariate associations among the dimensions and brain regions were then explored. Our findings revealed distinct yet overlapping neuroanatomical correlates throughout the left-hemisphere language network. Paraphasia and Logopenia were associated primarily with posterior regions, spanning both dorsal and ventral streams, which are critical for lexical-semantic access and phonological encoding. In contrast, Agrammatism and Motor speech were associated primarily with anterior regions of the dorsal stream that are involved in morphosyntactic construction and speech motor planning/execution respectively. Collectively, we view these results as constituting a brain-behavior model of naturalistic language production in aphasia, aligning with both historical and contemporary accounts of the neurobiology of spoken language production.

Individual differences in working memory impact the trajectory of non-native speech category learning.

What is the role of working memory over the course of non-native speech category learning? Prior work has predominantly focused on how working memory might influence learning assessed at a single timepoint. Here, we substantially extend this prior work by examining the role of working memory on speech learning performance over time (i.e., over several months) and leverage a multifaceted approach that provides key insights into how working memory influences learning accuracy, maintenance of knowledge over time, generalization ability, and decision processes. We found that the role of working memory in non-native speech learning depends on the timepoint of learning and whether individuals learned the categories at all. Among learners, across all stages of learning, working memory was associated with higher accuracy as well as faster and slightly more cautious decision making. Further, while learners and non-learners did not have substantially different working memory performance, learners had faster evidence accumulation and more cautious decision thresholds throughout all sessions. Working memory may enhance learning by facilitating rapid category acquisition in initial stages and enabling faster and slightly more careful decision-making strategies that may reduce the overall effort needed to learn. Our results have important implications for developing interventions to improve learning in naturalistic language contexts.

Checking Multiple Boxes: Themes Associated With Bicultural Identities

Multicultural individuals frequently feel they must check multiple boxes when responding to identity questions. In this investigation, we collected narratives from 402 bicultural individuals about their experiences with multicultural identities and used automated text-analytic tools to examine the valence of the narratives (sentiment analysis) and the most prevalent themes used (Meaning Extraction Method). In addition, we examined the relationships between the narratives and the two dimensions of Bicultural Identity Integration (BII) scales, Harmony and Blendedness. Findings suggested that the narratives were more emotionally positive than negative, and there were six most common themes. Furthermore, themes were significantly associated with BII scales. Our analyses provide deeper insights into the nature of bicultural identity using naturalistic language data.

A Brief Intervention to Teach Parents Naturalistic Language Facilitation Strategies.

This proof of concept study assessed the feasibility of a novel approach to teaching parents naturalistic language facilitation strategies in a single session. We investigated whether parents could learn to use the See and Say Sequence, which integrated responsive and language modeling strategies and measured the impacts that this intervention had on features of their input. Fourteen parent-child dyads participated in the study. Children ranged from 15 to 23 months of age and produced between three and 135 words. Five parents had concerns about their children's rate of language development. Parents were taught the See and Say Sequence during a brief single session (M = 18.98 min, SD = 2.65 min) using the Teach-Model-Coach-Review instructional process. We analyzed parents' use of the three See and Say Sequence components, total number of utterances, and mean turn length, as well as responsive and linguistic features of parent input before and after the brief intervention. Following intervention, parents significantly increased their use of the three See and Say Sequence components and decreased their total number of utterances and mean turn length. In addition, the use of the See and Say Sequence components substantially altered the overall composition of parent input. The results of this preliminary study demonstrate the feasibility of the See and Say Sequence in teaching responsive and language modeling strategies in a single session. We discuss the potential use and future evaluation of the See and Say Sequence as an option for early intervention service delivery.

The role of multimodal cues in second language comprehension

In face-to-face communication, multimodal cues such as prosody, gestures, and mouth movements can play a crucial role in language processing. While several studies have addressed how these cues contribute to native (L1) language processing, their impact on non-native (L2) comprehension is largely unknown. Comprehension of naturalistic language by L2 comprehenders may be supported by the presence of (at least some) multimodal cues, as these provide correlated and convergent information that may aid linguistic processing. However, it is also the case that multimodal cues may be less used by L2 comprehenders because linguistic processing is more demanding than for L1 comprehenders, leaving more limited resources for the processing of multimodal cues. In this study, we investigated how L2 comprehenders use multimodal cues in naturalistic stimuli (while participants watched videos of a speaker), as measured by electrophysiological responses (N400) to words, and whether there are differences between L1 and L2 comprehenders. We found that prosody, gestures, and informative mouth movements each reduced the N400 in L2, indexing easier comprehension. Nevertheless, L2 participants showed weaker effects for each cue compared to L1 comprehenders, with the exception of meaningful gestures and informative mouth movements. These results show that L2 comprehenders focus on specific multimodal cues – meaningful gestures that support meaningful interpretation and mouth movements that enhance the acoustic signal – while using multimodal cues to a lesser extent than L1 comprehenders overall.

Corpus of Mandarin Child Language: a preliminary study on the acquisition of semantic content categories in Mandarin-speaking preschoolers.

In studying language acquisition in children, sizable research studies have been focusing on the investigation of form and lexical semantics. This study aims to establish a child language database annotated both syntactically with part of speech and semantically with semantic content category to supplement the study of child language acquisition in the semantic domain beyond lexical level. The Corpus of Mandarin Child Language (CMCL) that documented the production of different semantic content categories by Mandarin-speaking children was established. Naturalistic language samples of 82 native Mandarin-speaking children aged 25-60 months, divided into three age groups, were obtained. The corresponding semantic content categories coded in each utterance were tagged according to previous studies, in addition to the annotations of part of speech. MLU and lexical diversity were examined and the usage and acquisition of different semantic content categories were also analyzed. The results regarding syntactic complexity and lexical diversity replicated the typical language acquisition pattern from previous studies, which supported the validity of the data obtained in the CMCL. To investigate the trajectory of acquisition of various semantic content categories by age, a 90% acquisition criterion was used. Our findings regarding the acquisition order of semantic content category were basically in line with previous studies in general, with some minor differences. This acquisition order observed is largely explained by the cognitive and syntactic complexity associated with the semantic content category, with additional influence from language specific properties and cultural specific factors of Mandarin. In addition, with the tags in both part-of-speech and semantic content category, the CMCL potentially provides a platform for examining the form-content interface in early child language acquisition, which also implies significantly on the theoretical and clinical ground.

Toward sophisticated models of naturalistic language behavior Comment on ”Beyond Simple Laboratory Studies” by A. Maselli et al.

Toward sophisticated models of naturalistic language behavior Comment on ”Beyond Simple Laboratory Studies” by A. Maselli et al.

Semantic-specific and domain-general mechanisms for integration and update of contextual information.

Recent research has highlighted the importance of domain-general processes and brain regions for language and semantic cognition. Yet, this has been mainly observed in executively demanding tasks, leaving open the question of the contribution of domain-general processes to natural language and semantic cognition. Using fMRI, we investigated whether neural processes reflecting context integration and context update-two key aspects of naturalistic language and semantic processing-are domain-specific versus domain-general. Thus, we compared neural responses during the integration of contextual information across semantic and non-semantic tasks. Whole-brain results revealed both shared (left posterior-dorsal inferior frontal gyrus, left posterior inferior temporal gyrus, and left dorsal angular gyrus/intraparietal sulcus) and distinct (left anterior-ventral inferior frontal gyrus, left anterior ventral angular gyrus, left posterior middle temporal gyrus for semantic control only) regions involved in context integration and update. Furthermore, data-driven functional connectivity analysis clustered domain-specific versus domain-general brain regions into distinct but interacting functional neural networks. These results provide a first characterisation of the neural processes required for context-dependent integration during language processing along the domain-specificity dimension, and at the same time, they bring new insights into the role of left posterior lateral temporal cortex and left angular gyrus for semantic cognition.

Statistical learning at a virtual cocktail party

Statistical learning – the ability to extract distributional regularities from input – is suggested to be key to language acquisition. Yet, evidence for the human capacity for statistical learning comes mainly from studies conducted in carefully controlled settings without auditory distraction. While such conditions permit careful examination of learning, they do not reflect the naturalistic language learning experience, which is replete with auditory distraction – including competing talkers. Here, we examine how statistical language learning proceeds in a virtual cocktail party environment, where the to-be-learned input is presented alongside a competing speech stream with its own distributional regularities. During exposure, participants in the Dual Talker group concurrently heard two novel languages, one produced by a female talker and one by a male talker, with each talker virtually positioned at opposite sides of the listener (left/right) using binaural acoustic manipulations. Selective attention was manipulated by instructing participants to attend to only one of the two talkers. At test, participants were asked to distinguish words from part-words for both the attended and the unattended languages. Results indicated that participants’ accuracy was significantly higher for trials from the attended vs. unattended language. Further, the performance of this Dual Talker group was no different compared to a control group who heard only one language from a single talker (Single Talker group). We thus conclude that statistical learning is modulated by selective attention, being relatively robust against the additional cognitive load provided by competing speech, emphasizing its efficiency in naturalistic language learning situations.

Expressive Language Profiles in a Clinical Screening Sample of Mandarin-Speaking Preschool Children With Autism Spectrum Disorder.

This cross-sectional study aimed to depict expressive language profiles and clarify lexical-grammatical interrelationships in Mandarin-speaking preschoolers with autism spectrum disorder (ASD) during the administration of the simplified Chinese Psychoeducational Profile-Third Edition screening. We collected naturalistic language samples from 81 (74 boys, seven girls) 2- to 7-year-old (Mage = 55.6 months, SD = 15.17) Mandarin-speaking children with ASD in clinician-child interactions. The child participants were divided into five age subgroups with 12-month intervals according to their chronological age. Computer-assisted part-of-speech tagging, constituency analysis, and dependency analysis addressed the developmental trajectories of early lexical and grammatical growth in each age subgroup. Significant within-ASD differences were observed in content words, function words, and lexical categories. Nouns and verbs were the predominant lexical categories, while noun types overwhelmed verb types in children over 3 years old. The grammatical development of 5- to 6-year-old Mandarin-speaking children with ASD was better than that of 3- to 4-year-old children. The trends of syntactic structures, grammatical relations, and grammatical complexity in each age group were similar. Mandarin-speaking preschoolers with ASD produce more lexicons with increasing age. They preserve the noun bias as a universal mechanism in early lexical learning. Moreover, their developmental trajectories of grammatical growth were comparable in each age subgroup. In addition, their lexicons and grammar were synchronically developed during early language acquisition.

Preliminary Data: Powered Mobility Intervention With Language Monitoring for Children With Cerebral Palsy

Purpose: Adapted ride-on cars offer an affordable mobility intervention that provides broader access to a child's environment. Such newfound independence is a unique opportunity to examine changes in environmental language and communicative participation as children acquire mobility. New technologies such as the Language Environmental Analysis (LENA) system hold promise for evaluating naturalistic language interactions but have not yet been applied to children with cerebral palsy with physical and/or communication impairments. This study sought to use LENA with a sample of children with cerebral palsy to determine the feasibility of monitoring environmental language patterns during longitudinal mobility intervention. Method: LENA's adult word counts, conversational turns, and child vocalizations were compared across three time points for participants aged 16–49 months with a range of communication skills (indexed by Communication Function Classification System Levels I–IV). Results: Feasibility was established with acceptable criterion validity during active drive time. Results had large effect sizes, likely diluted by sample size and heterogeneity. Conclusions: Limitations include small sample size, nonprescriptive use of powered mobility devices, and lack of control group. Future studies should incorporate these features to more closely examine communication patterns.

How do Autistic and Neurotypical Children’s Interests Influence their Accuracy During Novel Word Learning?

Word learning depends on attention – children must focus on the right things at the right times. However, autistic children often display restricted interests, limiting their intake of stimuli during word learning. This study investigates how category interests influence word learning in autism and neurotypical development. Autistic and neurotypical children matched on receptive vocabulary used a touch-screen computer to learn novel words associated with animals (high-interest stimuli) and objects (neutral-interest stimuli) via fast mapping. Response accuracy and speed were examined at referent selection, 5-minute retention, and 24-hour retention. Both groups identified meanings of novel words associated with unfamiliar animals and objects via mutual exclusivity with comparable accuracy. After 5 minutes, autistic children retained animal names with greater accuracy than neurotypical children. Autistic children showed a greater increase in their accuracy between 5-minute and 24-hour retention and outperformed neurotypical children across conditions after a night’s sleep. Across groups, 24-hour retention was predicted by number of target word repetitions heard at referent selection, indicating a relationship between fast mapping input and retention. However, autistic children were slower to respond correctly, particularly in the animal condition. For autistic children, superior word learning associated with high-interest stimuli was relatively short-term, as sleep appeared to consolidate their memory representations for neutral-interest stimuli. Although these results demonstrate that fundamental word learning mechanisms are not atypical in autism, slower response times may signal a speed-accuracy trade-off that could have implications for naturalistic language acquisition. Our findings also indicate favourable environmental conditions to scaffold word learning.

Mapping cortical activations underlying covert and overt language production using high-density diffuse optical tomography

Gold standard neuroimaging modalities such as functional magnetic resonance imaging (fMRI), positron emission tomography (PET), and more recently electrocorticography (ECoG) have provided profound insights regarding the neural mechanisms underlying the processing of language, but they are limited in applications involving naturalistic language production especially in developing brains, during face-to-face dialogues, or as a brain-computer interface. High-density diffuse optical tomography (HD-DOT) provides high-fidelity mapping of human brain function with comparable spatial resolution to that of fMRI but in a silent and open scanning environment similar to real-life social scenarios. Therefore, HD-DOT has potential to be used in naturalistic settings where other neuroimaging modalities are limited. While HD-DOT has been previously validated against fMRI for mapping the neural correlates underlying language comprehension and covert (i.e., “silent”) language production, HD-DOT has not yet been established for mapping the cortical responses to overt (i.e., “out loud”) language production. In this study, we assessed the brain regions supporting a simple hierarchy of language tasks: silent reading of single words, covert production of verbs, and overt production of verbs in normal hearing right-handed native English speakers (n = 33). First, we found that HD-DOT brain mapping is resilient to movement associated with overt speaking. Second, we observed that HD-DOT is sensitive to key activations and deactivations in brain function underlying the perception and naturalistic production of language. Specifically, statistically significant results were observed that show recruitment of regions in occipital, temporal, motor, and prefrontal cortices across all three tasks after performing stringent cluster-extent based thresholding. Our findings lay the foundation for future HD-DOT studies of imaging naturalistic language comprehension and production during real-life social interactions and for broader applications such as presurgical language assessment and brain-machine interfaces.