Measures Of Language Performance Research Articles

Examining Adolescent Language Performance in Discourse Production Across Four Elicitation Tasks.

Comprehensive spoken language assessment should include the evaluation of language use in naturalistic contexts. Discourse elicitation and analysis provides the opportunity for such an evaluation to occur. In this article, our overall aim was to describe adolescents' language performance on four elicitation tasks and determine if there are task-related differences across the elicitation tasks. Forty-four typically developing adolescents with ages ranging from 12;2 to 17;11 (years;months; M = 15;2; 21 boys and 23 girls) participated in the study. They completed four spoken discourse tasks: (a) story generation using a wordless picture book, (b) fable retell, (c) six personal narratives in response to emotion-based prompts, and (d) monologic response to two stories that contained a moral dilemma. Responses were transcribed and analyzed for four language performance measures tapping into language productivity, syntactic complexity, lexical diversity, and verbal facility. Despite individual variability in performance, mean scores were close to median scores for most measures, suggesting a symmetrical distribution. As expected, all four language performance measures were significantly different across the four elicitation tasks. The personal narrative task elicited the longest samples, with the highest verbal fluency. In contrast, both lexical diversity and syntactic complexity were the strongest in response to the fable retell and the moral dilemma tasks. This investigation provides speech-language pathologists with an overview of how task-related factors may impact adolescent language performance. These findings may be used to support their clinical decision-making processes in choosing a suitable discourse task when conducting a comprehensive spoken language assessment. Three hypothetical case examples are used to illustrate the decision-making process. https://doi.org/10.23641/asha.25761768.

Are subjective language complaints in memory clinic patients informative?

ABSTRACT To diagnose mild cognitive impairment, it is crucial to understand whether subjective cognitive complaints reflect objective cognitive deficits. This question has mostly been investigated in the memory domain, with mixed results. Our study was one of the first to address it for language. Participants were 55-to-93-year-old memory clinic patients (n = 163). They filled in a questionnaire about subjective language and memory complaints and performed two language tasks (naming-by-definition and sentence comprehension). Greater language complaints were associated with two language measures, thus showing a moderate value in predicting language performance. Greater relative severity of language versus memory complaints was a better predictor, associated with three language performance measures. Surprisingly, greater memory complaints were associated with better naming, probably due to anosognosia in further disease progression or personality-related factors. Our findings highlight the importance of relative complaint severity across domains and, clinically, call for developing self-assessment questionnaires asking specific questions about multiple cognitive functions.

Simulating Real‐World Context in an Email Writing Task: Implications for Task‐Based Language Assessment

Accomplishing a communication task in the real world requires the ability not only to do the task per se but also to manage aspects of the context in which it occurs. For this reason, simulations of target language use contexts have been incorporated into the design of communicative language tests as a way of enhancing the authenticity of assessment task performance. Although some contextual factors may increase extraneous cognitive load and distract learners from focusing on the task at hand (Sweller, 1994), they represent important design considerations in task‐based language assessment (TBLA), where the purpose of assessment is to determine what second language (L2) learners can do with the target language in the real world. In that sense, the extraneous cognitive load might well be part of the construct we are interested in assessing. Accordingly, the current study simulated aspects of a real‐world task performance context as part of an email writing task assessment. Simulated context was operationalized as (a) additional information about the task scenario, (b) a visual image to simulate the physical context, and (c) an audio to replicate the real‐world experience. A total of 276 L2 English learners performed the email task, half with simulated context and the other half without it. Findings revealed that, when presented with simulated context, the tasks were perceived by participants to have induced more time pressure and to be more interesting. In terms of performance effects, the provision of simulated context negatively affected the syntactic complexity of participants' writing but positively affected their syntactic fluency. It also led to greater discrimination among learners at different proficiency levels on various measures of language performance. The paper concludes by highlighting implications for task design and validity evaluation, especially in TBLA.

Online assessment of narrative macrostructure in adult Irish-English multilinguals.

BackgroundOnline assessment of narrative production and comprehension became an important component of language assessment during the COVID-19 pandemic. This study aimed to establish quantitative measures of narrative macrostructure in the production and comprehension of adult Irish-English bilinguals in an online assessment.MethodsA total of 30 Irish-English bilingual adults participated in an online assessment of oral narrative production and comprehension. Narratives were elicited using LITMUS-MAIN for Irish and English. Story-tell elicitation method was used for all stories. Twenty participants produced Baby Birds and Baby Goats story pairs while 10 participants produced Cat and Dog story pairs. Quantitative measures of story structure, comprehension score, and the overall number of Internal State Terms (ISTs) in production and comprehension were compared across the story pairs, languages, and the output type (production vs. comprehension).ResultsA general linear model indicated no differences in either story structure or story comprehension scores across languages for both sets of stories. Combined analysis for all participants and stories indicated no difference in the story structure scores or comprehension scores across the languages or the story pairs. While the overall number of ISTs was the same across languages, a higher number of ISTs was observed in comprehension relative to production in both languages for Cat and Dog story pair only, but not for Baby birds and Baby goats' stories. The major benefit of using online assessment was the accessibility of participants. The major drawback was the inability to control the environment and the quality of the internet connection.Conclusion and implicationsWhile online assessment increased the availability of participants, which is a significant factor in rural Ireland characterized by low population density and the high percentage of Irish speakers, the availability of stable internet connection limited the applicability of online assessment. Measures of narrative macrostructure were stable across the languages and the story pairs. This is important because of high variability in exposure to Irish, frequent code-switching, and a high number of morphosyntactic errors due to rapid language change that characterizes Irish-English bilinguals. Identifying reliable measures of language performance for Irish-English adult speakers is an important step toward establishing developmental norms for Irish-English bilinguals.

Structural disconnections associated with language impairments in chronic post-stroke aphasia using disconnectome maps

Inconsistent findings have been reported about the impact of structural disconnections on language function in post-stroke aphasia. This study investigated patterns of structural disconnections associated with chronic language impairments using disconnectome maps.Seventy-six individuals with post-stroke aphasia underwent a battery of language assessments and a structural MRI scan. Support-vector regression disconnectome-symptom mapping analyses were performed to examine the correlations between disconnectome maps, representing the probability of disconnection at each white matter voxel and different language scores. To further understand whether significant disconnections were primarily representing focal damage or a more extended network of seemingly preserved but disconnected areas beyond the lesion site, results were qualitatively compared to support-vector regression lesion-symptom mapping analyses.Part of the left white matter perisylvian network was similarly disconnected in 90% of the individuals with aphasia. Surrounding this common left perisylvian disconnectome, specific structural disconnections in the left fronto-temporo-parietal network were significantly associated with aphasia severity and with lower performance in auditory comprehension, syntactic comprehension, syntactic production, repetition and naming tasks. Auditory comprehension, repetition and syntactic processing deficits were related to disconnections in areas that overlapped with and extended beyond lesion sites significant in SVR-LSM analyses. In contrast, overall language abilities as measured by aphasia severity and naming seemed to be mostly explained by focal damage at the level of the insular and central opercular cortices, given the high overlap between SVR-DSM and SVR-LSM results for these scores.While focal damage seems to be sufficient to explain broad measures of language performance, the structural disconnections between language areas provide additional information on the neural basis of specific and persistent language impairments at the chronic stage beyond lesion volume. Leveraging routinely available clinical data, disconnectome mapping furthers our understanding of anatomical connectivity constraints that may limit the recovery of some language abilities in chronic post-stroke aphasia.

Event-Related Potentials to Speech Relate to Speech Sound Production and Language in Young Children

ABSTRACT This research was funded through the American Speech and Hearing Foundation’s 2012 StudentResearch Grant in Early Childhood Language Development awarded to Vanessa Harwood as well as an anonymous generous donation to Haskins Laboratories. Electrophysiological measures of language within early childhood provide important information about neurolinguistic development. We investigated associations between amplitude and latency of the P1 and N2 event-related potential components in response to spoken pseudowords, and clinical measures of language performance within a sample of 58 typically developing children between 24 and 48 months. N2 amplitude differences between repeated and new tokens were correlated with measures of expressive and receptive language and speech sound production. Phonemic sensitivity measured by the N2 component may reflect the integrity of neural networks that are important for speech perception and production in young children.

Correlation of natural language assessment results with health-related quality of life in adult glioma patients.

Impairments of speech are common in patients with glioma and negatively impact health-related quality of life (HRQoL). The benchmark for clinical assessments is task-based measures, which are not always feasible to administer and may miss essential components of HRQoL. In this study, the authors tested the hypothesis that variations in natural language (NL) correlate with HRQoL in a pattern distinct from task-based measures of language performance. NL use was assessed using audio samples collected unobtrusively from 18 patients with newly diagnosed low- and high-grade glioma. NL measures were calculated using manual segmentation and correlated with Quality of Life in Neurological Disorders (Neuro-QoL) outcomes. Spearman's rank-order correlation was used to determine relationships between Neuro-QoL scores and NL measures. The distribution of NL measures across the entire patient cohort included a mean ± SD total time speaking of 11.5 ± 2.20 seconds, total number of words of 27.2 ± 4.44, number of function words of 10.9 ± 1.68, number of content words of 16.3 ± 2.91, and speech rate of 2.61 ± 0.20 words/second. Speech rate was negatively correlated with functional domains (rho = -0.62 and p = 0.007 for satisfaction with social roles; rho = -0.74 and p < 0.001 for participation in social roles) but positively correlated with impairment domains (rho = 0.58 and p = 0.009 for fatigue) of Neuro-QoL. Assessment of NL at the time of diagnosis may be a useful measure in the context of treatment planning and monitoring outcomes for adult patients with glioma.

Characterizing the relationship between lesion-activation distance using fMRI and verbal measures in brain tumor patients

Functional resonance magnetic imaging (fMRI) allows for identification of eloquent cortex in pre-treatment planning. Previous studies have shown a correlation among lesion to activation distance (LAD) measures and morbidity and mortality. This study investigates the relationship between LAD, well-established language centers (Wernicke’s and Broca’s), and language performance measures. We included a sample population of brain tumor patients that received language fMRI (verbal fluency and sentence verification) for pre-treatment assessment (n = 51). LAD to the nearest language area was measured and divided into groups ≤ 10 mm and > 10 mm. Verbal fluency scores were compared between these groups. Additionally, patients were divided into similar groups based on LAD to either Broca’s or Wernicke’s areas, and the verbal fluency scores and sentence verification accuracy (n = 29) were subsequently compared between groups. Brain tumor patients with LAD ≤ 10 mm to either language area had significantly lower verbal fluency scores (p = 0.028). The difference in verbal fluency scores between groups with LAD ≤ 10 mm and > 10 mm to Wernicke’s area trends toward significance (p = 0.067). The sentence verification accuracy was significantly lower in patients with LAD ≤ 10 mm to either language area (p = 0.039). These findings suggest that there exists a significant relationship between LAD to language centers and measures; greater language deficits are seen when LAD ≤ 10 mm.

Inter-rater reliability, intra-rater reliability and internal consistency of the Brisbane Evidence-Based Language Test

Purpose To examine the inter-rater reliability, intra-rater reliability, internal consistency and practice effects associated with a new test, the Brisbane Evidence-Based Language Test. Methods Reliability estimates were obtained in a repeated-measures design through analysis of clinician video ratings of stroke participants completing the Brisbane Evidence-Based Language Test. Inter-rater reliability was determined by comparing 15 independent clinicians’ scores of 15 randomly selected videos. Intra-rater reliability was determined by comparing two clinicians’ scores of 35 videos when re-scored after a two-week interval. Results Intraclass correlation coefficient (ICC) analysis demonstrated almost perfect inter-rater reliability (0.995; 95% confidence interval: 0.990–0.998), intra-rater reliability (0.994; 95% confidence interval: 0.989–0.997) and internal consistency (Cronbach’s α = 0.940 (95% confidence interval: 0.920–1.0)). Almost perfect correlations (0.998; 95% confidence interval: 0.995–0.999) between face-to-face and video ratings were obtained. Conclusion The Brisbane Evidence-Based Language Test demonstrates almost perfect inter-rater reliability, intra-rater reliability and internal consistency. High correlation coefficients and narrow confidence intervals demonstrated minimal practice effects with scoring or influence of years of clinical experience on test scores. Almost perfect correlations between face-to-face and video scoring methods indicate these reliability estimates have direct application to everyday practice. The test is available from brisbanetest.org. Implications for Rehabilitation The Brisbane Evidence-Based Language Test is a new measure for the assessment of acquired language disorders. The Brisbane Evidence-Based Language Test demonstrated almost perfect inter-rater reliability, intra-rater reliability and internal consistency. High reliability estimates and narrow confidence intervals indicated that test ratings vary minimally when administered by clinicians of different experience levels, or different levels of familiarity with the new measure. The test is a reliable measure of language performance for use in clinical practice and research.

Using General Language Performance Measures to Assess Grammar Learning

Grammar is an important goal area for children with language impairment. The current article considers the use of General Language Performance Measures (GLPMs) to assess outcomes for this basic goal area and for 3 intermediate grammar goals that contribute to children's developing ability to construct increasingly longer and more complex sentences: production of word combinations, production of sentences with required and optional constituents, and production of complex sentences. These goals are very important because they address a significant deficit area for children with language impairment and enable these children to increase the informativeness of their utterances. Using GLPMs in addition to session data and probes enables speech–language pathologists to determine whether these forms are being incorporated into conversational speech.

Le langage oral en production chez les enfants bilingues : quels liens avec l’exposition ?

L’objectif de cette étude est d’observer les effets de l’exposition aux langues sur différentes variables linguistiques en production orale d’enfants bilingues français/anglais sans pathologie, scolarisés dans une école internationale publique (France), et répartis en deux groupes d’âges (groupe 1 :N= 10,M= 6;3; groupe 2 :N= 15,M= 10;2). Des questionnaires parentaux ont permis d’obtenir des mesures quant à l’exposition courante et cumulative aux langues ainsi qu’à la fréquence à laquelle les enfants lisent dans les deux langues. À partir de tests de vocabulaire réceptif standardisés et de productions narratives(Frog, where are you?), des données linguistiques françaises et anglaises ont été obtenues. Les narrations ont été transcrites en respectant les conventionsCHILDESdans le logicielCLAN, permettant l’observation de diverses mesures linguistiques relatives au lexique, à la morphosyntaxe et au discours. Des corrélations ont été effectuées entre les mesures d’exposition et celles des performances linguistiques. Les résultats révèlent que le vocabulaire réceptif et certaines mesures de productivité narrative semblent être sensibles au degré d’exposition de chaque langue. De plus, nos conclusions renforcent l’idée du rÔle essentiel de la lecture sur la performance linguistique, et ce d’autant plus pour les enfants les plus jeunes. Pour finir, cette étude souligne l’importance de travailler en étroite collaboration avec les équipes pédagogiques et les parents d’enfants bilingues afin de fournir des recommandations concrètes pour enrichir le langage oral.

Improving language comprehension in preschool children with language difficulties: a cluster randomized trial.

Children with language comprehension difficulties are at risk of educational and social problems, which in turn impede employment prospects in adulthood. However, few randomized trials have examined how such problems can be ameliorated during the preschool years. We conducted a cluster randomized trial in 148 preschool classrooms. Our intervention targeted language comprehension skills and lasted 1 year and 1 month, with five blocks of 6 weeks and intervention three times per week (about 75min per week). Effects were assessed on a range of measures of language performance. Immediately after the intervention, there were moderate effects on both near, intermediate and distal measures of language performance. At delayed follow-up (7 months after the intervention), these reliable effects remained for the distal measures. It is possible to intervene in classroom settings to improve the language comprehension skills of children with language difficulties. However, it appears that such interventions need to be intensive and prolonged.

Genome-wide association study of language performance in Alzheimer’s disease

Language impairment is common in prodromal stages of Alzheimer’s disease (AD) and progresses over time. However, the genetic architecture underlying language performance is poorly understood. To identify novel genetic variants associated with language performance, we analyzed brain MRI and performed a genome-wide association study (GWAS) using a composite measure of language performance from the Alzheimer’s Disease Neuroimaging Initiative (ADNI; n=1560). The language composite score was associated with brain atrophy on MRI in language and semantic areas. GWAS identified GLI3 (GLI family zinc finger 3) as significantly associated with language performance (p<5×10−8). Enrichment of GWAS association was identified in pathways related to nervous system development and glutamate receptor function and trafficking. Our results, which warrant further investigation in independent and larger cohorts, implicate GLI3, a developmental transcription factor involved in patterning brain structures, as a putative gene associated with language dysfunction in AD.

Measurement Issues: Assessing language skills in young children.

Language and communication skills are central to children's ability to engage in social relationships and access learning experiences. This paper identifies issues which practitioners and researchers should consider when assessing language skills. A range of current language assessments is reviewed. Current screening measures do not meet psychometric prerequisites to identify language problems. There are significant challenges in the interpretation of language assessments, where socioeconomic status, language status and dialect, hearing impairment and test characteristics impact results. Psychometrically sound assessments of language are an essential component of developing effective and efficient interventions. The language trajectories of preschool children vary substantially; current screening measures have significant limitations. Composite measures of language performance are better indicators of language problems and disorders than single measures of component skills.

Raters’ decisions, rating procedures and rating scales

This special issue of Language Testing explores raters’ evaluations of L2 proficiency and possible causes of variability of rater judgments. In addition to the analysis of rater behavior and rater consistency, we investigate the relationship between general measures of oral and written L2 performance concerning complexity, accuracy and fluency of L2 production and overall judgments of oral and written L2 performance by raters, based on holistic rating scales. Finally, the use of rating scales in different contexts and for different types of learners is also examined. In this introduction the three central themes presented in the various contributions are briefly discussed: rater behavior and rater consistency, rater judgments and measures of language performance, and the use of global rating scales.

Investigating language lateralization during phonological and semantic fluency tasks using functional transcranial Doppler sonography

Although there is consensus that the left hemisphere plays a critical role in language processing, some questions remain. Here we examine the influence of overt versus covert speech production on lateralization, the relationship between lateralization and behavioural measures of language performance and the strength of lateralization across the subcomponents of language. The present study used functional transcranial Doppler sonography (fTCD) to investigate lateralization of phonological and semantic fluency during both overt and covert word generation in right-handed adults. The laterality index (LI) was left lateralized in all conditions, and there was no difference in the strength of LI between overt and covert speech. This supports the validity of using overt speech in fTCD studies, another benefit of which is a reliable measure of speech production.

Multidimensional language performance in training teachers for secondary content instruction through English

Abstract This exploratory study focuses on four non-native English speaking secondary content teachers in a short-term immersion program aimed at introducing them to language teaching methods for secondary school content instruction through the medium of English. Such programs have been found to have largely mixed results for language performance. This may be especially true of programs for teachers because task adequacy in teaching may not be shown by improved scores on traditional language performance measures. This study uses multiple complexity, accuracy and fluency measures to analyze transcripts of paired peer microteaching tasks at the beginning and end of a ten-week program. Analysis of the results shows that in general the teachers' language performances improved, but that there were complex interactions among the measures. In general the performance measures reflected task adequacy, but they may have been affected by nonlinguistic factors such as teaching aims. Although the study is limited because of its small size, it suggests that teacher language performances are complex and dynamic, and that even short-term programs with non-linguistic aims can have a positive effect on language performances.

Augmentative and Alternative Communication and Language

Summary measures based on logfile datafrom language samples have their foundationin well-documented traditional measures oflanguage performance. Many of the utterance-based and word-based summary measuresfound in the AAC performance report(Figure 3) should be familiar to cliniciansroutinely analyzing language samples or usingSALT. These summary measures may includethe number of complete utterances, meanlength of utterance, number of differentword roots, or total number of words usedin the sample. An AAC performance reportcould be used to compare back to the EBPexample about the clinician interested in corevocabulary. Use of LAM allows the clinician togenerate a word frequency list to comparethe core vocabulary used by her client withthe core vocabulary from one or more ofthe studies found as evidence. In addition,Hill (2001) reported performance indiceson several SALT summary measures for theadults in the study. Consequently, an AACperformance report would provide severalrelated statistics to chart progress related tovocabulary activity with the AAC system.Many clinicians are concerned with vocab-ulary use by preschoolers rather than adults,but the same principles of EBP apply. Vo-cabulary lists generated using PeRT could be

General Language Performance Measures in Spoken and Written Narrative and Expository Discourse of School-Age Children With Language Learning Disabilities

Language performance in naturalistic contexts can be characterized by general measures of productivity, fluency, lexical diversity, and grammatical complexity and accuracy. The use of such measures as indices of language impairment in older children is open to questions of method and interpretation. This study evaluated the extent to which 10 general language performance measures (GLPM) differentiated school-age children with language learning disabilities (LLD) from chronological-age (CA) and language-age (LA) peers. Children produced both spoken and written summaries of two educational videotapes that provided models of either narrative or expository (informational) discourse. Productivity measures, including total T-units, total words, and words per minute, were significantly lower for children with LLD than for CA children. Fluency (percent T-units with mazes) and lexical diversity (number of different words) measures were similar for all children. Grammatical complexity as measured by words per T-unit was significantly lower for LLD children. However, there was no difference among groups for clauses per T-unit. The only measure that distinguished children with LLD from both CA and LA peers was the extent of grammatical error. Effects of discourse genre and modality were consistent across groups. Compared to narratives, expository summaries were shorter, less fluent (spoken versions), more complex (words per T-unit), and more error prone. Written summaries were shorter and had more errors than spoken versions. For many LLD and LA children, expository writing was exceedingly difficult. Implications for accounts of language impairment in older children are discussed.

The Breakfast Club: Results of a study examining the effectiveness of a multi-modality group communication treatment

Twenty mid-stage Alzheimer's patients at the Jewish Home and Hospital in the Bronx participatedfor 12 weeks each in four groups offive in a five-day-a-week program of structured multi-modality group communication intervention called “The Breakfast Club.” Twenty matched patients participated in a standard conversation group and served as controls. The Breakfast Club attempted to incorporate all that was currently known about the residual communication strengths of Alzheimer's patients and about previous treatments shown to be effective with this population. Results showed that Breakfast Club participants improved significantly on measures of language performance, functional independence and use of social communication while control subjects did not. Breakfast Club members also showed significant increases in “interest and involvement” and the use of procedural memories over the 12-week period.