Measures Of Language Proficiency Research Articles

Validation of an elicited Imitation test as a measure of Korean language proficiency

This study presents a modified version of the Korean Elicited Imitation (EI) test, designed to resemble natural spoken language, and validates its reliability as a measure of proficiency. The study assesses the correlation between average test scores and Test of Proficiency in Korean (TOPIK) levels, examining score distributions among beginner, intermediate, and advanced learner groups. Using item response theory (IRT), the study explores the influence of four key facets—learners, items, raters, and constructs—on performance evaluation. An explanatory item response modeling (EIRM) analysis identified linguistic factors impacting the EI test’s performance. Notably, the study uncovered a robust positive correlation between EI test scores and TOPIK levels. Significant score disparities were observed between beginner and intermediate, as well as beginner and advanced, learner cohorts. An IRT-based exploration of each facet revealed that item difficulty was comparatively lower in contrast to learners’ commendable performance, and raters exhibited a high degree of scoring consistency. The EIRM analysis underscores the significance of variables such as the number of syllables, vocabulary score, and content word density in influencing test performance.

Subjective versus objective language proficiency measures in the investigation of bilingual effects on cognitive control

Purpose: Bilingual language experience is thought to confer non-linguistic benefits in general cognition including improved cognitive control. These bilingual effects are most often observed in samples of bilinguals who are highly proficient in both languages. However, across the majority of previous studies, assessments of language proficiency are exclusively subjective. While evidence supports that subjective and objective measures of language proficiency are correlated, no studies have explored whether the use of either measure impacts on model results when investigating bilingual effects on cognitive control. Methodology: Mandarin-English bilingual young adults completed both subjective and objective assessments of language proficiency and a Simon task to measure differences in cognitive control. Data and Analysis: Data were analyzed using linear mixed-effects models to best account for differences in linguistic and non-linguistic variables as well as the repeated-measures nature of the Simon task. Findings and Conclusions: We report no evidence in support of improved cognitive control associated with higher levels of language proficiency. Crucially, results did not differ when either subjective or objective measures were included in our models. Results support that both subjective and objective assessments of language proficiency may be equivalent when modeling bilingual effects on cognitive control. Originality: This study is the first direct investigation of the influence of the proficiency assessment method on model results. Implications: The findings of this study have implications for the assessment of language proficiency in future investigations of bilingual effects.

The Use of Pronoun Interpretation Biases in L1 Spanish: The Role of Language Proficiency, Exposure, and Use in Heritage Speakers

Abstract In an offline sentence comprehension task, we test the interpretation biases for null and overt pronouns and anaphoric and cataphoric pronouns in heritage speakers (HSs) whose first language (L1) is Spanish (L2 English). Previous studies have shown that Spanish HSs can interpret overt anaphoric pronouns differently than monolingual speakers. A question that remains unexplored is how HSs interpret cataphoric pronouns and how L1 experience affects pronoun resolution biases in these individuals. We recruited 48 HSs and 48 monolingual Spanish speakers. The results show that pronoun interpretation in HSs differs from monolingual biases, with HSs demonstrating a higher subject antecedent preference for anaphoric and cataphoric pronouns. The results further show that a continuous measure of language proficiency, exposure, and use may contribute to explaining pronoun interpretation biases in heritage Spanish. We discuss the findings in light of recent research demonstrating that language experience can contribute to shape discourse patterns.

Examining the Reliability of the International English Language Testing System Design

Background: Despite the evolution of the International English Language Testing System (IELTS) to accommodate the evolving needs of test-takers and institutions, concerns have been raised regarding its reliability in assessing English language proficiency. Purpose: This study aimed to investigate the reliability of the International English Language Testing System (IELTS) design as a measure of English language proficiency. Methodology: To explore this issue, the research reviewed relevant literature and employed various methods, including frequency analyses, data analytics, and machine learning algorithms based on mathematical principles. Unlike previous studies, this work analysed a dataset of 33,505 authentic band scores, allowing for a comprehensive examination of IELTS test score patterns. Findings: The findings revealed that the concept of reliability is complex, and the statistical methods used by IELTS test providers, based on classical test theory, may provide approximate rather than precise scores. Relying solely on the reputation and authority of the IELTS test is insufficient to establish its reliability, as it involves logical fallacies and arguments from authority. Research implications: It is necessary to conduct further independent research to ensure the IELTS test's reliability and address the concerns expressed by researchers and test-takers.

The Effect of Language Dominance on Classic Semantic, Action, Emotional, and Phonemic Fluency in Unbalanced Bilinguals.

Verbal fluency evaluation in bilingual speakers should include dual-language assessment to obtain a comprehensive profile of word retrieval abilities. This study is the first to compare classic semantic, action, emotional, and phonemic fluency in terms of the magnitude of their performance gaps between the dominant and nondominant language in unbalanced bilingual speakers. We also examined the quantitative relationship between language dominance and verbal fluency performance. Twenty-six bilingual adults completed a comprehensive set of classic semantic ("animals," "vegetables"), action ("do"), emotional ("happy," "sad," "afraid"), and phonemic ("F," "A," "S") fluency tasks in their dominant language (English) and nondominant language (Spanish) in two sessions on separate days. Participants also completed subjective and objective measures of language proficiency. All tasks yielded fewer correct responses in the nondominant language. The between-languages performance gap was the largest for "animals" and the smallest for emotional fluency. "Happy" yielded the most balanced performance among all semantic tasks and a positivity bias that was unaffected by language dominance. Finally, language dominance scores computed by a newly developed formula indicated relationships between self-rated proficiency and fluency performance. This study provides preliminary, normative data of classic semantic, action, emotional, and phonemic fluency that could be used to gauge unbalanced bilingual speakers' performance. Significant impacts of language dominance on "animals" demand caution in using this widely used classic semantic category in evaluating bilingual speakers' performance. The data also underscore the robustness of positivity biases in emotional fluency and the validity of using subjective measures to supplement neuropsychological assessment of fluency performance.

Clinical Neuropsychology of Bilingual Mexican American Adults: Effect of Language Proficiency and Dominance.

To assess the association between language proficiency and absolute dominance on language versus visual-perceptual formatted and executive versus delayed memory neuropsychological measures in bilingual adults. Data from 55 bilingual, conversationally fluent, neurologically intact, Mexican American, consecutive, adults tested in separate sessions in Spanish and English in a counterbalanced order were analyzed. Age, years of education, self-reported language proficiency, Woodcock-Muñoz Language Survey-Revised (WMLS-R) picture vocabulary measures of language proficiency, and dominance (absolute Spanish-English WMLS-R difference scores) were correlated with 11 measures from La Batería Neuropsicólogica en Español and its original English language tests. Self-reported and WMLS-R measures of language proficiency were significantly correlated in each language. Absolute language dominance was not significantly associated with any Spanish or English neuropsychological raw score. The WMLS-R language proficiency, but not age or years of education, was significantly correlated with language-formatted neuropsychological measures of California Verbal Learning Test delayed free recall number of words (both languages), letter fluency (both languages), delayed story memory (in English), and Stroop interference (Spanish). Linear regression models using age, years of education, and WMLS-R picture vocabulary scores as predictors were significant for all these measures excepting the last. The WMLS-R language proficiency was not significantly associated with raw scores on any visual-perceptual formatted measure. Monolingual neuropsychological test norms for language-formatted tests likely overestimate bilingual Mexican Americans' performance.

FACTORS AFFECTING LISTENER PERCEPTION OF ACCENTED SPEECH: THE ROLE OF ACCENT FAMILIARITY AND LINGUISTIC TRAINING

ABSTRACT Listeners have been shown to judge second language (L2) users more negatively than L1 speakers on measures of language proficiency and even on personal qualities. However, less is known about what factors affect listener judgment which this study seeks to explore. In the study, one first language (L1) English speaker recorded her spontaneous speech (i.e. the baseline recording). Three other speakers of English (with Northern American, moderate Chinese, and heavy Chinese accents) read aloud two scripts controlled for content and language (one grammatical and one ungrammatical), resulting in six scripted recordings. Six-hundred and eighty-seven listeners with mixed L1s were randomly allocated into six groups. Each group listened to the same baseline recording and a different scripted recording. They evaluated the speakers’ proficiency on 9-point scales and reported their background information via an online questionnaire. Results suggested that speakers’ accents negatively influenced listeners’ proficiency judgment. However, a threshold was observed regarding this influence; specifically, the moderately-accented speaker was not perceived negatively compared to L1 speakers. Multiple regression showed that listeners’ accent familiarity and linguistic training were positively associated with their rating severity, meaning that these variables have the potential to affect listeners’ perceptual bias against L2 users.

LexMAL: A quick and reliable lexical test for Malay speakers.

Objective language proficiency measures have been found to provide better and more consistent estimates of bilinguals' language processing than self-rated proficiency (e.g., Tomoschuk et al., 2019; Wen & van Heuven, 2017a). However, objectively measuring language proficiency is often not possible because of a lack of quick and freely available language proficiency tests (Park et al., 2022). Therefore, quick valid vocabulary tests, such as LexTALE (Lemhöfer & Broersma, 2012) and its extensions (e.g., LexITA: Amenta et al., 2020; LEXTALE-FR: Brysbaert, 2013; LexPT: Zhou & Li, 2022) have been developed to reliably assess language proficiency of speakers of various languages. The present study introduces a Lexical Test for Malay Speakers (LexMAL), which estimates language proficiency for Malay first language (L1) and second language (L2) speakers. An initial 180-item LexMAL prototype was evaluated using 60 Malay L1 and 60 L2 speakers in Experiment 1. Sixty words and 30 nonwords with the highest discriminative power that span across the full difficulty range were selected for the final LexMAL based on point-biserial correlations and an item response theory analysis. The validity of LexMAL was demonstrated through a reliable discrimination between L1 and L2 speakers, significant correlations between LexMAL scores and performance on other Malay language tasks (i.e., translation accuracy and cloze test scores), and LexMAL outperforming self-rated proficiency. A validation study (Experiment 2) with the 90-item final LexMAL tested with a different group of Malay L1 (N = 61) and L2 speakers (N = 61) replicated the findings of Experiment 1. LexMAL is freely available for researchers at www.lexmal.org .

Language pedagogies and late-life language learning proficiency

Abstract Late-life language learning has gained considerable attention in recent years. Strikingly, additional language (AL) proficiency development is underinvestigated, despite it potentially being one of the main drivers for older adults to learn an AL. Our study investigates whether Dutch older adults learning English for three months significantly improve their AL skills, and if explicit or implicit language instruction is more beneficial. Sixteen learners participated in online weekly group lessons, five days of 60-min homework, and pre-post-retention tests. Half were randomly assigned to the mostly explicit condition and half to the mostly implicit condition. Data includes language proficiency measures and 201 dense-data spoken homework samples. Results show improvements in several areas for both conditions. For structural errors in homework, we found implicitly taught participants to make significantly more mistakes. Our exploratory data show that older adults significantly develop AL proficiency after a short language training, and, as we only found differences between conditions on one construct, that teaching pedagogies do not play a substantial role.

Using multiple measures of language dominance and proficiency in Farsi-English bilingual children

This paper aims to identify effective means of measuring dominance and proficiency in bilingual children. Thirty-seven Farsi-English Heritage language speaking children from 6;1 to 11;6 were assessed on their vocabulary, morphosyntax, and narrative microstructure skills in both languages to address whether there is a difference between their proficiency in Farsi as a heritage and English as a majority language, how the scores on the vocabulary, morphosyntax, and narrative microstructure tasks relate to one another, and based on the results of each task in both languages if any of the children are at risk for a Developmental Language Disorder. Vocabulary was assessed using the LITMUS- Cross-Linguistic Lexical Task (CLT), morphosyntax using the LITMUS-Sentence Repetition (SR) tasks, and Narrative microstructure using the LITMUS-Multilingual Assessment Instrument for Narratives (MAIN). Individual language proficiency was identified via an in-depth profile analysis for each participant who looked at their performance on all experimental tasks in both languages. The data demonstrated that on the vocabulary and narrative tasks the participants were more dominant in English than in Farsi, while on the sentence repetition task there were no significant differences between the two languages. Correlation analyses showed that vocabulary scores were strongly correlated to the sentence repetition scores and the microstructure scores. The English and Farsi sentence repetition scores also correlated moderately with the microstructure scores within each language. Profile analysis showed that no child within the study scored < 1.5 or 2 standard deviations below the mean on more than two tasks in both languages. However, interesting patterns emerged indicating that some participants had a greater proficiency in one language vs. the other language. The results from this study showed that measuring language within a single domain (e.g., morphosyntax) is not enough to identify a bilingual child's language dominance and/or proficiency. Instead, an in-depth profile analysis and language assessments across various language domains need to be done in order to appropriately measure language dominance and proficiency. Consequently, this study supports the importance of measuring language across multiple domains in studies of bilingual children. The clinical significance of appropriately identifying language dominance and proficiency was also shown, as such information would allow clinicians to make more appropriate clinical decisions.

Are people really less moral in their foreign language? Proficiency and comprehension matter for the moral foreign language effect in Russian speakers.

Previous work has demonstrated that people are more willing to sacrifice one person to save five in a foreign language (FL) than in their native tongue. This may be due to the FL either reducing concerns about sacrificial harm (deontological inclinations) or increasing concerns about overall outcomes (utilitarian inclinations). Moreover, proficiency in a foreign language (FL) may moderate results. To test these possibilities, we investigated the moral foreign language effect (MFLE) in a novel sample of Russian L1/English FL speakers. We employed process dissociation (PD)-a technique that independently assesses concerns about rejecting harm and maximizing outcomes in sacrificial dilemmas, and we assessed measures of objective and subjective foreign language proficiency and of dilemma comprehension. Results replicated the pattern of increased acceptance of sacrificial harm in FL demonstrated in earlier studies, but a PD analysis showed no evidence of increased concerns for utilitarian outcomes in a FL; instead, this pattern was driven by reduced concerns regarding sacrificial harm. However, people who reported better dilemma comprehension in the FL demonstrated both stronger deontological and utilitarian responding, and people with higher objective proficiency displayed stronger utilitarian responding in the FL than those with lower proficiency. These findings show that utilitarian inclinations are affected by reading dilemmas in a foreign language mainly in low-proficiency speakers, and that while emotional concerns for sacrifice are reduced in FL, better comprehension can increase such concerns as well as concern for outcomes.

Examining the comparability of parallel English and Chinese elicited imitation tasks

The suite of elicited imitation tasks (EITs) developed originally by Ortega et al. (2002) has been validated as an effective measure of second language (L2) proficiency and adopted in different L2 studies. However, the comparability of the crosslinguistic parallel EITs, all developed based on the English version, has yet to be explored. This exploratory study investigated the comparability of Ortega et al.’s parallel English and Chinese EITs by analyzing L2 English data collected from 82 L2 English participants in Wu et al. (2022) and L2 Chinese data collected from 80 L2 Chinese participants in Wu and Ortega (2013). Despite the crosslinguistic variation observed between the English and Chinese EITs, the converging evidence gathered from internal consistency, item discriminatory power, correlations with external proficiency measures, and abilities to discriminate four speaking levels indicates that the two parallel EITs tap roughly the same language proficiency components in each language. Crucially, the classification drawn from the same CAF measures across both languages suggests that the five proficiency levels of the EIT scores (i.e., Novice = 0–24; Low = 25–48; Intermediate = 49–72; High = 73–96; Advanced = 97–120) can consistently discriminate language abilities into distinctive levels across both languages. We hope the proposed five proficiency levels will enhance the application and interpretation ability of Ortega et al.’s parallel EITs in crosslinguistic, bilingual, and multilingual research.

Caregiver Experiences With Oral Bilingualism in Children Who Are Deaf and Hard of Hearing in the United States: Impact on Child Language Proficiency.

Best practices recommend promoting the use of the home language and allowing caregivers to choose the language(s) that they want to use with their child who is deaf or hard of hearing (DHH). We examined whether Spanish-speaking caregivers of children who are DHH receive professional recommendations on oral bilingualism that follow best practices. We also assessed whether professional recommendations, caregiver beliefs, and language practices had an impact on child language(s) proficiency. Sixty caregivers completed a questionnaire on demographic questions, language(s) use and recommendations, beliefs on bilingualism, and child language proficiency measures in English, Spanish, and American Sign Language (ASL). Professional recommendations on oral bilingualism were reported descriptively, and linear regression was used to identify the predictors of child language(s) proficiency. We found that only 23.3% of the caregivers were actively encouraged to raise their child orally bilingual. Language practices predicted child proficiency in each language (English, Spanish, and ASL), but professional recommendations and caregiver beliefs did not. Our results revealed that most caregivers received recommendations that do not follow current best practices. Professional training is still needed to promote bilingualism and increase cultural competence when providing services to caregivers who speak languages different from English. https://doi.org/10.23641/asha.21644846.

Assessing language background and experiences among heritage bilinguals.

The language backgrounds and experiences of bilinguals have been primarily characterized using self-report questionnaires and laboratory tasks, although each of these assessments have their strengths and weaknesses. The Electronically Activated Recorder (EAR), an audio recording device, has recently become more prominent as a method of assessing real-world language use. We investigated the relationships among these three assessment tools, to understand the shared variance in how these measures evaluated various aspects of the bilingual experience. Participants were 60 Southern California heritage bilingual college students who spoke a variety of heritage languages and began to learn English between the ages of 0-to 12-years. Participants completed both self-report and laboratory-based measures of language proficiency and use, and they wore the EAR for 4 days to capture representative samples of their day-to-day heritage language (HL) use. The results indicated that self-reported HL use and English age of acquisition were significant predictors of real-world language use as measured by the EAR. In addition, self-reported HL proficiency and laboratory-based HL proficiency, as measured by verbal fluency, were mutually predictive. While some variability was shared across different assessments, ultimately, none of the measures correlated strongly and each measure captured unique information about the heritage bilingual language experience, highlighting the dissociation between language experience measured at a single point in time and an accumulated life history with a heritage language. These findings may provide guidance for bilingualism researchers about which assessment tool, or combination of tools, may be best for their specific research questions.

The Role of Mother’s Education and Early Skills in Language and Literacy Learning Opportunities

The purpose of this study was to examine the relationship between language and literacy learning opportunities, early skills, socioeconomic status, and mother’s education in children’s early education. In addition, we attempted to determine the language domains needed for training as well as the ideal duration for language and literacy learning for children. We used a mixed methods design to assess early childhood language and literacy skills. The study involved 2550 early childhood learners, 235 teachers, and parents of learner participants from 112 kindergartens. The standard measure of language proficiency was used to measure the learners’ early language and literacy skills. Mother’s education and learners’ early abilities were positively correlated with learners’ language and literacy learning opportunities. Learners who have mothers with a high level of education receive significant opportunities to learn language and literacy compared to those with mothers with a low level of education. Therefore, their language skills are superior. There are 12 language domains needed for training in early childhood and the ideal duration of language learning for children is 26 minutes per day or 182 minutes a week. The language domains are determining factors for obtaining reading skills in the future. Mother’s education and child’s early skills really help children in language and literacy learning. The implication of this research is that literacy programs are needed for early childhood learners who have low initial abilities and children with mothers with a low level of education.

Dual-Language Testing of Emotional Verbal Fluency: A Closer Look at "Joy," "Sadness," "Fear," "Anger," and "Disgust".

Assessing neuropsychological functions of dual-language speakers with semantic verbal fluency should consider the impact of language proficiency. Much evidence for the proficiency effect is found in tasks targeting neutral words. The proficiency effect on emotional verbal fluency (Emo-VF) that targets emotional word retrieval, however, is far from conclusive. This study aimed to clarify the proficiency effect on Emo-VF, specifically the extent to which language proficiency impacted positive and negative word retrieval comparably. In addition, the study examined the extent to which dual-languages speakers produced duplicated items and unique, non-duplicated items in each of two languages tested. Thirty-two adult participants completed Emo-VF tasks that targeted a comprehensive set of basic emotions ("joy," "sadness," "fear," "anger," "disgust") in English and Spanish in two sessions separated by at least 72h. All participants exhibited greater proficiency in English than in Spanish according to subjective and objective measures of language proficiency. Verbal productivity was comparably lowered for all emotions in the less proficient language. Differences among categories were consistent between languages, with "joy" eliciting more words than other emotions, and "fear" yielding the highest productivity among negative emotions. Finally, dual-language speakers produced a significant number of unique items in different languages, especially for "fear." Language proficiency exerts a general effect on positive and negative word retrieval. The study extends previous work by revealing differences between "fear" and other negative categories in addition to replicating positivity biases in unbalanced dual-language speakers. Dual-language testing is valuable in capturing the richness of emotional word repertoire associated with different languages.

Developing a morphological awareness intervention through inquiry‐based learning

Secondary school students are increasingly exposed to texts containing morphologically complex words. Morphological knowledge may help with understanding those words. Appropriate educational materials to teach morphological awareness (MA) are scarce. A design-based approach was used to develop an MA-intervention for Dutch secondary school students. We (1) analysed the aforementioned problem through semi-structured discussions with teachers and a review of previous MA-intervention research; (2) developed a prototype of an MA-intervention in collaboration with teachers; (3) evaluated materials from Step 2 quantitatively through a pilot study with seventh grade students (N = 60) and qualitatively through a semi-structured discussion with teacher educators (N = 4); and (4) reflected on the proposed design principles. Insights obtained from teachers and the literature were implemented in our MA-intervention. Students' performance was related to a measure of general language proficiency. Finally, teachers, students, and teacher educators pointed out several strengths and weaknesses of the intervention. The present study highlights the importance of the following design principles: focus on the patterns in morphology in order for students to explicate their morphological knowledge, suitability of the intervention for students with varying levels of language proficiency, and availability of training and background information about MA for teachers. Collaborating with teachers provided indispensable insights into students' needs, knowledge of MA, how MA-interventions can be implemented and matched to teachers' pedagogical expertise and experience.

An Exploratory Study of Strategy Use on Elicited Imitation Tasks

Elicited imitation (EI) has gained popularity with recent interests in the quest for efficient and flexible measures of second language (L2) proficiency. Despite the surge of interests, questions remain as to what specific linguistic knowledge, skills, and strategies EI measures. To contribute to this line of inquiry, this study explored the nature of strategy use and its effect on EI performance to elucidate the constructs of EI. Twenty-four L2 learners and eight native speakers of Chinese completed an EI test of Chinese and a strategy use questionnaire after the test. Qualitative analyses of the questionnaire responses revealed that participants mainly employed five types of strategies, including approach strategies, cognitive strategies, metacognitive strategies, communication strategies, and test-wiseness strategies. While native speakers reported the least number of strategies, higher-proficiency L2 learners reported more strategies than lower-proficiency L2 learners. We further subjected strategy use, along with participant proficiency level, item length, and item complexity level, to linear mixed-effects regression analyses. The results showed that participant proficiency level, item length, and item complexity level explained the largest test score variance; in contrast, strategy use of different types only accounted for a smaller proportion. The total number of cognitive strategies had a significant, positive effect on EI performance whereas the total number of metacognitive strategies had a significant, negative effect. These findings offer some insights into the nature of speech comprehension and production on EI and provide validity evidence for the use of EI as a language proficiency measure.

Intricacies of the Multifaceted Triad-Complexity, Accuracy, and Fluency: A Review of Studies on Measures of Oral Production

Complexity, accuracy, and fluency (CAF) as measures of language proficiency have been used extensively in assessing second language production in English language studies related to task-based language teaching. The present review adds the following critical elements to the existing body of literature: (i) Provides synthesis and critical analysis of various measures of CAF used in empirical studies and reveals the variation with which the three constructs have been used; (ii) Lays down the most appropriate CAF measures for beginners, intermediates and advanced learners for a valid and robust assessment of oral production in English as a second and foreign language.

Evaluating the Efficacy of an English Language Development Program for Middle School English Learners

This paper reports the outcomes of an experimental evaluation of Direct Instruction Spoken English (DISE), an English language curriculum that focuses on developing English speaking, listening, and comprehension skills in English Learners (ELs). Twenty-nine middle schools in three states were randomly assigned to teach DISE Level 1 or their typical English language development program. Treatment teachers received two days of training and taught daily lessons. Project staff assessed 746 sixth and seventh-grade students with a proximal measure of English language proficiency and distal measures of language and oral reading fluency. Analyses of oral English language suggested differences between intervention conditions favored the DISE condition for students who began with lower English language proficiency and accumulated when taught over two years. Differences between intervention conditions were not observed after only one year of instruction and for students with advanced levels of English language proficiency. The findings suggest that an evidence-based English oral language program that included frequent teacher demonstrations and opportunities for students to practice speaking English may improve English oral language skills for middle school students. Supplemental data for this article is available online at https://doi.org/10.1080/10824669.2022.2045993.