Language Proficiency Assessment Research Articles

Register variation in spoken and written language use across technology-mediated and non-technology-mediated learning environments

In the realm of language proficiency assessments, the domain description inference and the extrapolation inference are key components of a validity argument. Biber et al.’s description of the lexicogrammatical features of the spoken and written registers in the T2K-SWAL corpus has served as support for the TOEFL iBT test’s domain description and extrapolation inferences. In the time since the T2K-SWAL corpus was collected, however, university learning environments have increasingly become technology-mediated. Accordingly, any description of the linguistic features of university language should account for the language produced in technology-mediated learning environments (TMLEs) in addition to non-technology-mediated learning environments (non-TMLEs). Kyle et al. recently began to address this issue by collecting a corpus of TMLE language use, which they then compared to language use in non-TMLEs using multidimensional analysis (MDA). The results indicated both similarities and substantive differences across the learning environments, but the study did not investigate the effects of particular registers on these results. In this study, we build on previous research by investigating lexicogrammatical features of specific spoken and written registers across technology-mediated and non-technology-mediated learning environments.

Current Practices and Challenges in Language Proficiency Assessment for English Learners with Complex Support Needs

The use of alternate English language proficiency (Alt-ELP) assessments is an emerging phenomenon with real-life consequences for students with complex support needs from language minority homes. Our examination of U.S. state practices during the 2017–2018 academic year for the identification of English learners (ELs) with complex support needs revealed that only 36 states and the District of Columbia used an Alt-ELP assessment. In addition, some states provided criteria for students to exit EL services, but others did not report or have established redesignation criteria. We found that a wide range of redesignation criteria were adopted by states, revealing a lack of consensus about what constitutes ELP for students with complex support needs. These data suggested substantial challenges remain in developing and implementing Alt-ELP assessments.

Special Needs Assessment in Bilingual School-Age Children in Germany

Educational and (psycho-)linguistic research on L1 and L2 acquisition in bilingual children sketches them as a group of language learners varying in many aspects. However, most studies to date have based evaluations of language proficiency or new assessment tools on data from heritage children, while studies on the appropriateness of assessment tools for school-age refugee children remain a notable exception. This study focuses on the standardized assessment tool BUEGA for primary school children, which is, among others, a widespread tool for the assessment of pedagogical support or special needs (SN) in Germany. We compare the performance of 12 typically developing monolinguals (MoTD: 7;3–12;1), 14 heritage-bilinguals (BiTD: 7;1–13;4, L1 Turkish and Arabic), 12 refugee- students (BiTD: 8;7–13;1, L1 Arabic), and 7 children with developmental language disorders (DLD: 7;7–13;9) on the subtests of grammar, word-reading, and spelling. Overall results show that refugee-BiTDs perform in the (monolingual) pathology range. No significant differences emerged between students with DLD and typically developing (TD) refugee students. Considering the assessment of school-related language performance, bilingual refugees are at risk of misdiagnosis, along with the well-known effects of educational disadvantage. This particularly applies to children with low socioeconomic status (SES). Looking beyond oral language competencies and using test combinations can help exclude language disorders in school-age children with limited L2 proficiency.

Impact of Formative Assessment on Students’ Motivation in Foreign Language Acquisition

The article focuses on introducing formative assessment to the educational process of professionally oriented English communication and the impact of assessment on students’ motivation to learn English. In the article, various types of motivation are discussed; high and low levels of formative assessment are presented; the relationship between motivation and assessment in learning English is studied. The model of high and low levels of formative assessment implementation in learning English for professional communication is tested. The levels of students’ motivation to English acquisition in the conditions of low and high levels of formative assessment are compared. Special attention is paid to specific pedagogical conditions under which formative assessment has a greater impact on students’ motivation. Experimental training was conducted among students of the pedagogical university who study English for professional communication. The authors compare and analyze the results of the motivation tests and diagnostic tests of language proficiency at the pre- and post- stages of experimental training. The study supports the idea about the relationship between students’ motivation and their English language proficiency assessment. The results show that a high level of formative assessment can be beneficial for overcoming students’ low motivation if students receive immediate feedback, evaluation of their work, teachers’ recommendations and support. It is observed that the reasonable integration of students’ self-assessment into teachers’ formative assessment plays a vital role in increasing students’ motivation to English acquisition. According to the study, regular practical implementation of high-level formative assessment methods in English acquisition increases the level of students’ motivation and, consequently, the level of students’ achievements in mastering professionally oriented English communication.

The Plight in Foreign Language Learning in Tunisian Context: Classroom Language Assessment v.s Foreign Language Learning Anxiety

The assessment of language proficiency from a psycholinguistics perspective has been a subject of considerable interest. Many literatures are devoted for the explanation of certain psychological phenomena related to first language acquisition and foreign language learning like language disorders/impairments, critical/sensitive period and language anxiety.
 This paper sheds the light on foreign language anxiety, which is in my conviction the hardest problem that concerns the foreign language learner as well as the teacher. The origin of this conviction is that foreign language anxiety hampers learner performance on one hand, and on the other hand effects, negatively, the classroom language assessment which in turn sharpens learner’s anxiety more and more. There is a significant negative correlation between foreign language anxiety and classroom language assessment. 
 Three issues are to be tackled in this paper: (i) The implication of ‘anxiety’ as a psychological issue in foreign language learning, (ii) classroom language assessment in Tunisian schools and (iii) the relation between foreign language anxiety and classroom language assessment.

The Plight in Foreign Language Learning in Tunisian Context: Classroom Language Assessment v.s Foreign Language Learning Anxiety

The assessment of language proficiency from a psycholinguistics perspective has been a subject of considerable interest. Many literatures are devoted for the explanation of certain psychological phenomena related to first language acquisition and foreign language learning like language disorders/impairments, critical/sensitive period and language anxiety.
 This paper sheds the light on foreign language anxiety, which is in my conviction the hardest problem that concerns the foreign language learner as well as the teacher. The origin of this conviction is that foreign language anxiety hampers learner performance on one hand, and on the other hand effects, negatively, the classroom language assessment which in turn sharpens learner’s anxiety more and more. There is a significant negative correlation between foreign language anxiety and classroom language assessment. 
 Three issues are to be tackled in this paper: (i) The implication of ‘anxiety’ as a psychological issue in foreign language learning, (ii) classroom language assessment in Tunisian schools and (iii) the relation between foreign language anxiety and classroom language assessment.

Examining validity issues in classifying and exiting English learners through dynamic views of language proficiency

With the growing English learner population, this study identifies issues of fairness around cut score setting practices that impact English learner (EL) classifications and reclassifications in the US context. High-stakes decisions regarding the education of ELs have relied mostly on English language proficiency (ELP) assessments and the ELP-related classifications. The ELP-related classifications are central to ELs’ life cycles in US schooling. These classifications impact decisions as to whom to designate as EL, whom to administer content assessments to, and who benefits from instructional support programmes. This study examines the practices in the US pre-and post- ‘Every Child Succeeds Act’ (ESSA) of 2015 and takes a dynamic multilingualism perspective to discuss what validity evidence is missing in the current classification practices and what could be done to better align the classification practices with the dynamic languaging practices of multilingual learners.

Language assessment frameworks as language policy

Reviewing the development and implementation of the Steps to English Proficiency (STEP) language proficiency assessment framework in Ontario, Canada, this article calls attention to how language proficiency is operationalised in English language proficiency (ELP) standards in ways that may institutionalise hierarchies among home, school and community language practices. Use of these standards for classroom-based language assessment requires teachers to play a defining role, to observe, interpret and make judgements about students’ bi/multilingual language practice. However, STEP was developed and validated based on teacher perceptions of students’ language use, revealing the monolingual ideologies at work in Ontario education. The study concludes by discussing opportunities for teachers to identify, observe and document students’ multilingual practice through teacher-inquiry to align curriculum and assessment policy and practice with the sociolinguistic reality of Ontario’s multilingual classrooms and communities.

Language profile of primary progressive aphasia in a linguistically diverse cohort in India

AbstractBackgroundThe accurate characterization of language and semantic memory deficits is crucial for diagnosis of primary progressive aphasia (PPA). In a linguistically diverse society like India, assessment with linguistically appropriate and culturally relevant tools is essential but challenging. This study aimed at developing language and culture specific tests to diagnose PPA and profile cognitive‐linguistic characteristics of a diverse cohort of PPA in India.MethodBased on typical characteristics of Indian languages: free‐word order, rich system of morphology and case‐marking that are features that differentiate Indian languages from English, a language‐specific assessment battery was developed in seven languages: Hindi, Bengali (Indo‐Aryan languages) and Kannada, Telugu, Malayalam and Tamil (Dravidian languages). In the background of the importance of case‐markers in the syntax of Indian languages, a unique syntax assessment test was specifically developed. To evaluate semantic memory, the Indian Semantic Memory battery was developed using stimuli appropriate for a pan‐Indian cultural context. All patients underwent assessment of language proficiency and comprehensive cognitive evaluation by examiners proficient in the language of testing.Result36 patients with PPA: 22 with semantic variant (svPPA), 12 nonfluent variant (nfvPPA) and 2 logopenic variant (lvPPA) participated. The cohort consisted of proficient speakers of seven Indian languages. A particular difficulty in the appropriate use of case markers on the new test of syntax was revealed in nfvPPA compared to svPPA. Further, the core diagnostic feature of ‘agrammatism’ for PPA in English‐speaking nfvPPA was not as dramatic in Indian patients owing to the morphological richness of Indian languages, leading to reduced contrast between fluent versus non‐fluent aphasia. Consistent with diagnosis, a significant reduction in fluency was demonstrated in nfvPPA (p< 0.006 on fluency tasks). Semantic memory was significantly affected in svPPA as revealed on non‐verbal tasks i.e., word‐picture matching (p<0.02)and semantic association task(p<0.01) of Indian Semantic Battery.ConclusionUsing language and culture specific assessment batteries tailored to the Indian context, a comprehensive clinical characteristics in PPA subtypes were identified. The need to re‐define agrammatism with respect to Indian languages with additional weightage to the use of case markers that assist in identifying the PPA subtype was a novel finding.

Predicting CEFR levels in learners of English: The use of microsystem criterial features in a machine learning approach

AbstractThis paper focuses on automatically assessing language proficiency levels according to linguistic complexity in learner English. We implement a supervised learning approach as part of an automatic essay scoring system. The objective is to uncover Common European Framework of Reference for Languages (CEFR) criterial features in writings by learners of English as a foreign language. Our method relies on the concept of microsystems with features related to learner-specific linguistic systems in which several forms operate paradigmatically. Results on internal data show that different microsystems help classify writings from A1 to C2 levels (82% balanced accuracy). Overall results on external data show that a combination of lexical, syntactic, cohesive and accuracy features yields the most efficient classification across several corpora (59.2% balanced accuracy).

CREATING VISUAL ASSETS OF KOREAN VOCABULARY CARD GAME NOLJA

The Korean vocabulary card game Nolja is a game designed to assist the learning process of Korean vocabulary in an interactive way and suitable for women aged 15-25 years. However, as a new game, Nolja has a fatal problem with its design, namely, the placement of illustrations on the player card which makes the game finish too quickly, there are no tools to assess language proficiency on the card, the typeface on the card is less legible, and strokes on the illustration are too thin, Therefore, it is necessary to design a visual asset for the Nolja vocabulary card game. Visual asset design for Nolja includes game theme design, illustration, educational game system, layout, and content arrangement. This design aims to produce a Nolja card game design that is interactive, educational, and suitable for women aged 15-25 years who have the motivation and function to convey Korean language subject matter well. The design is carried out by qualitative methods through interviews with experts and extreme users in the game and learning the Korean language, distributing surveys to 100 target markets. This design is also supported by data collection from literature studies from books, journals, and articles from the Internet. The design produces a visual asset for the Nolja vocabulary card game with a stronger educational game system and a design style that matches the target market's wishes so that it can be well received by the market.
 
 Keyword: Korean language, Visual learning media, Flashcards, Picture card games, Visual assets.

CREATING VISUAL ASSETS OF KOREAN VOCABULARY CARD GAME NOLJA

ABSTRACT 
 The Korean vocabulary card game Nolja is a game designed to assist the learning process of Korean vocabulary in an interactive way and suitable for women aged 15-25 years. However, as a new game, Nolja has a fatal problem with its design, namely, the placement of illustrations on the player card which makes the game finish too quickly, there are no tools to assess language proficiency on the card, the typeface on the card is less legible, and strokes on the illustration are too thin, Therefore, it is necessary to design a visual asset for the Nolja vocabulary card game. Visual asset design for Nolja includes game theme design, illustration, educational game system, layout, and content arrangement. This design aims to produce a Nolja card game design that is interactive, educational, and suitable for women aged 15-25 years who have the motivation and function to convey Korean language subject matter well. The design is carried out by qualitative methods through interviews with experts and extreme users in the game and learning the Korean language, distributing surveys to 100 target markets. This design is also supported by data collection from literature studies from books, journals, and articles from the Internet. The design produces a visual asset for the Nolja vocabulary card game with a stronger educational game system and a design style that matches the target market's wishes so that it can be well received by the market.
 
 Keyword: Korean language, Visual learning media, Flashcards, Picture card games, Visual assets.

Making the Case for the Quality and Use of a New Language Proficiency Assessment: Validity Argument for the Redesigned TOEIC Bridge® Tests

The redesigned TOEIC Bridge® tests were designed to evaluate test takers' English listening, reading, speaking, and writing skills in the context of everyday adult life. In this paper, we summarize the initial validity argument that supports the use of test scores for the purpose of selection, placement, and evaluation of a test taker's English skills. The validity argument consists of four major claims that provide a coherent narrative about the measurement quality and intended uses of test scores. Each major claim in the validity argument is supported by more specific claims and a summary of supporting evidence. By considering the claims and supporting evidence presented in the validity argument, readers should be able to better evaluate whether the TOEIC Bridge tests are appropriate for their situation.

How effective are lexical richness measures for differentiations of vocabulary proficiency? A comprehensive examination with clustering analysis

This study proposed an innovative automated approach to differentiation of the vocabulary proficiency of Chinese speakers. A robust K-means algorithm was designed to compare the oral proficiency between L1 and L2 Chinese speakers regarding lexical richness and how relatively effective the various lexical measures were in performing the differentiation task. Eighteen lexical richness measures were surveyed and compared using the clustering analysis. The effectiveness of each selected measure as well as an overall evaluation of all the measures for the concerned differentiation tasks were comprehensively calibrated. The results demonstrate that, while the L1 versus L2 group difference in lexical richness was observed with statistical significance for each of the chosen measures, the clustering and membership prediction accuracy of individual speakers varied greatly from one measure to another. The implication is that a more fully defined metric of lexical richness is still a worthwhile endeavor for language proficiency assessment, with optimal directions for such endeavors discussed in the concluding remarks.

The MINT Sprint: Exploring a Fast Administration Procedure with an Expanded Multilingual Naming Test.

The present study examined if time-pressured administration of an expanded Multilingual Naming Test (MINT) would improve or compromise assessment of bilingual language proficiency and language dominance. Eighty Spanish-English bilinguals viewed a grid with 80 MINT-Sprint pictures and were asked to name as many pictures as possible in 3 min in each language in counterbalanced order. An Oral Proficiency Interview rated by four native Spanish-English bilinguals provided independent assessment of proficiency level. Bilinguals also self-rated their proficiency, completed two subtests of the Woodcock-Muñoz, and a speeded translation recognition test. We compared scores after 2 min, a first-pass through all the pictures, and a second-pass in which bilinguals were prompted to try to name skipped items. The MINT Sprint and a subset score including original MINT items were highly correlated with Oral Proficiency Interview scores for predicting the degree of language dominance - matching or outperforming all other measures. Self-ratings provided weaker measures (especially of degree of balance - i.e., bilingual index scores) and did not explain any unique variance in measuring the degree of language dominance when considered together with second-pass naming scores. The 2-min scoring procedure did not improve and appeared not to hamper assessment of absolute proficiency level but prompting to try to name skipped items improved assessment of language dominance and naming scores, especially in the nondominant language. Time-pressured rapid naming saves time without significantly compromising assessment of proficiency level. However, breadth of vocabulary knowledge may be as important as retrieval speed for maximizing the accuracy in proficiency assessment.

Choosing assessment techniques in teaching English for specific purposes

The problem of adequate assessment of foreign language proficiency has always been in the focus of investigations dedicated to modern methods and technologies of teaching foreign languages. Recently, more attention has been paid to using alternative assessment methods generated by the teachers of foreign languages themselves and offering the system of multi-directional prolonged criteria based assessment of foreign language competences. It is argued that alternative assessment techniques better correspond to the teaching technologies used by individual teachers, stimulate brain activities, develop creative and logical thinking. This article considers and analyzes one of the models of authentic assessment which is becoming more and more popular – forming an educational portfolio. Examples of this authentic assessment model practical application in teaching business English to the students studying for a Bachelor’s degree in tourism and in teaching English for specific purposes to the students studying for a master’s degree in jewelry are provided and summarised.

Establishing a Nonlinear Dynamic Individual-Centered Language Assessment model: a dynamic systems theory approach

ABSTRACT The lack of comprehensive language proficiency assessment models recognizing the nonlinearity and dynamicity of individual differences, as well as the fundamental differences between paper- and computer-assisted language assessments (CALA) served as the rationale to conduct the present study. Drawing on dynamic systems theory, a Nonlinear Dynamic Individual-Centered Language Assessment (NDICLA) model was developed and tested for its applicability in the computer-assisted language learning context. To test the effectiveness of the model, a random assignment experiment was conducted among 73 Iranian EFL learners. Results show that language proficiency gains of the experimental group were statistically and practically greater than those of the control group. The experimental group held largely positive attitudes towards NDICLA assessment practices, in terms of engagement, competence, and performance. Lastly, the NDICLA model users perceived the model to be effective in terms of accessibility, content, and function. The integration of the computer-assisted language learning developments into a multifaceted CALA model is a significant step towards a dynamic, interactive, and adaptive assessment model, in response to the non-linear dynamic individual differences.

Use of Task-Based Activities in Improving Active Listening Comprehension Skills of Grade 8 ESL Students

Listening is a core receptive skill that facilitates language acquisition and proficiency, yet it is often neglected in the teaching and learning process. This research was conducted to investigate if there was any significant difference in the participants’ active listening comprehension skills before and after the implementation of the task-based activities anchored on the social learning theory. To achieve the research objectives, thirty 32 participants, 17 males, and 15 females were selected to participate and take the pre-and post-tests for Grades 9-10 adapted from the English Language Proficiency Assessments for California (ELPAC). Surveys, focus group discussions, and interviews were also conducted to identify and monitor any improvement in the participants’ active listening comprehension competencies. The quantitative results and analysis showed no significant difference in the pre-and post-tests. However, other findings indicate an important improvement in the participant’s behaviour as manifested in the FGD and interviews. This further implies that effective implementation of the SLT through TBAs could help language learners improve their active listening comprehension skills. English teachers may consider integrating real-life contexts in their classroom tasks to intensify dynamic and collaborative interactions using language-based instructions.

Critical Language Testing: Factors Influencing Students’ Decisions to (Not) Pursue the Seal of Biliteracy

In this investigation, Kristin J. Davin analyzes current and former emergent bilingual learners’ decisions to take or not take a language proficiency assessment in a home language to pursue a Seal of Biliteracy (SoBL). The SoBL is a policy adopted in forty states to counteract English-only ideologies by recognizing students who graduate high school bilingual and biliterate. Considering the power of assessments and the complexity of the decision to take a test of proficiency in one’s home language, this study uses the history-in-person framework to understand the factors that shape students’ decisions to take, or not take, the “seals test.” Davin’s findings point toward considerations and changes necessary to SoBL implementation to ensure that the policy meets the needs of the emergent bilingual learners it was intended to benefit.

State Assessment Score Reporting Practices for English Learner Parents

AbstractThis study sought to investigate how states communicate results for academic achievement and English language proficiency (ELP) assessments to parents who are English learners (EL). This objective was addressed by evaluating: (a) whether score reports and interpretive guides for state academic achievement and ELP assessments in each state were translated for EL parents; and (b) if so, whether recommended score reporting guidelines were followed in practice. Results demonstrated that for state achievement tests, 29 states had translated score reports and 28 had translated interpretive guides. Nearly every state translated these materials for their ELP assessments in a wide variety of languages. Across ELP and state achievement assessments, most states were found to limit statistical jargon, utilize figures/graphics to communicate test results, and include follow‐up information for parents, which represent improvements observed in prior reviews. However, states rarely provided personalization, statements on intended score use, a student's score history, or a direct link to their interpretive guide in their score reports. Recommendations on making score reports and interpretive guides more accessible and interpretable for EL parents are discussed.