Language Proficiency Assessment Research Articles

Assessing the Efficacy of Language Assessment Tools in Educational Settings

Language assessment plays a pivotal role in evaluating learners’ language proficiency and informing educational practices. This paper provides a comprehensive review of international standards guiding the assessment of language proficiency in educational settings. Drawing upon established frameworks such as the Common European Framework of Reference for Languages (CEFR) and the American Council on the Teaching of Foreign Languages (ACTFL) proficiency guidelines, this review examines the key principles, methodologies, and criteria for assessing the efficacy of language assessment tools. It explores the importance of validity, reliability, authenticity, and practicality in the design and implementation of language assessment instruments. Furthermore, it discusses the significance of considering socio-cultural and contextual factors in ensuring the fairness and equity of language assessment practices. The review also highlights emerging trends and challenges in language assessment, including the integration of technology and the need for continuous professional development for assessors. By synthesizing international standards and best practices, this paper offers valuable insights for educators, assessment specialists, and policymakers striving to enhance the quality and effectiveness of language assessment in educational contexts.

Technology's impact on language learning: Meta-analysis on variables and effectiveness

This study explores the correlation between technology utilization and language acquisition while analyzing the impact of moderating variables on this relation. Our meta-analysis approach analyzes data from 43 extracts out of 19 primary studies published between 2012 and 2021. Our data analysis employs a random-effect model utilizing a significance level of α = 0.05. Additionally, the authors examine four moderating variables: level of education, location of research, proficiency in language, and year of publication. Technology-based language acquisition outperforms traditional methods, indicating a significant and moderate impact on the learning process. This study enhances comprehension of the efficacy of technology in language acquisition by identifying various factors, such as the geographical location of research, methods of assessing language proficiency, and technology type employed. However, there is insufficient evidence to support the notion that educational level or sample size significantly impact technology-based language acquisition. This meta-analysis highlights the importance of considering nuanced factors when integrating technology into language learning. The findings emphasize the possibility of technology to transform methods of acquiring language and urge additional investigation into customized strategies that optimize its advantages.

A STUDY ON ROLE OF HRM PRACTICES OF EMPLOYEE PERFORMANCE IN LARSEN & TOUBRO CONSTRUCTION

The project conducted at L&T Construction aimed to understand employee views on HRM practices impacting employee performance. It examined key areas such as Recruitment and Selection, Training and Development, Performance Appraisal, Compensation, Employee Benefits and Employee Engagement Initiatives. The primary goal was to analyze how these HRM practices influence performance within Larsen and Toubro Construction. Secondary objectives included evaluating employee perceptions of the recruitment process, average selection time and talent acquisition effectiveness. Surveys were administered to L&T Construction employees in Chennai, with primary data collected via questionnaires. Secondary data was gathered from various sources. Analysis involved percentage analysis, correlation analysis, chi square tests and ANOVA using SPSS software. Results revealed that L&T utilizes several methods like merit-based systems, campus interviews, talent searches, consultancy services, language proficiency and job specific skills assessments to enhance employee performance. The study will offer recommendations based on the findings. Keywords: HRM Practices, Employee Performance, Analysis.

Assessment of Medical Spanish Language Proficiency in BSN Students: A Pilot Study

Assessment of Medical Spanish Language Proficiency in BSN Students: A Pilot Study

Raising the Bar: How Revising an English Language Proficiency Assessment for Initial English Learner Classification Affects Students’ Later Academic Achievements

ABSTRACT In 2006–2007, California revised its English language proficiency (ELP) assessment, the California English Language Development Test (CELDT), which resulted in more stringent criteria and higher cut scores for meeting proficiency. Using regression discontinuity designs and a difference-in-differences approach, this study examines the effects of the CELDT revision on students’ academic achievements in Grades 2 through 6 English language arts and mathematics. This study uses data from a large urban school district in California and follows two cohorts of Spanish-speaking students initially classified in kindergarten in 2002–2003 and 2006–2007. The CELDT revision did not significantly change the effects of being initially classified as ELs on students’ later outcomes. Before the change, being initially classified as ELs benefited students’ Grades 2 and 3 outcomes. After the revision, the positive effects of EL classification tended to decrease, but the impact was not sufficiently significant.

Challenges and Benefits of Developing CEFR for Languages-Aligned Writing Assessments for Thai EFL Learners: A Case Study on Yala Rajabhat University Test of English Proficiency

This research aims to explore the development of the local standardized tests, Yala Rajabhat University Test of English Proficiency or YRU-TEP, tailored to specific regions, and explore the test aligning with the Common European Framework of Reference for Languages compared to another standardized test, English Discoveries. This study focuses on the context of Thai English as a foreign language learners’ context. To write the test items, Can-Do statements were employed as the expert panel discussed and agreed on the objectives of each skill assessed in each item. In contrast, each item was evaluated through the Index of Item Objective Congruence by the experts. Three hundred experimental groups of test takers have been involved through purposive random sampling. To explore if the YRU-TEP aligns with the CEFR proficiency levels, another standardized test, English Discoveries developed by the ETS has been employed. The experimental test takers took both tests, and the scores were calculated following the cutoff scores to see if they were equivalent. The results showed that 98.67% of the scores had equally the same CEFR levels. However, challenges exist in implementing CEFR-aligned writing assessments in this context. Addressing these challenges involves adapting CEFR to the Thai context, ensuring cultural relevance, refining assessment criteria, and effective resource management. This endeavor contributes to standardizing language proficiency assessment practices and enhancing the quality of EFL education, especially in the case of local tests like the YRU-TEP.

How Cognitive Ability Shapes Personality Differentiation in Real Job Candidates: Insights from a Large-Scale Study.

The differentiation of personality by the cognitive ability hypothesis proposes that individuals with higher cognitive ability have more variability in their personality structure than those with lower cognitive ability. A large sample of actual job candidates (n = 14,462) who participated in an online proctored test session, providing socio-demographic information and completing cognitive ability, personality, and language proficiency assessments, was used to test this hypothesis. The total sample was divided into three equal groups (low, average, high) using percentiles as the cutoff point to investigate the effects of cognitive ability. An ANCOVA demonstrated the significant effect of cognitive ability on personality traits, controlling for language proficiency. Principal component analyses showed that the personality structure differed between the cognitive ability groups, with the high-cognitive-ability group having an additional personality component. Similarly, analyses across job complexity levels indicated more personality components for high-job-complexity positions. The implications, limitations, and future directions of this study are discussed.

Is the English Proficiency Level Always Reflected in Placement Tests?

In today's globalized world, the importance of language learning is paramount due to factors such as international travel, work opportunities, and cultural exchange. Accurately assessing language proficiency and placing learners in appropriate courses is crucial to maximize the benefits of language education. Language placement tests serve this purpose by determining learners' starting points based on their language skills, ensuring they are appropriately challenged and motivated. These tests are widely used in educational institutions and typically cover various language skills like grammar, vocabulary, reading, writing, speaking, and listening. By placing students at the right level, language placement tests enable instructors to tailor lessons to their individual needs, fostering growth and engagement. The integration of reliable placement tests has become indispensable in language education to provide fair assessment, effective instruction, and successful language acquisition. In this article, we present the results of a survey on the reliability of placement tests from the students’ perspective. The quantitative and qualitative study with 120 respondents was conducted in the spring semester of 2023 at Vytautas Magnus University. The study aimed to explore whether the participants believed that placement tests accurately gauge their actual language knowledge and skills and if VMU English Placement Test accurately identifies suitable language proficiency levels for students. The study also sought to collect data for recommendations to improve VMU English Placement Test. According to the respondents, VMU English Placement Test efficiently determines the test takers’ language proficiency levels. Nevertheless, a definitive answer to the question of whether English placement tests reliably indicate actual English language proficiency remains open, as the responses split evenly into positive and negative viewpoints. The study also revealed that all facets of English language competence, including writing, reading, grammar, and listening, must undergo assessment through placement tests.

Speech Fluency Production and Perception in L1 (Slovak) and L2 (English) Read Speech.

Research on fluency in native (L1) and non-native (L2) speech production and perception helps us understand how individual L1 speaking style might affect perceived L2 fluency and how this relationship might be reflected in L1 versus L2 oral assessment. While the relationship between production and perception of fluency in spontaneous speech has been studied, the information provided by reading has been overlooked. We argue that reading provides a direct and controlled way to assess language proficiency that might complement information gained from spontaneous speaking. This work analyzes the relationship between speech fluency production and perception in passages of L1 (Slovak) and L2 (English) read by 57 undergraduate Slovak students of English and rated for fluency by 15 English teachers who are Slovak natives. We compare acoustic production measures between L1 and L2 and analyze how their effect on perceived fluency differs for the two languages. Our main finding is that the articulation rate, the overall number of pauses, and the number of between-clause and mid-clause pauses predict ratings differently in L1 Slovak versus L2 English. The speech rate and durations of pauses predict ratings similarly in both languages. The contribution of our results to understanding fluency aspects of spontaneous and read speech, the relationship between L1 and L2, the relationship between production and perception, and to the teaching of L2 English are discussed.

Prevalence and quality of medical Spanish education in US osteopathic medical schools: a national survey.

Spanish is the language in the United States with the greatest language-concordant physician deficit. Allopathic medical Spanish programs have proliferated, but the national prevalence of medical Spanish education at osteopathic medical schools has never been evaluated. The objectives of this study are to describe the medical Spanish educational landscape at US osteopathic schools and evaluate program adherence to previously established basic standards. Between March and October 2022, surveys were sent to all 44 member schools of the American Association of Colleges of Osteopathic Medicine (AACOM). For nonrespondents, data were obtained from publicly available websites. Primary surveys were sent to deans or diversity, equity, and inclusion officers at each osteopathic school to determine whether medical Spanish was offered and to identify a medical Spanish leader. Medical Spanish leaders received the secondary survey. The main measures of this study were the prevalence of medical Spanish programs at osteopathic schools and the extent to which existing programs met each of the four basic standards: having a faculty educator, providing a curricular structure, assessing learner skills, and awarding institutional course credit. We gathered medical Spanish information from 90.9 % (40/44) of osteopathic schools. Overall, 88.6 % (39/44) offered medical Spanish, of which 66.7 % (26/39) had formal curricula, 43.6 % (17/39) had faculty educators, 17.9 % (7/39) assessed learner skills, and 28.2 % (11/39) provided course credit. Only 12.8 % (5/39) of osteopathic schools with medical Spanish programs metall basic standards. Urban/suburban schools were likelier to offer medical Spanish than rural schools (p=0.020). Osteopathic schools in states with the highest Spanish-speaking populations were more likely to offer student-run initiatives (p=0.027). Most osteopathic schools provide medical Spanish education, but work is needed to improve consistency, quality, and sustainability. Future research should focus on osteopathic student language proficiency assessment, improve medical Spanish accessibility for students at rural programs, and explore the unique content areas of osteopathic medical Spanish education.

A Comparative Study of Item Response Theory Models for Mixed Discrete-Continuous Responses.

Language proficiency assessments are pivotal in educational and professional decision-making. With the integration of AI-driven technologies, these assessments can more frequently use item types, such as dictation tasks, producing response features with a mixture of discrete and continuous distributions. This study evaluates novel measurement models tailored to these unique response features. Specifically, we evaluated the performance of the zero-and-one-inflated extensions of the Beta, Simplex, and Samejima's Continuous item response models and incorporated collateral information into the estimation using latent regression. Our findings highlight that while all models provided highly correlated results regarding item and person parameters, the Beta item response model showcased superior out-of-sample predictive accuracy. However, a significant challenge was the absence of established benchmarks for evaluating model and item fit for these novel item response models. There is a need for further research to establish benchmarks for evaluating the fit of these innovative models to ensure their reliability and validity in real-world applications.

Comparative Analysis of Chinese Language Proficiency Assessment: A Case Study of AP and IGCSE Examinations

With the growing global interest in learning Chinese, the proliferation of Chinese language exams by various institutions has become evident. This paper delves into a comparative analysis of the assessment systems of the College Board's AP Chinese Language and Culture Examination and the Cambridge IGCSE Chinese Examination across five dimensions: score categorization, paper composition, assessment methods, exam syllabus arrangement, and evaluation principles. The study aims to identify similarities and differences between the two exams and draw insights for language testing and assessment practices. The findings reveal that both exams provide valuable educational experiences and insights. Recommendations include refining assessment systems to comprehensively evaluate candidates' language proficiency, integrating cultural elements into examination materials, and leveraging modern technological means to enhance fairness in language testing and assessment processes.

Subjective versus objective language proficiency measures in the investigation of bilingual effects on cognitive control

Purpose: Bilingual language experience is thought to confer non-linguistic benefits in general cognition including improved cognitive control. These bilingual effects are most often observed in samples of bilinguals who are highly proficient in both languages. However, across the majority of previous studies, assessments of language proficiency are exclusively subjective. While evidence supports that subjective and objective measures of language proficiency are correlated, no studies have explored whether the use of either measure impacts on model results when investigating bilingual effects on cognitive control. Methodology: Mandarin-English bilingual young adults completed both subjective and objective assessments of language proficiency and a Simon task to measure differences in cognitive control. Data and Analysis: Data were analyzed using linear mixed-effects models to best account for differences in linguistic and non-linguistic variables as well as the repeated-measures nature of the Simon task. Findings and Conclusions: We report no evidence in support of improved cognitive control associated with higher levels of language proficiency. Crucially, results did not differ when either subjective or objective measures were included in our models. Results support that both subjective and objective assessments of language proficiency may be equivalent when modeling bilingual effects on cognitive control. Originality: This study is the first direct investigation of the influence of the proficiency assessment method on model results. Implications: The findings of this study have implications for the assessment of language proficiency in future investigations of bilingual effects.

Evaluating methodological enhancements to the Yes/No Angoff standard-setting method in language proficiency assessment

This study evaluated the efficacy of two proposed methods in an operational standard-setting study conducted for a high-stakes language proficiency test of the U.S. government. The goal was to seek low-cost modifications to the existing Yes/No Angoff method to increase the validity and reliability of the recommended cut scores using a convergent mixed-methods study design. The study used the Yes/No ratings as the baseline method in two rounds of ratings, while differentiating the two methods by incorporating item maps and an Ordered Item Booklet, each of which is an integral tool of the Mapmark and the Bookmark methods. The results showed that the internal validity evidence is similar across both methods, especially after Round 2 ratings. When procedural validity evidence was considered, however, a preference emerged for the method where panelists conducted the initial ratings unbeknownst to the empirical item difficulty information, and then such information was provided on an item map as part of the Round 1 feedback. The findings highlight the importance of evaluating both internal and procedural validity evidence when considering standard-setting methods.

The analysis of marking reliability through the approach of gauge repeatability and reproducibility (GR&R) study: a case of English-speaking test

Ensuring consistent and reliable scoring is paramount in education, especially in performance-based assessments. This study delves into the critical issue of marking consistency, focusing on speaking proficiency tests in English language learning, which often face greater reliability challenges. While existing literature has explored various methods for assessing marking reliability, this study is the first of its kind to introduce an alternative statistical tool, namely the gauge repeatability and reproducibility (GR&R) approach, to the educational context. The study encompasses both intra- and inter-rater reliabilities, with additional validation using the intraclass correlation coefficient (ICC). Using a case study approach involving three examiners evaluating 30 recordings of a speaking proficiency test, the GR&R method demonstrates its effectiveness in detecting reliability issues over the ICC approach. Furthermore, this research identifies key factors influencing scoring inconsistencies, including group performance estimation, work presentation order, rubric complexity and clarity, the student’s chosen topic, accent familiarity, and recording quality. Importantly, it not only pinpoints these root causes but also suggests practical solutions, thereby enhancing the precision of the measurement system. The GR&R method can offer significant contributions to stakeholders in language proficiency assessment, including educational institutions, test developers and policymakers. It is also applicable to other cases of performance-based assessments. By addressing reliability issues, this study provides insights to enhance the fairness and accuracy of subjective judgements, ultimately benefiting overall performance comparisons and decision making.

Language Lessons Learned? NATO Requirements and Norwegian Officer Cadets

This study establishes the extent to which Norwegian officer cadets have reached NATO English-language capability targets (NCT) on their graduation with a bachelor’s degree in Military Studies. It further investigates their opinions about the role of English language in the context of the officer profession. Cadets (n = 104) from the three military academies (Army, Navy, Air Force) filled in a Likert-scale questionnaire and afterwards tested on their language in accordance with NATO Standardization Agreement 6001. Analyses of test results showed that while very few cadets reached the NCT for officers working on international staffs; a larger number, however, reached the NCT for officer deployments to NATO operations, exercises, or training. While questionnaire respondents considered the English language instruction offered during their studies relevant to the military profession, the absence of a summative assessment of language proficiency would appear to influence motivation for language training efforts. We conclude that, in order to reach NCT, military educational institutions would do well to increase their focus on the NATO language standard and consider their future assessment practices.

해외거주 유아를 위한 스마트폰 앱 활용 가정 연계 한국어교육 실행연구

Objectives The purpose was to explore the changes experienced by parents and young children residing overseas through examining the execution process of Korean language education with a focus on smartphone app uti-lization and fostering home-school connections.
 Methods Following Elliot's (1991) action research model, a 15-week Korean language education program was conducted for ten preschoolers and their parents at a Korean school in City B, State A, USA. Data were collected through observations of children's behavior and language during class, parent-child interactions, teacher-parent conferences, peer teacher workshops, and Korean language proficiency assessments.
 Results In the first implementation, ‘Korean language education centered daily routines for young children’ and ‘home-school linked education encouraging parental participation through smartphone apps’ were conducted. The second implementation focused on ‘one-on-one support-based Korean language instruction’ and ‘intensive home-school linked education using portfolios’. Through these processes, the children progressed from ‘increased interest in Korean’ to ‘formation of confidence and improvement in Korean language ability’, while parents devel-oped ‘trust in Korean language schools’ and eventually ‘experienced the benefits of active participation in Korean language education’.
 Conclusions Korean language education for overseas-residing young children can facilitate emotional develop-ment, including improved self-confidence and self-identity formation, not only enhancing Korean language profi-ciency but also when communication and participation frequency with parents through home-school collabo-rations are increased.

Influence of Test Room Acoustics on Non-Native Listeners’ Standardized Test Performance

Understanding the impact of room acoustics on non-native listeners is crucial, particularly in standardized English as a foreign language (EFL) proficiency testing environments. This study aims to elucidate how acoustics influence test scores, considering variables overlooked in prior research such as seat position and baseline language proficiency. In this experiment, 42 Japanese university students’ performance on standardized EFL listening tests was assessed in two rooms with distinct acoustic qualities, as determined by the speech transmission index (STI) and reverberation time (RT). The rooms differed significantly in their STI values and RT measurements, with one exhibiting high speech intelligibility qualities of ≥0.66 STI and RT0.5–2kHz < 0.7 s and the other falling below these benchmarks. The findings revealed that listening test scores were consistently higher in the acoustically favorable room across all participants. Notably, the negative effect of poor acoustics was more pronounced for students with lower baseline language proficiency. No significant score differences were observed between front- and rear-seat positions, suggesting that overall room acoustics may be more influential than individual seating locations. The study concludes that acoustics play a significant role in the standardized EFL test performance, particularly for lower-proficiency learners. This highlights the necessity of standardized testing environments to be more carefully selected in order to ensure the fair and reliable assessment of language proficiency.

Kielitaidon arviointi on ideologisesti latautunutta toimintaa

Language assessment is an activity guided by prevailing language ideologies as well as more general ideas that permeate the entire society. In this chapter, we will examine the ideologies of language proficiency descriptors and assessment, based on the theoretical perspectives of critical language testing (e.g. Lynch 2001; Shohamy 2001). More specifically, we will present some of the key concepts of the Common European Framework of Reference for Languages (CEFR, Council of Europe 2001, 2020a) and assessment based on the framework, especially on test assessment. With the help of concrete examples, we will also show that a deep understanding of the effects of language assessment and the development of assessment require recognition of both its ideology, its origin, and the mechanisms of its effects.

The relationship between English language proficiency and academic performance among English learners: A subgroup analysis by home language background

In U.S. K-12 public education, states' English language proficiency (ELP) standards and assessments have undergone changes in their content to correspond to the language demands of new academic content-area standards (e.g., Common Core State Standards). However, empirical investigations about how English learner (EL) students perform on the new assessments are limited despite the continued use of ELP assessments for making various decisions about students' academic paths. Addressing this gap, this study focused on criterion-related validity evidence by examining the relationship between ELP and content assessment performances of fifth-grade EL students in one state. We examined all EL students and EL home language subgroups. The results suggest that the ELP Reading subdomain was a significant predictor of content assessment scores in English language arts and mathematics; however, the patterns and magnitude of ELP and content relationship differed across different home language groups. Implications for practice, policy, and research are discussed. Educational relevance statementStates' standards-based ELP assessments play a vital role in making decisions about individual EL students' academic paths as well as in school planning for resources and instruction for EL students. The findings of the present study provide important validity evidence to support the intended use of ELP assessments. The study also underscores the importance of investigating within group variations, as the EL group is tremendously heterogenous in terms of their cultural and linguistic backgrounds, socioeconomic status, length of U.S. residence, and formal schooling experience. The results of the study shed light on a more nuanced understanding of the relationship between ELP and the academic performance of EL students from different home language backgrounds.