English Language Proficiency Assessment Research Articles

Raising the Bar: How Revising an English Language Proficiency Assessment for Initial English Learner Classification Affects Students’ Later Academic Achievements

Raising the Bar: How Revising an English Language Proficiency Assessment for Initial English Learner Classification Affects Students’ Later Academic Achievements

The relationship between English language proficiency and academic performance among English learners: A subgroup analysis by home language background

In U.S. K-12 public education, states' English language proficiency (ELP) standards and assessments have undergone changes in their content to correspond to the language demands of new academic content-area standards (e.g., Common Core State Standards). However, empirical investigations about how English learner (EL) students perform on the new assessments are limited despite the continued use of ELP assessments for making various decisions about students' academic paths. Addressing this gap, this study focused on criterion-related validity evidence by examining the relationship between ELP and content assessment performances of fifth-grade EL students in one state. We examined all EL students and EL home language subgroups. The results suggest that the ELP Reading subdomain was a significant predictor of content assessment scores in English language arts and mathematics; however, the patterns and magnitude of ELP and content relationship differed across different home language groups. Implications for practice, policy, and research are discussed. Educational relevance statementStates' standards-based ELP assessments play a vital role in making decisions about individual EL students' academic paths as well as in school planning for resources and instruction for EL students. The findings of the present study provide important validity evidence to support the intended use of ELP assessments. The study also underscores the importance of investigating within group variations, as the EL group is tremendously heterogenous in terms of their cultural and linguistic backgrounds, socioeconomic status, length of U.S. residence, and formal schooling experience. The results of the study shed light on a more nuanced understanding of the relationship between ELP and the academic performance of EL students from different home language backgrounds.

Prereferral Intervention for English Learners: Important Considerations

Providing appropriate academic evaluation and intervention services for English learners remains a challenging goal for school support teams. Understanding English language proficiency assessment and universal screening processes can help teams make better decisions when considering possible continued intervention or referral for a comprehensive special education evaluation. This article will help school teams understand what information they should consider when determining whether to begin or continue an academic intervention or refer a student for a special education evaluation.

การรับรู้การใช้ผลคะแนน The Test of English for International Communication ก่อนเข้าศึกษาและก่อนลงทะเบียนรายวิชาสหกิจศึกษาของนักศึกษาสถาบันการบินพลเรือน

This research was carried out in order to investigate the perceptions of Civil Aviation Training Center (CATC) students about the use of the Test of English for International Communication (TOEIC) results before studying at CATC and before co-operative education enrollment. The objective of this research was to study the perception of using TOEIC results before studying and before cooperative education enrollment and the factors that affected the perception of learning English for the Test of English for International Communication (TOEIC) and study the development of their English language skills for TOEIC preparation in order to propose guidelines for the development of English learning to the TOEIC test by studying and collecting data from 378 participants of 1st and 3rd year undergraduate students of CATC, Bachelor of Technology in Aviation Program in Aviation Management and Bachelor of Engineering in the academic year 2019 by using questionnaires and a random interview of 68 students. The results showed that most respondents get 405-600 English TOEIC score or Elementary Proficiency Plus level. Students were aware that CATC has clearly publicized that TOEIC result would be used for consideration before enrolling and before joining cooperative education. They realized that TOEIC was the one of a reliable and useful English language proficiency assessment. Regarding self-test preparation, it was found that most of the students were good self-learners by being aware that it must be well prepared before the test. They set their own goals in the test and also prepared their own materials used in test preparation. In the analysis of one's own ability, students knew their weaknesses and were able to improve themselves. Students realized that they would receive a higher score after their good preparation. And we also found that the helps and advices from lecturers would make students felt more confident during the TOEIC preparation process.

English learners who are blind or visually impaired: A participatory design approach to enhancing fairness and validity for language testing accommodations

Standardizing accessible test design and development to meet students’ individual access needs is a complex task. The following study provides one approach to accessible test design and development using participatory design methods with school community members. Participatory research provides opportunities to empower collaborators by co-creating knowledge that is useful for assessment development. In this study, teachers of students who are visually impaired, students who are blind or are visually impaired, English language teachers, and test administrators provided feedback at critical stages of the development process to explore the construct validity of English language proficiency (ELP) assessments. Students who are blind or visually impared need to be able to show what they know and can do without impact from construct-irrelevant variance like language acquisition or disability characteristics. Building on our iterative accessible test design, development, and delivery practices, and as part of a large project on English-learner proficiency test accessibility and usability, we collected rich observation and interview data from 17 students who were blind or visually impaired and were enrolled in grades kindergarten through Grade 12. We examined the ratings and item metadata, including assistive technology preferences and interactions, while we used grounded theory approaches to examine qualitative thematic findings. Implications for research and practice are discussed.

Toward a systematic accessibility review process for English language proficiency tests for young learners

In order to make assessments as widely accessible as possible, including to young learners from diverse backgrounds with a wide range of individual needs and characteristics, some developers of standardized tests have resorted to offering accessibility tools (e.g., magnifying/zoom) and accommodations (e.g., extended response time) to test takers. These solutions have some limitations stemming from the fact that they are retroactively applied to tests. One possible avenue to meet the accessibility needs of diverse students is to proactively address accessibility in the development of test content for all learners. In this article, we report on the development and piloting of a systematic accessibility review process intended for all test content at an organization producing large-scale standardized English proficiency tests for elementary and secondary education students in the United States. We describe the theoretical and research foundations related to fairness, accessibility, and universal design that guided our development of the accessibility review process followed by the process used to develop the tool. The application of the accessibility checklist to a kindergarten English language proficiency assessment is described in detail. Finally, we share some considerations for how the accessibility checklist could be used in other assessment development contexts.

Quantitative Analysis of Online Teaching and Learning Techniques and the 2019 English Language Proficiency Assessment Levels in a Tri-State Region

The purpose of this quantitative, nonexperimental study was to determine the statistical differences among the delivery methods and among the proficiency levels among elementary schools across a tri-state area. Theories and concepts related to this research were English as a second language, vocabulary study in reading, English immersion, and blended learning and how these were applied in a virtual platform. The researcher applied chi square goodness-of-fit tests to analyze data which was collected from 2019 English proficiency assessments of 46,754 English language learners in 1037 different school districts, and online Qualtrics surveys of 137 English as a second language educators in grades two-five, within the tristate area of Pennsylvania, Ohio, and West Virginia. Future research should determine to what extent there is a relationship between proficiency assessments English language arts scores and suburban elementary English as a second language teachers who use more than 30 minutes of online learning and teaching techniques daily.

Universal tools activation in English language proficiency assessments: A comparison of Grades 1–12 English learners with and without disabilities

English learners (ELs) comprise approximately 10% of kindergarten to Grade 12 students in US public schools, with about 15% of ELs identified as having disabilities. English language proficiency (ELP) assessments must adhere to universal design principles and incorporate universal tools, designed to increase accessibility for all ELs, including those with disabilities. This two-phase mixed methods study examined the extent Grades 1–12 ELs with and without disabilities activated universal tools during an online ELP assessment: Color Overlay, Color Contrast, Help Tools, Line Guide, Highlighter, Magnifier, and Sticky Notes. In Phase 1, analyses were conducted on 1.25 million students’ test and telemetry data (record of keystrokes and clicks). Phase 2 involved interviewing 55 ELs after test administration. Findings show that ELs activated the Line Guide, Highlighter, and Magnifier more frequently than others. The tool activation rate was higher in listening and reading domains than in speaking and writing. A significantly higher percentage of ELs with disabilities activated the tools than ELs without disabilities, but effect sizes were small; interview findings further revealed students’ rationale for tool use. Results indicate differences in ELs’ activation of universal tools depending on their disability category and language domain, providing evidence for the usefulness of these tools.

Educational L2 constructs and diagnostic measurement

Various stakeholders, such as policymakers and educators, require diagnostic feedback and actionable test results. One particular context in which fine-grained test results are of utmost importance is English language proficiency (ELP) assessments as they are used for critical decisions for students. Diagnostic classification models (DCMs) afford finer levels of feedback to improve learning outcomes. This study is part of a research project which implemented DCMs to the reading domain of a K–12 ELP assessment for grades 6–8 to evaluate its feasibility and utility. The research project is U.S.-based. However, the investigation is of relevance to the language testing community broadly. The paper discusses how to address second language reading construct within DCMs. Specifically, it details the identification of attributes underlying the construct and the development and selection of alternative Q-matrices.

Aligning English Language Proficiency Assessments to Standards: Conceptual and Technical Issues

AbstractAlignment between standards and assessments is a fundamental component of the larger effort to ensure appropriate and fair use of standards‐based assessments and achievement of successful standards‐based reform. This article focuses on conceptual and technical issues as well as potential strategies in evaluating alignment between large‐scale, standardized English language proficiency assessments and English language proficiency standards in the U.S. K‐12 education context, as these assessments are used in high‐stakes decisions about English language learners. Areas for future research are also discussed.

Assessment of English Language Proficiency in Grade 7 Pakistan

Standardized tests help learners fulfil their natural desire to learn English and achieve self-actualization needs. The standardized Grade 7 test measured single subject maximum performance on lexical, vocabulary, reading, and writing items. Public schools of Pakistan were the population. Test takers included in this research (n=1500, boys =733, girls =770) were randomly sampled from 56 public schools across Pakistan. However, the pilot study sample was from Lahore district. The percentage of urban schools was 55%, while the rural school percentage was 45%. The percentage of test takers from Punjab was 42.2%, the Northern Areas 24.2%, Sindh 19.1%, and Baluchistan 10.8%. The percentage of female Grade 7 learners was 50.9%, and male learners 49.1%. The sample size for males and females was achieved at 95 % confidence level, margin of error ±5%, population proportion 70%. The objectives achieved during this research were development of a reliable standardized English test (Cronbach alpha 0.888). The findings indicated urban schools had higher scores on learners’ objective items compared to rural schools while the writing tasks indicated no sig. difference in the performance of urban and rural schools suggesting memorization of English essays. This research recommends standardized testing since objective items activate schema retrieval practices.

How Long-Term English Learners Perform on an English Language Proficiency Assessment During Grades 2 through 5: An Examination of Assessment Tasks and Features

Using two states’ longitudinal data, this study tracked long-term English learner (LTEL) students’ performance on an annual English language proficiency (ELP) assessment (the ELPA21) from their pre-LTEL period. We followed English learner (EL) students who were initially classified as EL students in Kindergarten or in Grade 1 until they reached Grade 5. We defined LTEL students as EL students who have not achieved English language proficiency after five years of their initial EL classification. We examined LTEL students’ overall-, domain-, task-, and item-level performance in the ELPA21 and compared how their performances differed from non-LTEL students. We also identified features of items in the Grades 2 through 5 ELPA21 that were particularly challenging for LTEL students. LTEL students showed significantly lower proficiency and slower development in reading and writing than in listening and speaking domains in Grades 2 through 5, suggesting the need for strengthening elementary EL services and targeting instruction for EL students at risk of becoming LTEL students. In addition, findings suggest variability of task types and item features across grade levels that presented more challenges for LTEL students. The findings have implications for EL policies, programs, and instruction, especially for elementary EL students at risk of becoming LTEL students.

A meta-analysis on the predictive validity of English language proficiency assessments for college admissions

For institutions where English is the primary language of instruction, English assessments for admissions such as the Test of English as a Foreign Language (TOEFL) and International English Language Testing System (IELTS) give admissions decision-makers a sense of a student’s skills in academic English. Despite this explicit purpose, these exams have also been used for the practice of predicting academic success. In this study, we meta-analytically synthesized 132 effect sizes from 32 studies containing validity evidence of academic English assessments to determine whether different assessments (a) predicted academic success (as measured by grade point average [GPA]) and (b) did so comparably. Overall, assessments had a weak positive correlation with academic achievement ( r = .231, p < .001). Additionally, no significant differences were found in the predictive power of the IELTS and TOEFL exams. No moderators were significant, indicating that these findings held true across school type, school level, and publication type. Although significant, the overall correlation was low; thus, practitioners are cautioned from using standardized English-language proficiency test scores in isolation in lieu of a holistic application review during the admissions process.

DEVELOPING EQUITABLE ELEMENTARY CLASSROOM PLACEMENT ROUTINES

The annual assignment of elementary students from one classroom to the next is a ubiquitous cycle in schools. Since student class assignment has an effect on students’ educational trajectory, understanding how placement procedures impact student equity is an important area of research with benefits for all stakeholders. In this action research project, survey data was collected to examine what student data points teachers found most meaningful as well as what elements participants considered the most effective and the least beneficial components of the previous piloted placement procedures at a central California elementary school. The findings of this research demonstrated that respondents valued intervention data, such as English Language Proficiency Assessments for California (ELPAC) levels or special education (SPED) status, English Language Arts (ELA)/Math achievement data, and behavior data. However, they also expressed hesitancy over certain narrative or subjective information. Respondents also reported that, while multi-grade level cooperation was a valuable component of the placement process, the procedures needed to be more clearly systematized. Findings suggest that by linking placement procedures to existing Multi-Tiered System of Supports (MTSS) structures, the placement process can more effectively address issues of student equity and achievement.

Technology-Enhanced Items in Grades 1–12 English Language Proficiency Assessments

ABSTRACT Technology-enhanced items (TEIs) are innovative, computer-delivered test items that allow test takers to better interact with the test environment compared to traditional multiple-choice items (MCIs). The interactive nature of TEIs offer improved construct coverage compared with MCIs but little research exists regarding students’ performance on TEIs versus MCIs in English language proficiency (ELP) assessments. This study examines the performance of Grades 1–12 English learners (ELs) on TEIs and MCIs in an online reading test. The test included TEI and MCI content-matched pairs that shared the same reading input but differed in response mode. We analyzed 1.2 million ELs’ scores across five different grade-level clusters: 1, 2–3, 4–5, 6–8, and 9–12. Items were evaluated for difficulty, discrimination, and information using Item Response Theory. Additionally, efficiency was investigated using the amount of information provided and item duration. TEIs were slightly more difficult than MCIs, but they did not differ in discriminative power. Notably, TEIs were more informative for ELs at higher grade or reading proficiency levels. TEIs generally had longer item durations than MCIs, making them less efficient, except for at Grades 6–8. Results provide insights for developing TEIs for ELP reading assessments.

Current Practices and Challenges in Language Proficiency Assessment for English Learners with Complex Support Needs

The use of alternate English language proficiency (Alt-ELP) assessments is an emerging phenomenon with real-life consequences for students with complex support needs from language minority homes. Our examination of U.S. state practices during the 2017–2018 academic year for the identification of English learners (ELs) with complex support needs revealed that only 36 states and the District of Columbia used an Alt-ELP assessment. In addition, some states provided criteria for students to exit EL services, but others did not report or have established redesignation criteria. We found that a wide range of redesignation criteria were adopted by states, revealing a lack of consensus about what constitutes ELP for students with complex support needs. These data suggested substantial challenges remain in developing and implementing Alt-ELP assessments.

Impact of Formative Assessment on Students’ Motivation in Foreign Language Acquisition

The article focuses on introducing formative assessment to the educational process of professionally oriented English communication and the impact of assessment on students’ motivation to learn English. In the article, various types of motivation are discussed; high and low levels of formative assessment are presented; the relationship between motivation and assessment in learning English is studied. The model of high and low levels of formative assessment implementation in learning English for professional communication is tested. The levels of students’ motivation to English acquisition in the conditions of low and high levels of formative assessment are compared. Special attention is paid to specific pedagogical conditions under which formative assessment has a greater impact on students’ motivation. Experimental training was conducted among students of the pedagogical university who study English for professional communication. The authors compare and analyze the results of the motivation tests and diagnostic tests of language proficiency at the pre- and post- stages of experimental training. The study supports the idea about the relationship between students’ motivation and their English language proficiency assessment. The results show that a high level of formative assessment can be beneficial for overcoming students’ low motivation if students receive immediate feedback, evaluation of their work, teachers’ recommendations and support. It is observed that the reasonable integration of students’ self-assessment into teachers’ formative assessment plays a vital role in increasing students’ motivation to English acquisition. According to the study, regular practical implementation of high-level formative assessment methods in English acquisition increases the level of students’ motivation and, consequently, the level of students’ achievements in mastering professionally oriented English communication.

Examining validity issues in classifying and exiting English learners through dynamic views of language proficiency

With the growing English learner population, this study identifies issues of fairness around cut score setting practices that impact English learner (EL) classifications and reclassifications in the US context. High-stakes decisions regarding the education of ELs have relied mostly on English language proficiency (ELP) assessments and the ELP-related classifications. The ELP-related classifications are central to ELs’ life cycles in US schooling. These classifications impact decisions as to whom to designate as EL, whom to administer content assessments to, and who benefits from instructional support programmes. This study examines the practices in the US pre-and post- ‘Every Child Succeeds Act’ (ESSA) of 2015 and takes a dynamic multilingualism perspective to discuss what validity evidence is missing in the current classification practices and what could be done to better align the classification practices with the dynamic languaging practices of multilingual learners.

Use of Task-Based Activities in Improving Active Listening Comprehension Skills of Grade 8 ESL Students

Listening is a core receptive skill that facilitates language acquisition and proficiency, yet it is often neglected in the teaching and learning process. This research was conducted to investigate if there was any significant difference in the participants’ active listening comprehension skills before and after the implementation of the task-based activities anchored on the social learning theory. To achieve the research objectives, thirty 32 participants, 17 males, and 15 females were selected to participate and take the pre-and post-tests for Grades 9-10 adapted from the English Language Proficiency Assessments for California (ELPAC). Surveys, focus group discussions, and interviews were also conducted to identify and monitor any improvement in the participants’ active listening comprehension competencies. The quantitative results and analysis showed no significant difference in the pre-and post-tests. However, other findings indicate an important improvement in the participant’s behaviour as manifested in the FGD and interviews. This further implies that effective implementation of the SLT through TBAs could help language learners improve their active listening comprehension skills. English teachers may consider integrating real-life contexts in their classroom tasks to intensify dynamic and collaborative interactions using language-based instructions.

State Assessment Score Reporting Practices for English Learner Parents

AbstractThis study sought to investigate how states communicate results for academic achievement and English language proficiency (ELP) assessments to parents who are English learners (EL). This objective was addressed by evaluating: (a) whether score reports and interpretive guides for state academic achievement and ELP assessments in each state were translated for EL parents; and (b) if so, whether recommended score reporting guidelines were followed in practice. Results demonstrated that for state achievement tests, 29 states had translated score reports and 28 had translated interpretive guides. Nearly every state translated these materials for their ELP assessments in a wide variety of languages. Across ELP and state achievement assessments, most states were found to limit statistical jargon, utilize figures/graphics to communicate test results, and include follow‐up information for parents, which represent improvements observed in prior reviews. However, states rarely provided personalization, statements on intended score use, a student's score history, or a direct link to their interpretive guide in their score reports. Recommendations on making score reports and interpretive guides more accessible and interpretable for EL parents are discussed.