Do decision rules matter? A descriptive study of English language proficiency assessment classifications for English-language learners and native English speakers in fifth grade

Abstract

English language proficiency assessments (ELPA) are used in the United States to measure annually the English language progress and proficiency of English-language learners (ELLs), a subgroup of language minority students who receive language acquisition support mandated and largely funded by Title III (NCLB, 2001). ELPA proficient and non-proficient classifications are determined by applying decision rules to combine the sub-domains of listening, speaking, reading, and writing in a conjunctive, compensatory, mixed or complementary manner in order that an ELP performance standard can be set. Although the ELP performance standard is used to set accountability objectives for federal reporting, it also is used to reveal students’ readiness for exit from English language services. This study operationalizes and tests the ELP performance standard for student-level decision making by describing to what extent students are classified as non-proficient under different models and rules and the effect of these differences on their eligibility for redesignation. Test performances from one state’s ELPA were gathered from a statewide sample of ELL ( n = 875) and randomly selected sample of native English speaker students (non-ELL, n = 92) in fifth grade. Findings indicate sizable differences in non-proficient classifications for ELLs, non-ELLs, and a constructed subgroup of academically high-performing students. There were also observed differences in redesignation eligibility in all groups suggesting that choice of model and decision rule can extend the length of time even high-performing students spend in English language services. Discussion includes implications for validation of high-stakes classification systems.