Threats to the validity of science performance assessments for English language learners

Abstract

In recent years, at the same time that performance assessments in science have become more popular, the number of English language learners (ELLs) (i.e., students whose native language is other than English) served by the U.S. educational system has also increased rapidly. While the research base is growing in each of these areas independently, little attention has been paid to their intersection. This case study of the use of a science performance assessment with 96 ELLs in five high school science classes investigated the face, construct, and consequential validity of this intersection. Qualitative and quantitative data analyses showed that both teachers and students had an overall favorable response to the assessment, although students' English comprehension and expression skills were determining factors for certain items. While most responses were reliably scored, ELL spelling and syntax on certain responses were significant sources of error. The degree of specificity of teachers' guidance also significantly affected students' scores. Recommendations from this study include increasing the clarity of an assessment's design, allowing ELLs more time to complete assessments, and scoring by raters who are knowledgeable about typical patterns in written English for this student population. Furthermore, it is recommended that the use of performance assessments with ELLs be exploratory until such time as their validity and reliability with this population can be more adequately established. J Res Sci Teach 34: 721–743, 1997.