Investigating student performance on an assessment tool to support the development of English learners’ argumentation skills

Abstract

The goal of this project was to design a scaffolded learning activity that develops English learners’ (ELs) argumentation skills. In this study, we investigated how middle school ELs interacted with a game-enhanced scenario-based assessment of argumentation skills (named Seaball). The target skills included identifying and classifying opinions on a controversial issue. Various features, such as language-based accommodations, were embedded in the assessment design to make the tasks more accessible to ELs. Immediate performance feedback was also provided in the assessment. We conducted a pilot study with 163 middle school students (including 96 ELs and 67 native English speakers as a control group) to both examine their performance and evaluate the effectiveness of the accommodations. Results indicated that English proficiency levels were significantly related to student performance in Seaball, and that ELs were outperformed by their English-speaking peers. Additionally, while survey data revealed that students considered the language accommodations embedded in the task to be helpful (e.g. the glossary accommodation), the accommodations had no significant impact on Seaball performance. Both ELs and non-ELs generally agreed that they benefited from receiving immediate feedback in the task. The findings suggest that the approach to designing EL accommodations for students engaging in ELA tasks employed in this work may be promising. Further, this approach has potential to guide the design of formative assessments intended to support ELs in learning ELA argumentation skills.