The Coding Stages Assessment: development and validation of an instrument for assessing young children’s proficiency in the ScratchJr programming language

Abstract

ABSTRACT Background and Context: Despite the increasing implementation of coding in early curricula, there are few valid and reliable assessments of coding abilities for young children. This impedes studying learning outcomes and the development and evaluation of curricula. Objective: Developing and validating a new instrument for assessing young children’s proficiency in the programming language ScratchJr, based on the Coding Stages framework. Method: We used an iterative, design-based research approach to develop the Coding Stages Assessment (CSA), a one-on-one assessment capturing children’s technical skills and expressivity. We tested 118 five-to-eight-year-olds and used Classical Test Theory and Item Response Theory to evaluate the assessment’s psychometric properties. Findings: The CSA has good to very good reliability. CSA scores were correlated with computational thinking ability, demonstrating construct validity. The items have good discrimination levels, and a variety of difficulty levels to capture different proficiency levels. Younger children tended to have lower scores, but even first graders can achieve the highest coding stage. There was no evidence of gender or age bias. Implications: The CSA allows testing learning theories and curricula, which supports the implementation of Computer Science as a school subject. The successful remote administration demonstrates that it can be used without geographical restrictions.