Development and Validation of the Middle Grades Computer Science Concept Inventory (MG-CSCI) Assessment

Abstract

The increasing interest in computer science (CS) and CS-integrated STEM teaching and learning has created a need for assessment instruments that can be used to evaluate the efficacy of innovative instructional approaches to K-12 CS education. However, there is a lack of validated assessment tools aligned to core CS concepts for younger students. This paper reports on the development and validation of a CS concept assessment for middle grades (ages 11-13) students. A total of 27 multiple-choice items were developed, guided by focal knowledge, skills and abilities associated with the concepts of variables, loops, conditionals, and algorithms. These items were administered to 457 middle grades students. The items were presented in form of block-based programming code and administered in a week-long computational modeling intervention. A combination of classical test theory and item response theory approaches were used to validate the assessment. Based on results, it was found that only 24 items are considered valid and reliable items to measure CS conceptual understanding. The results also suggested that the assessment can be used as a pre and post-test to investigate students’ learning gains. This work fills an important gap by providing a key resource for researchers and practitioners interested in assessing middle grades student CS conceptual understanding.