Developing and validating test items for first-year computer science courses

Abstract

We report on the development, validation, and implementation of a collection of test items designed to detect misconceptions related to first-year computer science courses. To this end, we reworked the development scheme proposed by Almstrum et al. (SIGCSE Bulletin 38(4):132–145, 2006) to include students’ artifacts and to simultaneously incorporate think-aloud interviews and flash tests. We also investigated to what extent the practical efficiency of detecting certain misconceptions could be increased without significantly affecting the sensitivity of the instrument, and present positive and negative results regarding this goal. The results of a first transfer and implementation study suggest that it is indeed possible to use the test items in a large-scale practical setting – both as diagnostic instruments and as interventions.