Measuring Information and Communication Technology Literacy using a performance assessment: Validation of the Student Tool for Technology Literacy (ST2L)

Abstract

This paper reports the validation scores of the Student Tool for Technology Literacy (ST2L), a performance-based assessment based on the National Educational Technology Standards for Students (NETS*S) used to measure middle grade students Information and Communication Technology (ICT) Literacy. Middle grade students (N = 5884) from school districts across the state of Florida were recruited for this study. This paper first provides an overview of various methods to measure ICT literacy and related constructs, and provides documented evidence of score reliability and validity. Following sound procedures based on prior research, this paper provides validity and reliability evidence for the ST2L scores using both item response theory and testlet response theory. This paper examines both the internal and external validity of the instrument. The ST2L, with minimal revision, was found to be a sound measure of ICT literacy for low-stakes assessment purposes. A discussion of the results is provided with emphasis on the psychometric properties of the tool and some practical insights on with whom the tool should be used in future research and practice.