Direct measures of digital information processing and communication skills in primary education: Using item response theory for the development and validation of an ICT competence scale

Abstract

In the past decade, several studies have measured ICT competences from the perspective of ICT self-efficacy. Such indirect measurements tend to have validity problems, as they depend on the pupils' ability to judge their own ICT competences. This study outlines the development of a performance-based digital test and the validation of a direct measure of ICT competence through the use of item response theory (IRT). More specifically, the test and the developed measure focus on primary-school pupils' proficiency in digital information processing and communication. 56 Items were administered to 560 pupils at the end of their primary-school education (age between 10.79 and 13.85 years old). The items were controlled for dimensionality, model-data fit, local item dependence and monotonicity. The final measure contains 27 items that refer to retrieving and processing digital information, and communication with a computer. The results indicate that the instrument is particularly reliable for low and median ability levels. Further refinement and possible future use of the instrument is discussed.