Adapting Kirkpatrick's evaluation model to technology enhanced learning

Abstract

Experiments, case studies and surveys are part of the standard toolbox researchers use for validating their proposed educational tools and methods. Yet the breadth of such evaluations is often limited to one dimension such as assessing the effect on learning outcomes by means of experiments, or evaluating user acceptance and perceived utility by means of surveys. Besides a positive effect in the classroom, it is equally important that students transfer their knowledge to their working environment, which constitutes yet another evaluation dimension. The lack of a widely accepted validation method encompassing a broad set of dimensions hampers the comparability and synthesis of research on modelling education. This study adapts Kirkpatrick's model of training evaluation to the assessment of Technology Enhanced Learning (TEL), i.e. learning settings where the teaching is supported by means of didactic tools. The adaptation proposes concrete metrics and instruments for each of Kirkpatrick's model level. The adaption is demonstrated by means of a case study on a TEL environment for User Interface (UI) modelling that supports the learning UI design principles.