Evaluation at scale: an approach to evaluate technology for informal workplace learning across contexts

Abstract

The implicit, spontaneous and hidden nature of informal learning in addition to the large and less predictable number of application scenarios challenge the evaluation of learning technology. A further challenge for evaluation is added if a user-centred design method had been employed that already had involved users in large numbers and has led to a high level of expectations regarding the potential of the tool to fulfil their needs. We propose an evaluation approach based on the ideas of active involvement of stakeholders, an iterative planning process, a continuous, open and collaborative interpretation of the data collected on the basis of representations shared across cases plus a pluralism of methods for evaluating learning technology and is particularly intended for large scale evaluations in workplace settings. We present a case, in which we applied the proposed evaluation approach in the context of a large scale research project on technology-enhanced-learning.