A layered framework for evaluating on-line collaborative learning interactions

Abstract

Evaluating on-line collaborative learning interactions is a complex task due to the variety of elements and factors that take place and intervene in the way a group of students comes together to collaborate in order to achieve a learning goal. The aim of this paper is to provide a better understanding of group interaction and determine how to best support the collaborative learning process. To that end, we propose a principled framework for the study and analysis of group interaction and group scaffolding which is built by combining different aspects and issues of collaboration, learning and evaluation. In particular, we define learning activity indicators at several levels of description which prompt to the application of a mixed interaction analysis scheme and the use of different data types and specific tools. At an initial layer, the basis of the approach is set by applying a qualitative process for evaluating the individual and group task performance as well as the group functioning and scaffolding. The interaction analysis process is completed by defining and applying two more layers: a social network analysis of the group activity and participation behaviour and a quantitative analysis of group effectiveness as regards task achievement and active interaction involvement. Our work defines a grounded and holistic conceptual model that describes on-line collaborative learning interactions sufficiently and applies it in a real, web-based, complex and long-term collaborative learning situation. An in-depth empirical evaluation of the conceptual model is fully discussed, which demonstrates the usefulness and value of the approach.