Assessing Collaboration Quality in Synchronous CSCL Problem-Solving Activities: Adaptation and Empirical Evaluation of a Rating Scheme

Abstract

The work described is part of an ongoing interdisciplinary collaboration between two research teams of the University of Patras, Greece and the University of Freiburg, Germany, which aims at the exchange of analysis tools and data sets in order to broaden the scope of analysis methods and tools available for Computer-Supported Collaborative Learning (CSCL) support. This article describes the adaptation, generalization and application of a rating scheme which had been developed by the Freiburg team for assessing collaboration quality on several dimensions [1]. The scheme was successfully adapted to suit data gathered by the Patras team in a different CSCL scenario. Collaboration quality is assessed by quantitative ratings of seven qualitatively defined rating dimensions. An empirical evaluation based on a dataset of 101 collaborative sessions showed high inter-rater agreement for all dimensions.KeywordsComputer-Supported Collaborative Learning (CSCL)rating schemeinteraction analysiscollaboration quality