Feedback Opportunities of Comparative Judgement: An Overview of Possible Features and Acceptance at Different User Levels

Abstract

Given the increasing criticism on common assessment practices (e.g. assessments using rubrics), the method of Comparative Judgement (CJ) in assessments is on the rise due to its opportunities for reliable and valid competence assessment. However, up to now the emphasis in digital tools making use of CJ has lied primarily on efficient algorithms for CJ rather than on providing valuable feedback. Digital Platform for the Assessment of Competences (D-PAC) investigates the opportunities and constraints of CJ-based feedback and aims to examine the potential of CJ-based feedback for learning. Reporting on design based research, this paper describes the features of D-PAC feedback available at different user levels: the user being assessed (assesse), the user assessing others (assessor) and the user who coordinates the assessment (Performance Assessment Manager (PAM)). Interviews conducted with different users in diverse organizations show that both the characteristics of D-PAC feedback and the acceptance at user level is promising for future use of D-PAC. Despite that further investigations are needed with regard to the contribution of D-PAC feedback for user learning, the characteristics and user acceptance of D-PAC feedback are promising to enlarge the summative scope of CJ to formative assessment and professionalization.