Towards an Annotation System for Collaborative Peer Review

Abstract

Peers providing feedback on their peers’ work is called peer review which has been shown to have beneficial effects on students’ learning. This article presents a novel approach to peer review where reviewers, reviewees, and lecturers alike have access to the same documents, and reviews are created in form of annotations which are automatically shared among all users, can be up- or downvoted, and can themselves be commented on. Working on a same document and seeing annotations immediately after they have been created enables various forms of collaboration among learners: Between reviewers who can agree or disagree with a review by up- or downvoting or comment on the review thus providing further insight, between reviewers and reviewees by allowing the reviewees to inquire about reviews and reviewers to provide clarifications and justifications, and between lecturers and reviewers, when lecturers take the role of reviewers and up- or downvote and comment on reviews. The contributions of this article are twofold: First, a report on the conception and implementation of a collaborative annotation environment that supports the collaborative peer review described in this article, and second, an analysis of the communication that happened during collaborative peer review in three courses pointing to the approach’s effectiveness but uncovering problems of the collaborative annotation system as well.