Reciprocity as a Source of Bias in Multiple Peer Assessment of Group Work

Abstract

Concerns about the use of peer ratings for assessment purposes are manifold. The issues which are raised by practitioners and researchers, and findings based on recent studies addressing these issues, are outlined. One of the most persistent criticisms is that peer ratings in group settings are prone to bias, resulting in unfairness of mark outcomes. The bias is seen to arise as a result of friendships and social interactions accompanying group task activities. Support for the belief that the validity and fairness of peer ratings are vitiated by 'relational effects' is found in the literature on small group behaviour and interactionist theory. Empirical studies in these two areas operationalise relational effects in what is termed 'reciprocation' - the tendency for two people who are involved in rating each other to be influenced in their rating behaviour by social interactions between the two. The effect of this on rating outcomes, referred to as 'reciprocity effects', is seen to be a major source of bias. The study reported here analyses a data set of multiple peer ratings which had been used as part of the assessment for a subject in community medicine. The analysis is based on cell-by-cell correlation analysis of reciprocal (rater/ratee) pairs, enabling an estimate of the proportion of the variance in scores accounted for by 'reciprocity effects'. The effect was found to be negligible, accounting for only 1% of the variance. Implications for practitioners and for small group behaviour research are discussed.