360° ratings: An analysis of assumptions and a research agenda for evaluating their validity

Abstract

This article argues that assumptions surrounding 360° ratings should be examined; most notably, the assumptions that different rating sources have relatively unique perspectives on performance and multiple rating sources provide incremental validity over the individual sources. Studies generally support the first assumption, although reasons for interrater disagreement across different organizational levels are not clear. Two research directions are suggested for learning more about why different organizational levels tend to disagree in their ratings and thus how to improve interpretation of 360° ratings. Regarding the second assumption, it is argued we might resurrect the hypothesis that low-to-moderate across organizational level interrater agreement is actually a positive result, reflecting different levels' raters each making reasonably valid performance judgments but on partially different aspects of job performance. Three approaches to testing this hypothesis are offered.