Potential sources of criterion bias in supervisor ratings used for test validation

Abstract

Four possible sources ofcriterion contamination were investigated in the supervisory performance ratings used for a predictive criterion-related validation study. Supervisors' liking for subordinates had a very large association with their performance ratings independent of the effects of employee ability. Also as hypothesized, expectations of employee qualifications were correlated significantly with initial (1-mo.) performance ratings but not with ratings made after 5 mos. Ethnicity was not associated with 1-mo. performance ratings, but after five months supervisors gave significantly higher ratings to subordinates of the same ethnic group as themselves. No evidence was found of sex bias in the ratings. Estimates of test validity were reduced substantially when the potential sources of criterion bias were controlled statistically. The data are interpreted in the contexts of construct relevance for ratings criteria, possible spurious inflation of employment test validities, and the developmental processes by which supervisor-subordinate relationships are established in the first few months of employment.