Work Sample Selection Tests and Expected Reduction in Adverse Impact: A Cautionary Note

Abstract

Almost all book chapters, review articles, and textbooks in the field of personnel selection suggest that work sample tests are associated with lower levels of ethnic group adverse impact than paper-and-pencil tests of cognitive ability. However, the empirical literature is heavily dependent on adverse impact estimates obtained from incumbent samples rather than applicant samples. As such, parameter estimates are subject to range restriction from prior selection and on-the-job experiences. Further, an emerging consensus in the selection literature indicates that any method of assessment can be associated with high or low levels of adverse impact – depending on the nature of the construct(s) being measured. To begin to examine these issues, we present two recent sets of applicant data from public sector jobs (for a management and entry-level job, each with technical and interpersonal skill requirements) and show that adverse impact of work sample exams might be more extensive than realized. We discuss the mismatch between what the field of employee selection “knows” and what is said in articles/summaries about work samples. Employers and other practitioners who depend on advice in academic overview articles may be overly optimistic and eventually disappointed by minimal reduction in adverse impact. Implications for workforce diversity and future research needs are also discussed.