Mend It, Don’t End It: An Alternate View of Assessment Center Construct-Related Validity Evidence

Abstract

The unitarian conceptualization of validity serves as the conceptual and logical basis for the so-called assessment center (AC) ‘‘construct-related validity paradox.’’ Within the unitarian framework, at a theoretical level, if a measurement tool demonstrates criterion-related and content-related validity evidence, as is widely accepted with ACs, then it should also be expected to demonstrate construct-related validity evidence (Binning & Barrett, 1989). And because ACs do not appear to do so, we have the resultant AC construct-related validity paradox. So, accepting the premise that the unitarian view is conceptually and logically sound, what is the explanation for the paradox? Why do AC dimension ratings appear not to ‘‘work’’ in terms of construct-related validity evidence? At a broad conceptual level, we present a view that is contrarian to Lance’s (2008) view of ‘‘why ACs don’t work the way they’re supposed to’’ and subsequently what to do about them. Our contrarian view is based on two key points, namely that the vast majority of the empirical AC research to date—particularly that which serves as the basis for calls for the ‘‘redesign of ACs toward taskor role-based ACs and away from traditional dimensionbased ACs’’ (Lance, 2008, p. 84)—is based on (a) espoused as opposed to actual constructs and (b) flawed analysis resulting from an overemphasis on postexercise dimension ratings as measures of AC dimensions. In our view, ACs in practice appear to be effectively designed to representatively sample from the job content domain and also predict criteria of interest, but they are woefully deficient in their construct explication and development. Consequently, we do not concur with Lance’s interpretation of the extant literature and his conclusions concerning what to do about it. Our position is that the issue is not one of a failure in ‘‘AC theory’’ but rather a failure to engage in appropriate tests of said theory. Until such tests have been undertaken, we think it is premature to abandon Correspondence concerning this article should be addressed to Winfred Arthur, Jr. E-mail: wea@psyc. tamu.edu Address: Department of Psychology, Texas AM Eric Anthony Day, Department of Psychology, The University of Oklahoma; David J. Woehr, Department of Management, The Universityof Tennessee. Industrial and Organizational Psychology, 1 (2008), 105–111. Copyright a 2008 Society for Industrial and Organizational Psychology. 1754-9426/08