Chronic Consequences of High‐Stakes Testing? Lessons from the Chinese Civil Service Exam

Abstract

In the field of educational assessment, the “validity of assessment” can be defined as the degree to which evidence and theory support the interpretations of test scores that are entailed by proposed uses of the test. Many concerns have arisen over what is termed the “consequential basis” of validity, the intended and unintended social consequences of testing. Theorists have taken distinctive positions regarding this issue. D. E. Wiley suggested that consequences of testing should not be considered a part of validity at all, as it overburdens the concept. In contrast, Lee Cronbach suggested that social consequences alone might influence validity by questioning the way in which a test is used. Lori Shepard added that unintended social consequences of test use represent rival hypotheses to the expressed use of the test and are thus an integral part of validity. Pam Moss argued that the study of consequences is an essential aspect of validity even for those who choose to limit the scope of validity to a test-based interpretation. In the latest edition of Standards for Educational and Psychological Testing, the Joint Committee on Testing Standards of the American Educational Research Association, the American Psychological Association, and the National Council on Measurement in Education clarified that “evidence about consequences can inform validity decisions. Here, however, it is important to distinguish between evidence that is directly relevant to validity and evidence that may inform decisions about social policy but which falls outside the realm of validity.” They