Standard Setting Issues and Methods

Abstract

Previous methodological reviews and the contro versy regarding the adequacy of standard-setting technology are summarized. The judgmental nature of all standard-setting methods is examined, and the debate about whether fallible standards are better than none is recast in the context of three different test uses: pupil diagnosis, pupil certification (for high school graduation or professional licensure), and program evaluation. Exemplary standard-set ting methods are reviewed, representing the follow ing major approaches: (1) judgments of test con tent ; (2) judgments about mastery-nonmastery groups; (3) norms and passing rates; (4) empirical methods for discovering standards; and (5) empiri cal methods for adjusting cutoff scores, given a standard on an external criterion measure. Stan dards based on the performance of judged mastery groups (the Contrasting Groups method) and cer tain uses of normative data are likened to Known Groups validation. Recommendations are made for selecting standard-setting techniques depending on test use, including pupil diagnosis, pupil certifica tion, and program evaluation. Future research on standard setting is discussed in the context of im proving practical aspects of judgmental methods.