“To develop and test:” The inventive difference between evaluation and experimentation

Abstract

The National Research Council (NRC) Report on Improving Evaluation of Anticrime Programs raises a fundamental question about the mission of evaluation research. The implicit premise of the report is that the mission of evaluation is to answer questions about programs developed by others; in short, to test anti-crime programs. In contrast, the mission of experimental criminology has, historically, been to develop anti-crime programs as well as to test them. There are times when an arm’s-length relationship between program and evaluation may be appropriate. Yet, such a separation necessarily produces a courtroom-like adjudication role for evaluators, rather than the laboratory-like, participant–inventor role that has characterized the best of experimental criminology. The recent case of the Chicago police’s “evaluating” the use of sequential suspect identification methods developed by academic psychologists shows the many flaws of the “testing-only” model. This suggests that providing “effective guidance of criminal justice policy and practice,” as the NRC report defines its focus [Lipsey, M. ed (2005). http://newton.nap.edu/pdf/0309097061/pdf_image/R1.pdf] will not only require evaluation research (defined as arm’s-length testing) but the full toolbox of experimental criminology to develop and test anti-crime programs.