Asking the right questions in criminal justice evaluations

Abstract

There is something seductive about the idea of well-designed ‘experimental’ evaluations in criminal justice. They draw their authority from medicine. They offer an unbiased means of estimating what would have happened anyway (the ‘counterfactual’). They enable effect size measurements, which can inform cost-benefit analysis. Randomly selected experimental and control groups from different studies can supposedly be aggregated to find statistically significant small effects, which may be absent in constituent small sample studies. Furthermore, experimental studies employ pleasingly simple, elegant, and common-sense methods, which all can understand and recognise. They are widely deemed to provide the ‘gold standard’ against which all others should be judged.