How Powerful is the Evidence in Criminology? On Whether We Should Fear a Coming Crisis of Confidence

Abstract

A crisis of confidence has struck the behavioral and social sciences. A key factor driving the crisis is the low levels of statistical power in many studies. Low power is problematic because it leads to increased rates of false-negative results, inflated false-discovery rates, and over-estimates of effect sizes. To determine whether these issues impact criminology, we computed estimates of statistical power by drawing 322 mean effect sizes and 271 average sample sizes from 81 meta-analyses. The results indicated criminological studies, on average, have a moderate level of power (mean = 0.605), but there is variability. This variability is observed across general studies as well as those designed to test interventions. Studies using macro-level data tend to have lower power than studies using individual-level data. To avoid a crisis of confidence, criminologists must not ignore statistical power and should be skeptical of large effects found in studies with small samples.