“Test-As-You-Go” for Hot Spots Policing: Continuous Impact Assessment with Repeat Crossover Designs

Abstract

Hot spots policing is rapidly changing its evidence-base. Instead of producing more results of one-off, conventional experiments that provide an evidence-base across police agencies (Braga et al., (Campbell Systematic Reviews, 15(3), 2019), hot spots policing (HSP) experiments in the UK are now providing continuous impact assessment (CIA) within police agencies, and within each hot spot. This new mission for experiments entails a change in research design, from the conventionally fixed assignment of each hot spot to a single treatment (in what is technically called a “parallel-track” design) to alternating treatments for each hot spot (in what is known as a “repeat crossover” design). Crossover design experiments are designed for an “on”-days-versus- “off”-days, “test-as-you-go” operating model, using test results in each hot spot for immediate operational feedback to improve performance. This feature can empower police supervisors with compelling evidence for officers about their impact on crime in recent weeks. This approach has great promise, but it also has a great threat. The promise is for integrating evidence more tightly with daily police operations. The threat is that crossover tests may underestimate the true benefits of HSP because they ignore the lingering, “residual deterrence” effects of HSP “on” days continuing into “off” days without HSP. That “carryover” effect of HSP in prior days may take up to 4 days to “wash out” (Barnes et al., 2020). Until it does, crime during HSP “off” days may be lower than if there had been no recent HSP “on” days, thus making HSP look less effective than it truly is. This problem is purely a matter of what analysts do, rather than what police officers do. As long as the officers deliver on their assigned tasks of which hot spots to patrol when, both research designs can have validity. But the problem of how to analyze the effects of these patrols is up to the analysts to solve. If the analysts handle the problem by deleting a certain number of days in between treatment changes for each hot spot—technically known as a “washout” period—they will provide valid impact assessments of HSP. More important, they can do that with a design that requires no long-term denial of service to large numbers of hot spots assigned to a control group, as in traditional random assignment to parallel tracks of treatment vs. control (e.g., Sherman & Weisburd, (Justice Quarterly, 625–648, 1995)).Repeat crossover trials are therefore an excellent improvement over parallel-track trials, subject to omitting crime measures from the washout period for eliminating carryover effects during crossover periods from one treatment condition to another. The following discussion shows how analysts and police leaders can use and implement crossover designs with high internal validity, without biased measures of crime on control days.