The Relative Incident Rate Ratio Effect Size for Count-Based Impact Evaluations: When an Odds Ratio is Not an Odds Ratio

Abstract

Area-based prevention studies often produce results that can be represented in a 2-by-2 table of counts. For example, a table may show the crime counts during a 12-month period prior to the intervention compared to a 12-month period during the intervention for a treatment and control area or areas. Studies of this type have used either Cohen’s d or the odds ratio as an effect size index. The former is unsuitable and the latter is a misnomer when used on data of this type. Based on the quasi-Poisson regression model, an incident rate ratio and relative incident rate ratio effect size and associated overdispersion parameter are developed and advocated as the preferred effect size for count-based outcomes in impact evaluations and meta-analyses of such studies.