Comparison of empirical Bayes and propensity score methods for road safety evaluation: A simulation study

Abstract

Statistical evaluation of road safety interventions can be undertaken using a variety of different approaches, typically requiring different assumptions to obtain causal identification. In this paper, we conduct a simulation study to compare the performance of empirical Bayes (EB) and propensity score (PS) based methods, which have featured prominently in the recent literature, in settings with and without violation of key assumptions. The estimators considered include EB, inverse probability weighting (IPW), and Doubly Robust (DR) estimation. We find that while the EB approach has good finite sample properties when model assumptions are met, the consistency of this estimator is substantially diminished when the reference and treated sites follow different functions. The IPW estimator performs well in large samples, but requires a correctly specified PS model with sufficient overlap in covariate distributions between treated and control units. The DR estimator allows for violation of assumptions in either the regression or PS model, but not both. We find that this added level of robustness affords overall better performance than attained via EB or IPW estimation.