Assessing quality of crash modification factors estimated by empirical Bayes before-after methods

Abstract

Before-after study with the empirical Bayes (EB) method is the state-of-the-art approach for estimating crash modification factors (CMFs). The EB method not only addresses the regression-to-the-mean bias, but also improves accuracy. However, the performance of the CMFs derived from the EB method has never been fully investigated. This study aims to examine the accuracy of CMFs estimated with the EB method. Artificial realistic data (ARD) and real crash data are used to evaluate the CMFs. The results indicate that: 1) The CMFs derived from the EB before-after method are nearly the same as the true values. 2) The estimated CMF standard errors do not reflect the true values. The estimation remains at the same level regardless of the pre-assumed CMF standard error. The EB before-after study is not sensitive to the variation of CMF among sites. 3) The analyses with real-world traffic and crash data with a dummy treatment indicate that the EB method tends to underestimate the standard error of the CMF. Safety researchers should recognize that the CMF variance may be biased when evaluating safety effectiveness by the EB method. It is necessary to revisit the algorithm for estimating CMF variance with the EB method.