The predictive validity of empirical Bayes estimates of road safety

Abstract

This paper examines the predictive validity of empirical Bayes (EB) estimates of road safety. The predictive performance of EB-estimates was tested by applying five versions of EB-estimates of road safety: (1) A simple estimate derived from the empirical distribution of accidents in a population of sites, by which the number of accidents predicted for period 2 for sites that recorded k accidents in period 1 equals the number of accidents for sites that recorded k + 1 accidents in period 1. (2) Estimates derived from the parameters of a negative binomial distribution fitted to an empirical distribution of accidents in a population of sites by means of the method of moments. (3) Estimates derived from the parameters of a negative binomial distribution fitted to an empirical distribution of accidents in a population of sites by means of the maximum likelihood technique. (4) Estimates derived by combining the predictions of an accident prediction model and the recorded number of accidents for a site. (5) Estimates derived by combining the predictions of a different version of an accident prediction model and the recorded number of accidents for a site. All versions of EB-estimates are compared to the traditional, naïve assumption of treating the recorded number of accidents as an unbiased estimate of the expected number of accidents. To test the predictive performance of EB-estimates, data for two periods was used. EB-estimates based on data for the first period were treated as predictions of the number of accidents in the second period for road sections that had 0, 1, 2, etc., accidents in the first period, the idea being that the more accurate the prediction, the more accurate the result of a before-and-after study. All versions of EB-estimates were found to give considerably more correct predictions of the number of accidents in the second period than relying on the count of accidents in the first period as a prediction of the count in the second period. Smaller prediction errors were associated with predictions based on accident prediction models than predictions not based on such models.