Making the Most of Limited Data in Evaluating Advanced Traveler Information Systems by Experimental Resampling

Abstract

Because of high data collection costs, analysts are commonly faced with the problem of limited data in the evaluation of intelligent transportation systems. How reliable are conclusions based on small samples? If limited data are available, how does one maximize their value? These questions were addressed to evaluate the potential benefits of prospective notification-based traveler information services used to deliver pre-trip travel time information to simulated drivers in a Cincinnati, Ohio, case study. In Cincinnati, travel time data were initially available for only 30 weekdays. An analysis that used this small data set indicated that an advanced traveler information system (ATIS) user would reduce disutility by 32% versus a comparable nonuser. However, since trip experiences on 30 weekdays may not characterize the typical experience of a commuter, conclusions drawn from the small sample may not accurately represent a more generalized assessment of the benefits of ATIS. Hence, an analogue of statistical resampling (experimental resampling) was applied to generate a large sample of days over which the effectiveness of ATIS could be evaluated. With experimental resampling, the reduction in disutility for an ATIS user was only 24%. It was concluded that experimental resampling provided a more reliable estimate of the benefit. To validate the claim, a more extensive study used 154 weekdays spanning a year. The validation analysis found that when compared with the small sample of 30 weekdays, the resampled cases were better predictors of the benefits for the large sample of 154 weekdays.