A critical evaluation of the Next Generation Simulation (NGSIM) vehicle trajectory dataset

Abstract

A clear understanding of car following behavior and microscopic relationships is critical for advancing traffic flow theory. Without empirical microscopic data, plausible but incorrect hypotheses perpetuate in the vacuum. The Next Generation Simulation (NGSIM) project was undertaken to collect such data and the NGSIM data set has become the de facto standard, underlying the vast majority of empirically based advances of the past decade. But there has been a growing minority of researchers who have found unrealistic relationships in the NGSIM data. To date, the critical findings have almost exclusively come from the existing NGSIM database itself. Unfortunately, as this paper shows, the NGSIM errors are beyond anything that could be corrected strictly through cleaning or interpolation of the reported NGSIM data.This paper takes the deepest evaluation yet of the NGSIM data. This research manually re-extracts the vehicle trajectories from a portion of the original NGSIM video to explicitly quantify NGSIM errors, e.g., piecewise constant speeds punctuated by brief periods of large acceleration exhibited by the NGSIM data were not evident in the newly extracted trajectories. This point is particularly troublesome for applications that rely on acceleration, e.g., most car following models. The magnitude of errors exhibit a dependency on speed, location and vehicle length. Examples are shown where a real vehicle stopped but the NGSIM trajectory does not and then overruns the location of the real leader. Needless to say, the re-extracted trajectories showed much cleaner speed-spacing relationships than the corresponding raw NGSIM trajectories. Finally, this work tracked the original NGSIM vehicles seen in one camera and added another 236 vehicles (11%) visible before/after the period of NGSIM tracking. As of publication, the manually re-extracted data from this paper will be released to the research community.