QUALITY CONTROL OF ARCHIVED INTELLIGENT TRANSPORTATION SYSTEMS DATA

Abstract

Previous research has reported on the need to implement quality control programs for ITS data to address critical issues such as suspicious or erroneous data, nature and extent of missing data, and accuracy and comparability of ITS data to similar data sources. This paper summarizes the work completed to address quality control and completeness issues associated with a very large archived ITS data set composed of some 3.4 billion 20-second lane detector data records from San Antonio’s TransGuide. The paper includes a description of the quality control tests utilized, the results of the analysis conducted, and a discussion of ITS data completeness issues. An evaluation of temporal variations in the distribution of quality control flags showed that, in most cases, the highest concentration of flagged records occurred at night when traffic was light. Consequently, there was a higher chance for time intervals without vehicles which produced abnormal detector readings. Finally, the researchers evaluated the data completeness both at the aggregate level (by server) and a more detailed individual detector level. While the analysis described in this paper uses data from one jurisdiction (San Antonio, Texas), the methodology is sufficiently generic to enable implementation at other traffic management centers.