A Methodology to Detect Traffic Data Anomalies in Automated Traffic Signal Performance Measures

Abstract

Automated traffic signal performance measures (ATSPMs) have garnered significant attention for their ability to collect and evaluate real-time and historical data at signalized intersections. ATSPM data are widely utilized by traffic engineers, planners, and researchers in various application scenarios. In working with ATSPM data in Utah, it was discovered that five types of ATSPM data anomalies (data switching, data shifting, data missing under 6 months, data missing over 6 months, and irregular curves) were present in the data. To address the data issues, this paper presents a method that enables transportation agencies to automatically detect data anomalies in their ATSPM datasets. The proposed method utilizes the moving average and standard deviation of a moving window to calculate the z-score for traffic volume data points at each timestamp. Anomalies are flagged when the z-score exceeds 2, which is based on the data falling within two standard deviations of the mean. The results demonstrate that this method effectively identifies anomalies within ATSPM systems, thereby enhancing the usability of data for engineers, planners, and all ATSPM users. By employing this method, transportation agencies can improve the efficiency of their ATSPM systems, leading to more accurate and reliable data for analysis.