Fast and Precise: Parallel Processing of Vehicle Traffic Videos Using Big Data Analytics

Abstract

Cities worldwide use camera systems that collect and store large amounts of images, which are used to study vehicle traffic conditions, facilitating traffic management authorities’ decision-making. Typically, the inspection of those images is performed manually, which prevents extracting relevant information in a timely manner. There is a lack of platforms to collect and analyze key data from traffic videos in an automatic and speedy way. Computer vision can be used in combination with parallel distributed systems to provide city authorities tools for automatic and fast processing of stored videos to determine the most significant driving patterns that cause traffic accidents while allowing to measure the traffic density. We use a Convolutional Neural Network (CNN) to detect vehicles captured by traffic cameras, which are then tracked using an algorithm that we designed, based on multi-tracking Kalman filters. To speed up analysis, we propose a low-cost distributed infrastructure based on Hadoop and Spark frameworks for data processing: videos are equally divided and distributed to multicore CPU nodes for analysis. However, splitting up videos could generate inaccuracies in vehicle counting, which were avoided through the use of an algorithm that we present in this work. We found that it is possible to rapidly determine traffic densities, identify dangerous driving maneuvers, and detect accidents with high accuracy by using low-cost commodity cluster computing. There is a lack of computing platforms to collect and analyze key data from traffic videos in an automatic and speedy way. Computer vision can be used in combination with parallel distributed systems to provide city authorities tools for automatic and fast processing of stored videos to determine the most significant driving patterns that cause traffic accidents while allowing to measure the traffic density. This study explores the integration of different tools such as parallel data processing, deep learning, and probabilistic models. We present an approach based on Convolutional Neural Network (CNN) and Kalman filters to detect and track vehicles captured by traffic cameras. To speed up analysis, we propose and evaluate a low-cost distributed infrastructure based on Hadoop and Spark frameworks and comprised of multicore CPU nodes for data processing. Finally, we present an algorithm to allow vehicle counting while avoiding inaccuracies generated when videos are split to be distributed for analysis. We found that it is possible to rapidly determine traffic densities, identify dangerous driving maneuvers, and detect accidents with high accuracy by using low-cost commodity cluster computing.