Real-Time Vehicle Traffic Prediction in Apache Spark Using Ensemble Learning for Deep Neural Networks

Abstract

Escalating traffic congestion in large and rapidly evolving metropolitan areas all around the world is one of the inescapable problems in our daily lives. In light of this situation, traffic monitoring and analytics is becoming the need of the hour in today's world. Real-time traffic analysis requires processing of data streams that are being generated continuously in real time to gain quick insights. The challenge of analyzing streaming data for real-time prediction can be overcome by exploiting deep learning techniques. Taking this as a motivation, this work aims to integrate big data technologies and deep learning techniques to develop a real-time data stream processing model for vehicle traffic forecast using ensemble learning approach. Real-time traffic data from an API is streamed using a distributed streaming platform called Kafka into Apache Spark where it is processed, and the traffic flow is predicted by a neural network ensemble model. This will reduce the travel time, cost, and energy through efficient decision making, thus having a positive impact on the environment.