Traffic Signal Control via Reinforcement Learning for Reducing Global Vehicle Emission

Bálint Kővári,Tamás Bécsi,Szilárd Aradi,Lászlo Szőke,Péter Gáspár

doi:10.3390/su132011254

Abstract

The traffic signal control problem is an extensively researched area providing different approaches, from classic methods to machine learning based ones. Different aspects can be considered to find an optima, from which this paper emphasises emission reduction. The core of our solution is a novel rewarding concept for deep reinforcement learning (DRL) which does not utilize any reward shaping, hence exposes new insights into the traffic signal control (TSC) problem. Despite the omission of the standard measures in the rewarding scheme, the proposed approach can outperform a modern actuated control method in classic performance measures such as waiting time and queue length. Moreover, the sustainability of the realized controls is also placed under investigation to evaluate their environmental impacts. Our results show that the proposed solution goes beyond the actuated control not just in the classic measures but in emission-related measures too.

Highlights

When testing new traffic control algorithms, it comes naturally to investigate their performance against sustainability measures, along with traditional measures such as travel time, waiting time, and queue length
A new rewarding concept is introduced for the single-intersection traffic signal control problem
The results suggest that both agents can outperform the Simulation of Urban MObility (SUMO)’s built-in time loss-based actuated control in every sustainability measure with the introduced rewarding concept

Summary

Introduction

According to the International Energy Agency (IEA), the transportation industry is responsible for 24% of the direct carbon dioxide emission, three-quarters of which came from road transportation in 2020. The emission of road transportation can be decreased in several ways, such as the utilization of less polluting fuels like hydrogen [2], more energy-efficient vehicles [3], or the application of intelligent transportation systems (ITS). ITS have the potential to mitigate congestion having a profound effect on several aspects of our lives. Such aspects are more productive hours, less emission, safer commute, and so forth. This paper focuses on the more efficient operation of signalized intersections because they are the prime bottlenecks of urban transportation environments. Signalized intersections can be controlled more productively by replacing the fixed-time operation approaches with actuated control techniques by considering the incoming traffic flow of the crossings, which is called the Traffic Signal Control (TSC) problem

Objectives

Methods

Results

Conclusion