EcoMRL: Deep reinforcement learning-based traffic signal control for urban air quality

Abstract

As greenhouse gas (GHG) emissions from road networks significantly impact the urban environment, there has been a growing interest in integrating environmental factors into traffic controls. Traffic signal control (TSC) is an effective method to reduce air pollution by managing traffic congestion. Accordingly, several recent studies have applied artificial intelligence to TSC to improve traffic efficiency. Despite these efforts, there are a few limitations on the existing studies. First of all, a number of recent studies focus only on traffic performance rather than environmental effects. However, in some cases, the objective function related to traffic efficiency can have an adverse effect on the environment. Furthermore, the variable nature of emissions based on traffic demand is rarely considered. To address these issues, we propose a multi-agent deep reinforcement learning-based TSC to minimize GHGs in traffic networks. To this end, we first identify vehicle’s stop and delay as key factors in traffic efficiency that critically influence air quality. Subsequently, we integrate these factors into our model’s reward function. To validate the proposed model, we conduct experiments under various demand scenarios featuring different traffic volume levels. The results show that the proposed model outperforms comparative control methods in two aspects: traffic performance and environmental sustainability. Especially at a high congestion level, the proposed model reduces average delay time by 11.9–16.1% and increases total throughput by 11.5%. Additionally, it significantly lowers total CO2 emissions by up to 8.9% and average emissions by up to 18.3%.