A Dueling Deep Q-Network method for low-carbon traffic signal control

Abstract

Improper traffic signal control will lead to long delays for vehicles and produce massive carbon emissions. The vast vehicle exhaust emissions will pollute the environment and exacerbate the earth’s greenhouse effect. Intersection signal optimization tends to start from the traditional view of improving traffic efficiency but ignores the perspective of reducing vehicle carbon emissions. Under the framework of a deep reinforcement learning strategy, this study proposes a novel signal control method to minimize the carbon emissions of vehicles at the intersection. To associate with carbon emissions and signal control plans, the method employs the negative value of vehicle’s carbon dioxide emissions as the reward and takes the feature vectors at different time points in the two decision action intervals as the state features. The fully connected neural network, convolutional neural network, and long short-term memory network are respectively adopted to extract the state features of the decision-making period and compare their Q-value estimation effects. Through the SUMO simulation platform, the proposed signal control method is comprehensively evaluated and compared with different baseline models. It has been proved that the proposed signal control approach can not only directly reduce vehicle carbon emissions but also improve the operational efficiency of the intersection.