Deep reinforcement learning and reward shaping based eco-driving control for automated HEVs among signalized intersections

Abstract

In a connected traffic environment with signalized intersections, eco-driving control needs to co-optimize fuel economy (fuel consumption), driving safety (collisions and red lights), and travel efficiency (total travel time) of automated hybrid electric vehicles. Thus, we proposed a deep reinforcement learning based eco-driving control strategy to co-optimize the fuel economy, driving safety, and travel efficiency. A twin-delayed deep deterministic policy gradient agent is implemented to plan vehicle speed in real-time. The multi-objective optimization function of the eco-driving control problem is transformed into the value function of the deep reinforcement learning algorithm by designing fuel reward, traffic light reward, and safety reward function. Specifically, we designed potential-based shaping functions to solve the problem that the intelligent agent cannot learn an optimal policy due to the sparse and delayed traffic environment. It can steer the agent to an optimal policy and guarantee policy invariance. Finally, the proposed method is verified in a real road traffic environment with signalized intersections. The results demonstrate that the proposed method can heavily reduce fuel consumption while satisfying the constraints of traffic lights and safety rules. Meanwhile, the proposed strategy shows certain feasibility for real-time application.