Energy efficient speed planning of electric vehicles for car-following scenario using model-based reinforcement learning

Abstract

Eco-driving is a term used to refer to a strategy for operating vehicles so as to minimize energy consumption. Without any hardware changes, eco-driving is an effective approach to improving vehicle efficiency by optimizing driving behavior, particularly for autonomous vehicles. Several approaches have been proposed for eco-driving, such as dynamic programming, Pontryagin’s minimum principle, and model predictive control; however, it is difficult to control the speed of the vehicle optimally in various driving situations. This study aims to derive an eco-driving strategy for reducing the energy consumption of a vehicle in diverse driving situations, including road slopes and car-following scenarios. A reinforcement learning-based energy efficient speed planning strategy is proposed for autonomous electric vehicles, which learn an optimal control policy through a data-driven learning process. A model-based reinforcement learning algorithm is developed for the eco-driving strategy; based on domain knowledge of the vehicle powertrain, a battery energy consumption model and longitudinal dynamics model of the vehicle are approximated from the driving data and are used for reinforcement learning. The proposed algorithm is tested using a vehicle simulation, and is compared to a global optimal solution obtained using an exact dynamic programming method. The simulation results show that the reinforcement learning algorithm can adjust the speed of the vehicle by considering driving conditions such as the road slope and a safe distance from the leading vehicle while minimizing energy consumption. The reinforcement learning algorithm achieves a near-optimal performance of 93.8% relative to the dynamic programming result.