Energy-optimal routing for electric vehicles using deep reinforcement learning with transformer

Abstract

This paper presents an end-to-end deep reinforcement learning (DRL) approach aimed at efficiently determining energy-optimal routes for a group of electric logistic vehicles, with the objective of minimizing operating costs. First, an Energy-Minimization Electric Vehicle Routing Problem (EM-EVRP) is formulated with an energy consumption model for electric vehicles, rather than Distance Minimization EVRP commonly favored in the literature. The energy consumption model incorporates several factors such as vehicle dynamics, road information, and charging losses. Then, the problem is reformulated based on the Markov decision process and solved using the transformer-based DRL method. The policy network is designed following the Transformer structure, including an encoder, a feature embedding module, and a decoder, where the feature embedding module is added to provide contextual information. Finally, extensive experiments demonstrate the superior of the proposed DRL method over existing learning-based methods and conventional methods, in solving both EM-EVRP and DM-EVRP. Notably, the formulated EM-EVRP achieves greater cost reduction than the traditional DM-EVRP.