Reinforcement learning-based heuristic planning for optimized energy management in power-split hybrid electric heavy duty vehicles

Abstract

In this work, we systematically integrate relevant expertise, specifically on the optimal brake-specific fuel consumption (BSFC) curve, battery characteristics and terrain information into the development of an energy management plan for heavy-duty power-split hybrid electric vehicles. We utilize deep deterministic policy gradient (DDPG) algorithm as one of the most sophisticated reinforcement learning technique. Initially, we begin by explaining the vehicle configuration's system modeling. Subsequently, we then present an energy management approach based on deep Q-learning concepts. A novel algorithm, Deep Deterministic Policy Gradient, for energy management control, have been created to combat the “curse of dimensionality” in reinforcement learning. The new AMSGrad optimization technique is used by DDPG algorithm to update the weight of neural networks. We robustly train the proposed control system in realistic driving environment. The Knowledge Incorporation (KI) based DDPG based system is compared systematically to the conventional DDDPG methodology and the benchmark Dynamic Programming (DP) method, the latter of which usually uses the RMSProp Optimizer in its formulation. The results show that as compared to the typical DQL policy, deep reinforcement learning approaches, notably expert Knowledge incorporation KI-DDPG and terrain information with the AMSGrad optimizer, achieve faster training speeds and reduced fuel consumption while maintaining the terminal state of charge (SOC).