An Online Correction Predictive EMS for a Hybrid Electric Tracked Vehicle Based on Dynamic Programming and Reinforcement Learning

Jinlong Wu,Yuan Zou,Xudong Zhang,Zehui Kong,Dingbo He,Teng Liu

doi:10.1109/access.2019.2926203

Abstract

Energy management strategy is critical in the development of hybrid electric vehicles. It is used to improve fuel economy and sustain battery state of charge by splitting the power demand to different power sources while satisfying various physical constraints and vehicle performance simultaneously. However, it is challenging to achieve an optimal control performance due to the complexity of the hybrid powertrain, the time-varying constraints, and stochastic of the load power. Focusing on these problems, this paper presents an online correction predictive energy management (OCPEM) strategy for a hybrid electric tracked vehicle based on dynamic programming (DP) and reinforcement learning (RL). First, a multi-time-scale prediction method is proposed to realize the short-period future driving cycle prediction. Then, the DP algorithm is applied to obtain the local control policy based on the short-period future driving cycle. The RL algorithm is combined with the fuzzy logic controller to optimize the control policy by eliminating the influence of imprecise prediction. Finally, the simulations are conducted in Matlab/Simulink to evaluate the control effectiveness and adaptability of the proposed method. The results indicate that the fuel economy of the proposed OCPEM is improved by 4% compared with the original predictive energy management and achieve 90.51% of that of the DP benchmark.

Highlights

Nowadays, electric vehicles (EVs), hybrid EVs (HEVs), and fuel cell HEVs (FCHEVs) have been widely considered to be the most promising solutions to environmental pollution and energy crisis [1]
This paper proposes a novel online correction predictive energy management (OCPEM) strategy to solve this issue, which integrates the reinforcement learning algorithm to achieve online correction based on a fuzzy logic controller for a hybrid electric tracked vehicle (HETV)
The results show that applying the reinforcement learning algorithm to correct the control policy of predictive EMS through fuzzy control can effectively improve the fuel economy of the vehicle

Summary

SYMBOLS

J. Wu et al.: Online Correction Predictive EMS for a Hybrid Electric Tracked Vehicle. Fuel consumption Penalty factor One-step transition probability Transition number from i to j Total transition number initiated from i One-step velocity prediction value tp ptij(n) x(n + t)p Error(k). Multi-step velocity prediction value Root mean square error of prediction horizon at k moment Action-value function in terms of s, a Learning rate of RL Discount factor Instant reward of the selected action a Optimal policy Adjustment factor

INTRODUCTION

Findings

CONCLUSION