Markov decision process of optimal energy management for plug-in hybrid electric vehicle and its solution via policy iteration

Abstract

This paper proposes a Markov decision process based optimal energy management strategy for plug-in hybrid electric vehicle with a hybrid energy storage system, which mainly consists of an ultracapacitor, a power battery pack and a DC/DC converter. Firstly, the mathematical model is built for the overall system. Secondly, based on modified worldwide harmonized light duty test cycle, the power transition probability matrix is constructed to describe the Markov property of driver’s power demand in natural driving. Then by combining the stochastic property of driver’s power demand and deterministic state transition of the hybrid powertrain system, the energy management optimization problem is formulated into a typical Markov decision process. Finally, the derived optimization problem is solved by policy iteration and the corresponding optimal control map is generated. The optimality of the solution is investigated and discussed in detail.