Online Markov Chain-based energy management for a hybrid tracked vehicle with speedy Q-learning

Abstract

This brief proposes a real-time energy management approach for a hybrid tracked vehicle to adapt to different driving conditions. To characterize different route segments online, an onboard learning algorithm for Markov Chain models is employed to generate transition probability matrices of power demand. The induced matrix norm is presented as an initialization criterion to quantify differences between multiple transition probability matrices and to determine when to update them at specific road segment. Since a series of control policies are available onboard for the hybrid tracked vehicle, the induced matrix norm is also employed to choose an appropriate control policy that matches the current driving condition best. To accelerate the convergence rate in Markov Chain-based control policy computation, a reinforcement learning-enabled energy management strategy is derived by using speedy Q-learning algorithm. Simulation is carried out on two driving cycles. And results indicate that the proposed energy management strategy can greatly improve the fuel economy and be employed in real-time when compared with the stochastic dynamic programming and conventional RL approaches.