A generalized energy management framework for hybrid construction vehicles via model-based reinforcement learning

Abstract

Hybrid construction vehicles (HCVs) have more specific tasks and highly repetitive patterns than on-road vehicles. Consequently, they are more suitable for model-based energy management. However, distinctions between work cycles result in adverse scenarios for generalizing model-based energy management. In this study, we solve this problem by proposing a generalized strategy using a model-based reinforcement learning framework. The generalized design highlights three aspects: 1) long-term stability, 2) self-learning ability, and 3) state transition model reuse. A reward function with a trend term is proposed to avoid the cumulative errors between operation cycles and improve the long-term stability of learning. In addition, Gaussian process regression is leveraged to approximate the value function, thereby reducing the computational load and improving the learning efficiency. To further enhance the reusability of the environmental model, a modelling method based on the Gaussian mixture model is put forward. Finally, a generalized HCV energy management framework that includes offline and online learning is designed, where a pre-learning model and an approximation function are adopted for reuse and dynamic learning. Simulation results demonstrate the superiority of the proposed framework to conventional model-based methods in terms of stability, generality, and adaptability, accompanied by a reduction of 5.9% in fuel consumption.