Real-time energy management controller design for a hybrid excavator using reinforcement learning

Abstract

Real-time energy management of a hybrid excavator is addressed using reinforcement learning (RL). Due to the computational complexity and need for a priori knowledge of the load cycles, a traditional optimal control method, like dynamic programming (DP), is not feasible for real-time control. Real-time controllers derived from traditional optimal control methods compute the solutions either in a cycle-dependent manner or far away from the optimal. An RL-based energy management controller is proposed to solve this problem. The simulation and experimental results demonstrate that the RL controller has a better performance than the widely used thermostat and equivalent consumption minimization strategy (ECMS) controllers. It also shows that the RL controller is cycle-independent. Pontryagin’s minimum principle (PMP) is used to obtain the analytical solution of the energy management problem, and this can help to reduce the iteration time in the design process.