Q-learning based energy management strategy for a hybrid multi-stack fuel cell system considering degradation

Abstract

The use of multi-stack fuel cells (FCs) is attracting considerable attention in electrified vehicles due to the added degrees of freedom in terms of efficiency and survivability. In a multi-stack FC hybrid electric vehicle, the power sources (FCs and the battery pack) have different energetic characteristics and their operation is influenced by the performance drifts caused by degradation. Hence, efficient power distribution for such a multi-source system is a critical issue. This paper proposes a three-layer online EMS for a recreational vehicle composed of three FCs and a battery pack. In the first layer, two online estimators are responsible for constantly updating the characteristics of each FC and the battery to be used by the power distribution algorithm. In the second layer, a rule-based method is developed to improve the calculation speed of the power distribution algorithm by deciding when it should be activated. The last layer performs the power distribution between FCs and battery using a model-free reinforcement learning (RL) algorithm called Q-learning. The proposed RL-based EMS attempts to meet the requested power while minimizing the costs of hydrogen consumption and degradation of all power sources. To justify the performance of the proposed strategy, a comprehensive benchmark with an offline EMS and two online strategies is performed under two driving cycles. In comparison with the online strategies, the proposed method based on RL reduces the defined trip cost up to 11.5 % and 13.08 % under the Real driving cycle while having a higher cost than the offline strategy by 4.78 %.