Data-driven energy management system for flexible operation of hydrogen/ammonia-based energy hub: A deep reinforcement learning approach

Abstract

In the context of carbon neutrality, multi-energy systems are being designed to enhance the integration of renewable energy, and the deployment of large-scale energy storage solutions is vital to ensure a flexible and cost-effective system operation. Hydrogen and ammonia are two promising energy carriers that can serve as storage media, feedstock, and fuels. This study designed an energy hub that integrates multiple energy resources and energy storage methods. Two pathways, power-to-gas-to-power (P2X2P) and biomass-to-gas-to-power (B2X2P), which use hydrogen and ammonia as energy carriers, are proposed for the energy hub. Considering the flexibility and profitability of the energy hub, scheduling of the energy hub was controlled using model-based mixed-integer linear programming and model-free deep reinforcement learning methods. A modified double-deep Q-network framework was trained on a monthly dataset and executed on a yearly dataset. The results indicate that the B2X2P pathway outperforms the P2X2P pathway in terms of profitability, while the P2X2P pathway has greater operational flexibility. Because the optimal biomass-to-ammonia-to-power has a 550.76 MUSD net present value and a five-year discounted payback period, ammonia is best suited for mass production and storage. Moreover, the modified double deep Q network framework demonstrates superior generality for solving similar optimal scheduling problems.