Deep Reinforcement Learning for Continuous Process Scheduling with Storage, Day-Ahead Pricing and Demand Uncertainty

Abstract

Abstract In this work, we evaluate the application of a Deep Reinforcement Learning (DRL) method for the scheduling of continuous process/energy systems under day-ahead electricity rate and demand forecast uncertainty. We employ the Soft Actor Critic (SAC) method, a stochastic, off-policy, actor-critic method with built-in entropy maximization that balances exploration and exploitation. We choose as a case study the dispatching of energy systems with storage, which can be posed as a continuous scheduling problem. Results from the computational case study demonstrate that the DRL agent is able to surpass a heuristic policy using very little data, and ultimately reaches a performance comparable to a model predictive control (MPC) solution. The effect of demand forecast uncertainty is further analysed and it is shown that, while the MPC performance degrades steadily as the forecast error and recalculation period increase, the DRL method exhibits a more robust performance.