Minimizing Energy Cost in PV Battery Storage Systems Using Reinforcement Learning

Abstract

This article addresses the development and tuning of an energy management for a photovoltaic (PV) battery storage system for the cost-optimized use of PV energy using of reinforcement learning (RL). An energy management concept based on the Proximal Policy Optimization algorithm in combination with recurrent Long Short-Term Memory neural networks is developed for data-based policy learning. As a reference system for the simulation-based investigations, a PV battery storage system is modelled, parametrized and implemented with an interface for the RL algorithm. To demonstrate the generalization capability of the learned energy management, 98 training and 12 evaluation episodes, each with a length of one year, are generated from an empirical dataset of global radiation and load power time series. To improve the convergence speed and stability of the RL algorithm as well as the learned policy with regards to techno-economic metrics, an extensive hyperparameter study is conducted by training 216 control policies with different hyperparameter configurations. Simulation-based benchmark tests of the learned energy management against conventional rule-based and model-predictive energy managements show that the RL-based concept can achieve slightly better results in terms of energy costs and the amount of energy fed into the grid than the commonly used model-predictive method.