Optimal home energy management strategy: A reinforcement learning method with actor-critic using Kronecker-factored trust region

Abstract

The global development trend of building decentralized and low-carbon energy systems has given rise to challenges in the demand side energy management. With the overwhelming increase in the domestic electricity demand and distributed energy penetration, scheduling residential energy consumption has become essential for decreasing environmental impact. To this end, the paper proposes an energy scheduling strategy optimization framework for a home energy management system devised with photovoltaic and storage to realize the optimal scheduling of household appliances. Uncertainties in energy usage behaviour and real-time electricity prices were considered and described using a Markov decision process model. Subsequently, to search for the optimal appliance scheduling policy in a complex and changing environment, a model-free energy scheduling approach was proposed based on actor-critic using the Kronecker-Factored Trust Region (ACKTR). Unlike conventional deep reinforcement learning methods, the proposed ACKTR-based approach has high sampling efficiency and can deal with both discrete and continuous control actions to jointly optimize the scheduling strategies of various types of appliances. Finally, the numerical results of case studies validate the effectiveness and demonstrate that the proposed approach can significantly reduce costs while maintaining satisfaction, with a remarkable 25.37% cost saving on a typical test day.