A novel deep reinforcement learning based business model arrangement for Korean net-zero residential micro-grid considering whole stakeholders’ interests

Abstract

In this paper, we put forward a deep reinforcement learning (DRL) based energy management system (EMS) solution for a typical Korean net-zero residential micro-grid (NZR-MG). We model NZR-MG EMS to extract a profitable business model that respects whole stakeholders’ interests and meets Korean power system regulations and specifications. We deployed the value-based DRL technique, dual deep Q-learning (DDQN), as a solution for our EMS problem since of its simplicity, stability in the learning process, and non-dependency on hyper-parameter selection compared to actor–critic methods. Due to the implementation of mixed-integer nonlinear programming (MINLP) to solve the reward function in this paper, DDQN, despite other DRL methods, provides precise, explicit, and meaningful rewards. In addition to encouraging the agent to choose profitable actions, this approach releases the proposed DRL-based method from the hindrance of redesigning the reward function experimentally in any future extension of the environment elements. Moreover, attaching transfer learning (TL) to the process of training DDQN agent defeat the MINLP imposed latency in training convergence. An extensive benchmark is proposed to test the superiority of the proposed method versus other DRL algorithms.