Reinforcement learning-based composite differential evolution for integrated demand response scheme in industrial microgrids

Abstract

The fourth industrial revolution is being propelled by the “Energy Internet,” which aims to encourage the integration of industrial multi-energy microgrids (MEMG) and renewable energy sources. The conventional demand response (DR) schemes only utilize a single energy source, which limits the industrial users (IUs) and prevents them from making full use of the demand side's communication capabilities. However, smart industrial multi-energy microgrids (MEMGs) give IUs additional options for meeting their energy needs by integrating diverse energy sources, including electricity, natural gas, and thermal power. This new approach to DR programs is known as “Integrated Demand Response” (IDR). This research work proposes a smart IDR program for a novel grid-connected industrial MEMG framework consisting of exhaust air wind turbines, concentrated photovoltaic-thermal panels, an electrical energy storage system (EES), a thermal energy storage (TES), a diesel generator, an indirect customer-to-customer energy trading platform, and typical electrical and thermal industrial loads. The proposed smart IDR considers the uncertainties of both wind and solar power generation and the buying and selling costs of electrical and thermal energy to automatically reduce the industry's total power consumption and battery EES degradation costs. A novel State-Action-Reward-State-Action (SARSA)-based composite different evolution (DE) method is proposed to solve a complex scenario-based non-convex optimization problem. It uses two selection strategies, three mutation strategies, and a positive feedback mechanism to change the states of the individuals. The strategies are coupled in pairs, resulting in a total of six distinct actions that may be performed by the SARSA agents. This allows an individual to not get stuck at a local optimum and adaptively benefit from all the mutation and parameter selection methods. Moreover, SARSA has introduced two more factors, the discount factor and the learning rate, which further improve the optimization performance. The proposed method is also compared with five other state-of-the-art methods to prove its effectiveness in minimizing industrial energy bills and battery degradation costs. The simulated results confirmed that the proposed SARSA-based composite DE algorithm has achieved the lowest total energy cost and battery degradation costs when compared with other state-of-the-art algorithms.