Chapter 5 - Online Adaptive Learning Control Schemes for Microgrids

Abstract

Reinforcement learning is an area of machine learning concerned with how an agent can pick its actions in a dynamic environment to transit to new states in such a way that optimizes the sum of cumulative reward. Online adaptive learning control schemes based on reinforcement learning are developed for single-agent and multiagent microgrid systems. A control scheme based on a value iteration algorithm is develop for an isolated autonomous microgrid. The proposed algorithm is based on heuristic dynamic programming. The control policy is selected by the solving of the governing microgrid discrete-time Bellman equation. The proposed algorithm is implemented online with use of actor-critic neural network structures, where only partial knowledge of the microgrid dynamics is required. In addition, an online multiagent policy iteration algorithm is developed to solve the dynamic graphical game in real time. Photovoltaic cells distributed on a communication graph network structure form a dynamic graphical game. The information flow between the photovoltaic cells is governed by a communication graph. The dynamic graphical game is a special class of the standard games. Cooperative control ideas are used to attain synchronization among the agents’ dynamics to the leader’s dynamics. The performance indices for the multiagent systems are developed to reflect the interplay between agents and the local distribution of information available to each agent. Online critic neural network structures are used to implement the solution to the dynamic graphical game in real time.