Energy Management in Solar Microgrid via Reinforcement Learning

ABSTRACTThis paper proposes a single-agent system towards solving energy management issues in solar microgrids. The proposed system consists of a photovoltaic (PV) source, a battery bank, a desalination unit (responsible for providing the demanded water) and a local consumer. The trade-offs and complexities involved in the operation of the different units, and the quality of services’ demanded from energy consumer units (e.g. the desalination unit), makes the energy management a challenging task. The goal of the agent is to satisfy the energy demand in the solar microgrid, optimizing the battery usage, in conjunction to satisfying the quality of services provided. It is assumed that the solar microgrid operates in island-mode. Thus, no connection to the electrical grid is considered. The agent collects data from the elements of the system and learns the suitable policy towards optimizing system performance by using the Q-Learning algorithm. The reward function is implemented by fuzzy system Sugeno type for improving the learning efficiency. Simulation results provided show the performance of the system.

The objective of this paper is to develop a Multi Agent System (MAS) for the integrated energy management of a solar micro-grid. We consider a grid connected solar micro-grid which contains two solar Photo Voltaic (PV) systems each contains a local consumer, a solar PV system and a battery. First we calculate the load patterns and solar power generated in the two solar units. Then we use Multi Agent System based distributed energy management of solar micro-grid with smart grid frame work. We develop a simulation model in Java Agent Development Environment (JADE) for dynamic model which considers the intermittent nature of solar power, randomness of load, dynamic pricing of grid and variation of critical loads and choose the best possible action every hour to stabilize and optimize the solar micro-grid. Further more, MAS increases the operational efficiency and thereby maximizes the power production of solar micro-grid and minimizes the operational cost. Thus MAS in micro-grid leads to economic and environmental optimization. Simulated operation of solar generators and loads are studied by performing simulations under different agent objectives. Outcome of the simulation studies demonstrates the effectiveness of proposed MAS in distributed energy management of micro-grid.

The objective of this paper is to develop a Multi Agent System (MAS) for advanced distributed energy management of a solar-wind interconnected micro-grid. This grid connected micro-grid also contains two solar Photo Voltaic (PV) systems, two Wind Turbines each contains a local consumer, a solar PV system and a battery unit. We also consider a Diesel Power Plant that provides considerable power. So, Initially we measure the load patterns, solar power, wind power generated in the two solar and wind units. Then we use Multi Agent System for advanced distributed energy management of this solar-wind micro-grid with smart grid frame work. We develop a simulation model in Java Agent Development Environment (JADE) for distributed, dynamic energy management, which considers the intermittent nature of solar power, wind power, randomness of load, dynamic pricing of grid and variation of critical loads and choose the best possible action every hour to stabilize and optimize the solar micro-grid. Furthermore, MAS increases the operational efficiency, due to decentralised approach and reduced timings. Thus MAS in solar micro-grid energy management leads to economic and environmental optimization. Simulated operation of solar generators and loads are studied by performing simulations under all possible agent objectives. Outcome of the simulation studies proves the effectiveness of proposed MAS in distributed energy management of solar-wind interconnected micro-grid.

In an optimization based control approach for solar microgrid energy management, consumer as an agent continuously interacts with the environment and learns to take optimal actions autonomously to reduce the power consumption from grid. Learning is built in directly into the consumer's behaviour so that he can decide and act in his own interest for optimal scheduling. The consumer evolves by interacting with the influencing variables of the environment. We consider a grid-connected solar microgrid system which contains a local consumer, a renewable generator (solar photovoltaic system) and a storage facility (battery). A model-free Reinforcement Learning algorithm, namely three-step-ahead Q-learning, is used to optimize the battery scheduling in dynamic environment of load and available solar power. Solar power and the load feed the reinforcement learning algorithm. By increasing the utility of battery and the solar power generator, an optimal performance of solar microgrid is achieved. Simulation results using real numerical data are presented for a reliability test of the system. The uncertainties in the solar power and the load are taken into account in the proposed control framework. Index Terms—Solar microgrid; Reinforcement learning; Q- learning; Battery scheduling; Optimization.

Multi Agent System (MAS) is a distributed system with loosely coupled agents; work together to solve problems that are beyond their individual capabilities.The objective of this paper is to implement MAS in Java Agent Development Environment (JADE) for distributed, autonomous energy management of a solar micro-grid. A grid connected solar micro-grid, which contains two solar Photo Voltaic (PV) systems, each contains a local consumer, a solar PV system and a battery, is considered. Load usage patterns and solar power generated in the two solar units are measured. Each constituents of the micro-grid is taken as an agent and these agents take decision autonomously as well as collectively for optimal energy management. The proposed approach autonomously manages the dynamics due to intermittent nature of solar power, randomness of load, dynamic pricing of grid and choose the best possible action to stabilize and optimize the solar micro-grid. MAS has the flexibility for plug and play and so solar power and load is added or removed seamlessly without affecting the micro-grid operations. Furthermore, MAS increases the operational efficiency, leading to economic and environmental optimization of solar micro-grid. All the smart grid features are implemented in the micro-grid, which is not attempted so far. The simulation outcomes demonstrate the effectiveness of proposed MAS in distributed, autonomous energy management of micro-grid, solving the constraints of using solar power. All agent actions for stabilizing the grid can be accomplished in less than ten milliseconds, taking automation of micro-grid to the next higher level.

The objective of this paper is to develop a model for distributed automation of micro-grid using Multi Agent System(MAS) for the advanced control and distributed energy management of a solar micro-grid. A grid connected solar micro-grid model, which contains two solar Photo Voltaic (PV) systems, one in department and other in hostel, each contains a local consumer, a solar PV system and a battery, is modelled in Simulink. Due to unstable nature of MATLAB, when dealing with multi threading environment, MAS operating in Java Agent Development Environment(JADE) is linked with the MATLAB using a middleware, Multi Agent Control using Simulink with Jade extension (MACSimJX). MACSimJX allows the solar micro-grid system designed with MATLAB to be controlled by solar micro-grid agents for realizing the advantages of decentralized approach of MAS. All the agents of solar micro-grid components are programmed in JADE and the results of the coordinated action of these agents are sent to the environment for distributed control and automation of the hardware. JADE leverages the advantage of MAS by its inherent features and hence the operational efficiency of solar micro-grid is further increased. Also the power exchange between main grid and solar micro-grid is optimized by effetive demand side management. Simulation is designed to evaluate impact of autonamous operations of agents.

Energy management in microgrid based on the multi objective stochastic programming incorporating portable renewable energy resource as demand response option

Clean and green energy production has become a major concern among researchers in recent years. In this context, microgrids can play a very important role and provide sustainable technological solution for this purpose. A microgrid is a small scale power network comprising of several distributed generation units, utilizing mostly renewable energy sources, such as solar PV and wind turbines, also incorporates battery storage, fuel cell and micro turbines. The main focus of this research work aims to develop a microgrid generation scheduling model using intelligent meta-heuristic algorithm. Here a modified cuckoo search algorithm is developed and utilized for energy management in microgrids. Case studies are conducted and results are presented for minimum operating costs without demand response participation and with demand response participation respectively. It represents the superiority of proposed demand side management scheduling model approach for effective energy management in microgrids.

This paper presents an economic–environmental power dispatch approach for a grid-connected microgrid (MG) with photovoltaic (PV) generation and battery energy storage systems (BESSs). The problem was formulated as a multiobjective optimization problem with functions such as minimizing fixed and variable generation costs, power losses, and CO2 emissions. This study addresses the problem of intelligent energy management in microgrids with PV generation and BESSs to optimize their performance based on multiple criteria. This study focuses on optimizing the Energy Management System (EMS) with metaheuristic algorithms to achieve practical implementation with simpler algorithms to solve a complex optimization problem. This study employs four multiobjective optimization algorithms: Nondominated Sorting Genetic Algorithm II (NSGA-II), Harris Hawks Optimization (HHO), multiverse optimizer (MVO), and Salp Swarm Algorithm (SSA), which are classified as robust techniques for obtaining Pareto fronts. The computational resources employed to simulate the problem are presented. The optimal dispatch obtained from the Pareto front achieved reductions of 0.067% in fixed costs, 0.288% in variable costs, 3.930% in power losses, and 0.067% in CO2 emissions, demonstrating the effectiveness of the proposed approach in optimizing both economic and environmental performance. The SSA stood out for its stability and computational efficiency, establishing itself as a promising method for energy management in urban and rural microgrids (MGs) and providing a solid framework for optimization in alternating current systems.

The objective of this paper is implementation of multiagent system (MAS) for the advanced distributed energy management and demand side management of a solar microgrid. Initially, Java agent development environment (JADE) frame work is used to implement MAS based dynamic energy management of solar microgrid. Due to unstable nature of MATLAB, when dealing with multithreading environment, MAS operating in JADE is linked with the MATLAB using a middle ware called Multiagent Control Using Simulink with Jade Extension (MACSimJX). MACSimJX allows the solar microgrid components designed with MATLAB to be controlled by the corresponding agents of MAS. The microgrid environment variables are captured through sensors and given to agents through MATLAB/Simulink and after the agent operations in JADE, the results are given to the actuators through MATLAB for the implementation of dynamic operation in solar microgrid. MAS operating in JADE maximizes operational efficiency of solar microgrid by decentralized approach and increase in runtime efficiency due to JADE. Autonomous demand side management is implemented for optimizing the power exchange between main grid and microgrid with intermittent nature of solar power, randomness of load, and variation of noncritical load and grid price. These dynamics are considered for every time step and complex environment simulation is designed to emulate the distributed microgrid operations and evaluate the impact of agent operations.

The objective of this paper is to develop Arduino-based multi-agent system (MAS) for advanced distributed energy management of a solar-wind micro-grid. High penetration of renewable energy resources needs new coordination and control approaches to meet the stochastic nature of the environment and dynamic loadings. We use multi-agent system for advanced distributed, autonomous energy management of micro-grid to dynamically and flexibly adapt to the changes in the environment as renewable energy resources are intermittent in nature. We consider that a micro-grid which contains two systems each contains solar photo voltaic (PV) system, wind generator system, local consumer, and a battery. We develop a simulation model using Java Agent Development Environment (JADE) in Eclipse IDE for dynamic energy management, which considers the intermittent nature of solar power, randomness of load, dynamic pricing of grid, and variation of critical loads, and choose the best possible action every hour to stabilize and optimize the micro-grid. Furthermore, environment variables are sensed through Arduino Mega micro-controller and given to agents of MAS. The agents take the strategic action, and the resulting actions are reflected in the LED outputs which can be readily deployed in the actual field. MAS increases responsiveness, stability, flexibility, and fault tolerance, thereby increasing operational efficiency and leading to economic and environmental optimization. All the smart grid features are tested using JADE simulations and practically verified through Arduino micro-controller to make micro-grid into smart micro-grid.

Novel data-driven energy management of a hybrid photovoltaic-reverse osmosis desalination system using deep reinforcement learning

Energy management in micro grids involves an integrated information and control system pivotal for optimizing energy flow from generation and distribution, minimizing operational costs. Energy Management Systems (EMS) are crucial for leveraging distributed energy resources, especially amidst variable generation and pricing. This paper introduces an Artificial Neural Network (ANN)-powered approach for managing a hybrid wind, solar, and Battery Storage System (BSS). Additionally, a 3 Port DC-DC Converter is proposed to sustain DC voltage. While renewable energy systems offer numerous benefits, their intermittent power generation poses challenges, resulting in grid power fluctuations. EMS seeks to mitigate these fluctuations while preserving the battery state of charge (SOC) within permissible limits to extend battery life. Implementation is conducted using the Simulink/Matlab platform. The efficacy of the proposed approach is demonstrated by comparing the Total Harmonic Distortion (THD) of the suggested controller (1.52%) against conventional controllers: ZSI-based PID (3.05%), PI (4.02%), and FO-PI (3.32%).

Energy management in hybrid microgrid with considering multiple power market and real time demand response

Developing Hybrid Demand Response Technique for Energy Management in Microgrid Based on Pelican Optimization Algorithm