Reinforcement Learning Based EV Charging Management Systems–A Review

Heba M Abdullah,Lazhar Ben-Brahim,Adel Gastli

doi:10.1109/access.2021.3064354

Heba M Abdullah, Lazhar Ben-Brahim + Show 1 more

Open Access

https://doi.org/10.1109/access.2021.3064354

Copy DOI

Abstract

To mitigate global warming and energy shortage, integration of renewable energy generation sources, energy storage systems, and plug-in electric vehicles (PEVs) have been introduced in recent years. The application of electric vehicles (EV) in the smart grid has shown a significant option to reduce carbon emission. However, due to the limited battery capacity, managing the charging and discharging process of EV as a distributed power supply is a challenging task. Moreover, the unpredictable nature of renewable energy generation, uncertainties of plug-in electric vehicles associated parameters, energy prices, and the time-varying load create new challenges for the researchers and industries to maintain a stable operation of the power system. The EV battery charging management system plays a main role in coordinating the charging and discharging mechanism to efficiently realize a secure, efficient, and reliable power system. More recently, there has been an increasing interest in data-driven approaches in EV charging modeling. Consequently, researchers are looking to deploy model-free approaches for solving the EV charging management with uncertainties. Among many existing model-free approaches, Reinforcement Learning (RL) has been widely used for EV charging management. Unlike other machine learning approaches, the RL technique is based on maximizing the cumulative reward. This article reviews the existing literature related to the RL-based framework, objectives, and architecture for the charging coordination strategies of electric vehicles in the power systems. In addition, the review paper presents a detailed comparative analysis of the techniques used for achieving different charging coordination objectives while satisfying multiple constraints. This article also focuses on the application of RL in EV coordination for research and development of the cutting-edge optimized energy management system (EMS), which are applicable for EV charging.

Highlights

Thanks to the recent advancements in the battery industry, and the growing pressure from climate change and greenhouse gas emission reduction policies, the concept of plug-in electric vehicles (PEVs) has been prompted
The introduction of PEVs in distributed energy systems can address the challenges of managing intermittent renewable energy resources
Uncertainties are emerged from introducing new flexibilities to the system

Summary

INTRODUCTION

Thanks to the recent advancements in the battery industry, and the growing pressure from climate change and greenhouse gas emission reduction policies, the concept of plug-in electric vehicles (PEVs) has been prompted. New PEV battery management systems need to be developed to overcome these challenges, to operate and control the charging mechanisms and energy flow in the grids with increasing and high penetration of PEVs. The coordinated charging problem is a relevant research topic that has been extensively studied in the literature, and various solutions have been [9], [10], [11]. The deep reinforcement learning agent interacts with the real-time variations in the electricity prices and EV charging schedules, while providing online solutions It has different execution methods including the simplex and hybrid algorithms, in their applications in EVs. The knowledge production within the research field of EV battery charging infrastructure management is accelerating at a tremendous speed while at the same time remaining fragmented and interdisciplinary. Discuss the limitations of the existing studies and identify future research directions

BACKGROUND

Findings

CONCLUSION