Learning-Based Predictive Control via Real-Time Aggregate Flexibility

Tongxin Li,Yue Chen,Adam Wierman,Steven H. Low,Bo Sun,Zixin Ye

doi:10.1109/tsg.2021.3094719

Abstract

Aggregators have emerged as crucial tools for the coordination of distributed, controllable loads. To be used effectively, an aggregator must be able to communicate the available flexibility of the loads they control, as known as the aggregate flexibility to a system operator. However, most of existing aggregate flexibility measures often are slow-timescale estimations and much less attention has been paid to real-time coordination between an aggregator and an operator. In this paper, we consider solving an online optimization in a closed-loop system and present a design of real-time aggregate flexibility feedback, termed the maximum entropy feedback (MEF). In addition to deriving analytic properties of the MEF, combining learning and control, we show that it can be approximated using reinforcement learning and used as a penalty term in a novel control algorithm -- the penalized predictive control (PPC), which modifies vanilla model predictive control (MPC). The benefits of our scheme are (1). Efficient Communication. An operator running PPC does not need to know the exact states and constraints of the loads, but only the MEF. (2). Fast Computation. The PPC often has much less number of variables than an MPC formulation. (3). Lower Costs. We show that under certain regularity assumptions, the PPC is optimal. We illustrate the efficacy of the PPC using a dataset from an adaptive electric vehicle charging network and show that PPC outperforms classical MPC.

Highlights

The uncertainty and volatility of renewable sources such as wind and solar power has created a need to exploit the flexibility of distributed energy resources (DERs) and aggregators have emerged as dominate players for coordinating these loads [1], [2]
To complement previous research, this paper considers a closed-loop control model formed by a system operator and an aggregator and propose a novel design of real-time aggregate flexibility feedback, called the maximum entropy feedback (MEF) that quantifies the flexibility available to an aggregator
Our experiments show that by sending simple action signals generated by the penalized predictive control (PPC), a system operator is able to coordinate with an electric vehicles (EVs) charging aggregator to satisfy almost all EV charging demands, while only knowing the MEF learned by a model-free off-policy Reinforcement learning (RL) algorithm

Summary

Introduction

The uncertainty and volatility of renewable sources such as wind and solar power has created a need to exploit the flexibility of distributed energy resources (DERs) and aggregators have emerged as dominate players for coordinating these loads [1], [2]. An ISO communicates a time-varying signal to an aggregator, e.g., a desired power profile, that optimizes ISO objectives and the aggregator coordinates with the DERs to collectively respond to the time-varying signal as faithfully as possible, e.g., by shaping their aggregate power consumption to follow ISO’s power profile, while satisfying DER constraints. These constraints are often private to the loads, e.g., satisfying energy demands of electric vehicles before their deadlines.

Objectives

Results

Conclusion