Event-based Optimization Research Articles

Event-Based EV Charging Scheduling in a Microgrid of Buildings

With the popularization of electric vehicles (EVs), EV charging demand is becoming an important load in the building. Considering the mobility of EVs from building to building and their uncertain charging demand, it is of great practical interest to control the EV charging process in a microgrid of buildings to optimize the total operation cost while ensuring transmission security between the microgrid and the main grid. We consider this important problem in this article and make the following contributions. First, we formulate this problem as a Markov decision process (MDP) to capture the uncertain supply and EV charging demand in the microgrid of buildings. Besides reducing the total operation cost of buildings, the model also considers the power exchange limitation to ensure transmission security. Second, this model is reformulated within an event-based optimization (EBO) framework to mitigate the impact of large state and action space. A randomized parametric event-based control policy is proposed for the microgrid controller to determine the charge ratio for each building. Then, a selecting-to-charging principle is implemented for each building controller to determine the charge control of each EV in the building. Third, a constrained gradient-based policy optimization method is developed to find the optimal event-based control policy. Furthermore, an adjustment mechanism is also introduced to ensure transmission security during optimization. Numerical experiments are conducted to analyze the structure and the performance of the event-based control policy for EV charging in a microgrid of three buildings.

On multi-scale event-based optimization

For most practical discrete event dynamic systems (DEDSs), the system structures are hierarchical and contain multiple subsystems, which leads to the fact that state transitions within different scales are differentiated. Directly applying standard Markov decision process (MDP) or event-based optimization (EBO) methods to model and optimize such problems is inefficient and often encounter issues like the curse of dimensionality. In this paper, a multi-scale event-driven optimization (MEBO) method is proposed. To accurately capture system state transitions at different scales, MEBO defines macro and micro events, respectively. With proper definition, the size of event space can be constant or grows only linearly with the problem size. Furthermore, two policy iteration algorithms are proposed to solve MEBO problems and the local optimality of the algorithms is proved theoretically. an example based on queueing system. Finally, the effectiveness of the MEBO methods are demonstrated by an example of admission control in queuing system. It is hoped that this method can be used as a supplement to EBO method and provide a new perspective for performance optimization of multi-scale DEDSs.

An Event Matching Energy Disaggregation Algorithm Using Smart Meter Data

Energy disaggregation algorithms disintegrate aggregate demand into appliance-level demands. Among various energy disaggregation approaches, non-intrusive load monitoring (NILM) algorithms requiring a single sensor have gained much attention in recent years. Various machine learning and optimization-based NILM approaches are available in the literature, but bulk training data and high computational time are their respective drawbacks. Considering these drawbacks, we devised an event matching energy disaggregation algorithm (EMEDA) for NILM of multistate household appliances using smart meter data. Having limited training data, K-means clustering was employed to estimate appliance power states. These power states were accumulated to generate an event database (EVD) containing all combinations of appliance operations in their various states. Prior to matching, the test samples of aggregate demand events were decreased by event-driven data compression for computational effectiveness. The compressed test events were matched in the sorted EVD to assess the contribution of each appliance in the aggregate demand. To counter the effects of transient spikes and/or dips that occurred during the state transition of appliances, a post-processing algorithm was also developed. The proposed approach was validated using the low-rate data of the Reference Energy Disaggregation Dataset (REDD). With better energy disaggregation performance, the proposed EMEDA exhibited reductions of 97.5 and 61.7% in computational time compared with the recent smart event-based optimization and optimization-based load disaggregation approaches, respectively.

Event-based optimization of service rate control in retrial queues

This paper deals with the problem of service rate control for a retrial queue where the controller can observe only the number of total customers in the system. Service rates are adjustable based on the partial information (the arrival or departure event) instead of the perfect system state (customer distribution among orbit and server). The goal is to find the optimal service rates that minimize the long-run average cost. This problem is formulated as an event-based optimization instead of a state-based optimization. Using the sensitivity-based optimization theory, we obtain interesting structures of the optimal service rate control policy, which show that the optimal policy is a bang-bang control and even has a threshold form under some mild conditions. The necessary and sufficient condition of optimal policies is also derived. Furthermore, by the difference formula of the system performances under any two policies, we develop a policy iteration-type algorithm to find the optimal policy for the case with general cost functions. With different initial values of service rates, our algorithm is demonstrated to efficiently find the optimal service rates. Moreover, we study the difference of system performances from the corresponding state-based optimization by numerical experiments, which indicates the managerial insights about the value of information for decision-makers.

Matching Uncertain Renewable Supply with Electric Vehicle Charging Demand—A Bi-Level Event-Based Optimization Method

The matching between dynamic supply of renewable power generation and flexible charging demand of the Electric Vehicles (EVs) can not only increase the penetration of renewables but also reduce the load to the state electric power grid. The challenges herein are the curse of dimensionality, the multiple decision making stages involved, and the uncertainty of both the supply and demand sides. Event-Based Optimization (EBO) provides a new way to solve large-scale Markov decision process. Considering different spatial scales, we develop a bi-level EBO model in this paper which can both catch the changes on the macro and micro levels. By proper definition, the size of event space stays fixed with the scale of the problem, which shows good scalability in online optimization. Then a bi-level Q-learning method is developed to solve the problem iteratively. We demonstrate the performance of the method by numerical examples. Our method outperforms other methods both in performance and scalability.

Optimization of stock trading with additional information by Limit Order Book

Nowadays stocks and futures are traded in electronic order-driven markets, in which orders to buy and sell are centralized in a Limit Order Book (LOB), and LOB provides more information about the stocks than their prices, such as the price dynamics and the prediction probability of the next move. In this paper, we investigate how and why Limit Order Book can improve the trading decision making. We propose a conditional binomial model for a stock price with conditional upward probability based on observations. We formulate the stock portfolio management as an optimization problem with observations, and optimal schemes are given for single-stock and multi-stock portfolios with observations. Compared with the strategies that do not consider the observations, we show that the final expected wealth achieves significant improvements. Finally, two real-data-based examples are conducted to demonstrate the efficiency of our schemes. Our work points to a new direction in improving the final wealth in portfolio management; and it illustrates the advantages of event-based optimization approach, which was successfully applied in discrete event dynamics systems.

Event-based optimization with random packet dropping

Event-based optimization (EBO) provides a general framework for policy optimization in many discrete event dynamic systems where decision making is triggered by events that represent state transitions with common features. Because the number of events can be defined by the user and usually increases linearly with respect to the system scale, EBO has good potential to address large-scale problems where the state space grows exponentially. However, in many practical systems, sensors are geographically distributed and connected to a central controller through imperfect communication channels. Therefore, events observed by the sensors may not reach the controller. Optimization methods of event-based policies in these cases have not yet been identified. In this paper, we consider this important problem and make three major contributions. First, we formulate a mathematical EBO model in which the communication between sensors and controllers is subject to random packet dropping. Second, we show that this EBO model can be converted to another EBO model with perfect communication. Then, the performance difference equation and the performance derivative equation for event-based policies are straightforward to develop. One gradient-based policy iteration algorithm is developed for problems where the state transition probabilities are explicitly known, while another for problems where they are unknown. Third, the performance of the algorithms and the impact of the packet dropping probability on policy performance are numerically demonstrated on a single-zone occupant level estimation problem in buildings.

Residential Household Non-Intrusive Load Monitoring via Smart Event-based Optimization

Home energy management requires accurate information about the appliances’ consumption pattern. This information can help consumers save energy, control their usage by shifting their usage to off-peak hours and reduce their electricity costs. Non-intrusive load monitoring (NILM) in which the power consumption profile of appliances are extracted from the aggregated signal of a household, provides this information. For the NILM problem, machine learning approaches as the training-based solutions require large training datasets for an accurate disaggregation and the optimization-based approaches employs prior information about the characteristics of appliances. This paper proposes a novel event-based optimization algorithm. In its first stage, the prior information about appliances is extracted from the events of the consumption profiles of appliances by means of clustering. Then, a new event-based down-sampling method and transition filtering are designed for decreasing the computation time of optimization. At the last stage of the proposed algorithm, post-processing considering ON duration of appliances and varying states are proposed to increase the accuracy of the power profile reconstruction. The proposed approach was successfully tested for the low-frequency dataset of a house from the REDD. Numerical results show the advantages of the proposed algorithm, marked improvement over classification-based NILM considering small training dataset and its applicability in disaggregating the power consumption measured by the smart meter.

Transactive Demand Response Operation at the Grid Edge using the IEEE 2030.5 Standard

Abstract This paper presents a transactive demand response (TDR) scheme for a network of residential customers with generation assets that emphasizes interoperability within a transactive energy architecture. A complete laboratory-based implementation provides the first (to our knowledge) realization of a comprehensive TDR use case that is fully compliant with the Institute of Electrical and Electronics Engineers (IEEE) 2030.5 standard, which addresses interoperability within a cybersecure smart energy profile (SEP) context. Verification is provided by a full system integration with commercial hardware using Internet Protocol (IP)-based (local area network (LAN) and Wi-Fi) communication protocols and transport layer security (TLS) 1.2 cryptographic protocol, and validation is provided by emulation using extensive residential smart meter data. The demand response (DR) scheme is designed to accommodate privacy concerns, allows customers to select their DR compliance level, and provides incentives to maximize their participation. The proposed TDR scheme addresses privacy through the implementation of the SEP 2.0 messaging protocol between a transactive agent (TA) and home energy management system (HEMS) agents. Customer response is handled by a multi-input multi-output (MIMO) fuzzy controller that manages negotiation between the customer agent and the TA. We take a multi-agent system approach to neighborhood coordination, with the TA servicing multiple residences on a common transformer, and use a reward mechanism to maximize customer engagement during the event-based optimization. Based on a set of smart meter data acquired over an extended time period, we engage in multiple TDR scenarios, and demonstrate with a fully-functional IEEE 2030.5-compliant implementation that our scheme can reduce network peak power consumption by 22% under realistic conditions.

A dynamic model for real-time track assignment at railway yards

In this paper, we study the real-time train assignment problem (RT-TAP) that arises from the unreliable arrival times of freight trains and the last-minute parking requests at railway yards. In the RT-TAP, the reassignment of trains to the yard is triggered every time a train arrives at the railway yard and needs to be assigned (event-based optimization). After introducing a problem formulation for the RT-TAP problem, we prove that RT-TAP is NP-Hard. In particular, the RT-TAP is modeled as a mixed integer program that strives to minimize the total weighted delay of trains. Because of its computational complexity and the time-critical nature of this problem, we introduce two real-time solution methods: (a) a problem-specific genetic algorithm (GA), (b) and a first-scheduled first-served (FSFS) heuristic. In small instances, we show that the GA returns a globally optimal solution which is identical to the solution of exact optimization methods. In larger problem instances, the heuristic approaches of FSFS and GA are tested at the Waalhaven Zuid railway yard in the Netherlands using two months of operational data. In the experimental results, the GA solutions reduce the average delays by more than 4 min compared to the solutions of the FSFS heuristic.

Event-based optimization approach for solving stochastic decision problems with probabilistic constraint

In many practical control systems or management systems, the manager of systems may allow that the statistic probability of system error or parameter deviation occurs within a certain range. The problem of decision optimization under probabilistic constraint is thus an issue needs to be addressed urgently. In this paper, we consider to develop an event-based approach which can solve the probabilistic constrained decision problems in discrete events dynamic systems. The framework of the event-based optimization is first introduced, and then with the methodology of the performance sensitivity analysis, we present an online event-based policy iteration algorithm based on the derived performance gradient formula. We apply the event-based idea and propose the concept of “risk state”, “risk event” and “risk index” which can be used to better describe the nature of the probabilistic constrained problem. Furthermore, by taking the Lagrangian approach, the constrained decision problem can be solved with two steps. Finally, numerical experiments are designed to verify the efficiency of the proposed method.

A Simulation-Based Policy Improvement Method for Joint-Operation of Building Microgrids With Distributed Solar Power and Battery

This paper investigates the joint-operation of building microgrids with distributed solar power and storage battery. The purpose of this paper is to improve the energy efficiency of the building microgrids taking advantage of the complementary building load profiles. Three major contributions are made in this paper. First, in order to incorporate the uncertainties in both the building load and solar power as well as in the communication system, the operation problem is formulated as a finite-stage event-based optimization model. Second, an on-line simulation-based policy improvement method is developed to improve the given control policy and an action aggregation method is applied to accelerate the computational speed. Third, a balanced battery operation strategy considering the expected life cycles is proposed. Numerical examples based on the building microgrids in a university campus are used to demonstrate the effectiveness of the proposed method in improving the building energy efficiency. The computational time in a practical system is also analyzed.

On distributed event-based optimization for shared economy in cyber-physical energy systems

On distributed event-based optimization for shared economy in cyber-physical energy systems

A novel POMDP-based server RAM caching algorithm for VoD systems

In contrast to other services on the Internet, streaming media service needs to fetch data from local disks more frequently, since it always lasts long and the bit rate is quite high. In addition, because of the much slower reading/writing speed of disk than random access memory (RAM), adopting advisable RAM caching policy can efficiently reduce disk I/O. In this paper, we study the problem of reducing disk I/O by using a novel approach. We first provide a new popularity estimate algorithm. Then a formal optimization problem about average disk I/O is presented, and a suboptimal caching algorithm for a special case of the problem is given. Furthermore, a partially observable Markov decision process (POMDP) model is constructed for the caching problem. Based on the model, popularity is taken advantage of to predict clients’ randomized behaviors, data replacing decisions are made when the defined observations occur, and the impact of caching actions on disk performance for future infinite steps is assessed. The method of event-based optimization is applied in search of the optimal stochastic policy. Disk I/O, as the long-run average performance measure, is optimized by applying the policy-gradient algorithm. The simulation results illustrate that data required by clients could be better predicted and lower disk I/O could be achieved by using the model proposed in this paper.

Distributed Model Predictive Control with Event-Based Optimization

Distributed model predictive control (DMPC) methods that are based on iterative optimization algorithms normally require a large number of communications between the controllers, especially when the number of subsystems is large. This can easily result in overloading the network which is normally shared between many different users. Therefore methods to reduce the load on the network while still satisfying convergence, feasibility, stability, and a certain level of performance are of a great importance. In this paper a novel event-based optimization algorithm is provided to reduce the number of communications in DMPC methods that are based on dual decomposition.

Performance of Deterministic and Probabilistic Hydrological Forecasts for the Short-Term Optimization of a Tropical Hydropower Reservoir

Hydropower is the most important source of electricity in Brazil. It is subject to the natural variability of water yield. One building block of the proper management of hydropower assets is the short-term forecast of reservoir inflows as input for an online, event-based optimization of its release strategy. While deterministic forecasts and optimization schemes are the established techniques for short-term reservoir management, the use of probabilistic ensemble forecasts and multi-stage stochastic optimization techniques is receiving growing attention. The present work introduces a novel, mass conservative scenario tree reduction in combination with a detailed hindcasting and closed-loop control experiments for a multi-purpose hydropower reservoir in a tropical region in Brazil. The case study is the hydropower project Tres Marias, which is operated with two main objectives: (i) hydroelectricity generation and (ii) flood control downstream. In the experiments, precipitation forecasts based on observed data, deterministic and probabilistic forecasts are used to generate streamflow forecasts in a hydrological model over a period of 2 years. Results for a perfect forecast show the potential benefit of the online optimization and indicate a desired forecast lead time of 30 days. In comparison, the use of actual forecasts of up to 15 days shows the practical benefit of operational forecasts, where stochastic optimization (15 days lead time) outperforms the deterministic version (10 days lead time) significantly. The range of the energy production rate between the different approaches is relatively small, between 78% and 80%, suggesting that the use of stochastic optimization combined with ensemble forecasts leads to a significantly higher level of flood protection without compromising the energy production.

Event-Based Optimization Within the Lagrangian Relaxation Framework for Energy Savings in HVAC Systems

Optimizing HVAC operation becomes increasingly important because of the rising energy cost and comfort requirements. In this paper, an innovative event-based approach is developed within the Lagrangian relaxation framework to minimize an HVAC's day-ahead energy cost. To solve the HVAC optimization problem based on events is challenging since with time-dependent uncertainties in weather, cooling load, etc., the optimal policy is not stationary. The nonstationary policy space is extremely large, and it is time consuming to find the optimal policy. To overcome the challenge, we develop an event-based approach to make the nonstationary optimal policy stationary in the planning horizon. The key idea is to augment state variables to include the time-dependent variables that make the optimal policy nonstationary and then define events based on the extended state variables. In addition, we develop within the Lagrangian relaxation framework a Q-learning method where Q-factors are used to evaluate event-action pairs and to obtain the optimal policy. Numerical results demonstrate that, as compared with time-based approaches, the event-based approach maintains similar levels of energy costs and human comfort, but reduces computational efforts significantly and has a much faster response to events.

Event-based optimization for admission control in distributed service system

In this paper, we consider the problem of admission control in a distributed service system. For the distributed structure of the system, service requests are assigned for service according to the node selection strategy. However, the admitted requests may be dropped since the service capacity of each service node in the system is limited. The desired admission control (AC) policy is to maximize the system performance while keep the dropping probability of the admitted requests below a certain threshold. The event-based optimization framework is applied to formulate the AC process. With the event-based policy, controllers at each service node make decisions only at the epoch when an event of request arrival occurs. The optimization of AC is thus modeled as a probabilistic constrained problem. We present the concept of risk-event to describe the event which happens may lead to dropping of the admitted request, and define the risk of system to denote the probability of a request been dropped. Using the Lagrangian approach, the original problem is converted to an unconstrained problem. Then, by applying the sensitivity analysis method, we derive the performance gradient formula with respect to the policy parameter based on event. Finally, numerical experiments are designed to verify the efficiency of the proposed method.

Optimal Control of Multiroom HVAC System: An Event-Based Approach

Building energy saving is of great practical interest due to the increasing energy consumption in buildings. The optimal control of the heating, ventilation, and air-conditioning (HVAC) systems leads to great energy saving potential. However, this problem is challenging due to the exponentially increasing state space and policy space. In this brief, we consider this important problem and make the following major contributions. First, we formulate the multiroom HVAC control problem as an event-based optimization, where decisions are made only when certain events occur. The size of the event space is significantly smaller than that of the state space. Second, to further simplify the calculation process, we develop an approximate solution method which focuses on local-event-based policies. These policies control the terminal devices in a room using solely the information in that room. Third, we demonstrate the performance of this method through two sets of numerical examples. In the small-scale two-room example, it is shown that our method can achieve a near-optimal solution. In the large-scale example, it is shown that the local-event-based approach can achieve a policy which is better than the threshold-based control method, hysteresis control method, and predictive control method.

Policy Gradient Approach of Event‐Based Optimization and Its Online Implementation

AbstractIn the theory of event‐based optimization (EBO), the decision making is triggered by events, which is different from the traditional state‐based control in Markov decision processes (MDP). In this paper, we propose a policy gradient approach of EBO. First, an equation of performance gradient in the event‐based policy space is derived based on a fundamental quantity called Q‐factors of EBO. With the performance gradient, we can find the local optimum of EBO using the gradient‐based algorithm. Compared to the policy iteration approach in EBO, this policy gradient approach does not require restrictive conditions and it has a wider application scenario. The policy gradient approach is further implemented based on the online estimation of Q‐factors. This approach does not require the prior information about the system parameters, such as the transition probability. Finally, we use an EBO model to formulate the admission control problem and demonstrate the main idea of this paper. Such online algorithm provides an effective implementation of the EBO theory in practice.