Dispatch Policy Research Articles

Near-Optimal Stochastic Bin-Packing in Large Service Systems with Time-Varying Item Sizes

In modern computing systems, jobs' resource requirements often vary over time. Accounting for this temporal variability during job scheduling is essential for meeting performance goals. However, theoretical understanding on how to schedule jobs with time-varying resource requirements is limited. Motivated by this gap, we propose a new setting of the stochastic bin-packing problem in service systems that allows for time-varying job resource requirements, also referred to as 'item sizes' in traditional bin-packing terms. In this setting, a job or 'item' must be dispatched to a server or 'bin' upon arrival. Its resource requirement varies over time as a Markov chain while in service. Our goal is to minimize the expected number of active servers, or 'non-empty bins', in the steady state. Under our problem formulation, we develop a job dispatch policy, named Join-Requesting-Server (JRS). We show that in the asymptotic regime where the job arrival rate scales large linearly with respect to a scaling factor r, JRS achieves an additive optimality gap of O(√r) in the objective value, where the optimal objective value is Θ(r). Our approach highlights a novel policy conversion framework that utilizes the solution of a single-server problem.

A bi-objective model for location, dispatch and relocation of ambulances with a revision of dispatch policies

PurposeUntimely responses to emergency situations in urban areas contribute to a rising mortality rate and impact society's primary capital. The efficient dispatch and relocation of ambulances pose operational and momentary challenges, necessitating an optimal policy based on the system's real-time status. While previous studies have addressed these concerns, limited attention has been given to the optimal allocation of technicians to respond to emergency situation and minimize overall system costs.Design/methodology/approachIn this paper, a bi-objective mathematical model is proposed to maximize system coverage and enable flexible movement across bases for location, dispatch and relocation of ambulances. Ambulances relocation involves two key decisions: (1) allocating ambulances to bases after completing services and (2) deciding to change the current ambulance location among existing bases to potentially improve response times to future emergencies. The model also considers the varying capabilities of technicians for proper allocation in emergency situations.FindingsThe Augmented Epsilon-Constrained (AEC) method is employed to solve the proposed model for small-sized problem. Due to the NP-Hardness of the model, the NSGA-II and MOPSO metaheuristic algorithms are utilized to obtain efficient solutions for large-sized problems. The findings demonstrate the superiority of the MOPSO algorithm.Practical implicationsThis study can be useful for emergency medical centers and healthcare companies in providing more effective responses to emergency situations by sending technicians and ambulances.Originality/valueIn this study, a two-objective mathematical model is developed for ambulance location and dispatch and solved by using the AEC method as well as the NSGA-II and MOPSO metaheuristic algorithms. The mathematical model encompasses three primary types of decision-making: (1) Allocating ambulances to bases after completing their service, (2) deciding to relocate the current ambulance among existing bases to potentially enhance response times to future emergencies and (3) considering the diverse abilities of technicians for accurate allocation to emergency situations.

DeepDispatch: Deep Reinforcement Learning-Based Vehicle Dispatch Algorithm for Advanced Air Mobility

Near-future air taxi operations with electric vertical takeoff and landing aircraft will be constrained by the need for frequent recharging and limited takeoff and landing pads in vertiports and will be subject to time-varying demand and electricity prices, making the dispatch problem unique and particularly challenging to solve. Previously, the authors have developed optimization models to address this problem. Such optimization models, however, suffer from prohibitively high computational run times when the scale of the problem increases, making them less practical for real-world implementation. To overcome this issue, the authors have developed two deep reinforcement learning-based dispatch algorithms, namely, single-agent and multi-agent double dueling deep Q-network dispatch algorithms, where the objective is to maximize operating profit. A passenger transportation simulation environment was built to assess the performance of these algorithms across 36 numerical cases with varying numbers of vehicles and vertiports and amounts of demand. The results indicate that the multi-agent dispatch algorithm can closely approximate the optimal dispatch policy with significantly less computational expenses compared to the benchmark optimization model. The multi-agent algorithm was found to outperform the single-agent counterpart with respect to both profits generated and training time. Additionally, we implemented a heuristic-based algorithm, faster but less effective in generating profits compared to our two deep reinforcement learning-based algorithms.

Assessment of Storage Sizing for Solar Tower Plants Using Model-Predictive Control for Dispatch Planning

Abstract Solar tower with thermal energy storage (ST-TES) represents a promising technology for large-scale exploitation of solar irradiation for electricity generation. A ST-TES has the potential to extend electricity generation to more favorable conditions, such as high electricity prices. The size of TES, however, constrains the flexibility of dispatching, especially when there is significant uncertainty in forecasts of solar irradiation and electricity prices. This study explores the impact of TES size when the plant uses model-predictive control (MPC) for dispatch planning. The performance of MPC is benchmarked against one perfect knowledge (PK) and two day-ahead strategies. The optimal achievable profit for each TES size is determined using the PK strategy. An analysis is conducted to evaluate the relative profit losses for all the other simulated strategies compared to the PK strategy. A case study is conducted for a hypothetical 115 MWe ST-TES in South Australia. For January and August, 100 tests are performed for each dispatch policy, with the TES size varying from 6 to 14 h. The revenue evaluation is conducted with both fixed and wholesale spot prices. The analysis shows that MPC-aided dispatching enables the adoption of a smaller TES compared to day-ahead policies while maintaining the same expected profit. The resulting TES size reduction from 14 to 10 h translates into approximately up to $45.4 million in capital cost savings. The findings of this study can inform the ST-TES plant’s design procedures and facilitate negotiations for electricity sales contracts.

Towards Risk-Aware Real-Time Security Constrained Economic Dispatch: A Tailored Deep Reinforcement Learning Approach

In the presence of increasing uncertainties brought by intermittent renewable energy sources, security and risk management continue to be the most critical concern in modern power system operation. Risk-aware, real-time security constrained economic dispatch (RT-SCED) provides an efficient solution towards promptly, economically and robustly responding to the changes in the power system operating state. Despite different model-based methods have been developed to handle uncertainties, significant computation burden arise to incorporate <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$N$</tex-math></inline-formula> -1 contingency constraints with a higher temporal resolution in RT-SCED. Driven by similar computational challenges, risk evaluation is often overlooked in the current application of deep reinforcement learning (DRL) based data-driven methods in RT-SCED. This paper proposes a DRL-based risk-aware RT-SCED methodological framework by incorporating a novel data-driven risk evaluation model to foster efficient agent-environment interactions. The real-time dispatch policies are constructed with an improved twin delayed deep deterministic policy gradient method. The policy network features a residual network architecture and incorporates an active power allocation mechanism to integrate empirical dispatch knowledge, preventing early termination and fostering more efficient learning behavior. Case studies validate the superior performance of the proposed method in risk-aware RT-SCED on cost efficiency, uncertainty adaptability and computational efficiency, through benchmarking against model-based and data-driven baseline methods.

Fast Coordinated Predictive Control for Renewable Energy Integrated Cascade Hydropower System Based on Quantum Neural Network

The increasing penetration of renewable energy poses intractable uncertainties in cascade hydropower systems, such that excessively conservative operations and unnecessary curtailment of clean energies can be incurred. To address these challenges, a quantum neural network (QNN)-based coordinated predictive control approach is proposed. It manipulates coordinated dispatch of multiple clean energy sources, including hydro, wind, and solar power, leverages QNN to conquer intricate multi-uncertainty and learn intraday predictive control patterns, by taking renewable power, load, demand response (DR), and optimal unit commitment as observations. This enables us to exploit the stability and exponential memory capacity of QNN to extrapolate diversified dispatch policies in a reliable manner, which can be hard to reach for traditional learning algorithms. A closed-loop warm start framework is finally presented to enhance the dispatch quality, where the decisions by QNN are fed to initialize the optimizer, and the optimizer returns optimal solutions to quickly evolve the QNN. A real-world case in the ZD sub-grid of the Sichuan power grid in China demonstrates that the proposed method hits a favorable balance between operational cost, accuracy, and efficiency. It realizes second-level elapsed time for intraday predictive control.

Learning-Based Optimal Admission Control in a Single-Server Queuing System

We consider a long-term average profit–maximizing admission control problem in an M/M/1 queuing system with unknown service and arrival rates. With a fixed reward collected upon service completion and a cost per unit of time enforced on customers waiting in the queue, a dispatcher decides upon arrivals whether to admit the arriving customer or not based on the full history of observations of the queue length of the system. Naor [Naor P (1969) The regulation of queue size by levying tolls. Econometrica 37(1):15–24] shows that, if all the parameters of the model are known, then it is optimal to use a static threshold policy: admit if the queue length is less than a predetermined threshold and otherwise not. We propose a learning-based dispatching algorithm and characterize its regret with respect to optimal dispatch policies for the full-information model of Naor [Naor P (1969) The regulation of queue size by levying tolls. Econometrica 37(1):15–24]. We show that the algorithm achieves an O(1) regret when all optimal thresholds with full information are nonzero and achieves an [Formula: see text] regret for any specified [Formula: see text] in the case that an optimal threshold with full information is 0 (i.e., an optimal policy is to reject all arrivals), where N is the number of arrivals. Funding: A. Cohen is partially supported by the National Science Foundation [Grant DMS-2006305]. V. Subramanian is supported in part by the NSF [Grants CCF-2008130, ECCS-2038416, CNS-1955777, and CMMI-2240981].

Robust and Chance-Constrained Dispatch Policies for Linear Power Systems

Several methods have recently been proposed to determine optimal feedback control policies for generator dispatch under uncertain grid in-feeds. This work presents and explains two such approaches based on a constrained linear power flow model in a unified notation, a robust method based on linear programming and a chance-constrained one based on polynomial chaos expansions and second order cone programming. This work compares both methods on the same small test example and discusses consequences for long-term grid expansion planning. While the robust method is easier to implement, it is also not more restrictive. For the chance-constrained method, we highlight a serious drawback in reformulations common in the literature. As an aside, we extend the existing formulation to cases where not all disturbances can be measured, and combine it with a robust pre-processing scheme.

An agent-based simulation study on the effect of ambulance dispatch policies on patient mortality in hospital systems: a disaster response test case

An agent-based simulation study on the effect of ambulance dispatch policies on patient mortality in hospital systems: a disaster response test case

Near-Optimal Stochastic Bin-Packing in Large Service Systems with Time-Varying Item Sizes

In modern computing systems, jobs' resource requirements often vary over time. Accounting for this temporal variability during job scheduling is essential for meeting performance goals. However, theoretical understanding on how to schedule jobs with time-varying resource requirements is limited. Motivated by this gap, we propose a new setting of the stochastic bin-packing problem in service systems that allows for time-varying job resource requirements, also referred to as 'item sizes' in traditional bin-packing terms. In this setting, a job or 'item' must be dispatched to a server or 'bin' upon arrival. Its resource requirement may vary over time while in service, following a Markovian assumption. Once the job's service is complete, it departs from the system. Our goal is to minimize the expected number of active servers, or 'non-empty bins', in steady state. Under our problem formulation, we develop a job dispatch policy, named Join-Reqesting-Server (JRS). Broadly, JRS lets each server independently evaluate its current job configuration and decide whether to accept additional jobs, balancing the competing objectives of maximizing throughput and minimizing the risk of resource capacity overruns. The JRS dispatcher then utilizes these individual evaluations to decide which server to dispatch each arriving job to. The theoretical performance guarantee of JRS is in the asymptotic regime where the job arrival rate scales large linearly with respect to a scaling factor r. We show that JRS achieves an additive optimality gap of O(√r) in the objective value, where the optimal objective value is Θ(r). When specialized to constant job resource requirements, our result improves upon the state-of-the-art o(r) optimality gap. Our technical approach highlights a novel policy conversion framework that reduces the policy design problem into a single-server problem.

Risk-averse stochastic dynamic power dispatch based on deep reinforcement learning with risk-oriented Graph-Gan sampling

The increasing penetration of renewable energy sources (RES) brings volatile stochasticity, which significantly challenge the optimal dispatch of power systems. This paper aims at developing a cost-effective and robust policy for stochastic dynamic optimization of power systems, which improves the economy as well as avoiding the risk of high costs in some critical scenarios with small probability. However, it is hard for existing risk-neutral methods to incorporate risk measure since most samples are normal. For this regard, a novel risk-averse policy learning approach based on deep reinforcement learning with risk-oriented sampling is proposed. Firstly, a generative adversarial network (GAN) with graph convolutional neural network (GCN) is proposed to learn from historical data and achieve risk-oriented sampling. Specifically, system state is modelled as graph data and GCN is employed to capture the underlying correlation of the uncertainty corresponding to the system topology. Risk knowledge is the embedded to encourage more critical scenarios are sampled while aligning with historical data distributions. Secondly, a modified deep reinforcement learning (DRL) with risk-measure under soft actor critic framework is proposed to learn the optimal dispatch policy from sampling data. Compared with the traditional deep reinforcement learning which is risk-neutral, the proposed method is more robust and adaptable to uncertainties. Comparative simulations verify the effectiveness of the proposed method.

Maximum Throughput Dispatch for Shared Autonomous Vehicles Including Vehicle Rebalancing

Shared autonomous vehicles (SAVs) provide on demand point-to-point transportation for passengers. This service has been extensively studied using dispatch heuristics and agent based simulations of large urban areas. However, these approaches make no mathematical guarantees of the passenger throughput for the SAV network. This study builds on the dynamic queuing model design of Kang and Levin which provides a maximum stability dispatch policy for SAVs. This model is extended to include rebalancing of empty vehicles to regions of high demand. The modified dispatch policy is proven to maximize throughput. Simulation results show that this dispatch policy reduces waiting times (between vehicle dispatch and passenger pickup) compared to the original formulation. However, vehicle time traveling empty increases in some scenarios. Simulation results also show that rebalancing often reduces passenger waiting times, but not when too many vehicles rebalance at once and are not available for dispatch.

Real-time dispatch of integrated electricity and thermal system incorporating storages via a stochastic dynamic programming with imitation learning

Coordinated dispatch of integrated electricity and thermal system (IETS) provides extra operation flexibility which is further improved by integration of electrical and thermal storages. However, the problem non-convexity and multiple uncertainties hinder its optimal real-time dispatch. This paper proposes an approximate dynamic programming with imitation learning (ADP-IL) based real-time dispatch policy for IETS with electrical and thermal storages, which is computationally efficient and adaptive to uncertainties while satisfying complex networks constraints. First, real-time dispatch of IETS is reformulated as a multistage stochastic sequential optimization and non-convex terms are addressed by mixed integer programming. Next, an ADP which exploits value function monotonicity is employed to temporally decompose original problem and an off-line pre-learning is incorporated to address dimension complexity. Then, imitation learning, which allows the algorithm to learn from expert demonstration, is introduced to further accelerate off-line pre-learning. After sufficient learning, ADP-IL provides high quality real-time solution with remarkable computation efficiency. Comprehensive studies verify optimality, adaptability, efficiency, and scalability of ADP-IL.

Rolling Horizon Robust Real-Time Economic Dispatch with Multi-Stage Dynamic Modeling

A multi-stage robust real-time economic dispatch model (MRRTD) for power systems is proposed in this paper. The MRRTD takes the dynamic form of multi-stage robust optimization as the framework to naturally simulate the operation of equipment that is temporally coupled, e.g., utility-level energy storage systems. For normal systems, the MRRTD can work directly in short time slots with a rolling horizon. For large-scale systems, the MRRTD expands the time-slot scale and generates optimal dispatch policies. With this guidance, the real-time dispatch decision can be swiftly made thereafter. In addition, a dynamic uncertainty set based on deep learning is proposed, which can dynamically refine the covering ability for probable occurred wind power scenarios. To efficiently solve the MRRTD, a novel fast robust dual dynamic programming method is employed. The effectiveness of the proposed model and solution algorithm, especially the improved scalability compared to several other dynamic economic dispatch methods, are demonstrated by simulation results from six benchmark test cases ranging from a modified IEEE 6-bus system to a 6495-bus system.

A spatio-temporality-enabled parallel multi-agent-based real-time dynamic dispatch for hydro-PV-PHS integrated power system

Integrated power system has emerged as a powerful alternative to penetrate renewables, due to its ability to reconcile energy discrepancy. However, due to limited mainstreams and complex mountainous meteorology, the dispatch of hydro-photovoltaic-pumped hydro storage (Hydro-PV-PHS) integrated power system (IPS) which are predominantly composed of cascaded daily-regulation and uncontrollable runoff hydropower stations and PVs still miss the expected clean energy utilization rates. To conquer the issue, a novel spatio-temporality-enabled parallel multi-agent-based dynamic dispatch method is proposed. At the outset, a temporal dispatch model fed by dynamic measurements of available PV generation and inflow is presented. To master the uncertainties, such model must be solved in real-time upon the incoming measurements. Whereupon the presented model is recast as Markov decision process for learning dispatch policies, parameterized by neural network agents. To manage the enormous spatial-temporal operating space of the hydro-PV-PHS IPS and to prevent conflict policies, a long short-term memory auto-encoder (LSTM-AE) combined unsupervised learning scheme is used to alleviate divergence and decrease stochasticity of renewables to pre-uncouple policies, which are then distributed to multiple parallel agents. Finally, distributed proximal policy optimization is conducted to produce dispatch policies in an offline parallel manner, with each agent responsible for dispatching the hydro-PV-PHS IPS within the respective operating subspace. The numerical studies in a real-world case demonstrate that the proposed scheme enables real-time and near-optimal dynamic dispatch for the concern IPS, and outperforms other rivals in terms of adaptability, robustness, and efficiency.

FMS-dispatch: a fast maximum stability dispatch policy for shared autonomous vehicles including exiting passengers under stochastic travel demand

Shared autonomous vehicles (SAVs) are a fleet of autonomous taxis that provide point-to-point transportation services for travellers, and have the potential to reshape the nature of the transportation market in terms of operational costs, environmental outcomes, increased tolling efficiency, etc. However, the number of waiting passengers could become arbitrarily large when the fleet size is too small for travel demand, which could cause an unstable network. An unstable network will make passengers impatient and some people will choose some other alternative travel modes, such as metro or bus. To achieve stable and reliable SAV services, this study designs a dynamic queueing model for waiting passengers and provides a fast maximum stability dispatch policy for SAVs when the average number of waiting for passengers is bounded in expectation, which is analytically proven by the Lyapunov drift techniques. After that, we expand the stability proof to a more realistic scenario accounting for the existence of exiting passengers. Unlike previous work, this study considers exiting passengers in stability analyses for the first time. Moreover, the maximum stability of the network doesn't require a planning horizon based on the proposed dispatch policy. The simulation results show that the proposed dispatch policy can ensure the waiting queues and the number of exiting passengers remain bound in several experimental settings.

A scalable graph reinforcement learning algorithm based stochastic dynamic dispatch of power system under high penetration of renewable energy

Due to the increasing penetration of renewable energy source, power system is facing significant uncertainties. How to fully consider those uncertainties in dynamic economic dispatch (DED) has become a crucial problem to the safe and economic operation of power system. Reinforcement learning (RL) based approaches can provide the dispatch policy in response to uncertainty. However, current RL uses traditional Euclidean data representation, which greatly reduces the scalability and computational efficiency of economic dispatch algorithms. To address such obstacle, this paper develops a novel graph reinforcement learning (GRL) method for DED of power system. Firstly, DED is formulated as a Markov decision process and formulated as a dynamic sequential decision problem. Secondly, a novel graph-based representation of system state is proposed. Using graph data to represent dispatch operation data with non-Euclidean characteristics can effectively capture the implicit correlation of the uncertainty corresponding to the system topology. Thirdly, a GRL algorithm is proposed to learn the optimal policy which maps graph represented system state to DED decision. Compared with the traditional deep reinforcement learning (DRL), the GRL proposed in this paper has certain generalization ability and scalability, and can achieve higher quality solutions in online operation. Case studies illustrate that the optimality of the proposed method is 98.04%, which is 15.13% higher than that of the existing learning methods. The algorithm is scalable and improves the sample efficiency.

Local integrated energy system operational optimization considering multi‐type uncertainties: A reinforcement learning approach based on improved TD3 algorithm

AbstractLocal integrated energy system (IES), usually a multi‐building heating and cooling system incorporating cogeneration systems and distributed energy resources (DERs), is becoming an efficient energy infrastructure for energy system decarbonization. However, for small and medium‐scale local IESs, the fluctuation of user loads seriously influences energy system operation, and the increasing DER penetration also enlarges the influence. Given the necessity of exploring an efficient way to handle the uncertainties in local IES, this paper proposes a reinforcement learning (RL) approach based on the improved TD3 algorithm. The mathematical model of local IES is first established considering supply‐ and load‐side flexible resources. The local IES dispatch problem is formulated as a Markov decision process (MDP), in which multi‐type uncertainties of renewable generation, electric load and heat load are considered. For solving the MDP, an improved twin delayed deep deterministic policy gradient (TD3) algorithm is proposed with a dynamic balance mechanism of exploration noise. Based on a local IES testbed in the Nantong Central Innovation District, China, a comparison analysis is conducted to verify the promoting effect of flexible resources on the operation economy and renewable energy consumption. The system operating cost reduces by 18.46%, and surplus renewable energy can all be accommodated considering flexible resources. The dispatch policies obtained by the deep deterministic policy gradient (DDPG), the improved TD3, the original TD3 and traditional optimization algorithms are also compared. The results reveal that the convergence stability and solving accuracy of the improved TD3 outperform the other two RL algorithms. Specifically, the system operating cost of the improved TD3 reduces by 2.76% compared with the DDPG, and the energy supply imbalance decreases by around 88%. Meanwhile, the improved TD3 exhibits better operation economy and adaptability to the uncertain environment than the deterministic optimization and intraday rolling algorithms.

A Survey on Market Based Economic Security Dispatch (MBESD) Policy in Supply Chain Power Market

Abstract: Power System safety and financial dispatch is principal intention to restructuring of electricity system. Economizing on gas price and minimizing emission of gases (CO, CO2, NO2, SO2) are one approach of to maintained safety of machine however solely this system are no longer archived this label of security. Now in pandemic state of affairs when energy financial increase is falling down then it is very tough to maintained desirable device security. In this pandemic scenario country wide load dispatching centers play essential position to overcome this situation. This paper is excessive lightening and mentioned some new current procedural schemes. We are additionally put into effect a new idea (MBED).This paper is exposes special method to enhancement of Power Policies. This method shall be relevant to all thermal ISGS (Inter –State – Generating Station) that are regional entities collaborating in RRAS mechanism and whose tariff is decided or adopted with the aid of the CERC (Central Electricity Regulatory Commission) for their full capacity. Here we are mentioned all on the foundation of NLDC, RLDC, RPCS, SCED exclusive adopting schemes and comparative studies.

Optimal economic dispatch policy for prosumer with energy storage considering self-consumption demand

This paper analyzed the effects of self-consumption demand on the joint economic dispatch of prosumers (energy consumers who are also producers), particularly for prosumers with both energy storage and distributed energy sources (DERs). Studies in the existing literature on the economic dispatch scheduling policy of energy storage, mostly from the perspective of electricity merchants, do not address the impacts of self-consumption demand. However, due to the intermittent and high levels of uncertainty regarding DERs generation and the dynamic demand of the prosumer, production and consumption are not always simultaneous; there are two possible scenarios in each period depending on whether DERs generation can meet prosumers' self-consumption or not. Incorporating the self-consumption demand will pose modeling challenges since these two scenarios cannot occur simultaneously in each period, and different scenarios require different decisions for prosumers. This paper analyzed the two scenarios separately to find the optimal storage scheduling strategy, and the results were combined to get the optimal global solution. We focused on prosumers' economic decision-making while considering self-consumption demand and the physical constraints of a battery based on dynamic programming. Our study showed that the feasible state of charge (SOC) range of storage can be segmented into several sub-ranges by SOC reference points under the above two scenarios. As a result, a prosumer's optimal scheduling can be uniquely and conveniently selected based on the sub-ranges within which the current SOC falls. The results, therefore, provided multistage decision-making guidance for prosumers with energy storage.