Stochastic Optimization Problem Research Articles

A two-stage stochastic collaborative planning approach for data centers and distribution network incorporating demand response and multivariate uncertainties

As a critical infrastructure to support the development of digital economy, the scale of data center (DC) power consumption has been growing continuously in recent years. The plug-in of a large number of DCs to the distribution network (DN) presents both opportunities and challenges for the secure, stable, and sustainable operation of the system. In order to address the planning issues arising from the integration of DCs into the DN, this paper proposes a two-stage (planning stage and operation stage) collaborative planning method for DCs and the DN while considering the source-demand uncertainty. Firstly, spatial-temporal demand response characteristic of DCs is explicitly elaborated, and based on the energy consumption model, a two-stage stochastic planning model considering conditional value at risk (CVaR) for DCs and the DN is constructed using probabilistic stochastic simulation, which obeys low-carbon and security, and satisfies the source-network-load constraints of the system. The siting and sizing for DCs and renewable generation, feeder expansion for the DN are determined to minimize the investment cost during the planning stage, while the power output for the devices considering uncertainties from load and renewable energy are optimized to minimize the operation cost during the operation stage. Secondly, the L-shaped decomposition algorithm is proposed to solve the two-stage stochastic optimization problem by reconstructing the problem into a master problem and subproblems, where the dual multiplies are calculated to generate feasibility cuts and optimality cuts to achieve iterative solving. Finally, numerical case studies on a 33-bus distribution system demonstrate that the participation of DCs in demand response can achieve a significant reduction in the investment and operation cost of the DN, and further promote the consumption of distributed renewable energy sources.

Optimal Service Time Windows

Because customers must usually arrange their schedules to be present for home services, they desire an accurate estimate of when the service will take place. However, even when firms quote large service time windows, they are often missed, leading to customer dissatisfaction. Wide time windows and frequent failures occur because time windows must be communicated to customers in the face of several uncertainties: future customer requests are unknown, final service plans are not yet determined, and when fulfillment is outsourced to a third party, the firm has limited control over routing procedures and eventual fulfillment times. Even when routing is performed in-house, time windows often do not receive explicit consideration. In this paper, we show how companies can communicate reliable and narrow time windows to customers in the face of arrival time uncertainty when time window decisions are decoupled from routing procedures. Under assumptions on the shape of arrival time distributions, our main result characterizes the optimal policy, identifying structure that reduces a high-dimensional stochastic nonlinear optimization problem to a root-finding problem in one dimension. The result inspires a practice-ready heuristic for the more general case. Relative to the industry standard of communicating uniform time windows to all customers, and to other policies applied in practice, our method of quoting customer-specific time windows yields a substantial increase in customer convenience without sacrificing reliability of service. Our results show that time windows should be tailored to individual customers, time window sizes should be proportional to the service level, larger time windows should be assigned to earlier requests and smaller time windows to later requests, larger time windows should be assigned to customers further from the depot of operation and smaller time windows to closer customers, high quality time windows can be identified even with limited data, and cost savings afforded by routing efficiency should be measured against potential losses to customer convenience. Funding: M. W. Ulmer’s work is funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) Emmy Noether Programme, [project 444657906]. Supplemental Material: The e-companion is available at https://doi.org/10.1287/trsc.2023.0004 .

Data-Driven Compositional Optimization in Misspecified Regimes

With a manifold growth in the scale and intricacy of systems, the challenges of parametric misspecification become pronounced. These concerns are further exacerbated in compositional settings, which emerge in problems complicated by modeling risk and robustness. In “Data-Driven Compositional Optimization in Misspecified Regimes,” the authors consider the resolution of compositional stochastic optimization problems, plagued by parametric misspecification. In considering settings where such misspecification may be resolved via a parallel learning process, the authors develop schemes that can contend with diverse forms of risk, dynamics, and nonconvexity. They provide asymptotic and rate guarantees for unaccelerated and accelerated schemes for convex, strongly convex, and nonconvex problems in a two-level regime with extensions to the multilevel setting. Surprisingly, the nonasymptotic rate guarantees show no degradation from the rate statements obtained in a correctly specified regime and the schemes achieve optimal (or near-optimal) sample complexities for general T-level strongly convex and nonconvex compositional problems.

A stochastic variance reduced gradient method with adaptive step for stochastic optimization

AbstractIn this paper, we propose a stochastic variance reduction gradient method with adaptive step size, referred to as the SVRG‐New BB method, to solve the convex stochastic optimization problem. The method could be roughly viewed as a hybrid of the SVRG algorithm and a new BB step mechanism. Under the condition that the objective function is strongly convex, we provide the linear convergence proof of this algorithm. Numerical experiment results show that the performance of the SVRG‐New BB algorithm can surpass other existing algorithms if parameters in the algorithm are properly chosen.

Optimal Control of Probability on a Target Set for Continuous-Time Markov Chains

In this paper, a stochastic optimal control problem is considered for a continuous-time Markov chain taking values in a denumerable state space over a fixed finite horizon. The optimality criterion is the probability that the process remains in a target set before and at a certain time. The optimal value is a superadditive capacity of target sets. Under some minor assumptions for the controlled Markov process, we establish the dynamic programming principle, based on which we prove that the value function is a classical solution of the Hamilton-Jacobi-Bellman equation on a discrete lattice space. We then prove that there exists an optimal deterministic Markov control under the compactness assumption of control domain. We further prove that the value function is the unique solution of the HJB equation. We also consider the case starting from the outside of target set and give the corresponding results. Finally, we apply our results to two examples.

Convexified Open-Loop Stochastic Optimal Control for Linear Systems With Log-Concave Disturbances

We consider open-loop solutions to the stochastic optimal control of a linear dynamical system with an additive non-Gaussian, log-concave disturbance. We propose a novel, sampling-free approach, based on characteristic functions and convex optimization, to cast the stochastic optimal control problem as a difference-of-convex program. Our method invokes higher moments, resulting in less conservatism compared to moment-based approaches. We employ piecewise affine approximations and the convex-concave procedure for efficient solution via standard conic solvers. We demonstrate that the proposed solution is competitive with sampling and moment based approaches, without compromising probabilistic constraints.

Duality for Nonlinear Filtering II: Optimal Control

This paper is concerned with the development and use of duality theory for a nonlinear filtering model with white noise observations. The main contribution of this paper is to introduce a stochastic optimal control problem as a dual to the nonlinear filtering problem. The mathematical statement of the dual relationship between the two problems is given in the form of a duality principle. The constraint for the optimal control problem is the backward stochastic differential equation (BSDE) introduced in the companion paper. The optimal control solution is obtained from an application of the maximum principle, and subsequently used to derive the equation of the nonlinear filter. The proposed duality is shown to be an exact extension of the classical Kalman-Bucy duality, and different from other types of optimal control and variational formulations given in literature.

Deep Reinforcement Learning-Based Security-Constrained Battery Scheduling in Home Energy System

The home energy system today involves multiple renewable energy sources and battery energy storage systems, which can be considered as a microgrid. The battery energy storage system is a key component in the home energy system for the sake of filling the gap between the user demand and volatile energy supplies to maximize the techno and economic performances. However, the battery scheduling must suffer the stochastic nature of renewable energy resources and loads, which results in an intractable multi-period stochastic optimization problem with security constraints. An improved actor-critic-based reinforcement learning is proposed for this issue, where a distributional critic net is applied for faster and more accurate reward estimation under uncertainties, and a policy net incorporating protective secondary control is adopted to satisfy security constraints, preventing the unsafe state of batteries during the trial-and-error process. Numerical tests show that the proposed approach outperforms conventional reinforcement learning algorithms, as well as the rule-based battery scheduling approach while guaranteeing safe operation. The robustness and adaptability of the proposed method are also verified in case studies with different optimization tasks.

Online Statistical Inference for Stochastic Optimization via Kiefer-Wolfowitz Methods

This article investigates the problem of online statistical inference of model parameters in stochastic optimization problems via the Kiefer-Wolfowitz algorithm with random search directions. We first present the asymptotic distribution for the Polyak-Ruppert-averaging type Kiefer-Wolfowitz (AKW) estimators, whose asymptotic covariance matrices depend on the distribution of search directions and the function-value query complexity. The distributional result reflects the tradeoff between statistical efficiency and function query complexity. We further analyze the choice of random search directions to minimize certain summary statistics of the asymptotic covariance matrix. Based on the asymptotic distribution, we conduct online statistical inference by providing two construction procedures of valid confidence intervals. Supplementary materials for this article are available online.

Value iteration algorithm for continuous-time linear quadratic stochastic optimal control problems

Value iteration algorithm for continuous-time linear quadratic stochastic optimal control problems

Convergence with rates for a Riccati-based discretization of SLQ problems with SPDEs

Abstract We consider a new discretization in space (parameter $h>0$) and time (parameter $\tau>0$) of a stochastic optimal control problem, where a quadratic functional is minimized subject to a linear stochastic heat equation with linear noise. Its construction is based on the perturbation of a generalized difference Riccati equation to approximate the related feedback law. We prove a convergence rate of almost ${\mathcal O}(h^{2}+\tau )$ for its solution, and conclude from it a rate of almost ${\mathcal O}(h^{2}+\tau )$ resp. ${\mathcal O}(h^{2}+\tau ^{1/2})$ for computable approximations of the optimal state and control with additive resp. multiplicative noise.

Optimum exploitation of multiple energy system using IGDT approach and risk aversion strategy and considering compressed air storage with solar energy

The concept of an energy hub within a microgrid has garnered significant attention from investors due to its potential for efficiently managing diverse energy sources and accurately forecasting energy prices. This study presents a model for an energy hub that facilitates the optimal operation of a microgrid with multiple energy carrier infrastructures under future-day scenarios. The primary objective of the optimization process is to minimize operating costs, environmental impacts, and technical limitations. The proposed energy hub model integrates both distributable generation, such as a Combined Cooling, Heating, and Power (CCHP) unit, and non-distributable generation sources, including wind turbines and photovoltaic systems. Additionally, it incorporates various energy storage components, namely Ice Storage Conditioner (ISC), Thermal Energy Storage (TES) systems, and Electric Vehicles (EVs). Furthermore, the study investigates the impact of a novel energy storage system called Solar Powered Compressed Air Energy Storage (SPCAES) on the operational and environmental performance of the energy hub. To address uncertainties stemming from predicted and actual variables, a two-level stochastic optimization problem is formulated based on the Information Gap Decision Theory (IGDT) with a Risk Aversion (RA) strategy. The overarching aim is to mitigate the risks associated with the information gap confronted by microgrid operators. Subsequently, the two-level stochastic optimization problem is transformed into a single-level problem using the Karush-Kuhn-Tucker method. The proposed framework employs the RA-IGDT method and utilizes the Elephant Herd Optimization (EHO) algorithm to effectively solve the optimization problem. By applying this proposed framework to a typical energy hub, this research demonstrates the effectiveness of energy storage systems in reducing operating costs and greenhouse gas emissions during daily energy management. The study highlights the potential of energy hubs and storage systems for cost-effective and eco-friendly microgrid energy management. Scenario 4, featuring the proposed energy hub with SPCAES and ISC storage, demonstrates a 15 % reduction in operating costs and a notable 20 % increase in peak-hour electrical power for sale, underscoring its economic and operational benefits.

Re-use of samples in stochastic annealing

The re-use of samples in stochastic black box optimisation is a double-edged sword. On the one hand it has the potential of substantially reducing the number of simulation runs required, on the other hand it introduces dependencies between iterations of the optimisation algorithm that may misguide the search. This paper proposes a principled way to re-use samples in stochastic annealing, a generalisation of simulated annealing for stochastic black-box optimisation problems. We propose three alternative algorithms that all, despite the stochastic errors when evaluating a solution via simulation, obey the detailed balance equation and thus have the same assurance of converging to the true optimum as standard simulated annealing in the deterministic case. This is achieved by optimising in the product space of solution and error. We compare our new algorithms in terms of their acceptance ratio as well as empirically on two stochastic combinatorial optimisation problems and find that the best algorithms are of order twice as efficient as the state-of-the-art. Furthermore, one algorithm we propose can work with non-Gaussian and even unknown error distributions.

Controlling a generalized Fokker–Planck equation via inputs with nonlocal action

This paper concerns an optimal control problem (P) related to a generalized Fokker–Planck equation. Basic properties of the solutions to the generalized FP equation are derived via a semigroup approach in the space H−1(Rd). Problem (P) is proven to be deeply related to a stochastic optimal control problem (PS) for a McKean–Vlasov equation. The existence of an optimal control is obtained for the deterministic problem (P). The existence of an optimal control is established for an approximating optimal control problem (Ph) related to a backward Euler approximation of the generalized FP equation (with a constant discretization step h). One proves under additional hypotheses that “(Ph) converges to (P)” in a certain sense. First order necessary conditions for (Ph) are derived as well.

Optimal Spatial–Temporal Scheduling for EVs With Stochastic Behaviors and Possible Inauthentic Information

This paper studies the spatial-temporal scheduling for electric vehicles with stochastic charging behaviors. A new spatial-temporal scheduling model is proposed to achieve both the spatial coordination and temporal coordination for EV users and is modeled as a non-convex optimization problem which can be solved by the coordinate descent method. Further, a new billing method of charging cost is proposed based on the Vickrey-Clarke-Groves to avoid possible inauthentic information from users. Specially, the individual rationality and incentive compatibility of the method are theoretically proved. Moreover, a stochastic optimization problem is proposed to describe the stochastic charging behavior of users, which can fit the real application better than the deterministic one. Consequently, an efficient charging strategy is obtained via chance constraint method. Simulation results and comparison studies show the effectiveness of both the model and the billing method.

Distributed Online Stochastic-Constrained Convex Optimization With Bandit Feedback.

This article studies the distributed online stochastic convex optimization problem with the time-varying constraint over a multiagent system constructed by various agents. The sequences of cost functions and constraint functions, both of which have dynamic parameters following time-varying distributions, are unacquainted to the agent ahead of time. Agents in the network are able to interact with their neighbors through a sequence of strongly connected and time-varying graphs. We develop the adaptive distributed bandit primal-dual algorithm whose step size and regularization sequences are adaptive and have no prior knowledge about the total iteration span T . The adaptive distributed bandit primal-dual algorithm applies bandit feedback with a one-point or two-point gradient estimator to evaluate gradient values. It is illustrated in this article that if the drift of the benchmark sequence is sublinear, then the adaptive distributed bandit primal-dual algorithm exhibits sublinear expected dynamic regret and constraint violation using both two kinds of gradient estimator to compute gradient information. We present a numerical experiment to show the performance of the proposed method.

Multi-agent reinforcement learning based optimal energy sensing threshold control in distributed cognitive radio networks with directional antenna

In CRNs, it is crucial to develop an efficient and reliable spectrum detector that consistently provides accurate information about the channel state. In this work, we investigate a CSS in a fully-distributed environment where all secondary users (SUs) are equipped with directional antennas and make decisions based solely on their local knowledge without information sharing between SUs. First, we establish a stochastic sequential optimization problem, which is an NP-hard, that maximizes the SU’s detection accuracy by the dynamic and optimal control of the energy sensing/detection threshold. It can enable SUs to select an available channel and sector without causing interference to the primary network. To address it in a distributed environment, the problem is transformed into a decentralized partially observed Markov decision process (Dec-POMDP) problem. Second, in order to determine the best control for the Dec-POMDP in a practical environment without any prior knowledge of state–action transition probabilities, we develop a multi-agent deep deterministic policy gradient (MADDPG)-based algorithm, which is referred to as MA-DCSS. This algorithm adopts the centralized training and decentralized execution (CTDE) architecture. Third, we analyzed its computational complexity and showed the proposed approach’s scalability by the polynomial computational complexity, in terms of the number of channels, sectors, and SUs. Lastly, the simulation confirms that the proposed scheme provides enhanced performance in terms of convergence speed, accurate detection, and false alarm probabilities when it is compared to baseline algorithms.

Lagrangian dual method for solving stochastic linear quadratic optimal control problems with terminal state constraints

A stochastic linear quadratic (LQ) optimal control problem with a pointwise linear equality constraint on the terminal state is considered. A strong Lagrangian duality theorem is proved under a uniform convexity condition on the cost functional and a surjectivity condition on the linear constraint mapping. Based on the Lagrangian duality, two approaches are proposed to solve the constrained stochastic LQ problem. First, a theoretical method is given to construct the closed-form solution by the strong duality. Second, an iterative algorithm, called augmented Lagrangian method (ALM), is proposed. The strong convergence of the iterative sequence generated by ALM is proved. In addition, some sufficient conditions for the surjectivity of the linear constraint mapping are obtained.

Optimal Control of Infinite-Dimensional Differential Systems with Randomness and Path-Dependence and Stochastic Path-Dependent Hamilton–Jacobi Equations

This paper is devoted to the stochastic optimal control problem of infinite-dimensional differential systems allowing for both path-dependence and measurable randomness. As opposed to the deterministic path-dependent cases studied by Bayraktar and Keller [J. Funct. Anal. 275 (2018) 2096-2161], the value function turns out to be a random field on the path space and it is characterized by a stochastic path-dependent Hamilton-Jacobi (SPHJ) equation. A notion of viscosity solution is proposed and the value function is proved to be the unique viscosity solution to the associated SPHJ equation.

Dynamic programming principle for one kind of stochastic recursive optimal control problem with Markovian switching

Dynamic programming principle for one kind of stochastic recursive optimal control problem with Markovian switching