Stochastic Optimization Problem Research Articles

Distributed optimal scheduling for virtual power plant with high penetration of renewable energy

The virtual power plants are aggregations of distributed generators, grid-connected devices in user side. The operation of virtual power plants affects the economic benefits and environmental benefits. However, due to the stochastic and fluctuating nature of power generation from renewable energy sources, the optimal scheduling problem for the virtual power plant containing high-dimensional random variables is difficult to solve. The conic power flow constraints are considered in the virtual power plant day-ahead optimal scheduling models to maximize the revenue by virtue of optimizing the flexible resources such as energy storage system and interruptible load. To quantify the uncertainties in the optimization problem, this paper proposes a network state-based power scenario reduction strategy for renewable energy generation, where typical scenarios are selected by the state of the grid voltage. The proposed day-ahead scheduling model is a mixed-integer, nonlinear, large-scale, stochastic optimization problem with high dimensional random variables, which is difficult to be solved directly by traditional centralized method. By temporally decoupling the charging and discharging model of energy storage systems, a distributed optimization algorithm is proposed based on multi-temporal power flow decoupling optimization model. The simulation results show that the proposed distributed optimization algorithm has high accuracy and good convergence, the virtual power plant can achieve day-ahead optimal scheduling and effectively promote renewable energy accommodation under the security and reliability of the grid.

Representation of stochastic optimal control problems with delay in the control variable

AbstractIn this manuscript we provide a representation in infinite dimension for stochastic optimal control problems with delay in the control variable. The main novelty consists in the fact that the representation can be applied also to dynamics where the delay in the control appears as a nonlinear term and in the diffusion coefficient. We then apply the representation to a LQ case where an explicit solution can be found.

Non-orthogonal multiple access-based MEC for energy-efficient task offloading in e-commerce systems

Mobile edge computing (MEC) reduces the latency for end users to access applications deployed at the edge by offloading tasks to the edge. With the popularity of e-commerce and the expansion of business scale, server load continues to increase, and energy efficiency issues gradually become more prominent. Computation offloading has received widespread attention as a technology that effectively reduces server load. However, how to improve energy efficiency while ensuring computing requirements is an important challenge facing computation offloading. To solve this problem, using non-orthogonal multiple access (NOMA) to increase the efficiency of multi-access wireless transmission, MEC supporting NOMA is investigated in the research. Computing resources will be divided into separate sub-computing that will be handled via e-commerce terminals or transferred to edge sides by reutilizing radio resources, we put forward a Group Switching Matching Algorithm Based on Resource Unit Allocation (GSM-RUA) algorithm that is multi-dimensional. To this end, we first formulate this task allocation problem as a long-term stochastic optimization problem, which we then convert to three short-term deterministic sub-programming problems using Lyapunov optimization, namely, radio resource allocation in a large timescale, computation resource allocating and splitting in a small-time frame. Of the 3 short-term deterministic sub-programming problems, the first sub-programming problem can be remodeled into a 1 to n matching problem, which can be solved using the block-shift-matching-based radio resource allocation method. The latter two sub-programming problems are then transformed into two continuous convex problems by relaxation and then solved easily. We then use simulations to prove that our GSM-RUA algorithm is superior to the state-of-the-art resource management algorithms in terms of energy consumption, efficiency and complexity for e-commerce scenarios.

Stochastic optimization problems with nonlinear dependence on a probability measure via the Wasserstein metric

Nonlinear dependence on a probability measure has recently been encountered with increasing intensity in stochastic optimization. This type of dependence corresponds to many situations in applications; it can appear in problems static (one-stage), dynamic with finite (multi-stage) or infinite horizon, and single- and multi-objective ones. Moreover, the nonlinear dependence can appear not only in the objective functions but also in the constraint sets. In this paper, we will consider static one-objective problems in which the nonlinear dependence appears in the objective function and may also appear in the constraint sets. In detail, we consider “deterministic” constraint sets, whose dependence on the probability measure is nonlinear, constraint sets determined by second-order stochastic dominance, and sets given by mean-risk problems. The last mentioned instance means that the constraint set corresponds to solutions which guarantee acceptable values of both criteria. To obtain relevant assertions, we employ the stability results given by the Wasserstein metric, based on the L1\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$ {{\\mathcal {L}}}_{1} $$\\end{document} norm. We mainly focus on the case in which a solution has to be obtained on the basis of the data and of investigating a relationship between the original problem and its empirical version.

Accelerated Driving-Training-Based Optimization for Solving Constrained Bi-Objective Stochastic Optimization Problems

The constrained bi-objective stochastic optimization problem (CBSOP) considers the optimization problem with stochastic bi-objective functions subject to deterministic constraints. The CBSOP is part of a set of hard combinatorial optimization problems regarding time complexity. Ordinal optimization (OO) theory provides a commonly recognized structure to handle hard combinatorial optimization problems. Although OO theory may solve hard combinatorial optimization problems quickly, the deterministic constraints will critically influence computing performance. This work presents a metaheuristic approach that combines driving-training-based optimization (DTBO) with ordinal optimization (OO), abbreviated as DTOO, to solve the CBSOP with a large design space. The DTOO approach comprises three major components: the surrogate model, diversification, and intensification. In the surrogate model, the regularized minimal-energy tensor product with cubic Hermite splines is utilized as a fitness estimation of design. In diversification, an accelerated driving-training-based optimization is presented to determine N remarkable designs from the design space. In intensification, a reinforced optimal computing budget allocation is used to find an extraordinary design from the N remarkable designs. The DTOO approach is applied to a medical resource allocation problem in the emergency department. Simulation results obtained by the DTOO approach are compared with three heuristic approaches to examine the performance of the DTOO approach. Test results show that the DTOO approach obtains an extraordinary design with higher solution quality and computational efficiency than the three heuristic approaches.

Hybrid-order distributed SGD: Balancing communication overhead, computational complexity, and convergence rate for distributed learning

Communication overhead, computation load, and convergence speed are three major challenges in the scalability of distributed stochastic optimization algorithms in training large neural networks. In this paper, we propose the approach of hybrid-order distributed stochastic gradient descent (HO-SGD) which strikes a better balance between these three than the previous methods, for a general class of non-convex stochastic optimization problems. In particular, we advocate that by properly interleaving zeroth-order and first-order gradient updates, it is possible to significantly reduce the communication and computation overheads while guaranteeing a fast convergence. The proposed method guarantees the same order of convergence rate as in the fastest distributed methods (i.e., fully synchronous SGD) while having significantly less computational complexity and communication overhead per iteration, and the same order of communication overhead as in the state-of-the-art communication-efficient methods, with order-wisely less computational complexity. Moreover, it order-wisely improves the convergence rate of zeroth-order SGD methods. Finally and remarkably, empirical studies demonstrate that the proposed hybrid-order approach provides significantly higher test accuracies and superior generalization than all the baselines, owing to its novel exploration mechanism.

Stochastic optimal control problems with delays in the state and in the control via viscosity solutions and applications to optimal advertising and optimal investment problems

AbstractIn this manuscript we consider optimal control problems of stochastic differential equations with delays in the state and in the control. First, we prove an equivalent Markovian reformulation on Hilbert spaces of the state equation. Then, using the dynamic programming approach for infinite-dimensional systems, we prove that the value function is the unique viscosity solution of the infinite-dimensional Hamilton-Jacobi-Bellman equation. We apply these results to problems coming from economics: stochastic optimal advertising problems and stochastic optimal investment problems with time-to-build.

A Bayesian approach to data-driven multi-stage stochastic optimization

Aimed at sufficiently utilizing available data and prior distribution information, we introduce a data-driven Bayesian-type approach to solve multi-stage convex stochastic optimization, which can easily cope with the uncertainty about data process’s distributions and their inter-stage dependence. To unravel the properties of the proposed multi-stage Bayesian expectation optimization (BEO) problem, we establish the consistency of optimal value functions and solutions. Two kinds of algorithms are designed for the numerical solution of single-stage and multi-stage BEO problems, respectively. A queuing system and a multi-stage inventory problem are adopted to numerically demonstrate the advantages and practicality of the new framework and corresponding solution methods, compared with the usual formulations and solution methods for stochastic optimization problems.

Optimal trading and competition with information in the price impact model

Information drives trading and hence affects price dynamics. We study how an informed trader optimally trades, how multiple traders compete with each other, and how their competition affects price dynamics as well as their inventories in the linear price impact model framework. We formulate the informed trading problem as a stochastic optimal control problem and a stochastic differential game in different cases. By virtue of different methods, problems are solved explicitly in all cases. In the case of a single trader, the risk-averse informed trader trades less in the beginning and then speeds up to reduce exposure to the volatility of dynamic information. In addition, the risk-averse trader reveals information more slowly and thus contributes less to price efficiency. When multiple risk-neutral traders compete with information, regardless of the competition structure, they tend to trade more rapidly in the early stage to fill orders at a good price. Competition and the presence of a leader help promote price efficiency, and competition also leads to the empirically observed mean-reverting behavior of price dynamics and order flows. Furthermore, among risk-averse informed traders, inventory management triggers predatory trading and active information leakage during informed trading, and the positions of traders with the same level of risk aversion tend to be identical and mean-reverting to zero under multiple rounds of competition.

Time Consistency for Multistage Stochastic Optimization Problems under Constraints in Expectation

Time Consistency for Multistage Stochastic Optimization Problems under Constraints in Expectation

Variance-Reduced Accelerated First-Order Methods: Central Limit Theorems and Confidence Statements

In this paper, we consider a strongly convex stochastic optimization problem and propose three classes of variable sample-size stochastic first-order methods: (i) the standard stochastic gradient descent method, (ii) its accelerated variant, and (iii) the stochastic heavy-ball method. In each scheme, the exact gradients are approximated by averaging across an increasing batch size of sampled gradients. We prove that when the sample size increases at a geometric rate, the generated estimates converge in mean to the optimal solution at an analogous geometric rate for schemes (i)–(iii). Based on this result, we provide central limit statements, whereby it is shown that the rescaled estimation errors converge in distribution to a normal distribution with the associated covariance matrix dependent on the Hessian matrix, the covariance of the gradient noise, and the step length. If the sample size increases at a polynomial rate, we show that the estimation errors decay at a corresponding polynomial rate and establish the associated central limit theorems (CLTs). Under certain conditions, we discuss how both the algorithms and the associated limit theorems may be extended to constrained and nonsmooth regimes. Finally, we provide an avenue to construct confidence regions for the optimal solution based on the established CLTs and test the theoretical findings on a stochastic parameter estimation problem. Funding: This work was supported by the Office of Naval Research [Grant N00014-22-1-2589] and Basic Energy Sciences [Grant DE-SC0023303]. The work of J. Lei was partially supported by the National Natural Science Foundation of China [Grants 72271187 and 62088101]. Supplemental Material: The online appendix is available at https://doi.org/10.1287/moor.2021.0068 .

Solving Contextual Stochastic Optimization Problems through Contextual Distribution Estimation

Stochastic optimization models always assume known probability distributions about uncertain parameters. However, it is unrealistic to know the true distributions. In the era of big data, with the knowledge of informative features related to uncertain parameters, this study aims to estimate the conditional distributions of uncertain parameters directly and solve the resulting contextual stochastic optimization problem by using a set of realizations drawn from estimated distributions, which is called the contextual distribution estimation method. We use an energy scheduling problem as the case study and conduct numerical experiments with real-world data. The results demonstrate that the proposed contextual distribution estimation method offers specific benefits in particular scenarios, resulting in improved decisions. This study contributes to the literature on contextual stochastic optimization problems by introducing the contextual distribution estimation method, which holds practical significance for addressing data-driven uncertain decision problems.

Life-cycle model with subsistence consumption constraint and state-dependent utilities

This paper studies optimal decision-making for investment, consumption and life insurance with state-dependent utilities in continuous time. It is supposed that an economic agent has a subsistence consumption constraint and an uncertain lifetime horizon. The martingale approach coupled with duality theory has been employed to study the stochastic optimization problem of the economic agent. Specifically, the original optimization problem is transformed into a dual maximization problem involving a system of linear equations. Optimal strategies for investment, consumption and life-insurance purchases are obtained by maximizing the expected discounted utilities from intertemporal consumption, legacy and terminal wealth. Economic insights that may be gained from the solution to the problem are illustrated through numerical examples and case studies based on real data. Specifically, for the real-data case studies, an attention is given to studying the impacts of Covid 19 and the recent inflation on optimal strategies for consumption, investment and life-insurance purchases implied by the solution of our proposed model.

Co-evolutionary traffic signal control using reinforcement learning for road networks under stochastic capacity

A co-evolutionary traffic signal control using reinforcement learning approach (CORLA) is proposed for time-varying road networks under stochastic capacity. Classic reinforcement learning based traffic signal control cannot effectively reduce traffic congestion for large-scale road networks while standard evolutionary metaheuristics often suffer from significantly high computational cost. A co-evolutionary decomposition algorithm (CODA) is proposed to improve traffic mobility for urban road networks with time-varying traffic flow. To capture time-varying spatial evolution of traffic flow inside road links, a stochastic traffic model is presented. To fully consider road users’ response, a stochastic bi-level optimization problem (SBOP) is given where road users’ route choice can be fully taken into account. To efficiently implement CORLA in a large-scale road network against high-consequence realization for stochastic capacity, a coordinated co-evolutionary two phase control (CCTPC) is proposed. Numerical experiments are performed at a real-data city road network and various sizable traffic grids. As compared to state-of-the-art traffic signal control for various traffic conditions, obtained results showed that CCTPC exhibits sufficient gain of achieving road network performance and suffers from the least computational cost in all cases.

Regression based stochastic energy management strategy for a grid connected microgrid using Artificial Electric Field Algorithm

Recent decade has witnessed steep technological advancement in the renewable energy sector due to growing concern of climate change and emission cut target. The lack of availability of fossil fuels and the high price of the same have made the use of Renewable Energy Sources (RES) as a promising approach for generating clean and sustainable energy at local level. This persuades the implementation of Microgrid (MG) supported with RES, to cater various types of load demand. However, planning and optimizing a MG operation have become challenging due to the intermittent nature of RES and uncertain loads. To address this issue, this paper introduces a Stochastic Energy Management Strategy (SEMS) for a grid connected MG comprising Photovoltaic, Wind Turbine, Fuel Cell, Micro Turbine, Battery Energy Storage and electrical as well as heat energy demand. The volatilities of RES power output and energy demand are modeled via Gaussian Process Regression (GPR) and Monte Carlo Simulation (MCS) respectively. The problem is framed as non-linear (NL) dynamic stochastic optimization problem and solved using ‘Artificial Electric Field Algorithm (AEFA)’. Here, minimization of expected value of operational and emission cost are taken as objective functions; while various practicality aspects are modeled as constraints in the problem. Further, the efficacy of the proposed approach is validated around the first central moment of the load. Various case studies are performed to estimate the day ahead optimum operating schedule of the Distributed Energy Resources (DER) and to assess the impact of load uncertainties on DER operation and total MG operation cost. Further, the efficacy of the GPR is verified through comparison with MCS based result.

Approximate propagation of normal distributions for stochastic optimal control of nonsmooth systems

We present a method for the approximate propagation of mean and covariance of a probability distribution through ordinary differential equations (ODE) with discontinuous right-hand side. For piecewise affine systems, a normalization of the propagated probability distribution at every time step allows us to analytically compute the expectation integrals of the mean and covariance dynamics while explicitly taking into account the discontinuity. This leads to a natural smoothing of the discontinuity such that for relevant levels of uncertainty the resulting ODE can be integrated directly with standard schemes and it is neither necessary to prespecify the switching sequence nor to use a switch detection method. We then show how this result can be employed in the more general case of piecewise smooth functions based on a structure preserving linearization scheme. The resulting dynamics can be straightforwardly used within standard formulations of stochastic optimal control problems with chance constraints.

Decomposition methods for monotone two-time-scale stochastic optimization problems

Decomposition methods for monotone two-time-scale stochastic optimization problems

Nonparametric statistical inference for stochastic optimal control problems and its applications for financial investment

Against a backdrop of financial decision-making, this study examines nonparametric statistical inference problems in stochastic optimal control problems. First, we construct a nonparametric estimation model using the wavelet estimation method for drift and diffusion coefficients. In this study, we focus on stochastic linear–quadratic (SLQ) problems, solved using Riccati equations, and apply this to a cash management problem within the linear–quadratic framework. The results show the effectiveness of the wavelet method using a numerical case.

On the adaptation of the Lagrange formalism to continuous time stochastic optimal control: A Lagrange-Chow redux

We show how the classical Lagrangian approach to solving constrained optimization problems from standard calculus can be extended to solve continuous time stochastic optimal control problems. Connections to mainstream approaches such as the Hamilton-Jacobi-Bellman equation and the stochastic maximum principle are drawn. Our approach is linked to the stochastic maximum principle, but more direct and tied to the classical Lagrangian principle, avoiding the use of backward stochastic differential equations in its formulation. Using infinite dimensional functional analysis, we formalize and extend the approach first outlined in Chow (1992) within a rigorous mathematical setting using infinite dimensional functional analysis. We provide examples that demonstrate the usefulness and effectiveness of our approach in practice. Further, we demonstrate the potential for numerical applications facilitating some of our key equations in combination with Monte Carlo backward simulation and linear regression, therefore illustrating a completely different and new avenue for the numerical application of Chow's methods.

Constrained minimum variance and covariance steering based on affine disturbance feedback control parameterization

AbstractThis paper deals with finite‐horizon minimum‐variance and covariance steering problems subject to constraints. The goal of the minimum variance problem is to steer the state mean of an uncertain system to a prescribed vector while minimizing the trace of its terminal state covariance whereas the goal in the covariance steering problem is to steer the covariance matrix of the terminal state to a prescribed positive definite matrix. The paper proposes a solution approach that relies on a stochastic version of the affine disturbance feedback control parametrization. In this control policy parametrization, the control input at each stage is expressed as an affine function of the history of disturbances that have acted upon the system. It is shown that this particular parametrization reduces the stochastic optimal control problems considered in this paper into tractable convex programs or difference of convex functions programs with essentially the same decision variables. In addition, the paper proposes a variation of this control parametrization that relies on truncated histories of past disturbances, which allows for sub‐optimal controllers to be designed that strike a balance between performance and computational cost. The suboptimality of the truncated policies is formally analyzed and closed form expressions are provided for the performance loss due to the use of the truncation scheme. Finally, the paper concludes with a comparative analysis of the truncated versions of the proposed policy parametrization and other standard policy parametrizations through numerical simulations.