Stochastic Optimization Methods for the Stochastic Storage Process Control

Abstract

AbstractMany stochastic optimal control problems have analytical solutions up to unknown numerical parameters. We demonstrate this fact with several examples from inventory theory, queuing theory, and risk theory. The paper reviews sufficient conditions for the existence of parametric optimal solutions to such problems in the stationary and nonstationary cases, for minimizing the average and discounted costs. The found parametric strategies are then substituted into the dynamic equations and the target functional of the primary optimal control problem. Thus, the original problem reduces to the problem of finite-dimensional stochastic programming concerning the unknown parameters. Since the optimal strategies are nonconvex, nonsmooth, or discontinuous functions of state variables and parameters, the corresponding stochastic programming problem may also be nonconvex, nonsmooth, or discontinuous. The paper proposes methods for calculating stochastic (quasi-)gradients (or their finite-difference analogs) of the objective function of the obtained stochastic programming problem and substantiates stochastic quasigradient methods for finding optimal parameter values. We illustrate the proposed solution approach by optimal inventory control and optimization of an energy accumulation system.