A New Approach to Solving Stochastic Optimal Control Problems

Pablo T Rodriguez-Gonzalez,Vicente Rico-Ramirez,Urmila M Diwekar,Ramiro Rico-Martinez

doi:10.3390/math7121207

Abstract

A conventional approach to solving stochastic optimal control problems with time-dependent uncertainties involves the use of the stochastic maximum principle (SMP) technique. For large-scale problems, however, such an algorithm frequently leads to convergence complexities when solving the two-point boundary value problem resulting from the optimality conditions. An alternative approach consists of using continuous random variables to capture uncertainty through sampling-based methods embedded within an optimization strategy for the decision variables; such a technique may also fail due to the computational intensity involved in excessive model calculations for evaluating the objective function and its derivatives for each sample. This paper presents a new approach to solving stochastic optimal control problems with time-dependent uncertainties based on BONUS (Better Optimization algorithm for Nonlinear Uncertain Systems). The BONUS has been used successfully for non-linear programming problems with static uncertainties, but we show here that its scope can be extended to the case of optimal control problems with time-dependent uncertainties. A batch reactor for biodiesel production was used as a case study to illustrate the proposed approach. Results for a maximum profit problem indicate that the optimal objective function and the optimal profiles were better than those obtained by the maximum principle.

Highlights

Several phenomena from various fields are described through dynamic models
The reactive system consists of the production of three molecules of methyl ester (ME) from one molecule of triglycerides in a series of three reversive reactions: in the first, a molecule of triglyceride (TG) is converted to one diglyceride and one ME; in the second, the DG is converted to one monoglyceride (MG) and one ME; the MG is converted to one glycerol (GL) and one ME [26]
The sampling is done through an efficient sampling technique called Hammersley sequence sampling (HSS) [48]; the number of samples, N, must be large enough to capture the effect that time-dependent uncertainties have on the objective function

Summary

Introduction

Due to the unsteady nature of these systems and the complexity involved in describing the parameters or variables included, the incorporation of uncertainties is usually needed to represent the inherent unsteady behavior. Stochastic processes result from the uncertain behavior of dynamic variables and are represented by using any probability law for the evolution of the variables x over time t, xt. < tn , one can calculate the probability that the corresponding variable values, x1 , x2 , x3 . When time t1 arrives and we observe the actual value of x1 , we can condition the probability of future events on this information [1]. One of the commonly used stochastic processes is the Brownian motions such as the one-dimensional random walk process, which is a random process where the value of the variable x at time t depends only and exclusively on the value in the previous time t − 1, plus an uncertain term with zero mean and Mathematics 2019, 7, 1207; doi:10.3390/math7121207 www.mdpi.com/journal/mathematics

Methods

Results

Conclusion