Concepts and Methods in Stochastic Control

Abstract

This chapter discusses the concepts and methods in stochastic control. In most control or multistage decision processes arising in engineering, economic, or biological systems, there are inherent uncertainties. These uncertainties prevent the exact determination of the effect of the controller's actions and deterministic control theory cannot be used. In many problems, the uncertainties that could arise in the system, as well as in the observations made on the system could be modeled as stochastic processes. These problems could be treated using stochastic control theory. The chapter discusses the problem of control of dynamic stochastic systems. The state of the type of system that is considered evolves according to the non-linear stochastic difference equations. The performance index is taken as the expected value of a cost function and is to be minimized by a sequence of controls. The choice of these controls, which is a multistage decision process, constitutes the stochastic control problem.