When control and state variations increase uncertainty: Modeling and stochastic control in discrete time

Abstract

This paper deals with the paradigm of controlling uncertain stochastic systems for which control and state variations increase uncertainty (CSVIU). The discrete time CSVIU model is particularly useful when an accurate dynamic model is unattainable, and only a rough model is available. Interesting features arise from the optimal control solution for the model. In particular, the optimal control action is to remain idle within a certain region of the state space — the inaction region. This feature, peculiar to the CSVIU approach, has ties to cautionary control policies found in economics. The convexity of the cost function holds for the discounted control problem with quadratic costs when the noise is Gaussian, and the control policy admits a solution in closed form in some regions of the state space. A linearly perturbed Lyapunov equation inside the inaction region, and a rational Riccati equation, asymptotically in far-off regions, characterize the control. We show the stochastic stability of solutions and provide numerical experiments that underline CSVIU model control’s interesting peculiarities.