Finite convergence of value iteration algorithm for discounted infinite horizon optimal control of stochastic logical systems

Abstract

This paper investigates the discounted infinite horizon optimal control problem for the stochastic multi-valued logical dynamical systems with finite states. After giving the equivalent descriptions of the stochastic logical dynamical system in terms of Markov decision process, the infinite horizon optimization problem is presented in an algebraic form. Based on the semi-tensor product of matrices and the increasing-dimension technique, it is proved that the optimal stationary policy is obtained by a finite horizon value iteration process, and an exact horizon length estimation for the finite horizon approach is derived. As an application, the optimization problem of Human-machine game is investigated.