The general maximum principle for discrete-time stochastic control problems

Abstract

In this paper, we discuss the problem of discrete-time stochastic control with multiplicative noise, where the control domains may not be convex. We propose a novel approach inspired by the classical spike variation in continuous-time cases, but adapted to the discrete-time setting by perturbing in a random scale instead of time scale. Our approach allows us to obtain the maximum principle recursively without the need for variation equations or adjoint equations, which are typically required in previous methods. Moreover, we provide two maximum principles, one for the general case and the other when feedback forms exist. Our approach covers the previous results when the control domains are convex. Finally, we demonstrate the effectiveness of our new results through an example.