Stochastic Optimal Control Using Gaussian Process Regression over Probability Distributions

Abstract

In this paper, we address optimal control of nonlinear stochastic systems under motion and measurement uncertainty with finite control input and measurement spaces. Such problems can be formalized as partially-observable Markov decision processes where the goal is to find policies via dynamic programming that map the information available to the controller to control inputs while optimizing a performance criterion. However, they suffer from intractability in scenarios with continuous state spaces and partial observability which makes approximations necessary. Point-based value iteration methods are a class of global approximate methods that regress the value function given the values at a set of reference points. In this paper, we present a novel point-based value iteration approach for continuous state spaces that uses Gaussian processes defined over probability distribution for the regression. The main advantages of the proposed approach is that it is nonparametric and therefore approximation quality can be adjusted by choosing the number and the position of reference points in the space of probability distributions. In addition, it provides a notion of approximation quality in terms of variance.