Robust Quadratic Programming for MDPs with uncertain observation noise

Abstract

The problem of Markov decision processes (MDPs) with uncertain observation noise has rarely been studied. This paper proposes a Robust Quadratic Programming (RQP) approach to approximate Bellman equation solution. Besides efficiency, the proposed algorithm exhibits great robustness against uncertain observation noise, which is essential in real world applications. We further represent the solution into kernel forms, which implicitly expands the state-encoded feature space to higher or even infinite dimensions. Experimental results well justify its efficiency and robustness. The comparison with different kernels demonstrates its flexibility of kernel selection for different application scenarios.