Learning and planning in partially observable environments without prior domain knowledge

Abstract

Planning in stochastic and partially observable environments is a central problem in artificial intelligence. To address this issue, an accurate model or a black-box simulator of the environment is usually needed in the literature. Although some recent approaches have been proposed for learning optimal behaviors under model uncertainty, prior knowledge about the environment is still required to guarantee the performance of the proposed algorithms. With the benefits of the Predictive State Representations (PSRs) approach for state representation and model prediction, in this paper, we introduce an approach for planning under partial observability with no prior domain knowledge, where an offline PSR model is firstly learned and then combined with online Monte-Carlo tree search for planning under model uncertainty. Furthermore, we also showed that with the proposed framework, the PSR models learned via other techniques, e.g., the online PSR model learning approach, can be integrated straightforwardly. By comparing with the state-of-the-art approach of planning under model uncertainty, we demonstrated the effectiveness of the proposed approaches along with the proof of their convergence.