A Sampling-Based Approach to Reducing the Complexity of Continuous State Space POMDPs by Decomposition Into Coupled Perceptual and Decision Processes

Abstract

In this paper, we propose a method to reduce the complexity of solving POMDPs in continuous state spaces by decomposing them into separate, coupled perceptual and decision processes which leads to a reduction of the state space size of the decision learning problem. In our method, we reduce the state space of the POMDP by handling some aspects of the state space outside of the decision POMDP. To achieve this, the whole problem state space is decomposed into separate state spaces for the decision and perceptual process. The Perceptual process just serves to estimate aspects of the belief state while the decision process estimates the remainder and determines a policy. As a result, the decision process is modeled as a reduced state space POMDP. To allow the application of this method to continuous state spaces, the decision and the perceptual processes are here both handled by a sampling method within which this separation makes it possible to represent the POMDP with a smaller state space which leads to smaller sample sets for the decision POMDP and as a result to reduced representational and decision learning complexity. The goal here is to focus decision learning on the aspects of the space that are important for decision making while the observations and attributes that are important for estimating the state of the decision process are handled separately by the perceptual process. In this way, the separation into different processes can significantly reduce the complexity of decision learning. In the proposed framework and algorithm, Monte Carlo based sampling methods and corresponding sample set representations are used for both the perceptual and decision processes to be able to deal efficiently with continuous domains. We show analytically and experimentally how much the complexity of solving a POMDP can be reduced to increase the range of decision learning tasks that can be addressed.