A New Pruning Method for Incremental Pruning Algorithm Using a Sweeping Scan-Line through the Belief Space

Abstract

AbstractThis paper introduces a new filtering technique to speed up computation for finding exact policies for Partially Observable Markov Decision Problems (POMDP). We consider a new technique, called Scan Line Filter (SCF) for the Incremental Pruning (IP) POMDP exact solver to introduce an alternative method to Linear Programming (LP) filter. This technique takes its origin from the scan line method in computer graphics. By using a vertical scan line or plane, we show that a high-quality exact POMDP policy can be found easily and quickly. In this paper, we tested this new technique against the popular Incremental Pruning (IP) exact solution method in order to measure the relative speed and quality of our new method. We show that a high-quality POMDP policy can be found in lesser time in some cases. Furthermore, SCF has solutions for several POMDP problems that LP could not converge to in 12 hours.KeywordsScan LineInitial BeliefDiscount RewardBelief SpaceObservable Markov Decision ProcessThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.