Clustering Based Approximation Procedure for Semi-Markov Decision Processes with Incomplete State Information

Abstract

It is well-known that in practice it is rather difficult to calculate the minimal value function and optimal policies for decision processes with countable state space. Thus in general, approximation procedures are investigated. In this paper we show how the concepts and use of clustering techniques for large data sets, by adequate grouping of objects, can be useful in decision making processes. Using a linear-time clustering algorithm for the state space decomposition, we introduce an approximation procedure for a numerical solution of a finite horizon incomplete state-information semi-Markov decision processes with countable state space, finite action and signal spaces. We introduce the concept of admissibility for the decomposition of the state space, and for the new semi-Markov decision processes obtained, we calculate the minimal value functions and the optimal policies by dynamic programming. Finally, the convergence theorems are also proved.