Minimal representations of some classes of dynamic programming

Abstract

It is known that various discrete optimization problems can be represented by finite state models called sequential decision processes (sdp's). A subclass of sdp's, the class of monotone sdp's (msdp's), is particularly important since the method of dynamic programming is applicable to obtain optimal policies. Several subclasses of msdp's have also been introduced from the viewpoint of computational complexity for obtaining optimal policies. For each of these classes of sdp's, optimal policies are usually obtained (if possible at all) in fewer steps if a given optimization problem is represented by a model with fewer states. Thus we are naturally led to the problem of finding a minimal (with the fewest states) representation of a given optimization problem by an sdp of a specified class. This paper investigates the existence or nonexistence of such minimization algorithms (in the sense of the theory of computation) for various classes of sdp's. It is shown that there exist minimization algorithms for some classes of sdp's, but there exist no algorithms for others. The nonuniqueness of a minimal representation is also proved for each class of sdp's.