Optimal Stopping in a Partially Observable Markov Process with Costly Information

Abstract

A problem of optimal stopping in a Markov chain whose states are not directly observable is presented. Using the theory of partially observable Markov decision processes, a model which combines the classical stopping problem with sequential sampling at each stage of the decision process is developed. Several results which characterize the optimal expected value function in terms of its parameters are given. An example is given which indicates that the best action to take as a function of the information currently available may not be of the intuitively appealing control limit type. The set of states at which it is optimal to purchase information need not be convex. The expected value of information as a function of the decision maker's knowledge is related to nonmonotone optimal policies.