Optimal stopping with a capacity constraint: Generalizing Shepp’s urn scheme

Abstract

We formulate an optimal stopping problem for a variant of Shepp’s urn model in which it is possible to sample more than one item at each stage. Using a Markov decision process model, we establish monotonicity of the optimal value function and show that the optimal policy is a monotone threshold policy that prescribes either not sampling, or sampling the maximum number of items permitted. A special case exhibits convexity and submodularity, but these properties do not hold in general.