Optimal stopping problems for multiarmed bandit processes with arms' independence

Abstract

This paper deals with the optimal stopping problem for multiarmed bandit processes. Under the assumption of independence of arms we show that optimal strategies and stopping times are expressed by the dynamic allocation indices for each arm. This paper reduces this problem to several independent one-parameter optimal stopping problems. On the basis of these results, we characterize optimal strategies and stopping times. Moreover, this paper also extends those to the case allowing time constraints. In the case where arm's state evolve according to Markov chains with finite state, linear programming calculation of optimal strategies and stopping times is discussed.