The K-armed bandit problem with multiple priors

Abstract

This paper studies the impact of ambiguity in the classic K-armed bandit problem. I show that two classic results are robust to ambiguity in the multiple-priors model: (i) In the one-armed bandit, the optimal plan is a switching strategy characterized by a multiple-priors Gittins–Jones index. (ii) The seminal Gittins–Jones theorem is generalized to the multiple-priors K-armed bandit case. Introducing ambiguity has two implications. First, in the K-armed bandit case, the incentive to experiment with an arm decreases in its own perceived ambiguity and increases in other arms’ ambiguity, differing from the comparative statics on risk. This suggests that ambiguity might explain the widely observed underexperimentation in new technologies and consumer products. Second, I characterize an upper bound for the multiple-priors Gittins–Jones index, as the lower envelope of the classic single-prior Gittins–Jones index for every prior lying in the multiple-priors set. I show with a counterexample that this upper bound can be strict, and I identify sufficient conditions under which this upper bound is exact.