Paradoxes in Learning and the Marginal Value of Information

Abstract

We consider the Bayesian ranking and selection problem, in which one wishes to allocate an information collection budget as efficiently as possible to choose the best among several alternatives. In this problem, the marginal value of information is not concave, leading to algorithmic difficulties and apparent paradoxes. Among these paradoxes is that when there are many identical alternatives, it is often better to ignore some completely and focus on a smaller number than it is to spread the measurement budget equally across all the alternatives. We analyze the consequences of this nonconcavity in several classes of ranking and selection problems, showing that the value of information is “eventually concave,” i.e., concave when the number of measurements of each alternative is large enough. We also present a new fully sequential measurement strategy that addresses the challenge that nonconcavity it presents.