Calibrating Prediction Regions

Abstract

Abstract Suppose that the variable X to be predicted and the learning sample Yn that was observed have a joint distribution, which depends on an unknown parameter θ. The parameter θ can be finite- or infinite-dimensional. A prediction region Dn for X is a random set, depending on Yn , that contains X with prescribed probability α. This article studies methods for controlling simultaneously the conditional coverage proability of Dn , given Yn , and the overall (unconditional) coverage probability of Dn . The basic construction yields a prediction region Dn , which has the following properties in regular models: Both the conditional and overall coverage probabilities of Dn converge to α as the size n of the learning sample increases. The convergence of the former is in probability. Moreover, the asymptotic distribution of the conditional coverage probability about α is typically normal; and the overall coverage probability tends to α at rate n −1. Can one reduce the dispersion of the conditional coverage pr...