Homeostasis phenomenon in conformal prediction and predictive distribution functions

Abstract

Conformal prediction is an attractive framework for prediction that is distribution free. In this article, we study in details its homeostasis property under a general regression setup and also introduce the concepts of upper and lower predictive distributions and predictive curve to establish connections to left-, right- and two-tailed hypothesis testing problems as well as the developments in confidence distributions. The homeostasis property is very attractive, since it states that under some conditions the prediction results remain valid even if the model used for learning is completely wrong. We show explicitly why the property holds in a model-based setup and also explore the boundary when the property breaks down. Beside the typical assumption used in conformal prediction that the response and covariate pairs (y,x) of all subjects are iid distributed, we also study the classical regression setting in which the design is fixed with given (non-random) covariates x. The trade-offs among learning model accuracy, prediction valid and prediction efficiency are discussed, leading to an emphasis of more efforts on developing better learning models.