Nonparametric predictive distributions based on conformal prediction

Vladimir Vovk,Valery Manokhin,Min-Ge Xie,Jieli Shen

doi:10.1007/s10994-018-5755-8

Vladimir Vovk, Valery Manokhin + Show 2 more

Open Access

https://doi.org/10.1007/s10994-018-5755-8

Copy DOI

Abstract

This paper applies conformal prediction to derive predictive distributions that are valid under a nonparametric assumption. Namely, we introduce and explore predictive distribution functions that always satisfy a natural property of validity in terms of guaranteed coverage for IID observations. The focus is on a prediction algorithm that we call the Least Squares Prediction Machine (LSPM). The LSPM generalizes the classical Dempster–Hill predictive distributions to nonparametric regression problems. If the standard parametric assumptions for Least Squares linear regression hold, the LSPM is as efficient as the Dempster–Hill procedure, in a natural sense. And if those parametric assumptions fail, the LSPM is still valid, provided the observations are IID.

Highlights

This paper applies conformal prediction to derive predictive distribution functions that are valid under a nonparametric assumption
After a brief discussion of Ridge Regression Prediction Machines, we prove Propositions 1–5 and find the explicit forms for the studentized, ordinary, and deleted Least Squares Prediction Machine (LSPM)
In the left-hand plot of Fig. 6 we show the first plot of Fig. 5 that is normalized by subtracting the true distribution function; this time, we show the output of both simplified and proper Oracles I and II; the difference is not large but noticeable

Summary

Introduction

This paper applies conformal prediction to derive predictive distribution functions that are valid under a nonparametric assumption. In our definition of predictive distribution functions and their property of validity we follow Shen et al (2018, Section 1), whose terminology we adopt, and Schweder and Hjort (2016, Chapter 12), who use the term “prediction confidence. The theory of predictive distributions as developed by Schweder and Hjort (2016) and Shen et al (2018) assumes that the observations are generated from a parametric statistical model. The more recent review by Gneiting and Katzfuss (2014) refers to the notion of validity used in this paper as probabilistic calibration and describes it as critical in forecasting; Gneiting and Katzfuss (2014, Section 2.2.3) give further references

Objectives

Results

Conclusion