Improper Priors Are Not Improper

Abstract

It is well known that improper priors in Bayesian statistics may lead to proper posterior distributions and useful inference procedures. This motivates us to give an elementary introduction to a theoretical frame for statistics that includes improper priors. Axioms that allow improper priors are given by a relaxed version of Kolmogorov’s formulation of probability theory. The theory of conditional probability spaces formulated by Renyi is closely related, but the initial axioms and the motivation differ. One consequence of the axioms is a general Bayes theorem which gives proper posterior distributions, and furthermore, the theory also gives a convenient frame for formulation of non-Bayesian statistical models. The results are in particular relevant for the current usage of improper priors in Markov chain Monte Carlo methods, and for methods for simulation from conditional distributions given sufficient statistics. This theory gives an alternative to ad hoc arguments without an underlying theory, and removes apparent paradoxes. Readers who acknowledge the need for a theoretical basis for statistical inference including improper priors are urged to consider the theory of conditional probability spaces as presented here.