Bayes Factors for Outlier Models Using the Device of Imaginary Observations

Abstract

Abstract Suppose we think that most observations in a sample have been generated from a distribution with density f(x) but we fear that a few outliers from a distribution with density g(x) may have contaminated our sample. In many situations, we might assume that f(x) is a density depending on a parameter θ and that g(x) is of the same form as f but with parameter θ + δ or θδ. A number of Bayesian models for this problem when f is normal have been discussed by Freeman. He points out that with a vague improper prior for contaminating parameters, most posterior weight is put on the model allowing for the largest number of outliers. He therefore confines attention to proper priors when trying to answer the question of “how many outliers?” However, in many situations we do not have very certain information on the contaminating parameters and would like to make inferences about outliers when using improper priors for the parameters of the model. In this article, we apply the ideas of Spiegelhalter and Smith to this problem. In particular, we use their idea of assigning the value of the constant in the improper prior for the parameter of the contaminating distribution by the device of an imaginary training sample. This enables us to calculate the Bayes factor comparing a model with no outliers to a model with one outlier. We also can extend the ideas to more than one outlier. We illustrate the method in the case of univariate normal distributions, simple linear regression, and exponential samples.