Bayesian asymptotics with misspecified models

Abstract

In this paper, we study the asymptotic properties of a sequence of poste- rior distributions based on an independent and identically distributed sample and when the Bayesian model is misspecified. We find a sufficient condition on the prior for the posterior to accumulate around the densities in the model closest in the Kullback-Leibler sense to the true density function. Examples are presented. This paper is concerned with asymptotics for Bayesian nonparametric mod- els. In particular, we consider generalizations of the recent literature on con- sistency; see for example, Barron, Schervish, and Wasserman (1999), Ghosal, Ghosh, and Ramamoorthi (1999), and Walker (2004). The standard assumption for consistency is that the true density function, which we denote by f0, is in the Kullback-Leibler support of the prior, denoted by Π. Further sufficient condi- tions on the prior are then established in order to ensure that the sequence of posterior distributions accumulate in suitable neighborhoods of f0. The three papers just cited deviate in the precise form of the further sufficient conditions. We make the support of the prior assumption more general now by assum- ing that the closest density in the support of the prior is a possibly non-zero Kullback-Leibler divergence away from f0; specifically, if f1 is the closest den- sity, in the Kullback-Leibler sense, in the support F of the prior (to be made more precise later), then δ1 is defined to be the Kullback-Leibler divergence between f0 and f1. We then look for further sufficient conditions under which the poste- rior distributions accumulate in suitable neighborhoods of f1. In particular, we work around the ideas presented in Walker (2004) and the sufficient conditions for accumulation at f1 can be seen as a generalization of the condition appearing in Walker (2004). The convenience of working in this setting is quite evident. When considering asymptotics, there are two possible scenarios: δ1 = 0 or δ1 > 0. The former involves a well specified model and the latter a misspecified model. Typically, the latter is more likely, though in reality it will be unknown. However, one can