Robust Bayes-like estimation: Rho-Bayes estimation

Abstract

We observe $n$ independent random variables with joint distribution ${\mathbf{P}}$ and pretend that they are i.i.d. with some common density $s$ (with respect to a known measure $\mu $) that we wish to estimate. We consider a density model $\overline{S}$ for $s$ that we endow with a prior distribution $\pi $ (with support in $\overline{S}$) and build a robust alternative to the classical Bayes posterior distribution which possesses similar concentration properties around $s$ whenever the data are truly i.i.d. and their density $s$ belongs to the model $\overline{S}$. Furthermore, in this case, the Hellinger distance between the classical and the robust posterior distributions tends to 0, as the number of observations tends to infinity, under suitable assumptions on the model and the prior. However, unlike what happens with the classical Bayes posterior distribution, we show that the concentration properties of this new posterior distribution are still preserved when the model is misspecified or when the data are not i.i.d. but the marginal densities of their joint distribution are close enough in Hellinger distance to the model $\overline{S}$.