Theoretical Properties of Bayesian Student-$T$ Linear Regression

Abstract

Student-$t$ linear regression is a commonly used alternative to the normal model in Bayesian analysis when one wants to gain robustness against outliers. The assumption of heavy-tailed error distribution makes the model more adapted to a potential presence of outliers by assigning higher probabilities to extreme values. Even though the Student-$t$ model is often used in practice, not a lot is known about its theoretical properties. In this paper, we aim to fill some gaps by providing analyses in two different asymptotic scenarios. In the first one, outliers are considered to be further and further away from the bulk of the data. The analysis allows to characterize the limiting posterior distribution, a distribution in which a trace of the outliers is present, making the approach partially robust. The impact of the trace is seen to increase with the degrees of freedom of the Student-$t$ distribution assumed. The second asymptotic scenario is one where the sample size increases and the normal model is the true generating process to be able to compare the efficiency of the robust estimator to the ordinary-least-squares one when the latter is the benchmark. The asymptotic efficiency is comparable, in the sense that the variance of the robust estimator is inflated but only by a factor, and this factor converges to 1 as the degrees of freedom increase. The trade-off between robustness and efficiency controlled through the degrees of freedom is thus precisely characterized (at least asymptotically).