Self-tuning density estimation based on Bayesian averaging of adaptive kernel density estimations yields state-of-the-art performance

Abstract

Non-parametric probability density function (pdf) estimation is a general problem encountered in many fields. A promising alternative to the dominating solutions, kernel density estimation (KDE) and Gaussian mixture modeling, is adaptive KDE where kernels are given individual bandwidths adjusted to the local data density. Traditionally the bandwidths are selected by a non-linear transformation of a pilot pdf estimate, containing parameters controlling the scaling, but identifying parameters values yielding competitive performance has turned out to be non-trivial.We present a new self-tuning (parameter free) pdf estimation method called adaptive density estimation by Bayesian averaging (ADEBA) that approximates pdf estimates in the form of weighted model averages across all possible parameter values, weighted by their Bayesian posterior calculated from the data.ADEBA is shown to be simple, robust, competitive in comparison to the current practice, and easily generalize to multivariate distributions. An implementation of the method for R is publicly available.