What Do Kernel Density Estimators Optimize?

Abstract

Some linkages between kernel and penalty methods of density estimation are explored. It is recalled that classical Gaussian kernel density estimation can be viewed as the solution of the heat equation with initial condition given by data. We then observe that there is a direct relationship between the kernel method and a particular penalty method of density estimation. For this penalty method, solutions can be characterized as a weighted average of Gaussian kernel density estimates, the average taken with respect to the bandwidth parameter. A Laplace transform argument shows that this weighted average of Gaussian kernel estimates is equivalent to a fixed bandwidth kernel estimate using a Laplace kernel. Extensions to higher order kernels are considered and some connections to penalized likelihood density estimators are made in the concluding sections.