Parallel distributed kernel estimation

Abstract

Non-parametric kernel methods are becoming more commonplace for data analysis, modeling, and inference. Unfortunately, these methods are known to be computationally burdensome. The burden increases as the amount of available data rises and can quickly overwhelm the computational resources present in modern desktop workstations. Approximation-based approaches exist which can dramatically reduce execution time, however, these approaches remain just that—approximations, however good they may be. Along with the approximate nature of such approaches, they do not admit multivariate kernel estimation with general bandwidths (fixed, variable, and adaptive). In this paper, I consider a parallel implementation of a number of popular kernel methods based on the MPI standard. MPI is a freely available parallel distributed library that runs on ‘commodity hardware’ such as a network of workstations typically found in many office environments. A simple demonstration indicates how one can dramatically reduce the computational burden often associated with kernel methods thereby achieving an almost ‘ideal’ parallel speed-up, while the approach is valid for multivariate kernel estimation with general bandwidths and does not rely on approximations. Some straightforward applications illustrate just how disarmingly simple the MPI library can be to use.