Nonparametric Additive Models for Billion Observations

Abstract

The nonparametric additive model (NAM) is a widely used nonparametric regression method. Nevertheless, due to the high computational burden, classic statistical techniques for fitting NAMs are not well-equipped to handle massive data with billions of observations. To address this challenge, we develop a scalable element-wise subset selection method, referred to as Core-NAM, for fitting penalized regression spline based NAMs. Specifically, we first propose an approximation of the penalized least squares estimation, based on which we develop an efficient variant of generalized cross-validation (GCV) to select the smoothing parameter and approximate the Bayesian confidence intervals for statistical inference. Theoretically, we show that the proposed estimator approximately minimizes an upper bound of the estimation mean squared error. Moreover, we provide a non-asymptotic approximation guarantee for the proposed estimator and establish the asymptotic optimality of the proposed variant of GCV. Extensive simulations demonstrate the superior accuracy and efficiency of the Core-NAM method. We also apply the proposed method to a total column ozone dataset containing nearly one billion observations, and the results indicate a speed-up by almost a thousand times with comparable performance compared to the full data approach. Supplementary materials for this article are available online.