Abstract
Wood et al. (J Am Stat Assoc 112(519):1199–1210, 2017) developed methods for fitting penalized regression spline based generalized additive models, with of the order of 10^4 coefficients, to up to 10^8 data. The methods offered two to three orders of magnitude reduction in computational cost relative to the most efficient previous methods. Part of the gain resulted from the development of a set of methods for efficiently computing model matrix products when model covariates each take only a discrete set of values substantially smaller than the sample size [generalizing an idea first appearing in Lang et al. (Stat Comput 24(2):223–238, 2014)]. Covariates can always be rounded to achieve such discretization, and it should be noted that the covariate discretization is marginal. That is we do not rely on discretizing covariates jointly, which would typically require the use of very coarse discretization. The most expensive computation in model estimation is the formation of the matrix cross product mathbf{X}^{mathsf{T}}{mathbf{WX}} where mathbf{X} is a model matrix and {mathbf{W}} a diagonal or tri-diagonal matrix. The purpose of this paper is to present a simple, novel and substantially more efficient approach to the computation of this cross product. The new method offers, for example, a 30 fold reduction in cross product computation time for the Black Smoke model dataset motivating Wood et al. (2017). Given this reduction in computational cost, the subsequent Cholesky decomposition of mathbf{X}^{mathsf{T}}{mathbf{WX}} and follow on computation of (mathbf{X}^{mathsf{T}}{mathbf{WX}})^{-1} become a more significant part of the computational burden, and we also discuss the choice of methods for improving their speed.
Highlights
A rate limiting step in computations involving large scale regression models is often the computation of weighted crossproducts, XTWX, of the model matrix, X, where W is diagonal
This paper provides new algorithms for computing XTWX from discretized covariates, that are more efficient than previous algorithms, thereby substantially reducing the computational burden of estimating large generalized additive models (GAM) of large data sets
The original Wood et al (2017) method implemented a parallel version of the block Cholesky method of Lucas (2004) followed by a parallel formation of (XTWX + Sλ)−1: the implementations scaled well and had good performance relative to LAPACK’s Cholesky routines based on the reference BLAS, but were poor compared to LAPACK using a tuned BLAS, such as OpenBLAS (Xianyi et al 2014)
Summary
A rate limiting step in computations involving large scale regression models is often the computation of weighted crossproducts, XTWX, of the model matrix, X, where W is diagonal (or in this paper sometimes tri-diagonal). In its most basic form a GAM is a generalized linear model in which the linear predictor depends on unknown smooth functions, f j , of covariates x j (possibly vector valued). The alternative is to find ways to exploit discretization when covariates are discretized individually (marginally), and Wood et al (2017) provide a set of algorithms to do this. These latter methods include the important case of model interaction terms. The columns of X relating to an interaction are given by a row-Kronecker product of a set of marginal model matrices for each marginal covariate of the interaction These marginal covariates and their marginal model matrices are discretized separately
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
