Abstract
We develop fast algorithms for estimation of generalized linear models with convex penalties. The models include linear regression, two-class logistic regression, and multinomial regression problems while the penalties include ℓ(1) (the lasso), ℓ(2) (ridge regression) and mixtures of the two (the elastic net). The algorithms use cyclical coordinate descent, computed along a regularization path. The methods can handle large problems and can also deal efficiently with sparse features. In comparative timings we find that the new algorithms are considerably faster than competing methods.
Highlights
The lasso [Tibshirani, 1996] is a popular method for regression that uses an 1 penalty to achieve a sparse solution
In this paper we extend the work of Friedman et al [2007] and develop fast algorithms for fitting generalized linear models with elastic-net penalties
We compared the speed of glmnet to the interior point method l1lognet proposed by Koh et al [2007], Bayesian Logistic Regression (BBR) due to Genkin et al [2007] and the Lasso Penalized Logistic (LPL) program supplied by Ken Lange [Wu and Lange, 2008b]
Summary
The lasso [Tibshirani, 1996] is a popular method for regression that uses an 1 penalty to achieve a sparse solution. In this paper we extend the work of Friedman et al [2007] and develop fast algorithms for fitting generalized linear models with elastic-net penalties. We compute the simple least-squares coefficient on the partial residual, apply soft-thresholding to take care of the lasso contribution to the penalty, and apply a proportional shrinkage for the ridge penalty. This algorithm was suggested by Van der Kooij [2007]
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
