Abstract
SummaryFor high-dimensional supervised learning, it is often beneficial to use domain-specific knowledge to improve the performance of statistical learning models. When the problem contains covariates which form groups, researchers can include this grouping information to find parsimonious representations of the relationship between covariates and targets. These groups may arise artificially, as from the polynomial expansion of a smaller feature space, or naturally, as from the anatomical grouping of different brain regions or the geographical grouping of different cities. When the number of features is large compared to the number of observations, one seeks a subset of the features which is sparse at both the group and global level.
Highlights
The sparse group lasso (Simon et al, 2013) is a penalized regression technique designed for exactly these situations
For the grid search strategy, our implementation is more efficient than using the base estimator with scikit-learn’s GridSearchCV because it makes use of warm-starting, where the model is fit along a pre-defined regularization path and the solution from the previous fit is used as the initial guess for the current hyperparameter value
Even without warm-starting, we find that the sequential model based optimization (SMBO) strategy usually outperforms grid search because far fewer evaluations are needed to arrive at the optimal hyperparameters
Summary
For high-dimensional supervised learning, it is often beneficial to use domain-specific knowledge to improve the performance of statistical learning models. When the problem contains covariates which form groups, researchers can include this grouping information to find parsimonious representations of the relationship between covariates and targets. The sparse group lasso (Simon et al, 2013) is a penalized regression technique designed for exactly these situations. It combines the original lasso (Tibshirani, 1996), which induces global sparsity, with the group lasso (Yuan & Lin, 2006), which induces group-level sparsity. It estimates a target variable yfrom a feature matrix X, using y = Xβ, as depicted, with color encoding the group structure of the covariates in X. Journal of Open Source Software, 6(58), 3024. https://doi.org/10. 1
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
