Spectral variable selection based on least absolute shrinkage and selection operator with ridge-adding homotopy

Abstract

The least absolute shrinkage and selection operator (LASSO) is an established sparse representation approach for variable selection, and its performance relies on finding a good value for the regularization parameter, typically through cross-validation. However, cross-validation is a computationally intensive step and requires a properly determined search range and step size. In the present study, the ridge-adding homotopy (RAH) algorithm is applied with LASSO to overcome the aforementioned shortcomings. The homotopy algorithm can fit the entire solution of the LASSO problem by tracking the Karush-Kuhn Tucker (KKT) conditions and yields a finite number of potential regularization parameters. Considering the singularities, a M×1 random ridge vector will be added to the KKT conditions, which ensures that only one element is added to or removed from the active set. Finally, we can select the optimal regularization parameter by traversing the potential parameters with modelling and evaluation metrics. The selected variables are the nonzero elements in the sparse regression coefficient vector derived by the optimal regularization parameter. The proposed method has been demonstrated on three near-infrared (NIR) datasets with regard to wavelength selection and calibration. The results suggested that the “RAH-LASSO ​+ ​PLS” outperforms “LASSO ​+ ​PLS” and “full-wavelength PLS” in most cases. Importantly, the RAH method provides a systematic, as opposed to trial-and-error, procedure to determine the regularization parameter in LASSO.