Abstract
The goal of supervised feature selection is to find a subset of input features that are responsible for predicting output values. The least absolute shrinkage and selection operator (Lasso) allows computationally efficient feature selection based on linear dependency between input features and output values. In this letter, we consider a feature-wise kernelized Lasso for capturing nonlinear input-output dependency. We first show that with particular choices of kernel functions, nonredundant features with strong statistical dependence on output values can be found in terms of kernel-based independence measures such as the Hilbert-Schmidt independence criterion. We then show that the globally optimal solution can be efficiently computed; this makes the approach scalable to high-dimensional problems. The effectiveness of the proposed method is demonstrated through feature selection experiments for classification and regression with thousands of features.
Highlights
Finding a subset of features in high-dimensional supervised learning is an important problem with many realworld applications such as gene selection from microarray data (Xing et al, 2001; Ding and Peng, 2005; Suzuki et al, 2009; Huang et al, 2010), document categorization (Forman, 2008), and prosthesis control (Shenoy et al, 2008).1.1 Problem DescriptionLet X (⊂ Rd) be the domain of input vector x and Y(⊂ R) be the domain of output data1 y
The least absolute shrinkage and selection operator (Lasso) (Tibshirani, 1996) allows computationally efficient feature selection based on the assumption of linear dependency between input features and output values
We use kernel regression (KR) (Scholkopf and Smola, 2002) with the Gaussian kernel for evaluating the mean squared error and the mean correlation when the top m = 10, 20, . . . , 50 features selected by each method are used
Summary
Let X (⊂ Rd) be the domain of input vector x and Y(⊂ R) be the domain of output data y. Suppose we are given n independent and identically distributed (i.i.d.) paired samples,. We denote the original data by X = [x1, . The goal of supervised feature selection is to find m features (m < d) of input vector x that are responsible for predicting output y. The least absolute shrinkage and selection operator (Lasso) (Tibshirani, 1996) allows computationally efficient feature selection based on the assumption of linear dependency between input features and output values. The Lasso optimization problem is given as min α∈Rd
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
