Regularized Regression With Fuzzy Membership Embedding for Unsupervised Feature Selection

Abstract

Although fuzziness universally diffuses in the real-world data, the fuzzy information is tricky to harness for feature selection such that it is rarely utilized. Therefore, how to efficiently exploit fuzzy information has become the major focus for feature selection recently. In this article, a novel unsupervised feature selection method is proposed via exploiting the sparse fuzzy membership efficiently. In general, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\ell _{2,1}$</tex-math></inline-formula> norm is utilized to induce sparsity, while Frobenius norm is used to prevent overfitting. To obtain sparsity and avoid overfitting simultaneously, adaptive loss regularization is introduced to the least-squares regression, such that a sparse and nontrivial projection matrix can be achieved via continuous interpolation between <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\ell _{2,1}$</tex-math></inline-formula> and Frobenius regularization. Additionally, the fuzzy <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -means problem is further embedded with the adaptive loss regression model to avoid the trivial solution caused by the linearity of fuzzy membership. Therefore, the fuzzy cluster structure of fuzzy <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -means is exploited for the efficient feature selection. By performing fuzzy clustering and subspace regression simultaneously, the embedded problem is then reformulated into a general quadratic problem with <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\ell _1$</tex-math></inline-formula> ball constraint. Equipped with an auxiliary variable and standard augmented Lagrangian method, the quadratic problem, i.e., the corresponding dual problem can be solved with the closed form solutions regarding the fuzzy membership and the projection matrix. Consequently, empirical results are provided to demonstrate the effectiveness of the proposed feature selection approach.