A Generalized Moreau Enhancement of $\ell_{2,1}$ -norm and Its Application to Group Sparse Classification

Abstract

In group sparse regularized least squares problem, the <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\ell_{2,1}$</tex> -norm is widely used to approximate convexly the <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\ell_{2,0}$</tex> pseudo-norm, but it causes largely biased estimates which are not desired for many applications. In this paper, we propose a nonconvex group sparse regularizer which can be seen as a generalized Moreau enhancement of the <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\ell_{2,1}$</tex> -norm. The proposed nonconvex regularizer promotes group sparsity more effectively than the <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\ell_{2,1}$</tex> -norm and can achieve the overall convexity of the regularized least squares model. We also propose to apply this model to the group sparse classification problem. The proposed classifier can utilize the label information of training samples in terms of the grouping information with smaller bias than the <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\ell_{2,1}$</tex> -norm, and thus is expected to improve the group sparse classification performance. Experimental results demonstrate that the proposed classifier certainly improves the performance of group sparse classification with <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\ell_{2,1}$</tex> regularizer, especially for unbalanced training data set.