A Continuous Exact $\ell_0$ Penalty (CEL0) for Least Squares Regularized Problem

Abstract

Within the framework of the $\ell_0$ regularized least squares problem, we focus, in this paper, on nonconvex continuous penalties approximating the $\ell_0$-norm. Such penalties are known to better promote sparsity than the $\ell_1$ convex relaxation. Based on some results in one dimension and in the case of orthogonal matrices, we propose the continuous exact $\ell_0$ penalty (CEL0) leading to a tight continuous relaxation of the $\ell_2-\ell_0$ problem. The global minimizers of the CEL0 functional contain the global minimizers of $\ell_2 - \ell_0$, and from each global minimizer of CEL0 one can easily identify a global minimizer of $\ell_2 - \ell_0$. We also demonstrate that from each local minimizer of the CEL0 functional, a local minimizer of $\ell_2 - \ell_0$ is easy to obtain. Moreover, some strict local minimizers of the initial functional are eliminated with the proposed tight relaxation. Then solving the initial $\ell_2 - \ell_0$ problem is equivalent, in a sense, to solving it by replacing the ...