Tractable ADMM schemes for computing KKT points and local minimizers for $$\ell _0$$-minimization problems

Abstract

We consider an $$\ell _0$$ -minimization problem where $$f(x) + \gamma \Vert x\Vert _0$$ is minimized over a polyhedral set and the $$\ell _0$$ -norm regularizer implicitly emphasizes the sparsity of the solution. Such a setting captures a range of problems in image processing and statistical learning. Given the nonconvex and discontinuous nature of this norm, convex regularizers as substitutes are often employed and studied, but less is known about directly solving the $$\ell _0$$ -minimization problem. Inspired by Feng et al. (Pac J Optim 14:273–305, 2018), we consider resolving an equivalent formulation of the $$\ell _0$$ -minimization problem as a mathematical program with complementarity constraints (MPCC) and make the following contributions towards the characterization and computation of its KKT points: (i) First, we show that feasible points of this formulation satisfy the relatively weak Guignard constraint qualification. Furthermore, if f is convex, an equivalence is derived between first-order KKT points and local minimizers of the MPCC formulation. (ii) Next, we apply two alternating direction method of multiplier (ADMM) algorithms, named (ADMM $$_{\mathrm{cf}}^{\mu , \alpha , \rho }$$ ) and (ADMM $$_{\mathrm{cf}}$$ ), to exploit the special structure of the MPCC formulation. Both schemes feature tractable subproblems. Specifically, in spite of the overall nonconvexity, it is shown that the first update can be effectively reduced to a closed-form expression by recognizing a hidden convexity property while the second necessitates solving a tractable convex program. In (ADMM $$_{\mathrm{cf}}^{\mu , \alpha , \rho }$$ ), subsequential convergence to a perturbed KKT point under mild assumptions is proved. Preliminary numerical experiments suggest that the proposed tractable ADMM schemes are more scalable than their standard counterpart while (ADMM $$_{\mathrm{cf}}$$ ) compares well with its competitors in solving the $$\ell _0$$ -minimization problem.