Iterative reweighted minimization methods for $$l_p$$ l p regularized unconstrained nonlinear programming

Abstract

In this paper we study general $$l_p$$ l p regularized unconstrained minimization problems. In particular, we derive lower bounds for nonzero entries of the first- and second-order stationary points and hence also of local minimizers of the $$l_p$$ l p minimization problems. We extend some existing iterative reweighted $$l_1$$ l 1 ( $$\mathrm{IRL}_1$$ IRL 1 ) and $$l_2$$ l 2 ( $$\mathrm{IRL}_2$$ IRL 2 ) minimization methods to solve these problems and propose new variants for them in which each subproblem has a closed-form solution. Also, we provide a unified convergence analysis for these methods. In addition, we propose a novel Lipschitz continuous $${\epsilon }$$ ∈ -approximation to $$\Vert x\Vert ^p_p$$ ? x ? p p . Using this result, we develop new $$\mathrm{IRL}_1$$ IRL 1 methods for the $$l_p$$ l p minimization problems and show that any accumulation point of the sequence generated by these methods is a first-order stationary point, provided that the approximation parameter $${\epsilon }$$ ∈ is below a computable threshold value. This is a remarkable result since all existing iterative reweighted minimization methods require that $${\epsilon }$$ ∈ be dynamically updated and approach zero. Our computational results demonstrate that the new $$\mathrm{IRL}_1$$ IRL 1 method and the new variants generally outperform the existing $$\mathrm{IRL}_1$$ IRL 1 methods (Chen and Zhou in 2012; Foucart and Lai in Appl Comput Harmon Anal 26:395---407, 2009).