On choosing initial values of iteratively reweighted ℓ1 algorithms for the piece-wise exponential penalty

Abstract

Computing the proximal operator of the sparsity-promoting piece-wise exponential (PiE) penalty [Formula: see text] with a given shape parameter [Formula: see text], which is treated as a popular nonconvex surrogate of [Formula: see text]-norm, is fundamental in feature selection via support vector machines, image reconstruction, zero-one programming problems, compressed sensing, neural networks, etc. Due to the nonconvexity of PiE, for a long time, its proximal operator is frequently evaluated via an iteratively reweighted [Formula: see text] algorithm, which substitutes PiE with its first-order approximation, however, the obtained solutions only are the critical point. Based on the exact characterization of the proximal operator of PiE, we explore how the iteratively reweighted [Formula: see text] solution deviates from the true proximal operator in certain regions, which can be explicitly identified in terms of [Formula: see text], the initial value and the regularization parameter in the definition of the proximal operator. Moreover, the initial value can be adaptively and simply chosen to ensure that the iteratively reweighted [Formula: see text] solution belongs to the proximal operator of PiE.