A sharp nonasymptotic bound and phase diagram of L 1/2 regularization

Abstract

We derive a sharp nonasymptotic bound of parameter estimation of the L 1/2 regularization. The bound shows that the solutions of the L 1/2 regularization can achieve a loss within logarithmic factor of an ideal mean squared error and therefore underlies the feasibility and effectiveness of the L 1/2 regularization. Interestingly, when applied to compressive sensing, the L 1/2 regularization scheme has exhibited a very promising capability of completed recovery from a much less sampling information. As compared with the L p (0 < p < 1) penalty, it is appeared that the L 1/2 penalty can always yield the most sparse solution among all the L p penalty when 1/2 ≤ p< 1, and when 0 < p< 1/2, the L p penalty exhibits the similar properties as the L 1/2 penalty. This suggests that the L 1/2 regularization scheme can be accepted as the best and therefore the representative of all the L p (0 < p< 1) regularization schemes.