Proximal variable metric method with spectral diagonal update for large scale sparse optimization

Abstract

We will tackle the l0-norm sparse optimization problem using an underdetermined system as a constraint in this research. This problem is turned into an unconstrained optimization problem using the Lagrangian method and solved using the proximal variable metric method. This approach combines the proximal and variable metric methods by substituting a diagonal matrix for the approximation of the full rank Hessian matrix. Hence, the memory requirement is reduced to O(n) storage instead of O(n2) storage. The diagonal updating matrix is derived from the same variational technique used in the derivation of variable metric or quasi-Newton updates but incorporated with some weaker form of quasi-Newton relation. Convergence analysis of this method is established. The efficiency of the proposed method is compared against existing versions of proximal gradient methods on simulated datasets and large real-world MNIST datasets using Python software. Numerical results show that our proposed method is more robust and stable for finding sparse solutions to the linear system.