Operator-Norm-Based Variable-Wise Diagonal Preconditioning for Automatic Stepsize Selection of A Primal-Dual Splitting Algorithm

Abstract

We propose a diagonal preconditioning method for automatically selecting the step sizes of a primal-dual splitting method (PDS). The conventional preconditioning method for PDS has several limitations, such as the need to directly access all the entries of the matrices representing the linear operators in the target optimization problem, and the possibility that the proximity operator cannot be solved analytically due to the element-wise preconditioning. In this paper, we establish operator norm-based variable-wise diagonal preconditioning (ON-VW) to resolve these issues. ON- VW has two features that are preferred in real applications. First, the preconditioners constructed by ON-VW are defined using only (an upper bound of) the operator norm of the linear operators, which eliminates the need for their explicit matrix representations. Furthermore, the stepsizes automatically selected by our preconditioners are variable-wise, which allows us to keep the proximity operator computable. We also prove that our preconditioners satisfy the convergence condition of PDS and demonstrate its effectiveness through its application to denoising of hyperspectral images.