Improving FISTA’s Speed of Convergence via a Novel Inertial Sequence

Abstract

The FISTA (fast iterative shrinkage-thresholding algorithm) is a well-known and fast (theoretical $\mathrm{O}(k^{-2})$ rate of convergence) procedure for solving optimization problems composed by the sum of two convex functions, such that one is smooth (differentiable) and the other is possible nonsmooth. FISTA can be understood as a first order method with one important aspect: it uses a suitable extragradient rule, i.e.: the gradient is evaluated at a linear combination of the past two iterates, whose weights, are usually referred to as the inertial sequence. While problem dependent, it has a direct impact on the FISTA’s practical computational performance. In this paper we propose a novel inertial sequence; when compared to well-established alternative choices, in the context of convolutional sparse coding and Wavelet-based inpainting, our proposed inertial sequence can reduce the number of FISTA’s global iterations (and thus overall computational time) by 30% $\sim 50$% to attain the same level of reduction in the cost functional.