Iterative shrinkage-thresholding algorithm with inertia and dry friction for convolutional dictionary learning

Abstract

The iterative shrinkage-thresholding algorithm (ISTA) is often used to address the convolutional dictionary learning problem. However, the ISTA algorithm is easy to fall into the local minimum, and some atoms in the convolution dictionary change little when updated. Therefore, in this paper, we propose an improved ISTA algorithm, iterative shrinkage-thresholding algorithm with inertia and dry friction (ISTA-IDF). An inertia is introduced to avoid falling into the local minimum but generating an oscillatory behavior since the values of the objective function is not monotonic decreasing, which can be alleviated by dry friction. From the perspective of dynamics, ISTA can be seen as an iteration algorithm deduced by a forward-backward Euler discretization of a first-order dynamic system, and ISTA-IDF can be regarded as the forward-backward Euler discretization of a second-order heavy-ball system with dry friction (HBDF). So using the dynamics theory, we establish the linear and finite convergence property of ISTA-IDF. The experimental results on image reconstruction and denoising tasks demonstrate the superior performance of the proposed ISTA-IDF compared with ISTA and FISTA.