Inexact block coordinate descent methods with application to non-negative matrix factorization

Abstract

This work is concerned with the cyclic block coordinate descent method, or nonlinear Gauss-Seidel method, where the solution of an optimization problem is achieved by partitioning the variables in blocks and successively minimizing with respect to each block. The properties of the objective function that guarantee the convergence of such alternating scheme have been widely investigated in the literature and it is well known that, without suitable convexity hypotheses, the method may fail to locate the stationary points when more than two blocks of variables are employed. In this paper the general constrained non convex case is considered and three contributions are given. First, a general method allowing an approximate solution of each block minimization subproblem is devised and the related convergence analysis is developed, showing that the proposed inexact method has the same convergence properties of the standard nonlinear Gauss-Seidel method. Then, a cyclic block gradient projection method is analyzed, proving that it leads to stationary points for every number of blocks. Finally, the cyclic block gradient method is applied to large scale problems arising from the nonnegative matrix factorization approach. The results of a numerical experimentation on image recognition problems are also reported.