Accelerated anti-lopsided algorithm for nonnegative least squares

Abstract

Nonnegative least squares (NNLS) problem has been widely used in scientific computation and data modeling, especially for low-rank representation such as nonnegative matrix and tensor factorization. When applied to large-scale datasets, first-order methods are preferred to provide fast flexible computation for regularized NNLS variants, but they still have the limitations of performance and convergence as key challenges. In this paper, we propose an accelerated anti-lopsided algorithm for NNLS with linear over-bounded convergence rate $$\left[ \left( 1 - \frac{\mu }{L}\right) \left( 1-\frac{\mu }{nL}\right) ^{2n}\right] ^k$$ in the subspace of passive variables where $$\mu $$ and L are always bounded as $$\frac{1}{2} \le \mu \le L \le n$$ , and n is the dimension size of solutions, which is highly competitive with current advanced methods such as accelerated gradient methods having sub-linear convergence $$\frac{L}{k^2}$$ , and greedy coordinate descent methods having convergence $$\left( 1 - \frac{\mu }{nL}\right) ^k$$ , where $$\mu $$ and L are unbounded. The proposed algorithm transforms the variable x into the new space satisfying the second derivative equals constant $$\frac{\partial ^2 f}{\partial x_i^2} = 1$$ for all variables $$x_i$$ to implicitly exploit the second-order derivative, and to guarantee that $$\mu $$ and L are always bounded in order to achieve over-bounded convergence of the algorithm, and to enhance the performance of internal processes based on exact line search, greedy coordinate descent methods, and accelerated search. The experiments on large matrices and real applications of nonnegative matrix factorization clearly show the higher performance of the proposed algorithm in comparison with the state-of-the-art algorithms.