Accelerated stochastic multiplicative update with gradient averaging for nonnegative matrix factorizations

Abstract

Nonnegative matrix factorization (NMF) is a powerful tool in data analysis by discovering latent features and part-based patterns from high-dimensional data, and is a special case in which factor matrices have low-rank nonnegative constraints. Applying NMF into huge-size matrices, we specifically address stochastic multiplicative update (MU) rule, which is the most popular, but which has slow convergence property. This present paper introduces a gradient averaging technique of stochastic gradient on the stochastic MU rule, and proposes an accelerated stochastic multiplicative update rule: SAGMU. Extensive computational experiments using both synthetic and real-world datasets demonstrate the effectiveness of SAGMU.