The gradient-based algorithms for non-negative matrix factorization with sparseness constraints

Abstract

Non-negative matrix factorization with sparseness constraints (SNMF) has become a widely used tool for keeping the main features of the original data as well as reducing the storage space. Most proposed SNMF problems are commonly solved using the multiplicative update rules. However, the popular multiplicative update rules have been shown to give poor convergence. In this paper, a comparative study of two gradient-based algorithms with strong optimization property and its implementation on SNMF are described. The objective of this study is to investigate whether these two gradient-based algorithms will provide an improvement over the multiplicative update rules in terms of convergence and sparseness. We conduct several experiments on the ORL face database. Through the experimental results, we can see that these two gradient-based algorithms do actually achieve stronger optimization property and higher sparseness of the factor matrices.