A stable approach for model order selection in nonnegative matrix factorization

Abstract

In order to find the correct model order in non-negative matrix factorization (NMF), an algorithm called automatic relevance determination (ARD) is proposed in Tan and Fevotte (2013). The algorithm explores the similarities of the NMF components and removes redundant ones iteratively. However, the algorithm can yield over-parsimonious representations where ground truth patterns can be grouped into one single component to cause superposition. In this paper, mixed entropy regularized NMF (MER-NMF) is proposed to overcome the above problem. In MER-NMF, the objective function of NMF is regularized by minimizing a mixed entropy of the coefficient matrix which is a weighted sum of two parts: the entropy of all the entries and the entropy of the row sums of the coefficient matrix. With the mixed entropy regularization, the algorithm tends to yield sharper activations of the components for each sample. By combining MER-NMF and ARD-NMF, correct number of components can always be selected according to our experiments.