Variational Bayesian Orthogonal Nonnegative Matrix Factorization Over the Stiefel Manifold.

Abstract

Nonnegative matrix factorization (NMF) is one of the best-known multivariate data analysis techniques. The NMF uniqueness and its rank selection are two major open problems in this field. The solutions uniqueness issue can be addressed by imposing the orthogonality condition on NMF. This constraint yields sparser part-based representations and improved performance in clustering and source separation tasks. However, existing orthogonal NMF algorithms rely mainly on non-probabilistic frameworks that ignore the noise inherent in real-life data and lack variable uncertainties. Thus, in this work, we investigate a new probabilistic formulation of orthogonal NMF (ONMF). In the proposed model, we impose the orthogonality through a directional prior distribution defined on the Stiefel manifold called von Mises-Fisher distribution. This manifold consists of a set of directions that comply with the orthogonality condition that arises in many applications. Moreover, our model involves an automatic relevance determination (ARD) prior to address the model order selection issue. We devised an efficient variational Bayesian inference algorithm to solve the proposed ONMF model, which allows fast processing of large datasets. We evaluated the proposed model, called VBONMF, on the task of blind decomposition of real-world multispectral images of ancient documents. The numerical experiments demonstrate its efficiency and competitiveness compared to the state-of-the-art approaches.