Dropout Non-negative Matrix Factorization for Independent Feature Learning

Abstract

Non-negative Matrix Factorization (NMF) can learn interpretable parts-based representations of natural data, and is widely applied in data mining and machine learning area. However, NMF does not always achieve good performances as the non-negative constraint leads learned features to be non-orthogonal and overlap in semantics. How to improve the semantic independence of latent features without decreasing the interpretability of NMF is still an open research problem. In this paper, we put forward dropout NMF and its extension sequential NMF to enhance the semantic independence of NMF. Dropout NMF prevents the co-adaption of latent features to reduce ambiguity while sequential NMF can further promote the independence of individual latent features. The proposed algorithms are different from traditional regularized and weighted methods, because they require no prior knowledge and bring in no extra constraints or transformations. Extensive experiments on document clustering show that our algorithms outperform baseline methods and can be seamlessly applied to NMF based models.