Manifold-respecting probabilistic matrix tri-factorization

Abstract

Probabilistic latent semantic analysis (PLSA) is a popular topic model for factor analysis of dyadic data, which is closely related to nonnegative matrix factorization (NMF) that seeks a 2-factor decomposition of a nonnegative data matrix. We previously proposed probabilistic matrix tri-factorization (PMTF) which is a probabilistic model for a 3-factor decomposition of a nonnegative data matrix, extending PLSA and NMF for co-clustering simultaneously columns and rows of dyadic data matrix. However, these methods do not take the local manifold structure of dyadic data into account. In this paper we present a method for manifold-respecting probabilistic matrix tri-factorization (MPMTF) where we incorporate a local manifold structure into PMTF, imposing smoothness constraints on posterior distributions over latent variables. We develop an EM algorithm to learn MPMTF. Our model handles both unlabeled and labeled data points, while existing methods considered unlabeled data only. Numerical experiments on document and image datasets confirm the useful behavior of our proposed method in the task of clustering.