A logistic non-negative matrix factorization approach to binary data sets

Abstract

An analysis of binary data sets employing Bernoulli statistics and a partially non-negative factorization of the related matrix of log-odds is presented. The model places several constraints onto the factorization process rendering the estimated basis system strictly non-negative or even binary. Thereby the proposed model places itself in between a logistic PCA and a binary NMF approach. We show with proper toy data sets that different variants of the proposed model yield reasonable results and indeed are able to estimate with good precision the underlying basis system which forms a new and often more compact representation of the observations. An application of the method to the USPS data set reveals the performance of the various variants of the model and shows good reconstruction quality even with a low rank binary basis set.