Automatic relevance determination for nonnegative dictionary learning in the Gamma-Poisson model

Abstract

In this paper a method for model order selection through automatic relevance determination in NMF is proposed. Overfitting is avoided by inferring the relevance of components in the dictionary and removing the irrelevant ones. To reduce the number of parameters in the model the activations are treated as missing data and marginalised out, the marginal posterior is maximised. Furthermore the hyper-parameters are found with full Bayesian inference, so they no longer have to be tuned. The proposed method solves two suboptimalities in a previously proposed method for automatic relevance determination based on the maximisation of the joint posterior. (1) In the joint posterior method the activations of the dictionary components are inferred by training them as parameters of the model and (2) the hyper-parameters are chosen by hand and have to be tuned. The proposed algorithm is extensively tested on a synthetic dataset and the swimmer dataset. Results from a face reconstruction task on the CBCL dataset and an unsupervised spoken keyword discovery task on the TIDIGITS dataset are also presented. The results show that the proposed algorithm outperforms previously proposed algorithms in most experiments.