Neural generative model for clustering by separating particularity and commonality

Abstract

Learning discriminative representation is essential in many machine learning tasks. Each category has intrinsic and particular features related to the label. However, objects from different categories often share some common patterns that are independent of the label. Therefore, separating the particular and shared features will facilitate representation learning and other downstream tasks. In this study, we attempt to improve clustering accuracy by disentangling these two types of features. We introduce a generative model based on a neural network to explain observations according to the assumed underlying structures and to perform clustering simultaneously. Specifically, our proposed model, named the disentangling generative model for clustering (DGC), assumes that the observed data are generated from the concatenation of latent particular and common features that are subject to a Gaussian mixture distribution and standard Gaussian distribution, respectively. For the inference in DGC, each observation is encoded into two parts with different networks, which correspond to the approximate posterior over the particular and common features. The former is fed into a classifier, and the result serves as the clustering assignment of the observation. The DGC is optimized within the variational autoencoder framework. The empirical results show that the proposed method exhibits performance comparable with those of state-of-the-art methods. In addition, the DGC can generate class-specific samples without any label information.