Disentangling the correlated continuous and discrete generative factors of data

Abstract

Real-world data typically include discrete generative factors, such as category labels and the existence of objects, as well as continuous generative factors. Continuous generative factors may be dependent on or independent of discrete generative factors. For instance, an intra-class variation of a category is dependent on the discrete generative factor, whereas a common variation of all categories is not. Most previous attempts to integrate discrete generative factors into disentanglement assumed statistical independence between the continuous and discrete variables. In this paper, we propose a Variational Autoencoder(VAE) model capable of disentangling both continuous generative factors. To represent these generative factors, we introduce two sets of continuous latent variables: a private variable and a public variable. The private and public variables represent the intra-class variations and common variations in categories, respectively. Our proposed framework models the private variable as a Gaussian mixture and the public variable as a Gaussian. Each mode of the private variable is responsible for a class of discrete variables. Our proposed model, called Discond-VAE, DISentangles the class-dependent CONtinuous factors from the Discrete factors by introducing private variables. The experiments showed that Discond-VAE could discover private and public factors from the data. Moreover, even under the dataset with only public factors, Discond-VAE does not fail and adapts private variables to represent public factors.