Enhancing diversity in GANs via non-uniform sampling

Abstract

Recent advances in Generative Adversarial Networks (GANs) have led to impressive results in generating realistic data. However, GANs training is still challenging, often leading to mode-collapse, where a certain type of samples dominates the generated output. To address this issue, we propose a novel training algorithm based on bidirectional GANs (BiGANs) that can be generalized to any implicit generative model. Our algorithm relies on a non-uniform sampling scheme, where data points in a minibatch are sampled with probability inversely proportional to their log-evidence. However, estimating log-evidence is computationally expensive. Instead, we propose to use the reconstruction error, which directly correlates with the log-evidence and only requires a BiGAN network evaluation. Additionally, we combine the aforementioned method with a regularization in the empirical distribution of the encoder that further boosts the performance. Our empirical results show that the proposed methods improve both the quality and diversity of the generated samples.