AutoInfo GAN: Toward a better image synthesis GAN framework for high-fidelity few-shot datasets via NAS and contrastive learning

Abstract

Background:Generative adversarial networks (GANs) are vital techniques for synthesizing high-fidelity images. Recent studies have applied them to generation tasks under small-data scenarios. Most studies do not directly train GANs on few-shot datasets, which have small data samples; instead, they borrow methods to transfer knowledge from large datasets to GANs with small ones. Partial fine-tuning of GANs is difficult to ensure the transfer performance, especially when the image domains are of great difference. FastGAN firstly trains GAN with small data samples by a carefully designed skip layer exception (SLE) connection to improve synthesis and an unsupervised discriminator to avoid overfitting. Problem. However, in FastGAN, different designs of SLE connections and operation settings would lead to great differences in synthesis performance. It is necessary to find the most appropriate ways of architecture design. Meanwhile, FastGAN merely improves discriminator learning, but ignores that the generator learning process is also insufficient due to small data samples. Objective:Based on FastGAN, this study aims to find the best generator designs and then improve the training process of it via unsupervised learning. Methods. This work applies a reinforcement learning neural architecture search method to find the optimal GAN architecture and an unsupervised contrastive loss function assisted by a discriminator to optimize generator learning. These two methods constitute our AutoInfoGAN. Results:Experiments were conducted on 11 datasets using AutoInfoGAN, covering a wide range of image domains, achieving better results than state-of-the-art (SOTA) models. Conclusion:The experimental results demonstrate the SOTA performance of our proposed AutoInfo GAN on few-shot datasets, and we are cautioned that although instance normalization (IN) improves synthesized image quality, it performed poorly in our mode-collapse test.