SAGAN: Skip attention generative adversarial networks for few-shot image generation

Abstract

The task of producing high-quality, realistic, and diverse images based on a few instances of newly emerging or long-tail categories is known as few-shot image generation. Despite prior works showing outstanding results, the quality and diversity of the outputs are still limited. In this paper, we tackle this problem by presenting a range of innovative fusion techniques based on attention mechanisms with generative adversarial networks. Our proposed framework introduces global skip attention that links matching residual blocks of symmetric encoder-decoder pairs to generate new instance objects. Additionally, we incorporate an alignment algorithm based on spatial transformer networks into our pipeline encoder to address feature misalignment. In the decoding phase of our attention-based decoder, we propose a novel attention mechanism within each fusion residual block, which leads to capturing long-range dependencies in feature maps. An attention reconstruction loss function has been proposed to balance adversarial training between the generator and discriminator, mitigate mode collapse, and guide the generator to focus on specific regions of interest within images. Finally, we apply a back summation to the decoding outputs, resulting in unified features through a weighted combination of similar characteristics. Extensive experiments conducted on five few-shot image datasets demonstrate the effectiveness of our proposed model. The source code of the proposed model can be found on GitHub https://github.com/Aldhubri/SAGAN.