S3Mix: Same Category Same Semantics Mixing for Augmenting Fine-grained Images

Abstract

Data augmentation is a common technique to improve the generalization performance of models for image classification. Although methods such as Mixup and CutMix that mix images randomly are indeed instrumental in general image classification, randomly swapping or masking regions is not friendly to fine-grained images, since the key to fine-grained image classification precisely lies in discriminative and informative regions, and it is unreasonable to generate labels solely consistent with the proportion of synthesis. Some erasing methods like Cutout even endanger fine-grained image classification because of erasing the discriminative regions by chance. In this article, we propose the Same Category Same Semantics Mixing method (S3Mix) corresponding to the characteristics of fine-grained images. Specifically, we limit the mixture to regions of the same category and semantics. The core of the method is two constraints. The exchange with the semantic region ensures the discrimination and semantics integrity of the generated image, and the exchange in the same class avoids the problem of unreasonable label generation. At the same time, we propose a homology loss to promote the semantic relationship between the generated positive image pairs. Experiments have been conducted on four fine-grained datasets, and the results show the proposed method is superior to the traditional image augmentation methods as well as some fine-grained data augmentation methods.