Vision Transformer with Information Bottleneck for Fine-Grained Visual Classification

Abstract

AbstractFine-grained visual classification focus on accurately identifying the subordinate categories from a base class. One key of this task is to find discriminative local parts. Convolutional neural network-based methods using attention mechanism can enhance the representation of local regions and improve the classification accuracy. Recently, vision transformer has shown great potential in traditional image classification with its self-attention mechanism, which can grab important regions. However, one main difficulty in fine-grained visual classification is the subtle inter-class variance. Different classes may share the same or similar features and these features may be identified by attention. This discriminative information is redundant for related classes and may damage the performance of network. To this end, we apply information bottleneck to vision transformer training to relieve the interference of redundant information. The experiments demonstrate the effectiveness of this simple method.KeywordsFine-grained visual classificationVision transformerInformation bottleneck