Enhancing Mixture-of-Experts by Leveraging Attention for Fine-Grained Recognition

Abstract

Differentiating subcategories of a common visual category is challenging because of the similar appearance shared among different classes in fine-grained recognition. Existing mixture-of-expert based methods divide the fine-grained space into some specific regions and solve the integrated problem by conquering subspace ones. However, it is not feasible to learn diverse experts directly through data partition strategy because of limited data available for fine-grained recognition problems. To address the issue, we leverage visual attention to learn an enhanced experts' mixture. Specifically, we introduce a gradually-enhanced learning strategy from model attention. The strategy promotes diversity among experts by feeding each expert with full-size data distinct in granularity. We further promote expert's learning by providing it with a larger data space, which is achieved by swapping attentive regions within positive pairs. Our method learns new experts on the dataset with the prior knowledge from former experts sequentially and enforces the experts to learn more diverse but discriminative representation. These enhanced experts are finally combined to make stronger predictions. We conduct extensive experiments on fine-grained benchmarks. The results show that our method consistently outperforms the state-of-the-art method in both weakly supervised localization and fine-grained image classification. Our code is publicly available at https://github.com/lbzhang/Enhanced-Expert-FGVC-Pytorch.git.