A Deformable Convolutional Neural Network with Oriented Response for Fine-Grained Visual Classification

Abstract

Fine-grained visual classification (FGVC) aims to classify images belonging to the same basic category in a more detailed sub-category. It is a challenging research topic in the field of computer vision and pattern recognition in recent years. The existing FGVC method conduct the task by considering the part detection of the object in the image and its variants, which rarely pays attention to the difference in expression of many changes such as object size, posture, and perspective. As a result, these methods generally face two major difficulties: 1) How to effectively pay attention to the latent semantic region, and reduce the interference caused by many changes in pose and perspective; 2) How to extract rich feature information for non-rigid and weak structure objects. In order to solve these two problems, this paper proposes a deformable convolutional neural network with oriented response for FGVC. The proposed method can be divided into three main steps: firstly, the local region of latent semantic information is localized based on a lightweight CAM network; then, the deformable convolutional ResNet-50 network and the rotation-invariant coding oriented response network are designed, which input the original image and local region into the feature network to learn the discriminant features of rotation invariance; finally, the learned features are embed into a joint loss to optimize the entire network end-to-end. Experiments are carried out on three challenging FGVC datasets, including CUB-200-2011, FGVC_Aircraft and Aircraft_2 datasets. The results show that the accuracy of the proposed method on all datasets is better than the comparison method, which can effectively improve the accuracy of weakly supervised FGVC.