Fine-grained Vehicle Recognition Research Articles

FedLVR: a federated learning-based fine-grained vehicle recognition scheme in intelligent traffic system

FedLVR: a federated learning-based fine-grained vehicle recognition scheme in intelligent traffic system

An efficient fine-grained vehicle recognition method based on part-level feature optimization

This paper presents an effective method for strengthening the discriminative ability of high-level deep features by enhancing and aggregating discriminative part-level features for the fine-grained vehicle recognition task. In general, the task of visual recognition concentrates more on the visual differences at the object level. However, for fine-grained object recognition, the visual differences between target objects typically exist in local discriminative areas, so it is more concerned about extracting fine-grained features from these part regions. In this context, we propose solving this issue with a novel feature extraction method from two perspectives: the generation of more feature descriptors of part regions through the learning process of deep networks and the aggregation of part-level discriminative features. This approach is designed to improve the backbone networks to generate finer-level part features through a part-level feature enhancement module and to investigate the intrinsic part-level features of the backbone networks with the help of a feature aggregation module. The enhancement module efficiently finds the finer features highly correlated to the part regions. Then the feature aggregation module builds correlations of similar part features through feature grouping and fusion. Moreover, our proposed method does not require additional parts annotations and achieves comparable performance on two widely-used benchmarks for recognizing fine-grained vehicle types. Experimental results and explainable visualizations demonstrate the effectiveness of the proposed method.

Fine-grained Vehicle Classification Based on Feature Augmentation

Fine-grained vehicle classification is a hot research topic in the field of computer vision, which aims to recognize different models of vehicles belonging to the same brand. Limited by the large appearance differences of the same vehicle (due to different attitudes and perspectives) and small appearance differences between different vehicles, the vehicle’s reorganization performance is not high and can’t meet the real application requirements. In this paper, a Fine-grained Vehicle Classification is proposed based on Feature Augmentation (FAFCC), which focuses on mining the key discriminative parts of the object. Specifically, the FAFCC first improves the weakly supervised data augmentation network (WSDAN) by using multi-layer feature extraction and fusion instead of single-layer features. Further, the FAFCC assigns local feature weights to select key features, which can solve the problem that the discriminant features in the channel are not prominent enough and realize the high-quality compact fine-grained vehicle recognition task. The performance of the FAFCC on the fine-grained image classification tasks for vehicles has been evaluated through algorithm comparison experiments and ablation experiments. Through visual analysis, it can be seen that the introduction of each new module has optimized the loss function value and convergence speed of the model. In general, our FAFCC algorithm has a certain improvement effect compared with the original algorithm.

Few-Shot Class-Incremental Learning via Compact and Separable Features for Fine-Grained Vehicle Recognition

Most of the existing deep learning-based fine-grained vehicle recognition methods collect a large-scale training set in advance and train a model based on the closed-world assumption. However, in the real world, new classes of vehicles are released over time but it is difficult to collect sufficient labeled data for new classes, which results in a typical few-shot class-incremental learning problem (FSCIL). To solve this problem, this work proposes a compact and separable feature learning method (CSFL) which exploits a decoupled learning scheme to prevent the feature extractor from updating during class-incremental learning. CSFL trains an initial model to learn discriminative features of fine-grained vehicles using deep metric learning. Then an incremental linear discriminant analysis algorithm is applied to the learned features to further discriminate potential confused classes. Specifically, the decoupled components share the same objective of enhancing intra-class compactness and inter-class separability, which is beneficial for classification. Extensive experiments on three fine-grained vehicle datasets demonstrate that the proposed CSFL achieves better results than state-of-the-art incremental learning methods, validating the importance of compact and separable features in the problem of FSCIL.

A Novel Smart Lightweight Visual Attention Model for Fine-Grained Vehicle Recognition

Vehicle Make and Model Recognition (VMMR) requires fast and accurate recognition of a vehicle’s information. Generally, the vision-based VMMR method recognizes different vehicle models that mainly rely on locating and extracting the discriminative part features of a vehicle. In this paper, we propose a Lightweight Recurrent Attention Unit (LRAU) to enhance the feature extraction ability of the standard Convolutional Neural Network (CNN) architectures for VMMR. The proposed LRAU extracts the discriminative part features by generating attention masks to locate the keypoints of a vehicle (e.g., logo, headlight). The attention mask is generated based on the feature maps received by the LRAU and the preceding attention state generated by the preceding LRAU. By adding LRAUs to receive the multi-scale feature maps generated by the standard CNN architecture, discriminative features of different scales can be efficiently extracted and combined. We conduct comprehensive experiments on three challenging VMMR datasets to evaluate the proposed VMMR models. Experimental results show our models have a stable performance under different environmental conditions. Our models achieve state-of-the-art results with 93.94% accuracy on the Stanford Cars dataset, 98.31% accuracy on the CompCars dataset, and 99.41% accuracy on the NTOU-MMR dataset. Moreover, we demonstrate that our models outperform the traditional machine learning-based VMMR models in terms of recognition accuracy and processing speed. In addition, we construct a one-stage Vehicle Detection and Fine-grained Recognition (VDFR) model by combining our LRAU with the general object detection model. Results show the proposed VDFR model can achieve excellent performance with real-time processing speed.

A novel part-level feature extraction method for fine-grained vehicle recognition

In this paper, we propose a novel part-level feature extraction method to enhance the discriminative ability of deep convolutional features for the task of fine-grained vehicle recognition. Generally, the challenges for fine-grained vehicle recognition are mainly caused by the subtle visual differences between part regions of vehicles. Therefore, it is essential to extract discriminative features from part regions. Many existing methods, especially deep convolutional neural networks (D-CNNs), tend to detect the discriminative part regions explicitly or learn the part information implicitly through network restructuring and neglect the abundant part-level information contained in the high-level features generated by CNNs. In light of this, we propose a simple and effective part-level feature extraction method to enhance the representation of part-level features within the global features of target object generated by the backbone networks. The proposed method is built on the deep convolutional layers from which the discriminative part features could be integrated and extracted accordingly. More specifically, a basic feature grouping module is adopted to integrate the feature maps of deep convolutional layers into groups in each of which the related discriminative parts are assembled. The feature grouping process is performed in a multi-stage manner to ensure the integration process. Then a fusion module follows to model the coarse-to-fine relationship of the part features and further ensure the integrity and effectiveness of the part features. We conduct comparison experiments on public datasets, and the results show that the proposed method achieves comparable performance with state-of-the-art algorithms.

Structure-guided attention network for fine-grained vehicle model recognition

In view of the difficulty in accurately recognizing vehicle models due to the wide variety of vehicle models and little differentiation between some models, a structure-guided attention network for weakly supervised learning fine-grained vehicle model recognition is proposed. First, we utilize the structure-guided attention location module to locate key features of images, extracting the strong and weak attention areas of the image by generating attention maps. Second, we investigate multichannel convolutional neural network for the learning of the strong-weak feature pairs and solve the problem of different network feature maps with different sizes by using feature map alignment. Third, an adaptive weighting method is proposed to balance the loss function. The experimental results show that our algorithm effectively improves the accuracy of the fine-grained vehicle recognition dataset and has universal applicability in other fine-grained recognition datasets.

A Viewpoint Aware Multi-Task Learning Framework for Fine-Grained Vehicle Recognition

The goal of fine-grained vehicle recognition is to identify the exact subtype of the vehicle from a given image. It plays an important role in the intelligent traffic surveillance system. Although fine-grained vehicle recognition has attracted more and more research interest, it remains as an open problem for vehicle images taken from arbitrary viewpoints. In this study, we present a one-stage deep multi-task learning framework for fine-grained vehicle recognition in traffic surveillance, which performs the fine-grained vehicle recognition and viewpoint estimation simultaneously. In the proposed framework, the fine-grained vehicle recognition is the main task which classifies images into different types, and the viewpoint estimation task is the auxiliary task to learn helpful viewpoint-aware features for improving the main task. We evaluate our method on two common used large-scale fine-grained vehicle recognition datasets, including BoxCars116k and CompCars. The experimental results testify that the proposed multi-task framework can improve the accuracy by incorporating the viewpoint information of the vehicles. In comparison to the state-of-the-art, our approach goes beyond, only requiring 3D bounding box for the training phase, which is important for future inferences using the trained model.

Global relative position space based pooling for fine-grained vehicle recognition

In the field of fine-grained vehicle recognition, large intra-class variation often affects the final prediction seriously. One of the principal reasons is unstable change of absolute positions for discriminative features, which can not be completely solved by the region of interest (ROI) localization for inevitable deviations. In this paper, we propose a global relative position space based pooling (GRPSP) method to replace the absolute position information with global relative position information, by which the discriminative features can be well aligned and thereby the intra-class variation is reduced effectively. Specifically, the discriminative features containing both absolute position information and appearance information are extracted by a convolution neural network (CNN). We then transform the absolute positions into global relative positions, and estimate position probabilities of locating at discrete representatives of global relative position space. Accordingly, the joint probabilities of different kinds of features locating at different representative positions can be calculated, and constitute the pooling vector. Lastly, the Support Vector Machine (SVM) is trained for classification. Experimental results on public and self-building challenging datasets both show that our proposed method can improve various CNNs and is the state-of-the-art.

Global Topology Constraint Network for Fine-Grained Vehicle Recognition

Fine-grained vehicle recognition is challenging due to the large intra-class variation in vehicle pose and viewpoint. Many existing methods, especially convolution neural network (CNN)-based methods, solve this problem via detecting and aligning parts individually, and do not consider the interaction between parts, which is very important for effective part detection and vehicle recognition. In this paper, we propose a global topology constraint network for fine-grained vehicle recognition, which adopts the constraint of global topology relationship to depict the interaction between parts and integrates it into CNN in an efficient way. Different CNNs for image classification truncated at intermediate layer can be used for part detection. The global topology relationship between parts is encoded into kernel of depthwise convolution layer, and can be learned from training images. The response under global topology constraint reflects the probability of topology relationship between parts existing in input image. All responses build the description for classification with translation invariance. Through training the whole network, the back-propagation of gradient information of kernel for global topology relationship will guide former layers to better detect useful parts, and thereby improve vehicle recognition. Our proposed method does not require additional annotation, such as bounding box or part annotation, and can be trained in an end-to-end way. We conduct comparison experiments on public Stanford Cars and CompCars datasets, which both show that our method achieves the state-of-the-art performance.

Location-aware fine-grained vehicle type recognition using multi-task deep networks

Distinct from conventional approaches to generic vehicle type recognition, this paper proposes an approach to fine-grained vehicle type recognition that leverages multiple cues jointly to convolutional neural networks (CNNs). Fine-grained vehicle recognition is inherently difficult with many challenges, including (i) how to incorporate multiple cues through joint optimization instead of separately handling, (ii) how to learn data-driven representation for vehicles instead of hand-crafted features and (iii) how to perceive and localize the vehicle region instead of using the whole image. Our multi-task CNNs localize vehicles in the first stage and recognize subclasses in the second stage, allowing us to handle samples with cluttered backgrounds and those where vehicles do not fill most of the image. Through collaborative feature learning from multiple tasks in each stage, our approach can handle subtle inter-class variations at the subordinate level. We also develop new vehicle-oriented data augmentation strategies in CNN training. To advance research on vehicle-centered tasks, we release a new vehicle dataset that provides both semantic labels and bounding boxes. This dataset can be used in vehicle localization, recognition, viewpoint estimation, and so on. Extensive experiments on two benchmark vehicle datasets demonstrate that our approach outperforms state-of-the-art algorithms.