Vehicle Make And Model Recognition Research Articles

Automated license plate authentication framework using multi-view vehicle images

The current framework for detecting Fake License Plates (FLP) in real-time is not robust enough for patrol teams. The objective of this paper is to develop a robust license plate authentication framework, based on the Vehicle Make and Model Recognition (VMMR) and the License Plate Recognition (LPR) algorithms that is implementable at the edge devices. The contributions of this paper are (i) Development of license plate database for 547 Indian cars, (ii) Development of an image dataset with 3173 images of 547 Indian cars in 8 classes, (iii) Development of an ensemble model to recognize vehicle make and model from frontal, rear, and side images, and (iv) Development of a framework to authenticate the license plates with frontal, rear, and side images. The proposed ensemble model is compared with the state-of-the-art networks from the literature. Among the implemented networks for VMMR, the Ensembling model with a size of 303.2 MB achieves the best accuracy of 89% . Due to the limited memory size, Easy OCR is chosen to recognize license plate. The total size of the authentication framework is 308 MB. The performance of the proposed framework is compared with the literature. According to the results, the proposed framework enhances FLP recognition due to the recognition of vehicles from side images. The dataset is made public at https://www.kaggle.com/ganeshmailecture/datasets.

A Novel FDLSR-Based Technique for View-Independent Vehicle Make and Model Recognition.

Vehicle make and model recognition (VMMR) is an important aspect of intelligent transportation systems (ITS). In VMMR systems, surveillance cameras capture vehicle images for real-time vehicle detection and recognition. These captured images pose challenges, including shadows, reflections, changes in weather and illumination, occlusions, and perspective distortion. Another significant challenge in VMMR is the multiclass classification. This scenario has two main categories: (a) multiplicity and (b) ambiguity. Multiplicity concerns the issue of different forms among car models manufactured by the same company, while the ambiguity problem arises when multiple models from the same manufacturer have visually similar appearances or when vehicle models of different makes have visually comparable rear/front views. This paper introduces a novel and robust VMMR model that can address the above-mentioned issues with accuracy comparable to state-of-the-art methods. Our proposed hybrid CNN model selects the best descriptive fine-grained features with the help of Fisher Discriminative Least Squares Regression (FDLSR). These features are extracted from a deep CNN model fine-tuned on the fine-grained vehicle datasets Stanford-196 and BoxCars21k. Using ResNet-152 features, our proposed model outperformed the SVM and FC layers in accuracy by 0.5% and 4% on Stanford-196 and 0.4 and 1% on BoxCars21k, respectively. Moreover, this model is well-suited for small-scale fine-grained vehicle datasets.

DeepCar 5.0: Vehicle Make and Model Recognition Under Challenging Conditions

Inspired by multi-agent systems (MASs) and ensemble models, a novel method of vehicle make and model recognition (VMMR) based on a vehicle’s front-view images is presented in this paper. By exploiting the attention mechanism in this work, it is demonstrated that most of the features used for classifying a vehicle are extracted from its headlight, grill, scoop, and bumper sections. These areas are designated as the regions of interest (ROIs) in our approach. Contrary to the other methods in which a whole ROI is fed to a deep convolutional neural network, in this scheme, different ROIs are extracted from each image, and then a preprocessing block and a distinctive network are designed for each ROI, which is considered as a single agent. Each agent is then trained separately, and a vehicle’s type is determined with the collaboration of these agents and based on the blackboard classification system. Also, a new dataset (DeepCar 5.0) is compiled by using the data from the top 50 automakers. This dataset contains 40,185 images of the front views and the front three-quarters of vehicles in 480 different classes, and all the parts of dataset are labeled manually. The proposed technique is able to achieve the accuracies of 92.14 and 96.72% in the automated and the manual scenarios, respectively; and contrary to the current methods, it can perform flawless classification even when just a portion of a vehicle’s front view image is available for processing. The dataset and parts of the code are available at: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/DeepCar/DeepCar5.0</uri> .

Framework for Vehicle Make and Model Recognition-A New Large-Scale Dataset and an Efficient Two-Branch-Two-Stage Deep Learning Architecture.

In recent years, Vehicle Make and Model Recognition (VMMR) has attracted a lot of attention as it plays a crucial role in Intelligent Transportation Systems (ITS). Accurate and efficient VMMR systems are required in real-world applications including intelligent surveillance and autonomous driving. The paper introduces a new large-scale dataset and a novel deep learning paradigm for VMMR. A new large-scale dataset dubbed Diverse large-scale VMM (DVMM) is proposed collecting image-samples with the most popular vehicle brands operating in Europe. A novel VMMR framework is proposed which follows a two-branch architecture performing make and model recognition respectively. A two-stage training procedure and a novel decision module are proposed to process the make and model predictions and compute the final model prediction. In addition, a novel metric based on the true positive rate is proposed to compare classification confusion of the proposed 2B-2S and the baseline methods. A complex experimental validation is carried out, demonstrating the generality, diversity, and practicality of the proposed DVMM dataset. The experimental results show that the proposed framework provides 93.95% accuracy over the more diverse DVMM dataset and 95.85% accuracy over traditional VMMR datasets. The proposed two-branch approach outperforms the conventional one-branch approach for VMMR over small-, medium-, and large-scale datasets by providing lower vehicle model confusion and reduced inter-make ambiguity. The paper demonstrates the advantages of the proposed two-branch VMMR paradigm in terms of robustness and lower confusion relative to single-branch designs.

A Novel Smart Lightweight Visual Attention Model for Fine-Grained Vehicle Recognition

Vehicle Make and Model Recognition (VMMR) requires fast and accurate recognition of a vehicle’s information. Generally, the vision-based VMMR method recognizes different vehicle models that mainly rely on locating and extracting the discriminative part features of a vehicle. In this paper, we propose a Lightweight Recurrent Attention Unit (LRAU) to enhance the feature extraction ability of the standard Convolutional Neural Network (CNN) architectures for VMMR. The proposed LRAU extracts the discriminative part features by generating attention masks to locate the keypoints of a vehicle (e.g., logo, headlight). The attention mask is generated based on the feature maps received by the LRAU and the preceding attention state generated by the preceding LRAU. By adding LRAUs to receive the multi-scale feature maps generated by the standard CNN architecture, discriminative features of different scales can be efficiently extracted and combined. We conduct comprehensive experiments on three challenging VMMR datasets to evaluate the proposed VMMR models. Experimental results show our models have a stable performance under different environmental conditions. Our models achieve state-of-the-art results with 93.94% accuracy on the Stanford Cars dataset, 98.31% accuracy on the CompCars dataset, and 99.41% accuracy on the NTOU-MMR dataset. Moreover, we demonstrate that our models outperform the traditional machine learning-based VMMR models in terms of recognition accuracy and processing speed. In addition, we construct a one-stage Vehicle Detection and Fine-grained Recognition (VDFR) model by combining our LRAU with the general object detection model. Results show the proposed VDFR model can achieve excellent performance with real-time processing speed.

Hierarchical Scheme for Vehicle Make and Model Recognition

A vehicle make and model recognition (VMMR) system is a common requirement in the field of intelligent transportation systems (ITS). However, it is a challenging task because of the subtle differences between vehicle categories. In this paper, we propose a hierarchical scheme for VMMR. Specifically, the scheme consists of (1) a feature extraction framework called weighted mask hierarchical bilinear pooling (WMHBP) based on hierarchical bilinear pooling (HBP) which weakens the influence of invalid background regions by generating a weighted mask while extracting features from discriminative regions to form a more robust feature descriptor; (2) a hierarchical loss function that can learn the appearance differences between vehicle brands, and enhance vehicle recognition accuracy; (3) collection of vehicle images from the Internet and classification of images with hierarchical labels to augment data for solving the problem of insufficient data and low picture resolution and improving the model’s generalization ability and robustness. We evaluate the proposed framework for accuracy and real-time performance and the experiment results indicate a recognition accuracy of 95.1% and an FPS (frames per second) of 107 for the framework for the Stanford Cars public dataset, which demonstrates the superiority of the method and its availability for ITS.

Vision-based Autonomous Vehicle Recognition

Vision-based Automated Vehicle Recognition (VAVR) has attracted considerable attention recently. Particularly given the reliance on emerging deep learning methods, which have powerful feature extraction and pattern learning abilities, vehicle recognition has made significant progress. VAVR is an essential part of Intelligent Transportation Systems. The VAVR system can fast and accurately locate a target vehicle, which significantly helps improve regional security. A comprehensive VAVR system contains three components: Vehicle Detection (VD), Vehicle Make and Model Recognition (VMMR), and Vehicle Re-identification (VRe-ID). These components perform coarse-to-fine recognition tasks in three steps. In this article, we conduct a thorough review and comparison of the state-of-the-art deep learning--based models proposed for VAVR. We present a detailed introduction to different vehicle recognition datasets used for a comprehensive evaluation of the proposed models. We also critically discuss the major challenges and future research trends involved in each task. Finally, we summarize the characteristics of the methods for each task. Our comprehensive model analysis will help researchers that are interested in VD, VMMR, and VRe-ID and provide them with possible directions to solve current challenges and further improve the performance and robustness of models.

Are We Ready for Accurate and Unbiased Fine-Grained Vehicle Classification in Realistic Environments?

Fine-grained vehicle classification from images, also known as Vehicle Make and Model Recognition (VMMR), has become an important research topic in the last years, with a growing number of scientific contributions in multiple application areas, such as autonomous vehicles, surveillance systems, traffic monitoring and management, among others. Recent techniques based on deep learning have proven to be very effective in addressing this problem. So effective that, based on the state-of-the-art results (above 95% accuracy), it would seem that the problem is practically solved. However, our main hypothesis is that the existing datasets to date have limited variability, which precludes good and unbiased generalisation of the models trained with them. In particular, it is observed that the test datasets are very similar in nature to those used for training and validation which makes these benchmarks prone to dataset bias and to overfitting. When these systems are tested with more challenging data or data from different datasets performance degrades considerably. In this paper, on the one hand, we evaluate state-of-the-art deep learning models to perform fine-grained vehicle classification and explore multiple training techniques, such as curriculum learning or weighted losses, to mitigate the bias between different makes and models and to assess the limits of current approaches. On the other hand, we analyse the existing datasets, present an additional dataset from a challenging scenario, and merge all the data into a cross-dataset that includes common samples and classes from the existing datasets. In this way, we can evaluate geographical, make and model biases, and performance and generalisation capabilities from a more realistic perspective. The obtained results suggest that we are still far from accurate and unbiased vehicle make and model recognition in realistic traffic and driving scenarios.

Vehicle Make and Model Recognition using Bag of Expressions.

Vehicle make and model recognition (VMMR) is a key task for automated vehicular surveillance (AVS) and various intelligent transport system (ITS) applications. In this paper, we propose and study the suitability of the bag of expressions (BoE) approach for VMMR-based applications. The method includes neighborhood information in addition to visual words. BoE improves the existing power of a bag of words (BOW) approach, including occlusion handling, scale invariance and view independence. The proposed approach extracts features using a combination of different keypoint detectors and a Histogram of Oriented Gradients (HOG) descriptor. An optimized dictionary of expressions is formed using visual words acquired through k-means clustering. The histogram of expressions is created by computing the occurrences of each expression in the image. For classification, multiclass linear support vector machines (SVM) are trained over the BoE-based features representation. The approach has been evaluated by applying cross-validation tests on the publicly available National Taiwan Ocean University-Make and Model Recognition (NTOU-MMR) dataset, and experimental results show that it outperforms recent approaches for VMMR. With multiclass linear SVM classification, promising average accuracy and processing speed are obtained using a combination of keypoint detectors with HOG-based BoE description, making it applicable to real-time VMMR systems.

Component-based feature extraction and representation schemes for vehicle make and model recognition

This paper presents a novel feature representation and recognition scheme for vehicle make and model recognition (VMMR) from the frontal image of the vehicle. In general, some domain knowledge is introduced and further exploited to accomplish this task. The modular components in the frontal appearance of vehicles present distinct visual characteristics, and the varying discrimination ability of them made the main discriminant region changes when vehicles compared at inter- or intra-brand level. Inspired by the peculiar vehicle properties, this paper focuses on the localized visual characteristics in the discriminant subregions, and make the representation of the frontal vehicle in a multi-scale spatial manner. This representation scheme encodes local spatial information and component-specific characteristics to the feature descriptors, which enhances the discrimination ability of the feature representation and mitigates the multiplicity problem of VMMR to some extent. Extensive experiments on a large-scale vehicle image dataset have shown the efficiency of the methods in this work.

Real-Time Vehicle Make and Model Recognition Using Unsupervised Feature Learning

Vehicle Make and Model Recognition (VMMR) systems provide a fully automatic framework to recognize and classify different vehicle models. Several approaches have been proposed to address this challenge; however, they can perform in restricted conditions. Here, in this paper, we formulate the VMMR as a fine-grained classification problem and propose a new configurable on-road VMMR framework. We benefit from the unsupervised feature learning methods, and in more details, we employ Locality-constraint Linear Coding (LLC) method as a fast feature encoder for encoding the input SIFT features. The proposed method can perform in real environments of different conditions. This framework can recognize 50 models of vehicles and has the advantage to classify every other vehicle not belonging to one of the specified 50 classes as an unknown vehicle. The proposed VMMR framework can be configured to become faster or more accurate based on the application domain. The proposed approach is examined on two datasets, including Iranian on-road vehicle (IORV) dataset and CompuCar dataset. The IORV dataset contains images of 50 models of vehicles captured in real situations by traffic-cameras in different weather and lighting conditions. The experimental results show the advantage of the real-time configuration of the proposed framework over the state-of-the-art methods on the IORV datatset and comparable results on CompuCar dataset with 97.5% and 98.4% accuracies, respectively and acceptable running time.

A Hierarchical Scheme for Vehicle Make and Model Recognition From Frontal Images of Vehicles

This paper presents a novel recognition scheme for vehicle make and model recognition (VMMR) from frontal images of vehicles. In general, we introduce some domain knowledge to cope with this task. The structural components contained in the frontal appearance of vehicles present different visual characteristics and their discriminating ability varies when vehicle models belonging to the same brand or different brands are compared. In light of the particularities, we take advantage of the varying discriminating ability of these structural components to perform the recognition task sequentially in two stages. At the first stage, the logo sub-region (which is one of the component-related sub-regions in the region of interest) is applied to classify the vehicle models at the brand level. Different from the traditional brand-level classification that the models of the same brand are considered as a single class, in this paper, multiple sub-classes in one brand class are allowed, since the intra-brand models also exhibit a certain degree of diversity. In this way, the problem of inter-class similarity is remitted. At the second stage, several customized classifiers are trained for each sub-class in the light of the discriminant ability of the remaining sub-regions. The proposed approach has been tested on a large-scale vehicle image database collected in this paper and has achieved the state-of-the-art results.

Real-Time Vehicle Make and Model Recognition System

A Vehicle Make and Model Recognition (VMMR) system can provide great value in terms of vehicle monitoring and identification based on vehicle appearance in addition to the vehicles’ attached license plate typical recognition. A real-time VMMR system is an important component of many applications such as automatic vehicle surveillance, traffic management, driver assistance systems, traffic behavior analysis, and traffic monitoring, etc. A VMMR system has a unique set of challenges and issues. Few of the challenges are image acquisition, variations in illuminations and weather, occlusions, shadows, reflections, large variety of vehicles, inter-class and intra-class similarities, addition/deletion of vehicles’ models over time, etc. In this work, we present a unique and robust real-time VMMR system which can handle the challenges described above and recognize vehicles with high accuracy. We extract image features from vehicle images and create feature vectors to represent the dataset. We use two classification algorithms, Random Forest (RF) and Support Vector Machine (SVM), in our work. We use a realistic dataset to test and evaluate the proposed VMMR system. The vehicles’ images in the dataset reflect real-world situations. The proposed VMMR system recognizes vehicles on the basis of make, model, and generation (manufacturing years) while the existing VMMR systems can only identify the make and model. Comparison with existing VMMR research demonstrates superior performance of the proposed system in terms of recognition accuracy and processing speed.

Vehicle joint make and model recognition with multiscale attention windows

Vehicle Make and Model Recognition (VMMR) deals with the problem of classifying vehicles whose appearance may vary significantly when captured from different perspectives. A number of successful approaches to this problem rely on part-based models, requiring however labor-intensive parts annotations. In this work, we address the VMMR problem proposing a deep convolutional architecture built upon multi-scale attention windows. The proposed architecture classifies a vehicle over attention windows which are predicted to minimize the classification error. Through these windows, the visual representations of the most discriminative part of the vehicle are aggregated over different scales which in fact provide more representative features for the classifier. In addition, we define a loss function accounting for the joint classification error across make and model. Besides, a training methodology is devised to stabilize the training process and to impose multi-scale constraints on predicted attention windows. The proposed architecture outperforms state-of-the-art schemes reducing the model classification error over the Stanford dataset by 1.7 % and improving the classification accuracy by 0.2 % and 0.3 % on model and make respectively over the CompCar dataset.

A Cascaded Part-Based System for Fine-Grained Vehicle Classification

Vehicle make and model recognition (VMMR) has become an important part of intelligent transportation systems. VMMR can be useful when license plate recognition is not feasible or fake number plates are used. VMMR is a hard, fine-grained classification problem, due to the large number of classes, substantial inner-class, and small inter-class distance. A novel cascaded part-based system has been proposed in this paper for VMMR. This system uses latent support vector machine formulation for automatically finding the discriminative parts of each vehicle category. At the same time, it learns a part-based model for each category. Our approach employs a new training procedure, a novel greedy parts localization, and a practical multi-class data mining algorithm. In order to speed up the system processing time, a novel cascading scheme has been proposed. This cascading scheme applies classifiers to the input image in a sequential manner, based on the two proposed criteria: confidence and frequency. The cascaded system can run up to 80% faster with analogous accuracy in comparison with the non-cascaded system. The extensive experiments on our data set and the CompCars data set indicate the outstanding performance of our approach. The proposed approach achieves an average accuracy of 97.01% on our challenging data set and an average accuracy of 95.55% on CompCars data set.

A cascading scheme for speeding up multiple classifier systems

The accuracy of multi-class classification problems is improving at a good pace. However, improving the accuracy often leads to slowing down the processing speed. Since employing a large number of classifiers or a combination of them is a time-consuming process, the sluggish behavior is more evident in multiple classifier systems. In this paper, a practical cascading scheme is proposed for boosting the speed of a multiple binary classifier system with no noticeable reduction in recognition rate. The proposed cascading scheme sets a sequence of binary classifiers and applies one classifier at a time to the input. Some effective criteria for a practical ordering of classifiers are introduced, and a fusion of them is verified to be the best. A vehicle make and model recognition (VMMR) system with multiple individual classifiers is presented briefly as a use case for a multi-class classification problem. The experiments done on this VMMR system using two completely different datasets confirm the effectiveness of our scheme. One of the configurations of the proposed scheme results in up to 30% speedup in comparison with the baseline VMMR system with analogous recognition rate. Another configuration of the proposed cascading scheme achieves up to 80% speedup with just a minor drop in accuracy.

Part‐based recognition of vehicle make and model

Fine-grained recognition is a challenge that the computer vision community faces nowadays. The main category of the object is known in this problem and the goal is to determine the subcategory or fine-grained category. Vehicle make and model recognition (VMMR) is a hard fine-grained classification problem, due to the large number of classes, substantial inner-class and small inter-class distance. In this study, a novel approach has been proposed for VMMR based on latent SVM formulation. This approach automatically finds a set of discriminative parts in each class of vehicles by employing a novel greedy parts localisation algorithm, while learning a model per class using both features extracted from these parts and the spatial relationship between them. An effective and practical multi-class data mining method is proposed to filter out hard negative samples in the training procedure. Employing these trained individual models together, the authors’ system can classify vehicles make and model with a high accuracy. For evaluation purposes, a new dataset including more than 5000 vehicles of 28 different makes and models has been collected and fully annotated. The experimental results on this dataset and the CompCars dataset indicate the outstanding performance of the authors’ approach.

Real-Time Vehicle Make and Model Recognition Based on a Bag of SURF Features

In this paper, we propose and evaluate unexplored approaches for real-time automated vehicle make and model recognition (VMMR) based on a bag of speeded-up robust features (BoSURF) and demonstrate the suitability of these approaches for vehicle identification systems. The proposed approaches use SURF features of vehicles' front- or rear-facing images and retain the dominant characteristic features ( codewords ) in a dictionary . Two schemes of dictionary building are evaluated: “ single dictionary ” and “ modular dictionary. ” Based on the optimized dictionaries, the SURF features of vehicles' front- or rear-face images are embedded into BoSURF histograms, which are used to train multiclass support vector machines (SVMs) for classification. Two real-time VMMR classification schemes are proposed and evaluated: a single multiclass SVM and an ensemble of multiclass SVM based on attribute bagging. The processing speed and accuracy of the VMMR system are affected greatly by the size of the dictionary. The tradeoff between speed and accuracy is studied to determine optimal dictionary sizes for the VMMR problem. The effectiveness of our approaches is demonstrated through cross-validation tests on a recent publicly accessible VMMR data set. The experimental results prove the superiority of our work over the state of the art, in terms of both processing speed and accuracy, making it highly applicable to real-time VMMR systems.

Local Tiled Deep Networks for Recognition of Vehicle Make and Model.

Vehicle analysis involves license-plate recognition (LPR), vehicle-type classification (VTC), and vehicle make and model recognition (MMR). Among these tasks, MMR plays an important complementary role in respect to LPR. In this paper, we propose a novel framework for MMR using local tiled deep networks. The frontal views of vehicle images are first extracted and fed into the local tiled deep networks for training and testing. A local tiled convolutional neural network (LTCNN) is proposed to alter the weight sharing scheme of CNN with local tiled structure. The LTCNN unties the weights of adjacent units and then ties the units k steps from each other within a local map. This architecture provides the translational, rotational, and scale invariance as well as locality. In addition, to further deal with the colour and illumination variation, we applied the histogram oriented gradient (HOG) to the frontal view of images prior to the LTCNN. The experimental results show that our LTCNN framework achieved a 98% accuracy rate in terms of vehicle MMR.

Comparative Analysis of Vehicle Make and Model Recognition Techniques

Vehicle Make and Model Recognition (VMMR) has emerged as a significant element of vision based systems because of its application in access control systems, traffic control and monitoring systems, security systems and surveillance systems, etc. So far a number of techniques have been developed for vehicle recognition. Each technique follows different methodology and classification approaches. The evaluation results highlight the recognition technique with highest accuracy level. In this paper we have pointed out the working of various vehicle make and model recognition techniques and compare these techniques on the basis of methodology, principles, classification approach, classifier and level of recognition. After comparing these factors we concluded that Locally Normalized Harris Corner Strengths (LHNS) performs best as compared to other techniques. LHNS uses Bayes and K-NN classification approaches for vehicle classification. It extracts information from frontal view of vehicles for vehicle make and model recognition.