A Novel Deep Learning Based ANPR Pipeline for Vehicle Access Control

Abstract

Computer Vision and Deep Learning technology are playing a key role in the development of Automatic Number Plate Recognition (ANPR) to achieve the goal of an Intelligent Transportation System (ITS). ANPR systems and pipelines presented in the literature often work on a specific layout of the number plate as every region has a unique plate configuration, font style, size, and layout formation. In this paper, we have developed a smart vehicle access control system considering a wide variety of plate formations and styles for different Asian and European countries and presented novel deep learning based ANPR pipeline that can be used for heterogeneous number plates. The presented improved ANPR pipeline detects vehicle front/rear view and subsequently localizes the number plate area using the YOLOv4 (You Only Look Once) object detection models. Further, an algorithm identifies the unique plate layout, which is either a single or double row layout in different countries, and the last step in the pipeline is to recognize the number plate label using a deep learning architecture (i.e., AlexNet or R-CNNL3). The results show that our trained YOLOv4 model for vehicle front/rear view detection achieves a 98.42% mAP score, and the number plate localization model achieves a 99.71% mAP score on a 0.50 threshold. The overall average plate recognition accuracy of our proposed deep learning-based ANPR pipeline using R-CNNL3 architecture achieved a single character recognition accuracy of 96%, while AlexNet architecture recognized a single character with a 98% accuracy. In contrast, the ANPR pipeline using the OCR method is found to be 90.94%, while latency is computed as 0.99 s/frame on Core i5 CPU and 0.42 s/frame on RTX 2060 GPU. The proposed ANPR system using a deep learning approach is preferred due to better accuracy, but it requires a high-performance GPU for real-time implementation. The presented pipeline is developed and implemented for smart vehicle access control, but it can be deployed for any ANPR application.