Robust license plate detection and recognition with automatic rectification

Abstract

We propose a robust license plate detection and recognition (LPDR) framework with automatic rectification. We explore the YOLOv2 object detector based on deep learning and train it to detect license plates (LPs) effectively. The LPs in natural scene images tend to be tilted and distorted because of the shooting angle or the geometric deformation of LPs. To solve the problem in which the LP tilt and distortion affect recognition accuracy, we introduce a spatial transformation network with thin-plate-spline transformation and propose a neural network called inverse compositional spatial transformer network-hierarchical spatial transformer network (ICSTN-CRNN). ICSTN-CRNN can automatically rectify and recognize LPs. Furthermore, we manually supplement the LP character annotations in PKUData. Our LPDR method achieves satisfactory results on three datasets, including Chinese City Parking Dataset, PKUData, and application-oriented license plate. Through a series of comparative experiments, we prove that our method is more accurate than other advanced methods.