Vehicle license plate detection and recognition using deep neural networks and generative adversarial networks

Abstract

This paper presents a deep learning-based framework for automatic license plate detection and recognition in nature scene images. To start with, a small model is developed for license plate detection, based on cascaded convolutional neural network (CNN). The CNN cascade works on multiple levels. The early levels quickly scan low-resolution candidate windows and reject most of the nonplate regions, and the late levels carefully evaluate a small number of candidate windows in high-resolution. The detected candidate regions are cropped from the original image for recognition. Next, we treat plate recognition as a sequence labeling problem and use a combination of CNN and recurrent neural network for feature extraction and learning. The output result is then decoded to a readable character sequence using a connectionist temporal classification layer. This plate recognition model is segment-free and can be trained end-to-end. Finally, the generative adversarial network is employed to automatically generate image samples for training the plate recognition model. Experimental results on extensive datasets prove the effectiveness and efficiency of the proposed framework.