Deep corner prediction to rectify tilted license plate images

Abstract

Skewness and obliqueness of vehicle plate images influence license plate recognition. The more tilted plate images are, the harder the recognition task is. To this end, if plate images are preprocessed to be aligned and rectified, the recognition performance would improve. We propose deep neural network models that can locate four corner plate positions, which can then be used to perform the perspective transformation that can be used to rectify plates. Such a transformation is called homography. The models consist of two sequential parts: a feature extraction part having convolution and a regression part with fully connected layers. The models are open in the sense that the feature extraction part can host other well-known models such as Mobilenet as long as they have the feature capture capability. We devise a loss function as the sum of Euclidean distance between predicted coordinates and ground truth and discuss image augmentation schemes. The experiment results show that the models with well-known object detection models are able to predict corner positions with relatively high precision.