An attention-based network for serial number recognition on banknotes

Abstract

The serial number recognition (SNR) on banknotes is essential for currency circulation. The performance of the existing SNR methods is significantly influenced by character segmentation, which is challenging due to uneven illumination and complex background. In this paper, we apply deep learning techniques to SNR by proposing an attention-based network, which can be end-to-end trained to avoid the problem of character segmentation. The proposed framework contains two parts: rectification and recognition. First, the rectification network, which can be trained in a weakly supervised manner without additional manual annotations, is built to automatically rectify the tilted and loosely-bounded images and reduce the difficulty of recognition. Then, the recognition network, an attention-based sequence model, recognizes serial number characters from the rectified image holistically, rather than segmenting and recognizing individual characters. To address the problem of complex textures on banknotes, we integrate the deformable convolution into the recognition network, which adaptively focuses on the character regions by using flexible receptive fields to accurately extract optimal character features, while ignoring redundant background information. Extensive experiments conducted on CNY, KRW, EUR and JPY banknotes, demonstrate that the proposed method achieves higher accuracy than the existing methods.