STAN: A sequential transformation attention-based network for scene text recognition

Abstract

Scene text with an irregular layout is difficult to recognize. To this end, a Sequential Transformation Attention-based Network (STAN), which comprises a sequential transformation network and an attention-based recognition network, is proposed for general scene text recognition. The sequential transformation network rectifies irregular text by decomposing the task into a series of patch-wise basic transformations, followed by a grid projection submodule to smooth the junction between neighboring patches. The entire rectification process is able to be trained in an end-to-end weakly supervised manner, requiring only images and their corresponding groundtruth text. Based on the rectified images, an attention-based recognition network is employed to predict a character sequence. Experiments on several benchmarks demonstrate the state-of-the-art performance of STAN on both regular and irregular text.