Sequence-to-Sequence Learning Using Deep Learning for Optical Character Recognition (OCR)

Abstract

In this paper, Convolution Neural Network (CNN) and a special variant of Recurrent Neural Network (RNN) named Long Short-Term Memory Model (LSTM) with peep hole connection is developed for optical character recognition (OCR). Data-set of mathematical equations known as Image to Latex 100K is retrieved from OPEN-AI and used for testing the model. First, the mathematical equations from the images are converted to Latex texts. Then this Latex text is used to render the mathematical equations. The proposed method uses the tokenized data, which is sequentially given to the deep learning network. The sequential process helps the algorithms to keep track of the processed data and yield high accuracy. A new variant of called LSTM with peephole connections and Stochastic Hard Attention model was used. The performance of the proposed deep learning neural network is compared with INFTY (which uses no RNN) and WYGIWYS (which uses RNN). The proposed algorithm gives a better accuracy of 76% as compared to 74% achieved by WYGIWYS.