An End-to-End Recognition System for Unconstrained Vietnamese Handwriting

Abstract

Inspired by recent successes in neural machine translation and image caption generation, we present an attention-based encoder–decoder model (AED) to recognize handwritten Vietnamese text. The model consists of two parts: a Convolutional Neural Network-based (CNN) encoder and a Long Short-Term Memory-based (LSTM) decoder. The encoder is based on DenseNet for extracting invariant features. The LSTM-based decoder with an attention model incorporated generates output text. The input of the encoder is a handwritten text image and the target of the decoder is the corresponding text of the input image. Our model is trained end-to-end to predict the text from a given input image since all the parts are differential components. In the experiment section, we evaluate our proposed AED model on the VNOnDB-Word and VNOnDB-Line datasets to verify its efficiency. The experiential results show that our model achieves 4.10% of character error eate (CER) and 10.24% of word error rate (WER) on the testing set of VNOnDB-Word and 4.67% of CER and 13.33% of WER on the testing set of VNOnDB-Line without using any language model. This result is higher than our previous system, Nguyen et al.’s system, and the system provided by Google in the Vietnamese Online Handwritten Text Recognition competition.