Classification and Localization of Arabic Handwritten Text in KHATT Dataset Based on Faster R-CNN

Abstract

Arabic handwriting recognition is an important research area in computer vision. Due to the complexity of the Arabic script, this is an arduous task. Several approaches have been proposed to address this challenge, including deep learning algorithms. Despite their efficiency, these approaches present some limitations such as the use of lexicon-driven models, the need of a lot of data for training and the huge computational cost. We propose, in this paper, two novel models based on the robust Faster Region-Convolution Neural Network (Faster R-CNN) to recognize Arabic handwritten sentences. The Faster R-CNN is commonly used to detect objects in images and depends on region proposal algorithms. These models use the pre-trained VGG 16 and ResNet50. Moreover, they use a soft non-maximum suppression strategy (Soft-NMS) in the post-processing stage instead of the traditional NMS to increase the detection rate of overlapping items. The models will be trained independently to identify words in the text lines and tested using sentences from the (KFUPM Handwritten Arabic TexT) KHATT dataset. It will be shown that the faster R-CNN models allow greater accuracy and effectiveness in handwriting recognition. They achieve accuracy rates of 99% and 98% for the two networks, VGG16 and ResNet50, respectively