Offline HWR Accuracy Enhancement with Image Enhancement and Deep Learning Techniques

Abstract

Handwriting recognition (HWR) is the ability of a machine to recognize handwritten text present in images, scanned or photographed. While much of the work reported in this field deals with scanned images, recognizing handwritten content from camera-captured images is still a challenge. While researchers are focusing their efforts to improve on the recognition process of such characters, the current work focuses efforts outside the recognition process.This work employs image enhancement techniques, noise removal, and binarization with adaptive thresholding & segmentation, to increase HWR accuracy. Additionally, the work of employs deep learning-based language models, such as BERT & BART, to further enhance the accuracy of recognized text. It is observed that HWR accuracy increases when smaller images such as word images are used as input to the HWR engine. On the post-processing front, BART is demonstrated to be superior in enhancing the accuracy of HWR-recognized text documents in a comparative study presented in this work. An accuracy improvement of 20+% is achieved with the BART model enhanced with transfer learning through a localized dataset to handle HWR-specific error patterns.