Transcript mapping for handwritten Chinese documents by integrating character recognition model and geometric context

Abstract

Creating document image datasets with ground-truths of regions, text lines and characters is a prerequisite for document analysis research. However, ground-truthing large datasets is not only laborious and time consuming but also prone to errors due to the difficulty of character segmentation and the large variability of character shape, size and position. This paper describes an effective recognition-based annotation approach for ground-truthing handwritten Chinese documents. Under the Bayesian framework, the alignment of text line images with text transcript, which is the crucial step of annotation, is formulated as an optimization problem by incorporating geometric context of characters and character recognition model. We evaluated the alignment performance on a Chinese handwriting database CASIA-HWDB, which contains nearly four million character samples of 7356 classes and 5091 pages of unconstrained handwritten texts. The experimental results demonstrate the superiority of recognition-based text line alignment and the benefit of integrating geometric context. On a test set of 1015 handwritten pages (10,449 text lines), the proposed approach achieved character level alignment accuracy 92.32% when involving under-segmentation errors and 99.04% when excluding under-segmentation errors. The tool based on the proposed approach has been practically used for labeling handwritten Chinese documents.