Further explorations in text alignment with handwritten documents

Abstract

Today's digital libraries increasingly include not only printed text but also scanned handwritten pages and other multimedia material. There are, however, few tools available for manipulating handwritten pages. Here, we extend our algorithm from [5]based on dynamic time warping (DTW) for a word by word alignment of handwritten documents with (ASCII) transcripts. We specifically attempt to incorporate language modelling and parameter training into our algorithm. In addition, we take a critical look at our evaluation metrics. We see at least three uses for such alignment algorithms. First, alignment algorithms allow us to produce displays (for example on the web) that allow a person to easily find their place in the manuscript when reading a transcript. Second, such alignment algorithms will allow us to produce large quantities of ground truth data for evaluating handwriting recognition algorithms. Third, such algorithms allow us to produce indices in a straightforward manner for handwriting material. We provide experimental results of our algorithm on a set of 100 pages of historical handwritten material–specifically the writings of George Washington. Our method achieves average F-measure values of 68.3 online by line alignment and 57.8 accuracy when aligning whole pages at time.