Neuromuscular Representation and Synthetic Generation of Handwritten Whiteboard Notes

Abstract

A fully automatic framework has been introduced recently for neuromuscular representation of complex handwriting patterns, such as gestures, signatures, and words, based on the Kinematic Theory of rapid human movements and its Sigma-Lognormal model. In this paper, we investigate the application of this framework to unconstrained whiteboard notes, taking into account a novel acquisition modality, multiple writers, natural language, and complete text lines. Although these conditions deviate strongly from the previously considered scenario of brief pen movements on tablet computers, we demonstrate that the Sigma-Lognormal model is still able to represent the handwriting accurately. In order to deal with longer handwriting patterns, we propose a robust component-wise representation of text lines that achieves a high model quality. Furthermore, we propose a stroke-wise distortion method to generate synthetic text lines from the Sigma-Lognormal representation of real specimens. For handwriting recognition on the IAM online database, it is demonstrated that the extension of the training set with the proposed synthesis method significantly increases current benchmark results achieved with recurrent neural networks.