Sense the pen: Classification of online handwritten sequences (text, mathematical expression, plot/graph)

Abstract

This paper has a threefold contribution. First, it presents a novel online handwriting database captured using a digital/sensor pen (Apple pencil) and digital/sensor screen (iPad). The captured data are continuous streams of multi-dimensional points, analyzed and processed to classify handwritten sequences into plain text, mathematical expressions, and plots/graphs. Second, a new feature set for online handwritten sequence classification is proposed. The said feature set is used to establish a benchmark for the proposed dataset using various machine-learning classifiers. Third, an ablation study is performed to look into the performance of the proposed feature set compared to the existing feature sets. Here, the proposed feature set has outperformed all the existing feature sets in various evaluation metrics. Furthermore, the proposed dataset and the feature set have been made publicly available along with the benchmark evaluation to enable further research in the field.