Contextual Stroke Classification in Online Handwritten Documents with Edge Graph Attention Networks

Abstract

The task of grouping strokes into different categories is an essential processing step in the automatic analysis of online handwritten documents. The technical challenge originates from the variation of the handwriting style, content heterogeneity and lack of prior layout knowledge. In this work, we propose the edge graph attention network (EGAT) to address the stroke classification problem. In this framework, the stroke classification problem is formulated as a node classification problem in a relational graph, which is constructed based on the temporal and spatial relationship of strokes. Then distributed node and edge features for classification are learned by stacking of multiple edge graph attention layers, in which various attention mechanisms are exploited to aggregate information between neighborhood nodes. In the task of text/nontext classification, the proposed model achieves accuracies 98.65% and 98.90% on the IAMOnDo and Kondate datasets, respectively. In the task of multi-class classification, the achieved accuracies are 95.81%, 97.36% and 99.05% on the IAMOnDo, FC and FA datasets, respectively. In addition, we conduct ablation experiments to quantitatively and qualitatively evaluate the key modules of our model.