In-air handwritten Chinese text recognition with temporal convolutional recurrent network

Abstract

As a new human-computer interaction way, in-air handwriting allows users to perform gesture-based writing in the midair. However, most existing in-air handwriting systems mainly focus on recognizing either isolated characters/words or only a small number of texts, making those systems far from practical applications. Instead, here we present a 3D in-air handwritten Chinese text recognition (IAHCTR) system for the first time, and construct the first public large-scale IAHCT dataset. Moreover, a novel architecture, named the temporal convolutional recurrent network (TCRN), is proposed for online HCTR. Specifically, the TCRN first applies the 1-dimensional convolution to extract local contextual features from low-level trajectories, and then it utilizes the recurrent network to capture long-term dependencies of high-level outputs. Compared with the state-of-the-art architecture, the TCRN not only avoids the domain-specific knowledge for feature image extraction, but also attains higher training efficiency with a more compact model. Empirically, this TCRN also outperforms the single recurrent network with faster prediction and higher accuracy. Experiments on CASIA-OLHWDB2 & ICDAR-2013 demonstrate that the TCRN yields the best result in comparison to the state-of-the-art methods for online HCTR.