A Temporal Convolutional Network for modeling raw 3D sequences and air-writing recognition

Abstract

There has been a steady increase in gesture-based applications interacting with various electronic devices. Characters and numerals are written in the air using air-writing applications. Due to the lack of stroke information and a reference point on the writing plane in the 3D space, the recognition process of 3D writing is more complex than conventional 2D writing in a harsh environment. However, these complexities can be evaded with thorough modeling of the 3D trajectories. Temporal Convolutional Networks (TCN) have been proposed for an air-writing recognition system. TCNs are variations of convolutional neural network tasks involving sequence modeling. The methodology was applied to three publicly available datasets containing air-written digits and characters by various writers. The results demonstrated the effectiveness of the temporal networks in the recognition process of 3D characters. An accuracy of 99.50% and 99.56% was observed for digits and characters using the TCNs technique.