Handwriting Recognition System Leveraging Vibration Signal on Smartphones

Abstract

The efficiency of human-computer interaction is greatly hindered by the small size of the touch screens on mobile devices, such as smart phones and watches. This has prompted widespread interest in handwriting recognition systems, which can be divided into active and passive systems. Active systems require additional hardware devices to perceive movements of handwriting or the tracking accuracy is not adequate for handwriting recognition. Passive methods use the acoustic signal of pen rubbing and are susceptible to environmental noise (above 60dB). This paper presents a novel handwriting recognition system based on vibration signals detected by the built-in accelerometer of smartphones. The proposed scheme is implemented in three stages: signal segmentation, signal recognition, and word suggestion. VibWriter is highly resistant to interference since the normal environmental noise (below 70dB) will not cause the vibration of the accelerometer. Extensive experiments demonstrated the efficacy of the system in terms of accuracy in letter recognition (75.3%), word recognition (86.4%) and number recognition (79%) in a variety of writing positions under a variety of environmental conditions.