Ultrasonic hand gesture recognition for mobile devices

Abstract

This paper describes a novel system for ultrasonic gesture recognition targeted at handheld devices, such as smartphones. Unlike previous proposals, the system obtains high update rate range estimates for the user’s hand. The range of the user’s hand is determined based on the Round Trip Time of ultrasonic pulses emitted by a transducer on the device, reflected by the hand and received at multiple sensors on the device. The range estimates, coupled with other information extracted from the reflected ultrasonic signals, are used for gesture recognition. Gesture recognition is performed by means of a multi-class hierarchical binary support vector machine. The high update rate is enabled by the use of compact wideband transducers. The ultrasonic pulses are short in duration and utilize Linear Frequency Modulation compression to achieve high resolution in Time Of Arrival estimation. The impact of multipath is reduced by the use of Frequency Hopping. A system prototype using one transmitter and four receivers was found to achieve gesture detection sensitivity and specificity of 99% and 99%, respectively, and classification accuracy of 96% for 7 users (5 males, 2 females) with around 500 repetitions per user for a set of 7 gesture types.