Abstract
The paper proposes a new HCI mechanism for device-free gesture recognition on the table using acoustic signal, which can extend the gesture input and interactions beyond the tiny screen of mobile device and allow users to provide input without blocking screen view. Previous researches have either relied on additional devices (e.g., special wearable device and mouse) or required active acoustic signals which demand additional cost and be less prone to popularize, while we explore the device-free gesture recognition using passive acoustic signals. This technology is more challenging due to the lack of an effective approach to eliminate the inherent ambient noise disturbances and extract stable gesture features. We fuse both short time energy (STE) and zero-crossing rate (ZCR) to identify the effective signals from the original input, and leverage the Mel frequency cepstral coefficients (MFCC), cochlear filter cepstral coefficients (CFCC) to extract the stable features from different gestures. The unique features in support vector machine (SVM) classifier achieve a high gesture recognition accuracy from the noisy scenarios and mismatched conditions. Implementation on the Android system has realized real-time processing of the feature extraction and gesture recognition. Extensive evaluations show our algorithm has a better noise tolerant performance and the system could recognize seven common gestures (click, flip left/right, scroll up/down, zoom in/out) on smart devices with an accuracy of 93.2%.
Highlights
Smart devices such as smartphones and smartwatches have become pervasive and play a pivotal role in our daily life
In state 0 or state 1, for each frame of the acoustic signal x(n), when the short time energy E(n) > TL or the zero-crossing rate Z (n) > ZCR, the frame signal is considered as the starting point of the signal and the parameter count is increased by 1, otherwise, the frame of acoustic segment will be regarded as noise and the algorithm will cut it off, the frame of acoustic signal will return to state 0
Considering the human hearing system is robust to the noisy conditions, we combine an auditory-based feature extraction algorithm which is modeled on the basic signal processing functions in the ear named cochlear filter cepstral coefficients (CFCC) [22]
Summary
Smart devices such as smartphones and smartwatches have become pervasive and play a pivotal role in our daily life. To solve the above challenges, we first design a dual-threshold scheme to identify the acoustic signal of the user’s input gesture and separate it from the ambient noise. Compared with previous methods which are based on energy detection and preset threshold [19], [21], our dual-threshold segmentation scheme can identify and extract the acoustic signal corresponding to the user’s gesture more effectively, and reduce the interference of ambient noise. In state 0 or state 1, for each frame of the acoustic signal x(n), when the short time energy E(n) > TL or the zero-crossing rate Z (n) > ZCR, the frame signal is considered as the starting point of the signal and the parameter count is increased by 1, otherwise, the frame of acoustic segment will be regarded as noise and the algorithm will cut it off, the frame of acoustic signal will return to state 0.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
