IDial

Abstract

Smart wearable devices have become pervasive and are playing a more important role in our everyday lives. However, the small screen size and very few buttons make the interaction and control cumbersome and inconvenient. Previous solutions to mitigate this problem either require extra dedicated hardware, or instrument the user's fingers with special purpose sensors, limiting their real-life applications. We present iDial, a novel real time approach that enables a virtual dial plate on the hand back, extending the interaction beyond the small screen of wearable devices. iDial only employs the already built-in microphone and motion sensors of the commercial-off-the-shelf (COTS) device to facilitate interactions between user and wearable, without any extra hardware involved. The key idea is to exploit the acoustic signatures extracted from passive subtle acoustic signals to accurately recognize the virtual keys input on the skin of the hand back. We innovatively locate the virtual keys on the 4 pieces of metacarpal bones to significantly reduce the possibility of casual inputs. iDial also takes advantages of the motion sensor fusion already available inside the wearable to achieve robustness against the ambient noise and human voices efficiently. We design and implement iDial on the Samsung Gear S2 smartwatch. Our extensive experiments show that iDial is able to achieve an average recognition accuracy of 96.7%, and maintain high accuracies across varying user behaviors and different environments. iDial achieves a below 0.5s end-to-end latency with all the computations and processes happening at the cheap commodity wearable.