Magic-hand: Turn a smartwatch into a mouse

Abstract

We present the design, implementation and evaluation of a smartwatch-based, freehand human–computer interaction system, called Magic-Hand, which consists of two modules: the operation detection module and the motion tracking module. In the operation detection module, five gestures are selected to represent typical operations including right-click, left-click and double-click, and we carefully design the detection algorithm to identify gestures even when the hand is in motion. In the motion tracking module, we map the movement of the hand to the screen cursor movement with the Sigmoid function. We verify the performance of Magic-Hand by comparing it with two existing human–computer interaction techniques: the mouse and the AirMouse. The experimental results show that the gesture recognition in the operation detection module can reach an accuracy of 95%, and the motion tracking is comparable with the benchmarks. Our work facilitates the use of the smartwatch as an effective interaction device in ubiquitous display environments.