Head mounted device for gesture recognition using acoustic time of flight sensor and 2D camera

Abstract

The goal of this work is to design a 3D imaging system with acoustical transducers, 2D cameras and real-time signal processing to enable gesture sensing for a Head Mounted Device (HMD) design. The system offers enhanced user interfaces using vivid spatial information for use on mobile platforms such as wearable electronics. Through an intuitive user interface, hand gestures and finger shapes can provide easy acquisition of instruction as a sign language between devices and users. To achieve this intuitive gesture-based communication, key characteristics are required from imaging systems: light insensitivity, and high spatial density (high depth rate and x-y plane resolution). For gesture recognition, a variety of sensing methods using infrared, laser, and acoustics have been developed, but they have a critical issue: light disturbance. This device overcomes sunlight interference that limits 3D camera performance outdoors. Multiple microphones with resonant frequency 40 kHz are used as a depth sensor for 3...