Human-defined, machine-optimized - gesture recognition using a mixed approach

Abstract

Wearable computing devices are now mainstream. Many such devices have capable MEMS sensors that can be exploited for recognizing dynamic, in-the-air gestures. The somewhat limited compute capacity and battery life of today's devices requires a computationally efficient approach to gesture recognition; one that can be effectively used inside an app running on standard, off-the-shelf hardware, such as an Android Smartwatch. The goal of this project is to test the feasibility of this idea. In a two-phased approach, a class of finite state machines (FSM1) for gesture recognition were first constructed and then the FSM were further tuned for higher accuracy with the help of some training data and a suitable optimization method, in the second phase. A novel approach is presented that leverages techniques from functional programming languages to define rich yet compact FSM. In order to demonstrate effectiveness, a prototype gesture recognition system for an automotive scenario using an Android Smartwatch app was developed. Then the system was tuned using a blended approach that combined a swarm-based evolutionary algorithm, Cultural Algorithms, and human parameter estimates with experimentally derived training data. The Blended approach using Cultural Algorithms achieved a 77% gesture recognition accuracy which is on par with more computationally intensive techniques such as Hidden Markov Models (HMM). The 'functional' FSM are human defined but machine optimized with Cultural Algorithms. By the blending of the two approaches, an improved balance between computational requirements and recognition accuracy was achieved.