Two gesture-computing approaches by using electromagnetic waves

Xianchao Wang,Hongyu Liu,Yukun Guo,Jingzhi Li

doi:10.3934/ipi.2019040

Abstract

We are concerned with a novel sensor-based gesture input/ instruction technology which enables human beings to interact with computers conveniently. The human being wears an emitter on the finger or holds a digital pen that generates a time harmonic point charge. The inputs/instructions are performed through moving the finger or the digital pen. The computer recognizes the instruction by determining the motion trajectory of the dynamic point charge from the collected electromagnetic field measurement data. The identification process is mathematically modelled as a dynamic inverse source problem for time-dependent Maxwell's equations. From a practical point of view, the point source should be assumed to move in an unknown inhomogeneous background medium, which models the human body and the surroundings. Moreover, a salient feature is that the electromagnetic radiated data are only collected in a limited aperture. For the inverse problem, we develop, from the respectively deterministic and stochastic viewpoints, a dynamic direct sampling method and a modified particle filter method. Both approaches can effectively recover the motion trajectory. Rigorous theoretical justifications are presented for the mathematical modelling and the proposed recovery methods. Extensive numerical experiments are conducted to illustrate the promising features of the two proposed recognition approaches.

Highlights

Among various human-computer interaction technologies, most people prefer to interact with computers in a more personal way, e.g., by using voice, touch and gesture rather than a mouse or a keyboard
For the dynamic inverse electromagnetic problem described above, we develop two methods for the trajectory identification: one is a direct imaging method motivated by our theoretical analysis and the other one is a modified particle filter method motived by the particle filter methods developed in [6, 16, 24] for various dynamic inverse problems
We develop a conceptual framework of some novel gesture recognition techniques using electromagnetic waves

Summary

Introduction

Among various human-computer interaction technologies, most people prefer to interact with computers in a more personal way, e.g., by using voice, touch and gesture rather than a mouse or a keyboard. Theorem 2.1 indicates that one should make use of low-frequency emission, and the human being that performs the input/instruction should keep a reasonable distance away from the computing device in the design of the proposed gesture-computing device This has been confirmed by our numerical experiments in what follows under practical and mild conditions on ω0 and L, not as restrictive as the theoretical assumptions in Theorem 2.1. We compare the particle filter technique for reconstructing a moving trajectory in homogeneous and inhomogeneous background medium, respectively. In the following figures concerning the problem geometry, some 2D projections (shadows with gray color) are added for better 3D visualizations Both methods are able to identify the moving trajectory even if the measured data has 10% noise. The implementation of the direct sampling method could be accelerated by incorporating the sequential or parallel local sampling strategies, see [26] for more details

Conclusions

Findings

Direct sampling Particle filter