Gesture Recognition With Ultrasounds and Edge Computing

Borja Saez,Encarnacion Castillo,Manuel Pegalajar,Diego P Morales,Javier Mendez,Miguel Molina

doi:10.1109/access.2021.3064390

Borja Saez, Encarnacion Castillo + Show 4 more

Open Access

https://doi.org/10.1109/access.2021.3064390

Copy DOI

Abstract

The aim of this work is to prove that it is possible to develop a system able to detect gestures based only on ultrasonic signals and Edge devices. A set of 7 gestures plus idle has been defined, being possible to combine them to increase the recognized gestures. In order to recognize them, Ultrasound transceivers will be used to detect the 2 dimensional gestures. The Edge device approach implies that the whole data is processed in the device at the network edge rather than depending on external devices or services such as Cloud Computing. The system presented in this paper has been proven to be able to measure Time of Flight (ToF) signals that can be used to recognize multiple gestures by the integration of two transceivers, with an accuracy between 84.18% and 98.4%. Due to the optimization of the preprocessing correlation technique to extract the ToF from the echo signals and our specific firmware design to enable the parallelization of concurrent processes, the system can be implemented as an Edge Device.

Highlights

The communication among humans is based on a multi-modal system, which includes verbal communication and face and body expressions to intensify the meaning of the verbal content
Due to the fact that all these techniques accomplish with the time restriction of the system, the compared parameter is the accuracy, which is measured in this experiment as correct classifications over all the classifications
The system presented in this paper has been proven to be able to measure Time of Flight (ToF) signals that can be later used to recognize multiple gestures by the integration of two transceivers

Summary

Introduction

The communication among humans is based on a multi-modal system, which includes verbal communication and face and body expressions to intensify the meaning of the verbal content. The Human System Interaction (HSI) trend is evolving, leading to the research of emerging technologies that mimic this natural communication, minimizing the use of interfaces like touchscreens, buttons or sliders. There are several systems that introduce gesture control to the system, i.e. SoundWave [1], AudioGest [2], Dolphin [3], or UltraGesture [4]. All of them use low frequency ultrasound signals to recognize between 5 and 12 gestures, which are mostly based on Doppler shift effect (frequency variation due to movement) while running the recognition algorithms on PC or Smartphones.

Objectives

Results

Conclusion