A Novel Gesture Recognition System Based on CSI Extracted from a Smartphone with Nexmon Firmware.

Tao Li,Pengpeng Chen,Chenqi Shi,Peihao Li

doi:10.3390/s21010222

Abstract

In this paper, we propose a novel gesture recognition system based on a smartphone. Due to the limitation of Channel State Information (CSI) extraction equipment, existing WiFi-based gesture recognition is limited to the microcomputer terminal equipped with Intel 5300 or Atheros 9580 network cards. Therefore, accurate gesture recognition can only be performed in an area relatively fixed to the transceiver link. The new gesture recognition system proposed by us breaks this limitation. First, we use nexmon firmware to obtain 256 CSI subcarriers from the bottom layer of the smartphone in IEEE 802.11ac mode on 80 MHz bandwidth to realize the gesture recognition system’s mobility. Second, we adopt the cross-correlation method to integrate the extracted CSI features in the time and frequency domain to reduce the influence of changes in the smartphone location. Third, we use a new improved DTW algorithm to classify and recognize gestures. We implemented vast experiments to verify the system’s recognition accuracy at different distances in different directions and environments. The results show that the system can effectively improve the recognition accuracy.

Highlights

Human gesture recognition and activity recognition are gradually becoming prominent promoters of human-computer interaction technology development
WiFi-based gesture recognition systems have emerged in recent years [7,8,9,10,11,12,13]
We introduced Cross Power Spectral Density (CPSD), which describes the statistical properties of Channel State Information (CSI) measurements in the frequency domain

Summary

Introduction

Human gesture recognition and activity recognition are gradually becoming prominent promoters of human-computer interaction technology development. Gesture recognition is usually implemented using technologies such as depth and infrared cameras [1,2,3] or wearable devices [4,5,6] These solutions require significant overhead and non-negligible costs when deployed. The wearable device-based method can achieve high accuracy, the wearable sensing device needs to be attached to the user’s hand or body, which may cause inconvenience in some cases and requires higher additional costs. To overcome these limitations, WiFi-based gesture recognition systems have emerged in recent years [7,8,9,10,11,12,13]

Results

Discussion

Conclusion