Seeing the unseen: Wifi-based 2D human pose estimation via an evolving attentive spatial-Frequency network

Abstract

Camera-based human pose estimation has become popular due to its wide applications and easy implementation. However, it is not applicable under several circumstances such as poor illumination, occlusion, and private protection. In this work, we utilize Wifi signals to estimate 2D human poses, which is challenging because Wifi signals are abstract and contain limited information. To address these challenges, we develop an evolving attentive spatial-frequency network to discover the relationship between signal variation and body movement for Wifi-based 2D human pose estimation. By first taking dilated CSI sequences as inputs, a spatial-frequency encoder is then introduced to effectively integrate static spatial information and dynamic frequency information from CSI signals. Finally, we design an evolving attention module to enable our model to attend to certain channels of features. Due to a lack of benchmarks, we propose two Wifi-based human pose estimation datasets, the General Pose Estimation dataset (GPE) and Specific Pose Estimation dataset (SPE), which have been released as a public download at project page. Extensive experiments on the proposed datasets show that our model outperforms the state-of-the-art method by at least 16% in terms of PCK@20 (percentage of correct keypoints).