WiFi-based Human Activity Recognition Research Articles

Democratizing Federated WiFi-Based Human Activity Recognition Using Hypothesis Transfer

Democratizing Federated WiFi-Based Human Activity Recognition Using Hypothesis Transfer

SAT: A Selective Adversarial Training Approach for WiFi-Based Human Activity Recognition

SAT: A Selective Adversarial Training Approach for WiFi-Based Human Activity Recognition

AutoDLAR: A Semi-supervised Cross-modal Contact-free Human Activity Recognition System

WiFi-based human activity recognition (HAR) plays an essential role in various applications such as security surveillance, health monitoring, and smart home. Existing HAR methods, though yielding promising performance in indoor scenarios, highly depend on a massive labeled dataset for training which is extremely difficult to acquire in practical applications. In this paper, we present an automatic data labeling and HAR system, termed AutoDLAR. Taking a semi-supervised cross-modal learning framework with a hybrid loss function as the core, AutoDLAR transfers rich visual information to automatically label WiFi signals for WiFi-based HAR. Specifically, we devise a lightweight and multi-view WiFi sensing model with a parallel feature embedding method to accurately identify activities and accelerate recognition speed. Then, we exploit the video data to fine-tune a well-established visual HAR model, generating effective pseudo-labels for guiding the WiFi model’s training. We also build a synchronized Video-WiFi dataset with seven types of human activities under different scenarios to enable training and validating the semi-supervised HAR system. Extensive experiments on our collected activity dataset and the emotion recognition benchmark demonstrate that AutoDLAR attains an average accuracy of over 95.89% without manual labeling and only spends the inference time of 3.35 ms, outperforming the state-of-the-art (SOTA) methods.

Feature decoupling and regeneration towards wifi-based human activity recognition

Gesture recognition using WiFi is vital for human–computer interaction, smart homes, smart spaces, and elderly care. WiFi signals are non-stationary, sensitive to the environment, and traditional pattern recognition-based HAR methods incur exorbitant training and deployment costs due to their reliance on the quality and quantity of training data. While there has been extensive research on enhancing HAR techniques, the efficiency of the system is still limited by the scarcity of training data and identity-action feature coupling. This research focuses on attaining gesture recognition across multiple users with a limited number of samples. Our findings show that the conventional data augmentation methods are incapable of facilitating the system in attaining sample diversity. As a result, we propose a training-free augmentation strategy as a means to provide adequate training data. Unlike the conventional data enhancement approach, this scheme aims to develop data processing methods that differentiate between various samples in practical applications. Consequently, it enhances data pertinent to specific HAR assignments effectively. To effectively extract action features in WiFi samples, an unsupervised cross-user domain sample generation (CUDSG) model is proposed. This model generates virtual gesture samples for new user domains by decoupling and recombining gesture and identity features. This extends the sensing boundary of the system to new user domains without requiring a significant number of users to participate. Model performance was evaluated using various classifiers, such as SVM, KNN and CNN. The results demonstrate a significant improvement in average classification accuracy from 57.3% to 98.4%. This indicates that CUDSG is a highly effective tool for enhancing the performance of existing gesture recognition techniques.

Wi-CHAR: A WiFi Sensing Approach with Focus on Both Scenes and Restricted Data.

Significant strides have been made in the field of WiFi-based human activity recognition, yet recent wireless sensing methodologies still grapple with the reliance on copious amounts of data. When assessed in unfamiliar domains, the majority of models experience a decline in accuracy. To address this challenge, this study introduces Wi-CHAR, a novel few-shot learning-based cross-domain activity recognition system. Wi-CHAR is meticulously designed to tackle both the intricacies of specific sensing environments and pertinent data-related issues. Initially, Wi-CHAR employs a dynamic selection methodology for sensing devices, tailored to mitigate the diminished sensing capabilities observed in specific regions within a multi-WiFi sensor device ecosystem, thereby augmenting the fidelity of sensing data. Subsequent refinement involves the utilization of the MF-DBSCAN clustering algorithm iteratively, enabling the rectification of anomalies and enhancing the quality of subsequent behavior recognition processes. Furthermore, the Re-PN module is consistently engaged, dynamically adjusting feature prototype weights to facilitate cross-domain activity sensing in scenarios with limited sample data, effectively distinguishing between accurate and noisy data samples, thus streamlining the identification of new users and environments. The experimental results show that the average accuracy is more than 93% (five-shot) in various scenarios. Even in cases where the target domain has fewer data samples, better cross-domain results can be achieved. Notably, evaluation on publicly available datasets, WiAR and Widar 3.0, corroborates Wi-CHAR's robust performance, boasting accuracy rates of 89.7% and 92.5%, respectively. In summary, Wi-CHAR delivers recognition outcomes on par with state-of-the-art methodologies, meticulously tailored to accommodate specific sensing environments and data constraints.

I-Sample: Augment Domain Adversarial Adaptation Models for WiFi-based HAR

Recently, using deep learning to achieve WiFi-based human activity recognition (HAR) has drawn significant attention. While capable of achieving accurate identification in a single domain (i.e., training and testing in the same consistent WiFi environment), it would become extremely tough when WiFi environments change significantly. As such, domain adversarial neural networks-based approaches have been proposed to handle such diversities across domains, yet often found to share the same limitation in practice: the imbalance between high-capacity of feature extractors and data insufficiency of source domains. This article proposes i-Sample, an intermediate sample generation-based framework, striving to tackle this issue for WiFi-based HAR. i-Sample is mainly designed as two-stage training, where four data augmentation operations are proposed to train a coarse domain-invariant feature extractor in the first stage. In the second stage, we leverage the gradients of classification error to generate intermediate samples to refine the classifiers together with original samples, making i-Sample also capable to be integrated into most domain adversarial adaptation methods without neural network modification. We have implemented a prototype system to evaluate i-Sample, which shows that i-Sample can effectively augment the performance of nowadays mainstream domain adversarial adaptation models for WiFi-based HAR, especially when source domain data is insufficient.

LiWi-HAR: Lightweight WiFi-Based Human Activity Recognition Using Distributed AIoT

Human activity recognition (HAR) based on WiFi channel state information (CSI) has received a lot of attentions recently due to its non-intrusive nature. Most CSI-based HAR systems use a WiFi router and a computing terminal for centralized processing, which makes it difficult to achieve real-time wide-range recognition. Recently, lightweight artificial intelligence internet of things (AIoT) devices are widely deployed. The equipped WiFi chips within such devices can collect and process CSI data in a distributed way. Thus, the AIoT devices extend the detection range of collecting CSI and enrich the applications. However, the memories of the AIoT devices are constrained and lack of appropriate lightweight CSI processing strategies. To address these challenges, we propose the LiWi-HAR system which employs a comprehensive lightweight CSI processing strategy in WiFi based AIoT devices. The proposed lightweight CSI processing strategy extracts the main related features while compressing the data size. Then, a double hidden layer BP neural network based on particle swarm optimization (PSO-BPNN) algorithm is developed for HAR. In this case, the computing memory occupation of the device is effectively reduced, and the real-time high accurate recognition is achieved. Extensive experimental results present that the efficiency of our system significantly outperforms other centralized deep learning based systems and the recognition accuracy achieves 91.7%.

WiFi-based human activity recognition through wall using deep learning

Wireless sensing is a promising method that integrates wireless mechanisms with strong sensing capabilities. The current focus of using WiFi Channel State Information (CSI) for human activity recognition (HAR) is the line-of-sight (LoS) path, which is mainly affected by human activities and is very sensitive to environmental changes. However, the signal on non-line-of-sight (nLoS) paths, particularly those passing through walls, is unpredictable due to the weak reflected signals destroyed by the wall. This work proposes a method to achieve high-accuracy wireless sensing based on CSI behavior recognition with low-cost resources by showing through-wall and wider-angle predictions using WiFi signals. The technique utilizes MIMO to exploit multipath propagation and increase the capability of signal transmission and receiving antennas. The signals captured by the multi-antenna are delivered into parallel channels with different spatial signatures. An RPi 4 B is attached to an ALFA AWUS 1900 adapter utilizing Nexmon firmware monitors and extracts CSI data with flexible C-based firmware for Broadcom/Cypress WiFi chips. Preprocessing techniques based on CSI are applied to improve the feature extraction from the amplitude data in an indoor environment. Furthermore, a deep learning algorithm based on RNN with an LSTM algorithm is used to classify the activity instances indoors, achieving up to 97.5% accuracy in classifying seven activities. The experiment shows CSI can achieve accurate wireless sensing in nLoS scenarios with extended antennas and a deep learning approach.

Enhancing CSI-Based Human Activity Recognition by Edge Detection Techniques

Human Activity Recognition (HAR) has been a popular area of research in the Internet of Things (IoT) and Human–Computer Interaction (HCI) over the past decade. The objective of this field is to detect human activities through numeric or visual representations, and its applications include smart homes and buildings, action prediction, crowd counting, patient rehabilitation, and elderly monitoring. Traditionally, HAR has been performed through vision-based, sensor-based, or radar-based approaches. However, vision-based and sensor-based methods can be intrusive and raise privacy concerns, while radar-based methods require special hardware, making them more expensive. WiFi-based HAR is a cost-effective alternative, where WiFi access points serve as transmitters and users’ smartphones serve as receivers. The HAR in this method is mainly performed using two wireless-channel metrics: Received Signal Strength Indicator (RSSI) and Channel State Information (CSI). CSI provides more stable and comprehensive information about the channel compared to RSSI. In this research, we used a convolutional neural network (CNN) as a classifier and applied edge-detection techniques as a preprocessing phase to improve the quality of activity detection. We used CSI data converted into RGB images and tested our methodology on three available CSI datasets. The results showed that the proposed method achieved better accuracy and faster training times than the simple RGB-represented data. In order to justify the effectiveness of our approach, we repeated the experiment by applying raw CSI data to long short-term memory (LSTM) and Bidirectional LSTM classifiers.

Towards a Dynamic Fresnel Zone Model to WiFi-based Human Activity Recognition

The passive WiFi sensing research has largely centered on activity sensing using fixed-location WiFi transceivers, leading to the development of several theoretical models that aim to map received WiFi signals to human activity. Of these models, the Fresnel zone model has shown to be particularly noteworthy. However, the growing popularity of mobile WiFi receivers has not been matched by corresponding research on mobile receiver-based theoretical models. This paper fills this gap by presenting the first theoretical model to quantify the impact of moving a moving receiver for WiFi sensing. We propose a novel dynamic Fresnel zone model in the free space of an indoor environment, which takes the form of a cluster of concentric hyperbolas centered on the transmitter and reflection subject. We examine three properties of this model, i.e., relating the variation in RF signals received by the receiver to the position and orientation of the human, the movement of the receiver, and the presence of other objects in the environment. To validate this model, we develop a prototype system and conduct extensive experiments. The results are consistent with our theoretical analysis, and the system is able to detect the direction of the transmitter with an accuracy of 10° or better, measure the receiver's relative motion displacement within 1 cm a millimeter-level accuracy, and classify five receiver-side activities with an accuracy of 98%. Our work moves a significant step forward in WiFi sensing and may potentially open up new avenues for future research.

PhaseAnti: An Anti-Interference WiFi-Based Activity Recognition System Using Interference-Independent Phase Component

Driven by a wide range of essential applications, significant achievements are made to explore WiFi-based Human Activity Recognition (HAR) techniques that utilize the information collected by commercial off-the-shelf WiFi infrastructures to infer human activities without the need for subjects to carry any devices. Although existing WiFi-based HAR systems achieve satisfactory performance in some instances, they are faced with a severe challenge that the impacts of ubiquitous Co-channel Interference (CCI) on WiFi signals are inevitable. This downgrades the performance of these HAR systems significantly. To address this challenge, we propose PhaseAnti, a novel WiFi-based HAR system to exploit the CCI-independent phase component, Nonlinear Phase Error Variation (NLPEV), of WiFi Channel State Information to cope with the negative effects of CCI. The stability of NLPEV data and the sensibility of this component to motions are rigorously analyzed. Furthermore, validated by extensive properly designed experiments, this phase component across subcarriers is invariant under various CCI scenarios while sufficiently distinct for different motions. Therefore, the NLPEV data can be used and processed effectively to perform HAR in CCI scenarios. Extensive experiments with various daily activities in different indoor rooms demonstrate the superior effectiveness and generalizability of the proposed PhaseAnti system under various CCI scenarios.

CeHAR: CSI-Based Channel-Exchanging Human Activity Recognition

Despite the intense effort from the research community, state-of-the-art WiFi-based human activity recognition performance remains unsatisfactory. Current approaches usually use individual characteristics of CSI, i.e., amplitude or phase measurements, to model the relationship between the changes of channel state information (CSI) and human activities, which lead to their failure to achieve satisfactory accuracy due to information loss. To deal with this issue, this paper proposes CeHAR, a CSI-based Human Activity Recognition using a channel-exchanging fusion network to deep fuse the CSI amplitude and phase features to obtain the informative features for human activity recognition. The proposed CeHAR is a parameter-free dual-characteristic fusion framework that dynamically exchanges channels between sub-networks of two kinds of characteristics to comprehensively learn informative features from both. Specifically, the proposed approach employs two sub-network using convolutional neural networks to learn features from each characteristic of CSI. The magnitude of Batch-Normalization (BN) scaling factor is used to determine the channel importance of each characteristic, and then guides the exchange process. The proposed CeHAR also shares convolutional filters, but keeps private BNs layers in different characteristics, which, as an added benefit, allows our characteristic fusion network to be nearly as compact as a single-characteristic network. Extensive real-world experiments have been conducted to evaluate the performance of our proposed CeHAR, and the experimental results illustrate that our proposed approach outperforms baselines.

Dual-Stream Contrastive Learning for Channel State Information Based Human Activity Recognition.

WiFi-based human activity recognition (HAR) has been extensively studied due to its far-reaching applications in health domains, including elderly monitoring, exercise supervision and rehabilitation monitoring, etc. Although existing supervised deep learning techniques have achieved remarkable performances for these tasks, they are however data-hungry and hence are notoriously difficult due to the privacy and incomprehensibility of WiFi-based HAR data. Existing contrastive learning models, mainly designed for computer vision, cannot guarantee their performance on channel state information (CSI) data. To this end, we propose a new dual-stream contrastive learning model that can process and learn the raw WiFi CSI data in a self-supervised manner. More specifically, our proposed method, coined as DualConFi, takes raw WiFI CSI data as input and incorporates channel and temporal streams to learn highly-discriminative spatiotemporal features under a mutual information constraint using unlabeled data. We exhibit the effectiveness of our model on three publicly available CSI data sets in various experiment settings, including linear evaluation, semi-supervised, and transfer learning. We show that DualConFi is able to perform favourably against challenging baselines in each setting. Moreover, by studying the effects of different transform functions on CSI data, we finally verify the effectiveness of highly-discriminative features.

Environment-Robust WiFi-Based Human Activity Recognition Using Enhanced CSI and Deep Learning

Deep learning has demonstrated its great potential in channel state information (CSI)-based human activity recognition (HAR), and hence has attracted increasing attention in both the industry and academic communities. While promising, most existing high-accuracy methodologies require to retrain their models when applying the previous-trained ones to a new/unseen environment. This issue has limited their practical usabilities. In order to overcome this challenge, this article proposes an innovative scheme, which combines an activity-related feature extraction and enhancement (AFEE) method and matching network (AFEE-MatNet). The proposed scheme is “one-fits-all,” meaning that the trained model can be directly applied in new/unseen environments without any retraining. We introduce the AFEE method to enhance CSI quality by eliminating noise. Specifically, the approach mitigates environmental noises unrelated to activity while better compressing and preserving the behavior-related information. Moreover, the size of feature signals generated by AFEE are reduced, which in turn significantly shortens the training time. For effective feature extraction, we propose to use the MatNet architecture to learn transferable features shared among source environments. To further improve the recognition performance, we introduce a prediction checking and correction scheme to rectify some classification errors that do not abide by the state transition of human behaviors. Extensive experimental results demonstrate that our proposed AFEE-MatNet significantly outperforms existing state-of-the-art HAR methods, in terms of both recognition accuracy and training time.

HHI-AttentionNet: An Enhanced Human-Human Interaction Recognition Method Based on a Lightweight Deep Learning Model with Attention Network from CSI.

Nowadays WiFi based human activity recognition (WiFi-HAR) has gained much attraction in an indoor environment due to its various benefits, including privacy and security, device free sensing, and cost-effectiveness. Recognition of human-human interactions (HHIs) using channel state information (CSI) signals is still challenging. Although some deep learning (DL) based architectures have been proposed in this regard, most of them suffer from limited recognition accuracy and are unable to support low computation resource devices due to having a large number of model parameters. To address these issues, we propose a dynamic method using a lightweight DL model (HHI-AttentionNet) to automatically recognize HHIs, which significantly reduces the parameters with increased recognition accuracy. In addition, we present an Antenna-Frame-Subcarrier Attention Mechanism (AFSAM) in our model that enhances the representational capability to recognize HHIs correctly. As a result, the HHI-AttentionNet model focuses on the most significant features, ignoring the irrelevant features, and reduces the impact of the complexity on the CSI signal. We evaluated the performance of the proposed HHI-AttentionNet model on a publicly available CSI-based HHI dataset collected from 40 individual pairs of subjects who performed 13 different HHIs. Its performance is also compared with other existing methods. These proved that the HHI-AttentionNet is the best model providing an average accuracy, F1 score, Cohen’s Kappa, and Matthews correlation coefficient of 95.47%, 95.45%, 0.951%, and 0.950%, respectively, for recognition of 13 HHIs. It outperforms the best existing model’s accuracy by more than 4%.

Two-Stream Convolution Augmented Transformer for Human Activity Recognition

Recognition of human activities is an important task due to its far-reaching applications such as healthcare system, context-aware applications, and security monitoring. Recently, WiFi based human activity recognition (HAR) is becoming ubiquitous due to its non-invasiveness. Existing WiFi-based HAR methods regard WiFi signals as a temporal sequence of channel state information (CSI), and employ deep sequential models (e.g., RNN, LSTM) to automatically capture channel-over-time features. Although being remarkably effective, they suffer from two major drawbacks. Firstly, the granularity of a single temporal point is blindly elementary for representing meaningful CSI patterns. Secondly, the time-over-channel features are also important, and could be a natural data augmentation. To address the drawbacks, we propose a novel Two-stream Convolution Augmented Human Activity Transformer (THAT) model. Our model proposes to utilize a two-stream structure to capture both time-over-channel and channel-over-time features, and use the multi-scale convolution augmented transformer to capture range-based patterns. Extensive experiments on four real experiment datasets demonstrate that our model outperforms state-of-the-art models in terms of both effectiveness and efficiency.

LSTM-CNN network for human activity recognition using WiFi CSI data

Human Activity Recognition (HAR) has had a diverse range of applications in various fields such as health, security and smart homes. Among different approaches of HAR, WiFi-based solutions are getting popular since it solves the problem of deployment cost, privacy concerns and restriction of the applicable environment. In this paper, we propose a WiFi-based human activity recognition system that can identify different activities via the channel state information from WiFi devices. A special deep learning framework, Long Short-Term Memory-Convolutional Neural Network (LSTM-CNN), is designed for accurate recognition. LSTM-CNN is going to be compared with the LSTM network and the experimental results demonstrate that LSTM-CNN outperforms existing models and has an average accuracy of 94.14% in multi-activity classification.

Data Augmentation and Dense-LSTM for Human Activity Recognition Using WiFi Signal

Recent research has devoted significant efforts on the utilization of WiFi signals to recognize various human activities. An individual’s limb motions in the WiFi coverage area could interfere with wireless signal propagation, that manifested as unique patterns for activity recognition. Existing approaches though yielding reasonable performance in certain cases, are ignorant of two major challenges. The performed activities of the individual normally have inconsistent speed in different situations and time. Besides that the wireless signal reflected by human bodies normally carries substantial information that is specific to that subject. The activity recognition model trained on a certain individual may not work well when being applied to predict another individual’s activities. Since only recording activities of limited subjects in a certain speed and scale, recent works commonly have a moderate amount of activity data for training the recognition model. The small-size data could often incur the overfitting issue that negative affect the traditional classification model. To address these challenges, we propose a WiFi-based human activity recognition system that synthesizes variant activities data through eight channel state information (CSI) transformation methods to mitigate the impact of activity inconsistency and subject-specific issues, and also design a novel deep-learning model that caters to the small-size WiFi activity data. We conduct extensive experiments and show synthetic data improve performance by up to 34.6% and our system achieves around 90% of accuracy with well robustness in adapting to small-size CSI data.

Device-free single-user activity recognition using diversified deep ensemble learning

WiFi-based human activity recognition (HAR) aims to recognize human activities in an off-the-shelf manner that only relies on the commercial Wi-Fi devices already installed in environments. The recent trend in HAR research is to train classifiers on top of statistical or deep neural features extracted from channel state information (CSI) data. Unfortunately, existing methods only take into account the temporal-correlation within each CSI subcarrier, while ignoring the spatial-correlation between different subcarriers. This issue has not been fully exploited yet, resulting a limited performance. To address this issue, we propose WiAReS, a WiFi-based device-free activity recognition system that takes both temporal-correlation and spatial-correlation into account. WiAReS embarks on diversified deep ensemble methods 2̌for single-user activity recognition where one user performs a single activity at a given time. More specifically, it adopts convolutional neural network (CNN) to automatically extract features from CSI measurements with the preservation of the locality of both spatial patterns and temporal patterns. To further improve recognition accuracy upon CNN-extracted features, we propose a novel ensemble architecture that fuses a multiple layer perception (MLP), a random forest (RF) and a support vector machine (SVM). Our system obtains the CSI data in PHY layer of off-the-shelf WiFi devices by installing Atheros-CSI-Tool on AR9590 based WiFi network interface cards (NICs). Comprehensive experiments have been conducted in three real environments with environmental variation to evaluate the performance of the proposed WiAReS. The experimental results demonstrate that the proposed WiARes system significantly outperforms existing methods.

Temporal-Frequency Attention-Based Human Activity Recognition Using Commercial WiFi Devices

Human activity recognition has been growing for decades in a variety of technological disciplines. However, in the existing WiFi-based human activity recognition systems, there are the following problems: Firstly, in the processing of channel state information (CSI) data, mainly for the removal of noise in the superimposed signal, there is no effective removal of useless multipath signals; Secondly, the data segmentation algorithm based on the empirical threshold requires manual adjustment of the threshold in different environments, resulting in poor robustness and unstable segmentation; Thirdly, simple learning classification is applied without specific design for CSI data structure and sufficiently abstracting information features. In this paper, a device-free human activity recognition system with a temporal-frequency attention mechanism is proposed, which can be deployed on commercial WiFi devices to identify human's daily activities. Firstly, the multipath signal affected by the channel change is extracted by using the difference of the propagation delay of different multipath, thereby eliminating the delay and invalid multipath signals that have undergone multiple reflections and refractions. Secondly, a neural network model based on attention mechanism is proposed, which assigns different weights to different characteristics and sequences by imitating the human brain to dedicate more attention to important information. Then, the long short-term memory (LSTM) model is used to learn the correlation features of different dimensions to realize human activity recognition. Finally, the system performance is evaluated in different environments, and the experimental results show that our syetem holds a better performance in both line-of-sight (LOS) and non-line-of-sight (NLOS) than the existing human activity recognition systems.