Why Partitioning Matters: Revealing Overestimated Performance in WiFi-CSI-Based Human Action Recognition

Human activity recognition (HAR) is a broad research area. While there exist solutions based on sensors and vision-based technologies, these solutions suffer from considerable limitations. Thus in order to mitigate or avoid these limitations, device free solutions based on radio signals like (home) WiFi, in particular 802.11 are considered. Recently, channel state information (CSI), available in WiFi 802.11n networks have been proposed for fine-grained analysis. We are able to detect human activities like Walk, Sit, Stand, Run (in the sequel, any human activity used for classification is capitalised, i.e. is denoted by its corresponding label. For example, “standing“ is denoted as Stand, the activity “sitting“ is denoted by Sit and so on), etc. in a line-of-sight (LOS) scenario and a non-line-of-sight (N-LOS) scenario within an indoor environment. We propose two algorithms—one using a support vector machine (SVM) for classification and another one using a long short-term memory (LSTM) recurrent neural network. While the former uses sophisticated pre-processing and feature extraction techniques based on wavelet analysis, the latter processes the raw data directly (after denoising). We show that it is possible to characterize activities and/or human body presence with high accuracy and we compare both approaches with regard to accuracy and performance. Furthermore, we extend the experimental setup to detect human falls, too which is a relevant use-case in the context of ambient assisted living (AAL) and show that with the developed algorithms it is possible to detect falls with high accuracy. In addition, we also show that the algorithms can be used to count the number of people in a room based on the CSI-data, which is a first step towards detecting more complex social behavior and activities. Our paper is an extended version of the paper (Damodaran and Schäfer, Device free human activity recognition using wifi channel state information, in: 16th IEEE International Conference on Ubiquitous Intelligence and Computing (UIC 2019), 5th IEEE Smart World Congress, Leicester, vol 16, IEEE, 2019).

Human activity recognition can benefit various applications including healthcare services and context awareness. Since human actions will influence WiFi signals, which can be captured by the channel state information (CSI) of WiFi, WiFi CSI based human activity recognition has gained more and more attention. Due to the complex relationship between human activities and WiFi CSI measurements, the accuracies of current recognition systems are far from satisfactory. In this paper, we propose a new deep learning based approach, i.e., attention based bi-directional long short-term memory (ABLSTM), for passive human activity recognition using WiFi CSI signals. The BLSTM is employed to learn representative features in two directions from raw sequential CSI measurements. Since the learned features may have different contributions for final activity recognition, we leverage on an attention mechanism to assign different weights for all the learned features. Real experiments have been carried out to evaluate the performance of the proposed ABLSTM for human activity recognition. The experimental results show that our proposed ABLSTM is able to achieve the best recognition performance for all activities when compared with some benchmark approaches.

CSITime: Privacy-preserving human activity recognition using WiFi channel state information

Human activity recognition (HAR) is a fundamental sensing and analysis technique that supports diverse applications, such as smart homes and healthcare. In device-free and non-intrusive HAR, WiFi channel state information (CSI) captures wireless signal variations caused by human interference without the need for video cameras or on-body sensors. However, current CSI-based HAR performance is hampered by incomplete CSI recordings due to fixed window sizes in CSI collection and human/machine errors that incur missing values in CSI. To address these issues, we propose DiffAR, a temporal-augmented HAR approach that improves HAR performance by augmenting CSI. DiffAR devises a novel Adaptive Conditional Diffusion Model (ACDM) to synthesize augmented CSI, which tackles the issue of fixed windows by forecasting and handles missing values with imputation. Compared to existing diffusion models, ACDM improves the synthesis quality by guiding progressive synthesis with step-specific conditions. DiffAR further exploits an ensemble classifier for activity recognition using both raw and augmented CSI. Extensive experiments on four public datasets show that DiffAR achieves the best synthesis quality of augmented CSI and outperforms state-of-the-art CSI-based HAR methods in recognition performance. The source code of DiffAR is available at https://github.com/huangshk/DiffAR.

WiFi-based human activity recognition through wall using deep learning

Human activity recognition (HAR) is vital for applications in fields such as smart homes, health monitoring, and navigation, particularly in GNSS-denied environments where satellite signals are obstructed. Wi-Fi channel state information (CSI) has emerged as a key technology for HAR due to its wide coverage, low cost, and non-reliance on wearable devices. However, existing methods face challenges including significant data fluctuations, limited feature extraction capabilities, and difficulties in recognizing complex movements. This study presents a novel solution by integrating a multi-sensor array of Wi-Fi CSI with deep learning techniques to overcome these challenges. We propose a 2 × 2 array of Wi-Fi CSI sensors, which collects synchronized data from all channels within the CSI receivable range, improving data stability and providing reliable positioning in GNSS-denied environments. Using the CNN-LSTM-attention (C-L-A) framework, this method combines short- and long-term motion features, enhancing recognition accuracy. Experimental results show 98.2% accuracy, demonstrating superior recognition performance compared to single Wi-Fi receivers and traditional deep learning models. Our multi-sensor Wi-Fi CSI and deep learning approach significantly improve HAR accuracy, generalization, and adaptability, making it an ideal solution for GNSS-denied environments in applications such as autonomous navigation and smart cities.

Human activity recognition (HAR) has emerged as a significant area of research due to its numerous possible applications, including ambient assisted living, healthcare, abnormal behaviour detection, etc. Recently, HAR using WiFi channel state information (CSI) has become a predominant and unique approach in indoor environments compared to others (i.e., sensor and vision) due to its privacy-preserving qualities, thereby eliminating the need to carry additional devices and providing flexibility of capture motions in both line-of-sight (LOS) and non-line-of-sight (NLOS) settings. Existing deep learning (DL)-based HAR approaches usually extract either temporal or spatial features and lack adequate means to integrate and utilize the two simultaneously, making it challenging to recognize different activities accurately. Motivated by this, we propose a novel DL-based model named spatio-temporal convolution with nested long short-term memory (STC-NLSTMNet), with the ability to extract spatial and temporal features concurrently and automatically recognize human activity with very high accuracy. The proposed STC-NLSTMNet model is mainly comprised of depthwise separable convolution (DS-Conv) blocks, feature attention module (FAM) and NLSTM. The DS-Conv blocks extract the spatial features from the CSI signal and add feature attention modules (FAM) to draw attention to the most essential features. These robust features are fed into NLSTM as inputs to explore the hidden intrinsic temporal features in CSI signals. The proposed STC-NLSTMNet model is evaluated using two publicly available datasets: Multi-environment and StanWiFi. The experimental results revealed that the STC-NLSTMNet model achieved activity recognition accuracies of 98.20% and 99.88% on Multi-environment and StanWiFi datasets, respectively. Its activity recognition performance is also compared with other existing approaches and our proposed STC-NLSTMNet model significantly improves the activity recognition accuracies by 4% and 1.88%, respectively, compared to the best existing method.

Human action recognition using WiFi channel state information (CSI) has gained attention due to its non-intrusive nature and potential applications in healthcare, smart environments, and security. However, the reliability of methods developed for CSI-based action recognition is often contingent on the quality of the datasets and evaluation protocols used. In this paper, we uncovered a critical data leakage issue, which arises from improper data partitioning, in a widely used WiFi CSI benchmark dataset. Specifically, the benchmark fails to separate individuals between the training and test sets, leading to inflated performance metrics as models inadvertently learn individual-specific features rather than generalizable action patterns. We analyzed this issue in depth, retrained several benchmarked models using corrected data partitioning methods, and demonstrated a significant drop in accuracy when individuals were properly separated across training and testing. Our findings highlight the importance of rigorous data partitioning in CSI-based action recognition and provide recommendations for mitigating data leakage in future research. This work contributes to the development of more robust and reliable human action recognition systems using WiFi CSI.

Recently, device-free human activity recognition has become a research hotspot, and great progress has been made in ubiquitous computing. Among the different kinds of implementations, activity recognition based on WiFi channel state information (CSI) has attracted enormous attention for its superiority compared with conventional approaches. In this article, a device-free human continuous activity recognition system based on WiFi CSI is proposed. First, the CSI phase difference expansion matrix is constructed as a more obvious activity recognition feature, and a method based on threshold combined with labeling is used to achieve continuous activity segmentation. Then, the Gaussian mixture model–hidden Markov model (GMM–HMM) is used to model the CSI feature data of each activity, which is originally used for human 3-D skeleton-based activity recognition. The approach is of great value not only for its high accuracy compared with other classification approaches, such as long short-term memory (LSTM) and convolutional neural network (CNN), but also for its tremendous advantage that a pretty short CSI time series could be used to identify human activities, thus saving computer memory, reducing system calculation time greatly, and improving the error tolerance rate of the segmentation. Experiments on measured activity datasets and methods comparison demonstrate the effectiveness and superiority of the proposed system. The factors affecting system performance, such as the length of the CSI time series, have been discussed in this article.

Wi-Fi channel state information (CSI)-based human action recognition systems have garnered significant interest for their non-intrusive monitoring capabilities. However, the integrity of these systems can be compromised by data leakage, particularly when improper dataset partitioning strategies are employed. This paper investigates the presence and impact of data leakage in three published Wi-Fi CSI-based human action recognition methods that utilize deep learning techniques. The original studies achieve precision rates of 95% or higher, attributed to the lack of human-based dataset splitting. By re-evaluating these systems with proper subject-based partitioning, our analysis reveals a substantial decline in performance, underscoring the prevalence of data leakage. This study highlights the critical need for rigorous dataset management and evaluation protocols to ensure the development of robust and reliable human action recognition systems. Our findings advocate for standardized practices in dataset partitioning to mitigate data leakage and enhance the generalizability of Wi-Fi CSI-based models.

Human activity recognition (HAR) based on WiFi channel state information (CSI) has received a lot of attentions recently due to its non-intrusive nature. Most CSI-based HAR systems use a WiFi router and a computing terminal for centralized processing, which makes it difficult to achieve real-time wide-range recognition. Recently, lightweight artificial intelligence internet of things (AIoT) devices are widely deployed. The equipped WiFi chips within such devices can collect and process CSI data in a distributed way. Thus, the AIoT devices extend the detection range of collecting CSI and enrich the applications. However, the memories of the AIoT devices are constrained and lack of appropriate lightweight CSI processing strategies. To address these challenges, we propose the LiWi-HAR system which employs a comprehensive lightweight CSI processing strategy in WiFi based AIoT devices. The proposed lightweight CSI processing strategy extracts the main related features while compressing the data size. Then, a double hidden layer BP neural network based on particle swarm optimization (PSO-BPNN) algorithm is developed for HAR. In this case, the computing memory occupation of the device is effectively reduced, and the real-time high accurate recognition is achieved. Extensive experimental results present that the efficiency of our system significantly outperforms other centralized deep learning based systems and the recognition accuracy achieves 91.7%.

Using Wi-Fi Channel State Information (CSI) is a novel way of environmental sensing and human activity recognition (HAR). These methods can be used for several safety and security applications by (re)using Wi-Fi routers without the need for additional costly hardware required for vision-based approaches, known also to be particularly privacy-intrusive. This work introduces a full pipeline of a Wi-Fi CSI-based system for human activity recognition that assesses and compares two deep learning methods. We analyze how different hardware configurations affect WiFi CSI signals. We contribute a novel and more realistic data collection process, in which human activity recognition is seamlessly integrated in real-life, resulting in more reliable assessments of the model classification performance. We analyze how InceptionTime and LSTM-based classification models perform in human activity recognition. The source code and collected dataset are made publicly available for reproducibility and encouraging further research in the community.

In the recent past, human activity recognition research has focused on using WiFi channel state information (CSI) as a viable alternative to legacy systems like video and sensor-based activity recognition having limitations such as privacy invasion, obtrusiveness, and the inconvenience of wearing sensory devices. While the performance of CSI-based activity recognition models is impressive, many of the models are built using offline processed data from regulated settings which hinders their application in real-time. However, real-life human activity recognition requires models to be responsive to identifying activities in real-time. To address the shortcoming of CSI-based activity recognition models, we propose a deep learning object detection framework and instance segmentation for multiple human activity recognition using WiFi signals. The real-time CSI data from the signal is captured on a sliding window and converted into time-frequency domain images of the activity stream using continuous wavelet transform (CWT). Since it is impossible to pre-segment activities within a stream in real-time, the power profile from the transformed images is exploited to provide insights for deep learning instance segmentation to identify each unique human activity. The evaluation is carried out using real-time CSI data with single and multiple human activities. The results show that real-time model classification accuracy is 93.80% on average and instance segmentation accuracy of 90.73%.

Wireless sensing represented by WiFi channel state information (CSI) is now enabling various fields of applications such as person identification, human activity recognition, occupancy detection, localization, and crowd estimation these days. So far, those fields are mostly considered as separate topics in WiFi CSI-based methods, on the contrary, some camera and vision-based crowd estimation systems intuitively estimate both crowd size and location at the same time. Our work is inspired by the idea that WiFi CSI also may be able to perform the same as the camera does. In this paper, we construct <i>Wi-CaL</i>, a simultaneous crowd counting and localization system by using ESP32 modules for WiFi links. We extract several features that contribute to dynamic state (moving crowd) and static state (location of the crowd) from the CSI bundles, then assess our system by both conventional machine learning (ML) and deep learning (DL). As a result of ML-based evaluation, we achieved 0.35 median absolute error (MAE) of counting and 91.4&#x0025; of localization accuracy with five people in a small-sized room, and 0.41 MAE of counting and 98.1&#x0025; of localization accuracy with 10 people in a medium-sized room, by leave-one-session-out cross-validation. We compared our result with percentage of non-zero elements metric (PEM), which is a state-of-the-art metric for crowd counting, and confirmed that our system shows higher performance (0.41 MAE, 81.8&#x0025; of within-1-person error) than PEM (0.62 MAE, 66.5&#x0025; of within-1-person error).

Human Activity Recognition (HAR) has attracted considerable attention in recent years due to its potential applications in healthcare, smart homes, and security. Wi-Fi Channel State Information (CSI) is a promising sensor modality for HAR, providing a device-free and low-cost solution. However, building environment-independent models for HAR using Wi-Fi CSI remains a significant challenge. In this paper, we present a deep learning-based activity recognition system that exploits CSI measurements obtained from one or more environments to deliver consistent and accurate performance even in unseen environments. Our system employs a multi-task learning approach that is based on an encoder-decoder network architecture. This enables the encoder part of this architecture to mitigate the environment-dependent factors and extract a rich and environment-invariant representation. To evaluate the proposed system, we collected CSI samples for six activities pursued by three participants in four distinct environments. The results demonstrate the efficacy of the proposed system in achieving environment-independent HAR with an average accuracy of 80%. Additionally, the results validate the superiority of our method over environment-specific models by a minimum margin of 6% in cases of limited data.