A federated learning system with enhanced feature extraction for human activity recognition

Abstract

With the rapid growth of mobile devices, wearable sensor-based human activity recognition (HAR) has become one of the hottest topics in the Internet of Things. However, it is challenging for traditional approaches to achieving high recognition accuracy while protecting users’ privacy and sensitive information. To this end, we design a federated learning system for HAR (HARFLS). Based on the FederatedAveraging method, HARFLS enables each user to handle its activity recognition task safely and collectively. However, the recognition accuracy largely depends on the system’s feature extraction ability. To capture sufficient features from HAR data, we design a perceptive extraction network (PEN) as the feature extractor for each user. PEN is mainly composed of a feature network and a relation network. The feature network, based on a convolutional block, is responsible for discovering local features from the HAR data while the relation network, a combination of long short-term memory (LSTM) and attention mechanism, focuses on mining global relationships hidden in the data. Four widely used datasets, i.e., WISDM, UCI_HAR 2012, OPPORTUNITY, and PAMAP2, are used for performance evaluation. Experimental results demonstrate that PEN outperforms 14 existing HAR algorithms on these datasets in terms of the F1-score; HARFLS with PEN obtains better recognition results on the WISDM and PAMAP2 datasets, compared with 11 existing federated learning systems with various feature extraction structures.