Federated Feature Selection for Horizontal Federated Learning in IoT Networks

Abstract

Under horizontal federated learning (HFL) in IoT scenarios, different user datasets have significant similarities on the feature spaces, the final goal is to build a high-performance global model. However, not all features are great contributors when training the global HFL model, some features even impair the HFL. Besides, the curse of dimension will delay the training time and cause more energy consumption (EC). In this case, it is critical to remove irrelevant features from the local and select the useful overlapping features from a federated global perspective. In addition, the uncertainty of data being labelled and the non-independent and identically distributed (non-IID) client data should also consider. This paper introduces an unsupervised federated feature selection approach (named FSHFL) for HFL in IoT networks. First, a feature relevance outlier detection method is applied to the HFL participants to remove the useless features, which combines with the improved one-class support vector machine. Besides, a feature relevance hierarchical clustering (FRHC) algorithm is proposed for HFL overlapping feature selection. Experiment results on four IoT datasets show that the proposed methods can select better federated feature sets among HFL participants, thus improving the performance of the HFL system. Specifically, the global model accuracy improves up to 1.68% since fewer irrelevant features. Moreover, FSHFL can lower the average training time as high as 6.9%. Last, when the global model gets the same test accuracy, FSHFL can decrease the average EC of training the model by approximately 2.85% compared to Fed-Avg and roughly 68.39% compared to Fed-SGD.