An embedded vertical‐federated feature selection algorithm based on particle swarm optimisation

Abstract

AbstractIn real life, a large amount of data describing the same learning task may be stored in different institutions (called participants), and these data cannot be shared among participants due to privacy protection. The case that different attributes/features of the same instance are stored in different institutions is called vertically distributed data. The purpose of vertical‐federated feature selection (FS) is to reduce the feature dimension of vertical distributed data jointly without sharing local original data so that the feature subset obtained has the same or better performance as the original feature set. To solve this problem, in the paper, an embedded vertical‐federated FS algorithm based on particle swarm optimisation (PSO‐EVFFS) is proposed by incorporating evolutionary FS into the SecureBoost framework for the first time. By optimising both hyper‐parameters of the XGBoost model and feature subsets, PSO‐EVFFS can obtain a feature subset, which makes the XGBoost model more accurate. At the same time, since different participants only share insensitive parameters such as model loss function, PSO‐EVFFS can effectively ensure the privacy of participants' data. Moreover, an ensemble ranking strategy of feature importance based on the XGBoost tree model is developed to effectively remove irrelevant features on each participant. Finally, the proposed algorithm is applied to 10 test datasets and compared with three typical vertical‐federated learning frameworks and two variants of the proposed algorithm with different initialisation strategies. Experimental results show that the proposed algorithm can significantly improve the classification performance of selected feature subsets while fully protecting the data privacy of all participants.