Data heterogeneous federated learning algorithm for industrial entity extraction

Abstract

Entity extraction is an important part to realize digital transformation in the industrial field. Building an entity extraction model in the industrial field requires a lot of data. The parties in industry often cannot share data due to commercial competition and security and privacy issues, thus forming “Data Island”. Federated learning provides a solution to this problem. Federated learning is a distributed machine learning framework that allows each party to train locally and independently using their own private data. The model parameters or gradient information of each party will be aggregated to the central server, thus forming a model jointly trained by all parties. This approach can not only protect the security and privacy of data from all parties, but also fully utilize their data resources. Federated learning can effectively solve the problem of data island, but it still faces some problems and challenges, among which the most typical problem is data heterogeneity. To address the data islanding problem and data heterogeneity problem faced by industrial entity extraction, this paper uses a federated learning framework to solve the data islanding problem and proposes the FedDP algorithm. This algorithm assigns weights based on the data quality performance of each participant. Participants with relatively good data quality performance have higher weights in the aggregation stage, while participants with relatively poor data quality performance have lower weights in the aggregation stage, thus optimizing the performance of federated learning in heterogeneous data scenarios.