Heterogeneous Model Fusion Federated Learning Mechanism Based on Model Mapping

Abstract

The computing power of various Internet of Things (IoT) devices is quite different. To enable IoT devices with lower computing power to perform machine learning, all nodes can only train smaller models, which results in the waste of computing power for high-performance devices. In this article, a heterogeneous model fusion federated learning (HFL) mechanism is proposed. Each node trains learning models of different scales according to its own computing capabilities. After receiving the gradient trained by each node, the parameter server (PS) corrects the received gradient with the repeat matrix, and then update the corresponding region of the global model according to the mapping matrix. After all update operations are over, the PS assigns the compressed model to the corresponding node. This article uses a variety of experimental schemes to evaluate the proposed method, including three data sets, two model structures, and three computational complexity levels. The proposed method have been proved it not only maximizes the use of the unbalanced computing power of edge nodes, but also enables different structural models to compensate for the shortcomings of others, improving the overall performance.