Coalition Formation Game in the Cross-Silo Federated Learning System

Abstract

In cross-silo federated learning (FL), organizations cooperatively train a global model with their local data. The organizations, however, own different datasets and may be heterogeneous in terms of their expectation on the precision of the global model. Meanwhile, the cost of secure global model aggregation, including computation and communication, is proportional to the square of the number of organizations in the FL system. In this paper, we consider all organizations in the FL system as a grand coalition. We introduce a novel concept from coalition game theory which allows the dynamic formation of coalitions among organizations. A simple and distributed merge and split algorithm for coalition formation is constructed. The aim is to find an ultimate coalition structure that allows cooperating organizations to maximize their utilities in consideration of the coalition formation cost. Through this novel game theoretical framework, the FL system is able to self-organize and form a structured network composed of disjoint stable coalitions. To fairly distribute cost in each formed coalition, a cost sharing mechanism is proposed to align members' individual utility with their coalition's utility. In FL systems, training data has a significant impact on model performances, i.e., it should lead to a more precise global model if organizations with greater data complementarity are grouped. Numerical evaluations are presented to verify the proposed models.