Federated fusion learning with attention mechanism for multi-client medical image analysis

Abstract

Federated Learning (FL) has gained significant attention because of its potential for privacy-preserving distributed learning. However, statistical heterogeneity and label scarcity remain major issues in multi-client scenarios. Furthermore, existing FL algorithms do not consider the unique data distribution of each client, which is essential for enhancing the overall performance of a model. In this study, a new FL algorithm called federated fusion learning (FFL) is proposed to address the challenges of statistical heterogeneity and label scarcity in multiclient data fusion. In FFL, each device calculates its data distribution parameters and sends them to the server in a single communication round. The server then uses the received distribution parameters to generate synthetic data for each client and combines them to create a larger dataset. FFL comprises three modules that identify the latent space of each client, intelligently fuse various modalities at the server, and construct global weights. The study also introduces a novel refinement mechanism that embeds training images into the W space, thereby fusing the latent spaces of different clients using a custom multiclient latent space fusion (MCLSF) module. The attention mechanism used in FFL enables effective multiclient fusion by leveraging the inherent correlations between various data modalities. Ten datasets sourced from MedMNIST encompassing diverse imaging modalities such as X-rays, computed tomography (CT), and optical coherence tomography (OCT) were considered. In addition, detailed ablation studies were conducted on multiple client configurations and non-independent and identical (non-IID) setups to ensure the generalizability of the proposed approach. The results demonstrate the enhanced generalizability and overall performance of the global model compared to existing state-of-the-art methods, confirming its broader applicability and effectiveness.