Fostering Trustworthiness of Federated Learning Ecosystem through Realistic Scenarios

Abstract

The present study thoroughly evaluates the most common blocking challenges faced by the federated learning (FL) ecosystem and analyzes existing state-of-the-art solutions. A system adaptation pipeline is designed to enable the integration of different AI-based tools in the FL system, while FL training is conducted under realistic conditions using a distributed hardware infrastructure. The suggested pipeline and FL system’s robustness are tested against challenges related to tool deployment, data heterogeneity, and privacy attacks for multiple tasks and data types. A representative set of AI-based tools and related datasets have been selected to cover several validation cases and distributed to each edge device to closely reflect real-world scenarios. The study presents significant outcomes of the experiments and analyzes the models’ performance under different realistic FL conditions, while highlighting potential limitations and issues that occurred during the FL process.