TORR: A Lightweight Blockchain for Decentralized Federated Learning

Abstract

Federated learning has received considerable attention because it allows multiple devices to train models locally without revealing sensitive data. Well-trained local models are transmitted to a parameter server for further aggregation. The dependence on a trusted central server makes federated learning vulnerable to the single point of failure or attack. Blockchain is regarded as a state-of-the-art solution to decentralize the central server and provide attractive features simultaneously, such as immutability, traceability and accountability. However, current popular blockchain systems cannot be combined with federated learning seamlessly. Since all local models should be collected before aggregation, the latency of federated learning is determined by the slowest device. The consensus process required by blockchain will increase the latency further especially when a large block is required for including the model. Moreover, forever-growing blockchain together with models will take up a lot of storage space, making it impractical to be deployed on lightweight devices. To address these problems, we propose a lightweight blockchain TORR for federated learning. A novel consensus protocol Proof of Reliability is designed to achieve fast consensus while mitigating the impact of stragglers. A storage protocol is designed based on erasure coding and periodic storage refreshing policy. With erasure coding we take full advantage of the limited storage space of devices. With periodic storage refreshing policy we reduce the requirement for storage. Compared to the common blockchain-based federated learning system, TORR reduces the system latency, overall storage overhead and peak storage overhead by up to 62%, 75.44% and 51.77%, respectively.