An LWE-Based Multi-Key Privacy-Preserving Distributed Deep Learning

Abstract

Privacy preservation in distributed deep learning (DDL) has received a lot of attention recently. One key approach to preserving privacy for DDL is the use of Homomorphic Encryption (HE), which allows computation on ciphertexts without decryption. Although current applications of HE provide some level of privacy to DDL, they are not post-quantum resistant and multi-key compliant. The use of same key by all participants could lead to collusion attacks and possible leakage of sensitive information from a participant's local dataset in the distributed deep learning setting. Furthermore, most of the privacy solutions provided for the DDL assume that all learning participants must be present before the start of the protocol. Hence, such solutions do not allow a new participant to benefit from the utility of the joint learning of other participants, for which it was not present. To address the above problems, we propose a new privacy preserving solution for DDL that uses an LWE-based Multi-key approach. Our proposed privacy preserving DDL solution is multi-key compliant, post-quantum resistant and has an additional property of allowing a new participant to benefit from the utility of an already trained collaborative deep learning system. We prove that our privacy-preserving distributed deep learning system is secured. We note that our proposed solution generally provides a robust privacy-preserving solution for the DDL and at the same time has great potential to reduce communication and computation costs associated with similar systems.