Privacy-preserving distributed deep learning via LWE-based Certificateless Additively Homomorphic Encryption (CAHE)

Abstract

Current privacy-preserving schemes proposed for distributed deep learning (DDL) are based on public key infrastructure (PKI) which has management of keys problem and also key escrow problem for those that may use identity-based public key cryptography (ID-PKC). This presents the possibility for the privacy guarantee of the distributed/collaborative deep learning system to be flawed due to the presence of a Trusted Third Party (TTP) (i.e. a certificate authority (CA) for public key cryptography (PKC) and a key generation center (KGC) for an ID-PKC), that must be trusted. Additionally, the presence of high communication cost associated with the transmission of homomorphically encrypted gradients from participants to the cloud server in distributed deep learning needs more attention. Furthermore, few schemes have been proposed for DDL that have post-quantum robustness. We, therefore, propose and design a new privacy-preserving distributed deep learning solution. First, a new LWE-based certificateless additively homomorphic encryption scheme called CAHE is proposed and designed. Secondly, partial sharing algorithms based on random selection and selection of the largest (or topmost) gradients in terms of magnitude are proposed. Thirdly, we put forward a new privacy-preserving distributed deep learning framework that combines the certificateless additively homomorphic encryption and partial sharing in the DDL. Our approach provides better privacy (i.e. not relying solely on the TTP) and also reduces communication cost for distributed deep learning. We note emphatically that this is the first time certificateless HE is being proposed, designed, and implemented in DDL. Results from our work indicate that the accuracy of the CAHE-based deep learning does not deteriorate (97.20%) while greatly preserving the privacy of participant’s dataset. With partial sharing, our solution records 97.17% and 97.12% accuracy for the selection of topmost gradients and random selection of gradients scenarios respectively, and with a great reduction in communication cost for transmitting encrypted gradients.