Towards high performance homomorphic encryption for inference tasks on CPU: An MPI approach

Abstract

Fully homomorphic encryption (FHE) is a powerful tool enabling secure outsourcing to untrusted third-parties. As AI technologies mature and companies start offering AI-as-a-service (AIaaS), the privacy of their customers’ data needs to be considered but the high overhead of existing FHE schemes is a major limitation. Prior work tried to address this but suffered accuracy degradation, lack of scalability, ciphertext expansion, and impractical performance issues. In this paper, we explore the utility of High-Performance Computing systems for accelerating the secure evaluation of deep convolutional neural networks (CNN) with FHE. We examine previous encrypted neural network architectures and propose a holistic re-design for the HPC environment, which introduces a new consideration for the latency of inter-node communication. The new architecture was implemented with MPI and OpenMP and its performance benchmarked against state-of-the-art FHE-based CNN evaluation methods. Our solution achieved improvements of between 6−34× for various CIFAR10 CNN architectures and more than 1500× for AlexNet, bringing FHE-based evaluation of AlexNet down from several hundred days to within 14 h. Such findings pave the way towards practical inference on encrypted medium resolution images through the use of HPC systems.