PvCNN: Privacy-Preserving and Verifiable Convolutional Neural Network Testing

Abstract

We propose a new approach for privacy-preserving and verifiable convolutional neural network (CNN) testing in a distrustful multi-stakeholder environment. The approach is aimed to enable that a CNN model <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">developer</i> convinces a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">user</i> of the truthful CNN performance over non-public data from <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">multiple testers</i> , while respecting model and data privacy. To balance the security and efficiency issues, we appropriately integrate three tools with the CNN testing, including collaborative inference, homomorphic encryption (HE) and zero-knowledge succinct non-interactive argument of knowledge (zk-SNARK). We start with strategically partitioning a CNN model into a private part kept locally by the model developer, and a public part outsourced to an outside server. Then, the private part runs over the HE-protected test data sent by a tester, and transmits its outputs to the public part for accomplishing subsequent computations of the CNN testing. Second, the correctness of the above CNN testing is enforced by generating zk-SNARK based proofs, with an emphasis on optimizing proving overhead for two-dimensional (2-D) convolution operations, since the operations dominate the performance bottleneck during generating proofs. We specifically present a new quadratic matrix program (QMP)-based arithmetic circuit with <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a single multiplication gate</i> for expressing 2-D convolution operations between multiple filters and inputs in a batch manner. Third, we aggregate multiple proofs with respect to a same CNN model but different testers’ test data ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e</i> ., different statements) into one proof, and ensure that the validity of the aggregated proof implies the validity of the original multiple proofs. Lastly, our experimental results demonstrate that our QMP-based zk-SNARK performs nearly 13.9× faster than the existing quadratic arithmetic program (QAP)-based zk-SNARK in proving time, and 17.6× faster in Setup time, for high-dimension matrix multiplication. Besides, the limitation on handling a bounded number of multiplications of QAP-based zk-SNARK is relieved.