Practical Private Computation and Zero-Knowledge Tools for Privacy-Preserving Distributed Data Mining

Abstract

In this paper we explore private computation built on vector addition and its applications in privacy-preserving data mining. Vector addition is a surprisingly general tool for implementing many algorithms prevalent in distributed data mining. Examples include linear algorithms like voting and summation, as well as non-linear algorithms such as SVD, PCA, k-means, ID3, machine learning algorithms based on Expectation Maximization (EM), etc., and all algorithms in the statistical query model [27]. The non-linear algorithms aggregate data only in certain steps, such as conjugate gradient, which are linear in the data. We introduce a new and highly efficient VSS (Verifiable Secret-Sharing) protocol in a special but widely-applicable model that allows secret-shared arithmetic operations in such aggregation steps to be done over small fields (e.g. 32 or 64 bits). There are two major advantages: (1) in this framework private arithmetic operations have the same cost as normal arithmetic and (2) the scheme admits extremely efficient zero-knowledge (ZK) protocols for verifying properties of user data. As a concrete example, we present a very efficient zero-knowledge method based on random projection for verification that uses a linear number of inexpensive small field operations, and only a logarithmic number of large-field (1024 bits or more) cryptographic operations. Our implementation shows that the approach can achieve orders of magnitude reduction in running time over standard techniques (from hours to seconds) for large scale problems. The ZK tools provide efficient mechanisms for dealing with actively cheating users, a realistic threat in distributed data mining which has been lacking practical solutions.