Using Convex Relaxations for Efficiently and Privately Releasing Marginals

Abstract

Differential privacy is a definition giving a strong privacy guarantee even in the presence of auxiliary information. In this work we pursue the application of geometric techniques for achieving differential privacy, a highly promising line of work initiated by Hardt and Talwar [26], focusing on the problem of marginal release. Here, a database is a collection of the data of n individuals, each characterized by d binary attributes. A k-way marginal query is specified by a subset S of k attributes, together with a |S|-dimensional binary vector β specifying their values. The true answer to this query is a count of the number of people in the database whose attribute vector restricted to S agrees with β. Information theoretically, the error complexity of marginal queries -- how wrong do the answers have to be in order to preserve differential privacy -- is well-understood: the perquery additive error is known to be at least Ω(min{√n,dk/2}) and at most O(√nd1/4,dk/2). However, no polynomial time algorithm with error complexity as low as the information theoretic upper bound is known for small n. We present a polynomial time algorithm that matches the best known information-theoretic bounds when k = 2; more generally, by reducing to the case k = 2, for any distribution on marginal queries, our algorithm achieves average error at most O(√nd[k/2]/4), an improvement over previous work on when k is small and when error o(n) is desirable. Using private boosting we are also able to give nearly matching worst-case error bounds. Our algorithms are based on the geometric techniques of Nikolov, Talwar, and Zhang [29], wherein a vector of sufficiently noisy answers is projected onto a particular convex body. We reduce projection, which is expensive, to a simple geometric question: given (a succinct representation of) a convex body K, find a containing convex body L that one can efficiently optimize over, while keeping the Gaussian width of L small. This reduction is achieved by a careful use of the Frank-Wolfe algorithm.