Privacy-Guaranteed Two-Agent Interactions Using Information-Theoretic Mechanisms

Abstract

This paper introduces a multi-round interaction problem with privacy constraints between two agents that observe correlated data. The data are assumed to have both public and private features, and the goal of the interaction is to share the public data subject to utility constraints (bounds on the distortion of public feature) while ensuring bounds on the information leakage of the private data at the other agent. The agents alternately share data with one another for a total of $K$ rounds such that each agent initiates sharing over $K/2$ rounds. The interactions are modeled as a collection of $K$ random mechanisms (mappings), one for each round. The goal is to jointly design the $K$ private mechanisms to determine the set of all achievable distortion-leakage pairs at each agent. Arguing that a mutual information-based leakage metric can be appropriate for streaming data settings, this paper: 1) determines the set of all achievable distortion-leakage tuples; 2) shows that the $K$ mechanisms allow for precisely composing the total privacy budget over $K$ rounds without loss; and 3) develops conditions under which interaction reduces the net leakage at both agents and illustrates it for a specific class of sources. The paper then focuses on log-loss distortion to better understand the effect on leakage of using a commonly used utility metric in learning theory. The resulting interaction problem leads to a non-convex sum-leakage-distortion optimization problem that can be viewed as an interactive version of the information bottleneck problem. A new merge-and-search algorithm that extends the classical agglomerative information bottleneck algorithm to the interactive setting is introduced to determine a provable locally optimal solution. Finally, the benefit of interaction under log-loss is illustrated for specific source classes, and the optimality of one-shot is proved for Gaussian sources under both mean-square and log-loss distortions constraints.