VPP: Privacy Preserving Machine Learning via Undervolting

Abstract

Machine Learning (ML) systems are susceptible to membership inference attacks (MIAs), which leak private information from the training data. Specifically, MIAs are able to infer whether a target sample has been used in the training data of a given model. Such privacy breaching concern motivated several defenses against MIAs. However, most of the state-of-theart defenses such as Differential Privacy (DP) come at the cost of lower utility (i.e, classification accuracy). In this work, we propose Privacy Preserving Volt $(V_{PP})$, a new lightweight inference-time approach that leverages undervolting for privacy-preserving ML. Unlike related work, V <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">PP</inf> maintains protected models’ utility without requiring re-training. The key insight of our method is to blur the MIA differential analysis outcome by comprehensively garbling the model features using random noise. Unlike DP, which injects noise within the gradient at training time, V <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">PP</inf> injects computational randomness in a set of layers’ during inference through carefully designed undervolting Specifically, we propose a bi-objective optimization approach to identify the noise characteristics that yield privacypreserving properties while maintaining the protected model’s utility. Extensive experimental results demonstrate that V <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">PP</inf> yields a significantly more interesting utility/privacy tradeoff compared to prior defenses. For example, with comparable privacy protection on CIFAR-10 benchmark, V <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">PP</inf> improves the utility by 32.93% over DP-SGD. Besides, while related noisebased defenses are defeated by label-only attacks, V <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">PP</inf> shows high resilience to such adaptive MLA. More over, V <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">PP</inf> comes with a by-product inference power gain of up to 61%. Finally, for a comprehensive analysis, we propose a new adaptive attacks that operate on the expectation over the stochastic model behavior. We believe that V <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">PP</inf> represents a significant step towards practical privacy preserving techniques and considerably improves the state-of-the-art.