SafeNet: The Unreasonable Effectiveness of Ensembles in Private Collaborative Learning

Abstract

Secure multiparty computation (MPC) has been proposed to allow multiple mutually distrustful data owners to jointly train machine learning (ML) models on their combined data. However, by design, MPC protocols faithfully compute the training functionality, which is susceptible to poisoning and privacy attacks, as shown by the adversarial ML community. In this work, we argue that model ensembles, implemented in our framework called SafeNet, are a highly MPC-amenable way to avoid many adversarial ML attacks. The natural partitioning of data amongst owners in MPC training allows this approach to be highly scalable at training time, provide provable protection from poisoning attacks, and provable defense against a number of privacy attacks. We demonstrate SafeNet's efficiency, accuracy, and resilience to poisoning on several machine learning datasets and models trained in end-to-end and transfer learning scenarios. For instance, SafeNet reduces backdoor attack success significantly, while achieving 39× faster training and 36× less communication than the four-party MPC framework of Dalskov et al. [31]. Our experiments show that ensemble learning retains these benefits even in many non-iid settings. The simplicity, cheap setup, and robustness properties of ensemble learning make it a strong first choice for training ML models privately in MPC.