Membership Inference Attacks Research Articles

Анализ безопасности проекта национального стандарта «Нейросетевые алгоритмы в защищенном исполнении. Автоматическое обучение нейросетевых моделей на малых выборках в задачах классификации»

We propose a membership inference attack against the neural classification algorithm from the draft national standard developed by the Omsk State Technical University under the auspices of the Technical Committee on Standardization «Artificial Intelligence» (TC 164). The attack allows us to determine whether the data were used for neural network training, and aimed at violating the confidentiality property of the training set. The results show that the protection mechanism of neural network classifiers described by the draft national standard does not provide the declared properties. The results were previously announced at Ruscrypto’2023 conference.

Performance analysis of various machine learning models for membership inference attack

Performance analysis of various machine learning models for membership inference attack

Subject-Level Membership Inference Attack via Data Augmentation and Model Discrepancy

Subject-Level Membership Inference Attack via Data Augmentation and Model Discrepancy

I-GWAS: Privacy-Preserving Interdependent Genome-Wide Association Studies

Genome-wide Association Studies (GWASes) identify genomic variations that are statistically associated with a trait, such as a disease, in a group of individuals. Unfortunately, careless sharing of GWAS statistics might give rise to privacy attacks. Several works attempted to reconcile secure processing with privacy-preserving releases of GWASes. However, we highlight that these approaches remain vulnerable if GWASes utilize overlapping sets of individuals and genomic variations. In such conditions, we show that even when relying on state-of-the-art techniques for protecting releases, an adversary could reconstruct the genomic variations of up to 28.6% of participants, and that the released statistics of up to 92.3% of the genomic variations would enable membership inference attacks. We introduce I-GWAS, a novel framework that securely computes and releases the results of multiple possibly interdependent GWASes. I-GWAS continuously releases privacy-preserving and noise-free GWAS results as new genomes become available.

TEAR: Exploring Temporal Evolution of Adversarial Robustness for Membership Inference Attacks Against Federated Learning

TEAR: Exploring Temporal Evolution of Adversarial Robustness for Membership Inference Attacks Against Federated Learning

Performance Analysis of Various Machine Learning Models for Membership Inference Attack

Performance Analysis of Various Machine Learning Models for Membership Inference Attack

Defense against membership inference attack in graph neural networks through graph perturbation.

Graph neural networks have demonstrated remarkable performance in learning node or graph representations for various graph-related tasks. However, learning with graph data or its embedded representations may induce privacy issues when the node representations contain sensitive or private user information. Although many machine learning models or techniques have been proposed for privacy preservation of traditional non-graph structured data, there is limited work to address graph privacy concerns. In this paper, we investigate the privacy problem of embedding representations of nodes, in which an adversary can infer the user's privacy by designing an inference attack algorithm. To address this problem, we develop a defense algorithm against white-box membership inference attacks, based on perturbation injection on the graph. In particular, we employ a graph reconstruction model and inject a certain size of noise into the intermediate output of the model, i.e., the latent representations of the nodes. The experimental results obtained on real-world datasets, along with reasonable usability and privacy metrics, demonstrate that our proposed approach can effectively resist membership inference attacks. Meanwhile, based on our method, the trade-off between usability and privacy brought by defense measures can be observed intuitively, which provides a reference for subsequent research in the field of graph privacy protection.

An Understanding of the Vulnerability of Datasets to Disparate Membership Inference Attacks

Recent efforts have shown that training data is not secured through the generalization and abstraction of algorithms. This vulnerability to the training data has been expressed through membership inference attacks that seek to discover the use of specific records within the training dataset of a model. Additionally, disparate membership inference attacks have been shown to achieve better accuracy compared with their macro attack counterparts. These disparate membership inference attacks use a pragmatic approach to attack individual, more vulnerable sub-sets of the data, such as underrepresented classes. While previous work in this field has explored model vulnerability to these attacks, this effort explores the vulnerability of datasets themselves to disparate membership inference attacks. This is accomplished through the development of a vulnerability-classification model that classifies datasets as vulnerable or secure to these attacks. To develop this model, a vulnerability-classification dataset is developed from over 100 datasets—including frequently cited datasets within the field. These datasets are described using a feature set of over 100 features and assigned labels developed from a combination of various modeling and attack strategies. By averaging the attack accuracy over 13 different modeling and attack strategies, the authors explore the vulnerabilities of the datasets themselves as opposed to a particular modeling or attack effort. The in-class observational distance, width ratio, and the proportion of discrete features are found to dominate the attributes defining dataset vulnerability to disparate membership inference attacks. These features are explored in deeper detail and used to develop exploratory methods for hardening these class-based sub-datasets against attacks showing preliminary mitigation success with combinations of feature reduction and class-balancing strategies.

On the Privacy–Utility Trade-Off in Differentially Private Hierarchical Text Classification

Hierarchical text classification consists of classifying text documents into a hierarchy of classes and sub-classes. Although Artificial Neural Networks have proved useful to perform this task, unfortunately, they can leak training data information to adversaries due to training data memorization. Using differential privacy during model training can mitigate leakage attacks against trained models, enabling the models to be shared safely at the cost of reduced model accuracy. This work investigates the privacy–utility trade-off in hierarchical text classification with differential privacy guarantees, and it identifies neural network architectures that offer superior trade-offs. To this end, we use a white-box membership inference attack to empirically assess the information leakage of three widely used neural network architectures. We show that large differential privacy parameters already suffice to completely mitigate membership inference attacks, thus resulting only in a moderate decrease in model utility. More specifically, for large datasets with long texts, we observed Transformer-based models to achieve an overall favorable privacy–utility trade-off, while for smaller datasets with shorter texts, convolutional neural networks are preferable.

LDP‐Fed+: A robust and privacy‐preserving federated learning based classification framework enabled by local differential privacy

AbstractAs a distributed learning framework, Federated Learning (FL) allows different local learners/participants to collaboratively train a joint model without exposing their own local data, and offers a feasible solution to legally resolve data islands. However, among them, the data privacy and model security are two challenges. The former means that, if original data are used for trained FL models, various methods can be used to deduce the original data samples, thereby causing the leakage of data. The latter implies that unreliable/malicious participants may affect or destroy the joint FL model, through uploading wrong local model parameters. Therefore, this paper proposes a novel distributed FL training framework, namely LDP‐Fed+, which takes into account differential privacy protection and model security defense. Specifically, firstly, a local perturbation module is added at the local learner side, which perturbs the original data of local learners through feature extraction, binary encoding and decoding, and random response. Then, through using the perturbed data, local neural network model is trained to obtain the network parameters that meet local differential protection, to effectively deal with model inversion attacks. Secondly, a security defense module is added on the server side, which uses the auxiliary model and differential index mechanism to select an appropriate number of local disturbance parameters for aggregation to enhance model security defense and deal with membership inference attacks. The experimental results show that, compared with other federated learning models based on differential privacy, LDP‐Fed+ has stronger robustness for model security and higher accuracy for model training while ensuring strict privacy protection.

Toward Selective Membership Inference Attack against Deep Learning Model

Toward Selective Membership Inference Attack against Deep Learning Model

Understanding and defending against White-box membership inference attack in deep learning

Membership inference attacks (MIA) exploit the fact that deep learning algorithms leak information about their training data through the learned model. It has been treated as an indicator which reveals the privacy leakage of machine learning models. In this work, we aim to understand the advantage achieved by White-box MIA, and defend against White-box MIA. Firstly, we estimate the KL divergence on the hidden layers’ features between training set and test set as the internal generalization gap. By comparing the internal generalization gap and the generalization gap on the output layer, we raise two insights including (1) the existence of larger generalization gaps on hidden layers and (2) feasibility of generalization gap minimization in defending White-box MIA. Based on our insights, we further design a novel defense method named Nirvana. It intentionally minimizes generalization gap to defend White-box MIA. Formally, Nirvana works by selecting a hidden layer with large generalization gaps and executing a multi-samples convex combination among features on the layer during the training to defend against White-box MIA. Finally, we empirically evaluate Nirvana with state-of-the-art defense methods on CIFAR100 dataset, Purchase100 dataset, and Texas dataset. The experiment results show that Nirvana achieves a trade-off between utility and privacy. It can defend both White-box MIA and Black-box MIA while the test accuracy of the model is maintained. It outperforms previous defense methods in defending against White-box MIA.

Privacy‐preserving generative framework for images against membership inference attacks

Machine learning has become an integral part of modern intelligent systems in all aspects of life. Membership inference attacks (MIAs), as the significant model attacks, also jeopardize the privacy of the intelligent systems. Previous works on defending MIAs concentrate on the model output perturbation or tampering with the training process. However, data and model reuse are common in intelligent systems, which results in the lack of scalability of previous defending works. This paper proposes a new privacy-preserving framework for images to transform source data into synthetic data to train models against MIAs. The synthetic data makes it easy to defend MIAs during data and model reuse to improve the scheme's scalability. The framework generates synthetic data satisfying differential privacy through the variational autoencoder model's information extraction and data generation capabilities to improve model accuracy. A noise addition mechanism with metric privacy for the latent code generated from source data is proposed, where noise is the product of Γ-distribution and unit hyper-sphere samples. Moreover, it is proved that the synthetic data also satisfies metric privacy. The experimental evaluations demonstrate that the framework reduces MIAs' attack accuracy to about 0.5 and maintains higher utility than DP-SGD under the same setting.

Evaluating Membership Inference Through Adversarial Robustness

Abstract The usage of deep learning is being escalated in many applications. Due to its outstanding performance, it is being used in a variety of security and privacy-sensitive areas in addition to conventional applications. One of the key aspects of deep learning efficacy is to have abundant data. This trait leads to the usage of data which can be highly sensitive and private, which in turn causes wariness with regard to deep learning in the general public. Membership inference attacks are considered lethal as they can be used to figure out whether a piece of data belongs to the training dataset or not. This can be problematic with regard to leakage of training data information and its characteristics. To highlight the significance of these types of attacks, we propose an enhanced methodology for membership inference attacks based on adversarial robustness, by adjusting the directions of adversarial perturbations through label smoothing under a white-box setting. We evaluate our proposed method on three datasets: Fashion-MNIST, CIFAR-10 and CIFAR-100. Our experimental results reveal that the performance of our method surpasses that of the existing adversarial robustness-based method when attacking normally trained models. Additionally, through comparing our technique with the state-of-the-art metric-based membership inference methods, our proposed method also shows better performance when attacking adversarially trained models. The code for reproducing the results of this work is available at https://github.com/plll4zzx/Evaluating-Membership-Inference-Through-Adversarial-Robustness.

Budget Distributed Support Vector Machine for Non-ID Federated Learning Scenarios

In recent years, there has been remarkable growth in Federated Learning (FL) approaches because they have proven to be very effective in training large Machine Learning (ML) models and also serve to preserve data confidentiality, as recommended by the GDPR or other business confidentiality restrictions that may apply. Despite the success of FL, performance is greatly reduced when data is not distributed identically (non-ID) across participants, as local model updates tend to diverge from the optimal global solution and thus the model averaging procedure in the aggregator is less effective. Kernel methods such as Support Vector Machines (SVMs) have not seen an equivalent evolution in the area of privacy preserving edge computing because they suffer from inherent computational, privacy and scalability issues. Furthermore, non-linear SVMs do not naturally lead to federated schemes, since locally trained models cannot be passed to the aggregator because they reveal training data (they are built on Support Vectors), and the global model cannot be updated at every worker using gradient descent. In this article, we explore the use of a particular controlled complexity (“Budget”) Distributed SVM (BDSVM) in the FL scenario with non-ID data, which is the least favorable situation, but very common in practice. The proposed BDSVM algorithm is as follows: model weights are broadcasted to workers, which locally update some kernel Gram matrices computed according to a common architectural base and send them back to the aggregator, which finally combines them, updates the global model, and repeats the procedure until a convergence criterion is met. Experimental results using synthetic 2D datasets show that the proposed method can obtain maximal margin decision boundaries even when the data is non-ID distributed. Further experiments using real-world datasets with non-ID data distribution show that the proposed algorithm provides better performance with less communication requirements than a comparable Multilayer Perceptron (MLP) trained using FedAvg. The advantage is more remarkable for a larger number of edge devices. We have also demonstrated the robustness of the proposed method against information leakage, membership inference attacks, and situations with dropout or straggler participants. Finally, in experiments run on separate processes/machines interconnected via the cloud messaging service developed in the context of the EU-H2020 MUSKETEER project, BDSVM is able to train better models than FedAvg in about half the time.

Membership Inference Attacks on Machine Learning Model

Machine learning(ML) models today are vulnerable to several types of attacks. In this work, we will study a category of attack known as membership inference attack and show how ML models are susceptible to leaking secure information under such attacks. Given a data record and a black box access to a ML model, we present a framework to deduce whether the data record was part of the model’s training dataset or not. We achieve this objective by creating an attack ML model which learns to differentiate the target model’s predictions on its training data from target model’s predictions on data not part of its training data. In other words, we solve this membership inference problem by converting it into a binary classification problem. We also study mitigation strategies to defend the ML models against the attacks discussed in this work. In this paper evaluation method on real world datasets: (1) CIFAR-10 and (2) UCI Adult (Census Income) using classification as the task performed by the target ML models built on these datasets.

Leveraging Domain Adaptation as a Defense Against Membership Inference Attacks

Deep Learning (DL) techniques allow ones to train models from a dataset to solve tasks. DL has attracted much interest given its fancy performance and potential market value, while security issues are amongst the most colossal concerns. However, the DL models may be prone to the membership inference attack, where an attacker determines whether a given sample is from the training dataset. Efforts have been made to hinder the attack but unfortunately, they may lead to a major overhead or impaired usability. In this article, we propose and implement DAMIA, leveraging Domain Adaptation (DA) as a defense aginist membership inference attacks. Our observation is that during the training process, DA obfuscates the dataset to be protected using another relate and similar dataset, and derives a model that underlyingly extracts the features from both datasets. Seeing that the model is obfuscated, membership inference fails, while the extracted features provide supports for usability. Extensive experiments have been conducted to validates our intuition. The model trained by DAMIA has a negligible footprint to the usability and introduces slight overhead compared with other defenses. Our experiment also excludes factors that may hinder the performance of DAMIA, and comparisons with other defenses, providing a potential guideline to vendors and researchers to benefit from our solution in a timely manner.

Private Data Inference Attacks against Cloud: Model, Technologies, and Research Directions

Machine learning models are established with a variety of data collected from individual users who are concerned about their privacy. Various cloud service providers (e.g., Amazon, Google, Alibaba) commercialize their models by selling them or providing limited access via APIs. However, existing studies have shown that such cloud-based models are susceptible to training data inference attacks (i.e., membership inference attacks, attribute inference attacks, and model inversion attacks) since the trained models remember information about the training data. In this article, we perform an exhaustive investigation on the current training data inference attacks against cloud-based models. According to the attack scenario, we divide the existing inference attacks into two categories: centralized and collaborative learning scenarios. We first give a comprehensive review of attacks in each category and then give a qualitative comparison of the existing attacks. Further, we quantitatively compare the performance of different attack strategies by experiments. Last but not least, we pinpoint some open issues and potential future directions that need to be investigated in this area.

KD‐GAN: An effective membership inference attacks defence framework

KD‐GAN: An effective membership inference attacks defence framework

Label‐only membership inference attacks on machine unlearning without dependence of posteriors

Label‐only membership inference attacks on machine unlearning without dependence of posteriors