Model Inversion Attacks Research Articles

Privacy-SF: An encoding-based privacy-preserving segmentation framework for medical images

Deep learning is becoming increasingly popular and is being extensively used in the field of medical image analysis. However, the privacy sensitivity of medical data limits the availability of data, which constrains the advancement of medical image analysis and impedes collaboration across multiple centers. To address this problem, we propose a novel encoding-based framework, named Privacy-SF, aimed at implementing privacy-preserving segmentation for medical images. Our proposed segmentation framework consists of three CNN networks: 1) two encoding networks on the client side that encode medical images and their corresponding segmentation masks individually to remove the privacy features, 2) a unique mapping network that analyzes the content of encoded data and learns the mapping from the encoded image to the encoded mask. By sequentially encoding data and optimizing the mapping network, our approach ensures privacy protection for images and masks during both the training and inference phases of medical image analysis. Additionally, to further improve the segmentation performance, we carefully design augmentation strategies specifically for encoded data based on its sequence nature. Extensive experiments conducted on five datasets with different modalities demonstrate excellent performance in privacy-preserving segmentation and multi-center collaboration. Furthermore, the analysis of encoded data and the experiment of model inversion attacks validate the privacy-preserving capability of our approach.

Privacy-Preserving Task-Oriented Semantic Communications Against Model Inversion Attacks

Privacy-Preserving Task-Oriented Semantic Communications Against Model Inversion Attacks

The Role of Class Information in Model Inversion Attacks Against Image Deep Learning Classifiers

The Role of Class Information in Model Inversion Attacks Against Image Deep Learning Classifiers

Aligning the domains in cross domain model inversion attack

Model Inversion Attack reconstructs confidential training dataset from a target deep learning model. Most of the existing methods assume the adversary has an auxiliary dataset that has similar distribution with the private dataset. However, this assumption does not always hold in real-world scenarios. Since the private dataset is unknown, the domain divergence between the auxiliary dataset and the private dataset is inevitable. In this paper, we use Cross Domain Model Inversion Attack to represent the distribution divergence scenario in MIA. With the distribution divergence between the private images and auxiliary images, the distribution between the feature vectors of the private images and those of the auxiliary images is also different. Moreover, the outputted prediction vectors of the auxiliary images are also misclassified. The inversion attack is thus hard to be performed. We perform both the feature vector inversion task and prediction vector inversion task in this cross domain setting. For feature vector inversion, Domain Alignment MIA (DA-MIA) is proposed. While performing the reconstruction task, DA-MIA aligns the feature vectors of auxiliary images with the feature vectors of private images in an adversarial manner to mitigate the domain divergence between them. Thus, semantically meaningful images can be reconstructed. For prediction vector inversion, we further introduce an auxiliary classifier and propose Domain Alignment MIA with Auxiliary Classifier (DA-MIA-AC). The auxiliary classifier is pretrained by the auxiliary dataset and fine-tuned during the adversarial training stage. Thus, the misclassification problem caused by domain divergence can be solved, and the images can be reconstructed correctly. Various experiments are performed to show the advancement of our methods, the results show that DA-MIA can improve the SSIM score of the reconstructed images for up to 191%, DA-MIA-AC can increase the classification accuracy score of the reconstructed images from 9.18% to 81.32% in Cross Domain Model Inversion Attack.

Targeted Model Inversion: Distilling style encoded in predictions

Previous model inversion (MI) research has demonstrated the feasibility of reconstructing images representative of specific classes, inadvertently revealing additional feature information. However, there are two remaining challenges for practical black-box MI: (1) minimizing the number of queries to the target model, and (2) reconstructing a high-quality input image tailored to an observed prediction vector. We introduce Targeted Model Inversion (TMI), a practical black-box MI attack. Our approach involves altering the mapping network in StyleGAN, which projects an observed prediction vector into a StyleGAN latent representation. Later, TMI leverages a surrogate model that is also derived from StyleGAN to guide instance-specific MI by optimizing the latent representation. These mapping and surrogate networks work together to conduct high-fidelity MI while significantly decreasing the number of necessary queries. Our experiments demonstrate that TMI outperforms state-of-the-art MI methods, demonstrating a new upper bound on the susceptibility to black-box MI attacks.

Game Theory and Adversarial Machine Learning: Analyzing Strategic Interactions in Cybersecurity

When it comes to cybersecurity, the way that attackers and defenders work together strategically is becoming more and more like how games work in general. Adversarial machine learning (AML) has become an important area of hacking. In AML, attackers use complex methods to avoid being caught and take advantage of flaws in machine learning models. The goal of this study is to give a full picture of how strategies combine in cybersecurity by looking at where game theory and AML meet. You can think of the strategic exchanges in cybersecurity as a game between attackers and defenders. Defenders want to keep systems and data safe, and enemies want to break into them for bad reasons. In this game, players have to make a lot of decisions. Defenders have to think ahead and prepare for possible attacks, while attackers change their plans all the time to avoid being caught and take advantage of defenses' weaknesses. Game theory gives us a way to formally model these interactions, which lets us look at the best tactics and results that are in balance. During AML, defenders use machine learning models to find and stop security risks, while attackers change these models using methods like escape attacks, poisoning attacks, and model inversion attacks. In these hostile tactics, there is a strategic element added to cybersecurity, where defenders must think about the goals and skills of attackers when creating and using defenses. This paper looks at the research that has already been done on using game theory to study hacking and how these models can be used to study how strategies combine in AML. It talks about different types of games, like rigid and dynamic games, and what they mean for protecting against threats from other players. Besides that, it looks into how uneven knowledge, doubt, and strategy learning affect the results of computer games. This paper helps us understand the strategy problems that both attackers and defenders face in cybersecurity better by combining game theory with AML. It shows how important it is to think strategically and build weapons that can change to deal with risks that change, and it sets the stage for future study in this field, which is changing very quickly.

[formula omitted]: Self-sovereign identity based privacy-preserving federated learning

Traditional federated learning (FL) raises security and privacy concerns such as identity fraud, data poisoning attacks, membership inference attacks, and model inversion attacks. In the conventional FL, any entity can falsify its identity and initiate data poisoning attacks. Besides, adversaries (AD) holding the updated global model parameters can retrieve the plain text of the dataset by initiating membership inference attacks and model inversion attacks. To the best of our knowledge, this is the first work to propose a self-sovereign identity (SSI) and differential privacy (DP) based FL namely SSI−FL for addressing all the above issues. The first step in the SSI−FL framework involves establishing a secure connection based on blockchain-based SSI. This secure connection protects against unauthorized access attacks of any AD and ensures the transmitted data's authenticity, integrity, and availability. The second step applies DP to protect against model inversion attacks and membership inference attacks. The third step focuses on establishing FL with a novel hybrid deep learning to achieve better scores than conventional methods. The SSI−FL performance analysis is done based on security, formal, scalability, and score analysis. Moreover, the proposed method outperforms all the state-of-art techniques.

Machine Learning in Cybersecurity : Applications, Challenges and Future Directions

Machine learning (ML) is transforming cybersecurity by enabling advanced detection, prevention and response mechanisms. This paper provides a comprehensive review of ML's role in cybersecurity, examining both theoretical frameworks and practical implementations. It outlines the emerging threats targeting ML models, such as adversarial attacks, data poisoning and model inversion attacks and discusses state-of-the-art defense strategies, including adversarial training, robust architectures and differential privacy. Additionally, the paper explores various ML applications in cybersecurity from intrusion detection to malware classification, highlighting their impact on enhancing security measures. An anomaly inference algorithm is proposed for the early detection of cyber-intrusions at the substations. Cybersecurity has become a vital research area. The paper concludes with a discussion on the key research directions and best practices for creating secure and resilient ML systems in a data-driven world. This paper delves into how Machine Learning (ML) revolutionizes cybersecurity, empowering advanced detection, prevention, and response mechanisms. It offers a thorough exploration of ML's pivotal role in cybersecurity, encompassing theoretical frameworks and practical applications. It addresses emerging threats like adversarial attacks and data poisoning, alongside cutting-edge defense strategies such as adversarial training and robust architectures.

C2FMI: Corse-to-Fine Black-Box Model Inversion Attack

Privacy-preserving machine learning requires that models do not reveal any private information about their training data. However, model inversion attacks (MIAs), which aim to recover the features of training data, pose a huge threat to the security of AI models. Most existing MIAs assume that the target model is white-box, but most models deployed in reality are black-box, and these models can only be accessed like an oracle. There are a few studies for black-box scenarios, but their performance is limited. In this paper, we firstly formulate the MIA problem completely in Bayesian perspective. Second, we propose a novel two-stage MIA approach, the Coarse-to-Fine Model Inversion Attack (C2FMI), which efficiently addresses the MIA problem in the black-box scenario. In stage I of C2FMI, we design a reverse network that constrains the recovered images (also named attacked images) to fall near the manifold space of the training data. In stage II, we design a black-box oriented strategy which further facilitates the attacked images to approach the training data. Empirically, C2FMI achieves a performance that even surpasses existing white-box attack methods. Furthermore, we design the stability analysis method for analyzing the stability of C2FMI along with existing MIAs. Finally, we explore the potential countermeasures which could defend against our attacks.

Boosting Model Inversion Attacks With Adversarial Examples

Boosting Model Inversion Attacks With Adversarial Examples

Concealing Sensitive Samples against Gradient Leakage in Federated Learning

Federated Learning (FL) is a distributed learning paradigm that enhances users' privacy by eliminating the need for clients to share raw, private data with the server. Despite the success, recent studies expose the vulnerability of FL to model inversion attacks, where adversaries reconstruct users’ private data via eavesdropping on the shared gradient information. We hypothesize that a key factor in the success of such attacks is the low entanglement among gradients per data within the batch during stochastic optimization. This creates a vulnerability that an adversary can exploit to reconstruct the sensitive data. Building upon this insight, we present a simple, yet effective defense strategy that obfuscates the gradients of the sensitive data with concealed samples. To achieve this, we propose synthesizing concealed samples to mimic the sensitive data at the gradient level while ensuring their visual dissimilarity from the actual sensitive data. Compared to the previous art, our empirical evaluations suggest that the proposed technique provides the strongest protection while simultaneously maintaining the FL performance. Code is located at https://github.com/JingWu321/DCS-2.

IPRemover: A Generative Model Inversion Attack against Deep Neural Network Fingerprinting and Watermarking

Training Deep Neural Networks (DNNs) can be expensive when data is difficult to obtain or labeling them requires significant domain expertise. Hence, it is crucial that the Intellectual Property (IP) of DNNs trained on valuable data be protected against IP infringement. DNN fingerprinting and watermarking are two lines of work in DNN IP protection. Recently proposed DNN fingerprinting techniques are able to detect IP infringement while preserving model performance by relying on the key assumption that the decision boundaries of independently trained models are intrinsically different from one another. In contrast, DNN watermarking embeds a watermark in a model and verifies IP infringement if an identical or similar watermark is extracted from a suspect model. The techniques deployed in fingerprinting and watermarking vary significantly because their underlying mechanisms are different. From an adversary's perspective, a successful IP removal attack should defeat both fingerprinting and watermarking. However, to the best of our knowledge, there is no work on such attacks in the literature yet. In this paper, we fill this gap by presenting an IP removal attack that can defeat both fingerprinting and watermarking. We consider the challenging data-free scenario whereby all data is inverted from the victim model. Under this setting, a stolen model only depends on the victim model. Experimental results demonstrate the success of our attack in defeating state-of-the-art DNN fingerprinting and watermarking techniques. This work reveals a novel attack surface that exploits generative model inversion attacks to bypass DNN IP defenses. This threat must be addressed by future defenses for reliable IP protection.

Random forest evaluation using multi-key homomorphic encryption and lookup tables

In recent years, machine learning (ML) has become increasingly popular in various fields of activity. Cloud platforms have also grown in popularity, as they offer services that are more secure and accessible worldwide. In this context, cloud-based technologies emerged to support ML, giving rise to the machine learning as a service (MLaaS) concept. However, the clients accessing ML services in order to obtain classification results on private data may be reluctant to upload sensitive information to cloud. The model owners may also prefer not to outsource their models in order to prevent model inversion attacks and to protect intellectual property. The privacy-preserving evaluation of ML models is possible through multi-key homomorphic encryption (MKHE), that allows both the client data and the model to be encrypted under different keys. In this paper, we propose an MKHE evaluation method for decision trees and we extend the proposed method for random forests. Each decision tree is evaluated as a single lookup table, and voting is performed at the level of groups of decision trees in the random forest. We provide both theoretical and experimental evaluations for the proposed method. The aim is to minimize the performance degradation introduced by the encrypted model compared to a plaintext model while also obtaining practical classification times. In our experiments with the proposed MKHE random forest evaluation method, we obtained minimal (less than 0.6%) impact on the main ML performance metrics considered for each scenario, while also achieving reasonable classification times (of the order of seconds).

Privacy-Preserving Neural Networks for Smart Manufacturing

Abstract The rapid advance in sensing technology has expedited data-driven innovation in manufacturing by enabling the collection of large amounts of data from factories. Big data provides an unprecedented opportunity for smart decision-making in the manufacturing process. However, big data also attracts cyberattacks and makes manufacturing systems vulnerable due to the inherent value of sensitive information. The increasing integration of artificial intelligence (AI) within smart factories also exposes manufacturing equipment susceptible to cyber threats, posing a critical risk to the integrity of smart manufacturing systems. Cyberattacks targeting manufacturing data can result in considerable financial losses and severe business disruption. Therefore, there is an urgent need to develop AI models that incorporate privacy-preserving methods to protect sensitive information implicit in the models against model inversion attacks. Hence, this paper presents the development of a new approach called mosaic neuron perturbation (MNP) to preserve latent information in the framework of the AI model, ensuring differential privacy requirements while mitigating the risk of model inversion attacks. MNP is flexible to implement into AI models, balancing the trade-off between model performance and robustness against cyberattacks while being highly scalable for large-scale computing. Experimental results, based on real-world manufacturing data collected from the computer numerical control (CNC) turning process, demonstrate that the proposed method significantly improves the ability to prevent inversion attacks while maintaining high prediction performance. The MNP method shows strong potential for making manufacturing systems both smart and secure by addressing the risk of data breaches while preserving the quality of AI models.

RVE-PFL: Robust Variational Encoder-Based Personalized Federated Learning Against Model Inversion Attacks

RVE-PFL: Robust Variational Encoder-Based Personalized Federated Learning Against Model Inversion Attacks

F-NIDS — A Network Intrusion Detection System based on federated learning

The rise of IoT networks has presented fresh challenges in terms of scalability and security for distributed Network Intrusion Detection Systems (NIDS) due to privacy concerns. While some progress has been made in addressing these challenges, there are still unanswered questions regarding how to achieve a balance between performance and robustness to ensure privacy in a distributed manner. Additionally, there is a need to develop a reliable and scalable architecture for distributed NIDS that can be effectively deployed in various IoT scenarios. These questions about robustness relied mainly on choosing privacy-secured and distributed Machine Learning techniques. In this work, we propose the F-NIDS, an intrusion detector that utilizes federated artificial intelligence and asynchronous communication techniques between system entities to provide horizontal scalability, along with differential privacy techniques to address data confidentiality concerns. The architecture of F-NIDS is designed to be adaptable for usage in IoT networks, suited to be used in cloud or fog-based environments. Results from our experiments have shown that the confidential detection model employed in F-NIDS – considering multi-class accuracy, binary accuracy, precision, and recall metrics – was capable of predicting and determining the nature of attacks when they occur. In order to determine optimal parameters that strike a balance between data privacy and classification performance, three strategies were employed, each evaluated for its corresponding robustness performance. Firstly, models were trained with varying Gaussian noise values, and subjected to membership inference black box rule-based attacks. Secondly, regular membership inference black box attacks were performed, utilizing different stolen samples with varying sizes to determine the maximum amount of data that could be securely stored on the detection agents for training tasks. Lastly, the robustness of the trained models was evaluated against a model inversion attack, and the results were compared through graphical comparisons. Based on these evaluations, Gaussian noise level and sample size values of 21 were obtained for each detection agent in the system, with sample sizes ranging from 10K to 25K.

Model Inversion Attacks Against Graph Neural Networks

Many data mining tasks rely on graphs to model relational structures among individuals (nodes). Since relational data are often sensitive, there is an urgent need to evaluate the privacy risks in graph data. One famous privacy attack against data analysis models is the model inversion attack, which aims to infer sensitive data in the training dataset and leads to great privacy concerns. Despite its success in grid-like domains, directly applying model inversion attacks on non-grid domains such as graph leads to poor attack performance. This is mainly due to the failure to consider the unique properties of graphs. To bridge this gap, we conduct a systematic study on model inversion attacks against Graph Neural Networks (GNNs), one of the state-of-the-art graph analysis tools in this paper. Firstly, in the white-box setting where the attacker has full access to the target GNN model, we present GraphMI to infer the private training graph data. Specifically in GraphMI, a projected gradient module is proposed to tackle the discreteness of graph edges and preserve the sparsity and smoothness of graph features; a graph auto-encoder module is used to efficiently exploit graph topology, node attributes, and target model parameters for edge inference; a random sampling module can finally sample discrete edges. Furthermore, in the hard-label black-box setting where the attacker can only query the GNN API and receive the classification results, we propose two methods based on gradient estimation and reinforcement learning (RL-GraphMI). With the proposed methods, we study the connection between model inversion risk and edge influence and show that edges with greater influence are more likely to be recovered. Extensive experiments over several public datasets demonstrate the effectiveness of our methods. We also evaluate our attacks under two defenses: one is the well-designed differential private training, and the other is graph preprocessing. Our experimental results show that such defenses are not sufficiently effective and call for more advanced defenses against privacy attacks.

Get Your Foes Fooled: Proximal Gradient Split Learning for Defense Against Model Inversion Attacks on IoMT Data

The past decade has seen a rapid adoption of Artificial Intelligence (AI), specifically the deep learning networks, in Internet of Medical Things (IoMT) ecosystem. However, it has been shown recently that the deep learning networks can be exploited by adversarial attacks that not only make IoMT vulnerable to the data theft but also to the manipulation of medical diagnosis. The existing studies consider adding noise to the raw IoMT data or model parameters which not only reduces the overall performance concerning medical inferences but also is ineffective to the likes of deep leakage from gradients method. In this work, we propose proximal gradient split learning (PSGL) method for defense against the model inversion attacks. The proposed method intentionally attacks the IoMT data when undergoing the deep neural network training process at client side. We propose the use of proximal gradient method to recover gradient maps and a decision-level fusion strategy to improve the recognition performance. Extensive analysis show that the PGSL not only provides effective defense mechanism against the model inversion attacks but also helps in improving the recognition performance on publicly available datasets. We report 14.0$\%$, 17.9$\%$, and 36.9$\%$ gains in accuracy over reconstructed and adversarial attacked images, respectively.

Homomorphic Encryption-Based Privacy-Preserving Federated Learning in IoT-Enabled Healthcare System

In this work, the federated learning mechanism is introduced into the deep learning of medical models in Internet of Things (IoT)-based healthcare system. Cryptographic primitives, including masks and homomorphic encryption, are applied for further protecting local models, so as to prevent the adversary from inferring private medical data by various attacks such as model reconstruction attack or model inversion attack, etc. The qualities of the datasets owned by different participants are considered as the main factor for measuring the contribution rate of the local model to the global model in each training epoch, instead of the size of datasets commonly used in deep learning. A dropout-tolerable scheme is proposed in which the process of federated learning would not be terminated if the number of online clients is not less than a preset threshold. Through the analysis of the security, it shows that the proposed scheme satisfies data privacy. Computation cost and communication cost are also analyzed theoretically. Finally, skin lesion classification using training images provided by the HAM10000 medical dataset is set as an example of healthcare applications. Experimental results show that compared with existing schemes, the proposed scheme obtained promising results while ensuring privacy preserving.

FedHEONN: Federated and homomorphically encrypted learning method for one-layer neural networks

Federated learning (FL) is a distributed approach to developing collaborative learning models from decentralized data. This is relevant to many real applications, such as in the field of the Internet of Things, since the models can be used in edge computing devices. FL approaches are motivated by and designed to protect privacy, a highly relevant issue given current data protection regulations. Although FL methods are privacy-preserving by design, recently published papers show that privacy leaks do occur, caused by attacks designed to extract private data from information interchanged during learning. In this work, we present an FL method based on a neural network without hidden layers that incorporates homomorphic encryption (HE) to enhance robustness against the above-mentioned attacks. Unlike traditional FL methods that require multiple rounds of training for convergence, our method obtains the collaborative global model in a single training round, yielding an effective and efficient model that simplifies management of the FL training process. In addition, since our method includes HE, it is also robust against model inversion attacks. In experiments with big data sets and a large number of clients in a federated scenario, we demonstrate that use of HE does not affect the accuracy of the model, whose results are competitive with state-of-the-art machine learning models. We also show that behavior in terms of accuracy is the same for identically and non-identically distributed data scenarios.