Adversary's Knowledge Research Articles

LG-AKD: Application of a lightweight GCN model based on adversarial knowledge distillation to skeleton action recognition

Human action recognition, a pivotal topic in computer vision, is a highly complex and challenging task. It requires the analysis of not only spatial dependencies of targets but also temporal changes in these targets. In recent decades, the advancement of deep learning has led to the development of numerous action recognition methods based on deep neural networks. Given that the skeleton points of the human body can be treated as a graph structure, graph neural networks (GNNs) have emerged as an effective tool for modeling such data, garnering significant interest from researchers. This paper aims to address the issue of low test speed caused by over-complicated deep graph convolutional models. To achieve this, we compress the network structure using knowledge distillation from a teacher-student architecture, leading to a compact and lightweight student GNN. To enhance the model’s robustness and generalization capabilities, we introduce a data augmentation mechanism that generates diverse action sequences while maintaining consistent behavior labels, thereby providing a more comprehensive learning basis for the model. The proposed model integrates three distinct knowledge learning paths: teacher networks, original datasets, and derived data. The fusion of knowledge distillation and data augmentation enables lightweight student networks to outperform their teacher networks in terms of both performance and efficiency. Experimental results demonstrate the efficacy of our approach in the context of skeleton-based human action recognition, highlighting its potential to simplify state-of-the-art models while enhancing their performance.

Graph-Based Model Compression for HSR Bogies Fault Diagnosis at IoT Edge via Adversarial Knowledge Distillation

Graph-Based Model Compression for HSR Bogies Fault Diagnosis at IoT Edge via Adversarial Knowledge Distillation

Clean, performance‐robust, and performance‐sensitive historical information based adversarial self‐distillation

AbstractAdversarial training suffers from poor effectiveness due to the challenging optimisation of loss with hard labels. To address this issue, adversarial distillation has emerged as a potential solution, encouraging target models to mimic the output of the teachers. However, reliance on pre‐training teachers leads to additional training costs and raises concerns about the reliability of their knowledge. Furthermore, existing methods fail to consider the significant differences in unconfident samples between early and late stages, potentially resulting in robust overfitting. An adversarial defence method named Clean, Performance‐robust, and Performance‐sensitive Historical Information based Adversarial Self‐Distillation (CPr & PsHI‐ASD) is presented. Firstly, an adversarial self‐distillation replacement method based on clean, performance‐robust, and performance‐sensitive historical information is developed to eliminate pre‐training costs and enhance guidance reliability for the target model. Secondly, adversarial self‐distillation algorithms that leverage knowledge distilled from the previous iteration are introduced to facilitate the self‐distillation of adversarial knowledge and mitigate the problem of robust overfitting. Experiments are conducted to evaluate the performance of the proposed method on CIFAR‐10, CIFAR‐100, and Tiny‐ImageNet datasets. The results demonstrate that the CPr&PsHI‐ASD method is more effective than existing adversarial distillation methods in enhancing adversarial robustness and mitigating robust overfitting issues against various adversarial attacks.

Attacking Delay-Based PUFs With Minimal Adversarial Knowledge

Attacking Delay-Based PUFs With Minimal Adversarial Knowledge

A Survey of Privacy Attacks in Machine Learning

As machine learning becomes more widely used, the need to study its implications in security and privacy becomes more urgent. Although the body of work in privacy has been steadily growing over the past few years, research on the privacy aspects of machine learning has received less focus than the security aspects. Our contribution in this research is an analysis of more than 45 papers related to privacy attacks against machine learning that have been published during the past seven years. We propose an attack taxonomy, together with a threat model that allows the categorization of different attacks based on the adversarial knowledge, and the assets under attack. An initial exploration of the causes of privacy leaks is presented, as well as a detailed analysis of the different attacks. Finally, we present an overview of the most commonly proposed defenses and a discussion of the open problems and future directions identified during our analysis.

Actuator Anomaly Detection in Linear Parabolic Distributed Parameter Cyber-Physical Systems

Distributed parameter cyber-physical systems (DPCPSs) are characterized by two features: 1) they exhibit spatio-temporal (distributed parameter) dynamics and 2) there exist cyber components in the form of communication channels between the physical plant and control system. DPCPSs are vulnerable to anomalies such as nonmalicious physical faults and malicious cyber-attacks. Although the safety and security aspects of CPSs modeled by ordinary differential equations (ODEs) have been extensively explored during the past decade, security of DPCPSs has not received its due attention despite its safety-critical nature. In this work, we explore anomaly diagnostics in DPCPSs from a system theoretic viewpoint. Specifically, we focus on DPCPSs modeled by linear parabolic partial differential equations (PDEs) subject to physical faults and cyber-attacks in the actuation channel. First, we explore the detectability of malicious anomalies (i.e., cyber-attacks) and derive conditions for stealthy attacks in the context of output-feedback-based monitoring. Such stealthy attacks essentially illustrate the fundamental limitations of feedback-based monitoring algorithms given perfect adversarial knowledge. Subsequently, focusing on the anomalies that are detectable, we develop a design framework for anomaly detection algorithms based on output injection observers. Such detection algorithm explicitly considers the conflicting objectives of robustness to uncertainties and sensitivity to anomalies in their design. Finally, theoretical analysis and simulation studies are performed to illustrate the effectiveness of the proposed approach.

Online adversarial knowledge distillation for graph neural networks

Knowledge distillation, a technique recently gaining popularity for enhancing model generalization in Convolutional Neural Networks (CNNs), operates under the assumption that both teacher and student models are trained on identical data distributions. However, its effect on Graph Neural Networks (GNNs) is less than satisfactory since the graph topology and node attributes are prone to evolve, thereby leading to the issue of distribution shift. In this paper, we tackle this challenge by simultaneously training a group of graph neural networks in an online distillation fashion, where the group knowledge plays a role as a dynamic virtual teacher and the structure changes in graph neural networks are effectively captured. To improve the distillation performance, two types of knowledge are transferred among the students to enhance each other: local knowledge reflecting information in the graph topology and node attributes, and global knowledge reflecting the prediction over classes. We transfer the global knowledge with KL-divergence as the vanilla knowledge distillation does, while exploiting the complicated structure of the local knowledge with an efficient adversarial cyclic learning framework. Extensive experiments verified the effectiveness of our proposed online adversarial distillation approach. The code is published at https://github.com/wangz3066/OnlineDistillGCN.

Your Labels are Selling You Out: Relation Leaks in Vertical Federated Learning

Vertical federated learning (VFL) is an emerging privacy-preserving paradigm that enables collaboration between companies. These companies have the same set of users but different features. One of them is interested in expanding new business or improving its current service with others' features. For instance, an e-commerce company, who wants to improve its recommendation performance, can incorporate users' preferences from another corporation such as a social media company through VFL. On the other hand, graph data is a powerful and sensitive type of data widely used in industry. Their leakage, e.g., the node leakage and/or the relation leakage, can cause severe privacy issues and financial loss. Therefore, protecting the security of graph data is important in practice. Though a line of work has studied how to learn with graph data in VFL, the privacy risks remain underexplored. In this paper, we perform the first systematic study on <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">relation inference attacks</i> to reveal VFL's risk of leaking samples' relations. Specifically, we assume the adversary to be a semi-honest participant. Then, according to the adversary's knowledge level, we formulate three kinds of attacks based on different intermediate representations. Particularly, we design a novel numerical approximation method to handle VFL's encryption mechanism on the participant's representations. Extensive evaluations with four real-world datasets demonstrate the effectiveness of our attacks. For instance, the AUC of relation inference can reach more than 90%, implying an impressive relation inference capability. Furthermore, we evaluate possible defenses to examine our attacks' robustness. The results show that their impacts are limited. Our work highlights the need for advanced defenses to protect private relations and calls for more exploration of VFL's privacy and security issues.

Optimal feedback control law for automated vehicles in the presence of cyberattacks: A min–max approach

While automated vehicles (AVs) are expected to bring a wide range of benefits to future transportation systems, emerging AV technologies open a door for cyberattacks, where a select number of AVs are compromised to drive in an adversarial manner. This could result in network-wide increases in traffic congestion and energy consumption, degrading the performance of transportation systems. Hence, it is necessary to develop attack mitigation strategies for AVs as AVs gradually become a reality. However, this is rather challenging due to the lack of knowledge of adversaries. Limited prior studies have assumed attacks with a given probability distribution without considering their malicious intention, which is far from realistic due to the adversarial nature of potential attacks.In this study, we derive an optimal feedback control law for AVs in the presence of cyberattacks. Notably, attacks are only assumed to have a bounded magnitude without being subject to any specific statistical distribution, which is not only of theoretical interest but also relaxes the assumptions seen in prior studies. More importantly, to deal with lack of knowledge of malicious attacks, we, for the first time, formulate a min–max control problem to minimize the worst-case disturbance to traffic flow. Specifically, we present a mathematical framework for describing mixed-autonomy traffic involving AVs and human-driven vehicles (HVs), considering two typical types of attacks, namely false data injection attack on sensor measurements and malicious attack on AV control commands. Based on the framework presented, we derive a set of necessary conditions of optimality for the min–max control problem, based on which an iterative computational algorithm is developed for determining the optimal control (driving) strategy of AVs. The effectiveness of the proposed approach is demonstrated via extensive numerical simulations.

A Transfer Learning based Cross-subject Generic Model for Continuous Estimation of Finger Joint Angles from a New User.

Continuous estimation of finger joints based on surface electromyography (sEMG) has attracted much attention in the field of human-machine interface (HMI). A couple of deep learning models were proposed to estimate the finger joint angles for specific subject. When applied onto a new subject, however, the performance of the subject-specific model would degrade significantly due to the inter-subject differences. Therefore, a novel cross-subject generic (CSG) model was proposed in this study to estimate continuous kinematics of finger joints for new users. Firstly, a multi-subject model based on the LSTA-Conv network was built by using sEMG and finger joint angles data from multiple subjects. Then, the subjects adversarial knowledge (SAK) transfer learning strategy was adopted to calibrate the multi-subject model with the training data from a new user. With the updated model parameters and the testing data from the new user, multiple finger joint angles could be estimated afterwards. The overall performance of the CSG model for new users was validated on three public datasets from Ninapro. The results showed that the newly proposed CSG model significantly outperformed five subject-specific models and two transfer learning models in terms of Pearson correlation coefficient, root mean square error, and coefficient of determination. Comparison analysis showed that both the long short-term feature aggregation (LSTA) module and the SAK transfer learning strategy contributed to the CSG model. Moreover, increasing number of subjects in training set improved the generalization capability of the CSG model. The novel CSG model would facilitate the application of robotic hand control and other HMI settings.

Egocentric Early Action Prediction via Adversarial Knowledge Distillation

Egocentric early action prediction aims to recognize actions from the first-person view by only observing a partial video segment, which is challenging due to the limited context information of the partial video. In this article, to tackle the egocentric early action prediction problem, we propose a novel multi-modal adversarial knowledge distillation framework. In particular, our approach involves a teacher network to learn the enhanced representation of the partial video by considering the future unobserved video segment, and a student network to mimic the teacher network to produce the powerful representation of the partial video and based on that predicting the action label. To promote the knowledge distillation between the teacher and the student network, we seamlessly integrate adversarial learning with latent and discriminative knowledge regularizations encouraging the learned representations of the partial video to be more informative and discriminative toward the action prediction. Finally, we devise a multi-modal fusion module toward comprehensively predicting the action label. Extensive experiments on two public egocentric datasets validate the superiority of our method over the state-of-the-art methods. We have released the codes and involved parameters to benefit other researchers. 1

Adversarial Knowledge Distillation Based Biomedical Factoid Question Answering.

Biomedical factoid question answering is an essential application for biomedical information sharing. Recently, neural network based approaches have shown remarkable performance for this task. However, due to the scarcity of annotated data which requires intensive knowledge of expertise, training a robust model on limited-scale biomedical datasets remains a challenge. Previous works solve this problem by introducing useful knowledge. It is found that the interaction between question and answer (QA-interaction) is also a kind of knowledge which could help extract answer accurately. This research develops a knowledge distillation framework for biomedical factoid question answering, in which a teacher model as the knowledge source of QA-interaction is designed to enhance the student model. In addition, to further alleviate the problem of limited-scale dataset, a novel adversarial knowledge distillation technique is proposed to robustly distill the knowledge from teacher model to student model by constructing perturbed examples as additional training data. By forcing the student model to mimic the predicted distributions of teacher model on both original examples and perturbed examples, the knowledge of QA-interaction can be learned by student model. We evaluate the proposed framework on the widely used BioASQ datasets, and experimental results have shown the proposed method's promising potential.

RecurrentHAR: A Novel Transfer Learning-Based Deep Learning Model for Sequential, Complex, Concurrent, Interleaved, and Heterogeneous Type Human Activity Recognition

Smartphone sensor-based Human Activity Recognition (HAR) is a significant facilitator for many real-world applications such as smart homes, personal healthcare, and illness detection because it has the potential to discover activity patterns in daily life. While solutions for detecting sequential activities like walking or running are already matured, however recognizing additional forms of complex, concurrent, interleaved, and heterogeneous human activities remains a research challenge. Furthermore, several solutions have been developed for specific types of activity complexity. According to the best of our knowledge, there is no common solution for detecting all types of action-based activities. In this paper, we address the problem of recognizing all types of action-based human activities i.e. sequential, complex, concurrent, interleaved, and heterogeneous activities. The intention behind this paper is to present a generic methodology for HAR by leveraging transfer learning and evaluating the performance on a plethora of HAR datasets. In this regard, we have proposed a model called RecurrentHAR, which stands for Recurrent layers for HAR, a novel adversarial knowledge transfer approach that uses the Gated Recurrent Units (GRUs) architecture for smartphone sensor-based HAR. The performance of the proposed model is evaluated through extensive experiments using three public datasets namely WISDM (sequential activities), PAMAP2 (complex, concurrent, and interleaved activities), and KU-HAR (Heterogeneous activities), and the RecurrentHAR model has outperformed compared with other state-of-the-art approaches. The proposed model achieved 96.26%, 94.77%, and 98.98% of F1-score for WISDM, PAMAP2, and KU-HAR datasets respectively. The experimental results provide insight into the applicability of the proposed model and future research possibilities.

Similarity‐based adversarial knowledge distillation using graph convolutional neural network

This letter presents an adversarial knowledge distillation based on graph convolutional neural network. For knowledge distillation, many methods have been proposed in which the student model individually and independently imitates the output of the teacher model on the input data. Our method suggests the application of a similarity matrix to consider the relationship among output vectors, compared to the other existing approaches. The similarity matrix of the output vectors is calculated and converted into a graph structure, and a generative adversarial network using graph convolutional neural network is applied. We suggest similarity-based knowledge distillation in which a student model simultaneously imitates both of output vector and similarity matrix of the teacher model. We evaluate our method on ResNet, MobileNet and Wide ResNet using CIFAR-10 and CIFAR-100 datasets, and our results outperform results of the baseline model and other existing knowledge distillations like KLD and DML.

Privacy risk quantification in education data using Markov model

AbstractWith Big Data revolution, the education sector is being reshaped. The current data‐driven education system provides many opportunities to utilize the enormous amount of collected data about students' activities and performance for personalized education, adapting teaching methods, and decision making. On the other hand, such benefits come at a cost to privacy. For example, the identification of a student's poor performance across multiple courses. While several works have been conducted on quantifying the re‐identification risks of individuals in released datasets, they assume an adversary's prior knowledge about target individuals. Most of them do not utilize all the available information in the datasets. For example, event‐level information that associates multiple records to the same individual and correlation between attributes. In this work, we propose a method using a Markov Model (MM) to quantify re‐identification risks using all available information in the data under a more realistic threat model that assumes different levels of an adversary's knowledge about the target individual, ranging from any one of the attributes to all given attributes. Moreover, we propose a workflow for efficiently calculating MM risk which is highly scalable to large number of attributes. Experimental results from real education datasets show the efficacy of our model for re‐identification risk. Practitioner notesWhat is already known about this topic? There has been a number of works/research conducted on privacy risk quantification in datasets and in the Web. Majority of them have strong assumption about adversary's prior knowledge of target individual(s). Most of them do not utilize all the available information in the datasets, eg, event‐level or duplicate records and correlation between attributes. What this paper adds? This paper proposes a new re‐identification risk quantification model using Markov models. Our model addresses the shortcomings of existing works, eg, strong assumption about adversary's knowledge, unexplainable model, and utilizing available information in the datasets. Specifically, our proposed model not only focuses on the uniqueness of data points in the datasets (as most of the other existing methods), but also takes into account uniformity and correlation characteristics of these data points. Re‐identification risk quantification is computationally expensive and is not scalable to large datasets with increasing number of attributes. This paper introduces a workflow for data custodians to use to efficiently evaluate the worst‐case re‐identification risk in their datasets before releasing. It presents extensive experimental evaluation results of the proposed model for quantifying re‐identification risks on several real education datasets. Implications for practice and/or policy? Empirical results on real education datasets validate the significance and efficacy of the proposed model for re‐identification risk quantification compared to existing approaches. Our model can be used by the data custodians as a tool to evaluate the worst‐case risk of a dataset. It empowers data custodians to make informed decisions on appropriate actions to mitigate these risks (eg, data perturbation) before sharing or releasing their datasets to third parties. A typical use case would be one where the data custodians is an online course/program provider, which collects data about students' engagement with their courses and would like to share it with third parties for them to run learning analytics that would provide value‐added benefits back to the data custodian. We specifically study the privacy risk quantification for education data; however, our model is applicable to any tabular data release.

Tactics And Techniques Classification In Cyber Threat Intelligence

Abstract Completing the classification of tactics and techniques in cyber threat intelligence (CTI) is an important way to obtain tactics, techniques and procedures (TTPs) and portray the behavior of cyber attacks. However, the high level of abstraction of tactics and techniques information and their presence in CTI, usually in the form of natural language text, make it difficult for traditional manual analysis methods and feature engineering-based machine learning methods to complete the classification of tactics and techniques effectively. Meanwhile, flat deep learning methods do not perform well in classifying more fine-grained techniques due to their inability to exploit the hierarchical relationship between tactics and techniques. Therefore, this paper regards the tactics and techniques of TTPs defined in Adversarial Tactics, Techniques and Common Knowledge knowledge base as labels and proposes a Convolutional Neural Network (CNN) model based on hierarchical knowledge migration and attention mechanism for classifying tactics and techniques in CTI, named HM-ACNN (CNN based on hierarchical knowledge migration and attention mechanism). HM-ACNN classifies tactics and techniques into two phases, and the underlying network model for both phases is the Attention-based CNN network. The first step in HM-ACNN is converting the CTI text into a two-dimensional image based on the word embedding model, and then start training the classification of tactics through the CNN structure based on the attention mechanism before the classification of techniques. Secondly, after the tactics classification training is completed, the tactic-to-technique knowledge migration is then completed by transforming the parameters of the CNN layer and the attention layer in the tactics classification process based on the special hierarchical relationship between tactics and techniques. Then, the classification of techniques is finished by fine-tuning. The experimental results show that HM-ACNN performs well in the tactics and techniques classification tasks, and the metric F1 values reach 93.66% and 86.29%, which are better than other models such as CNN, Recurrent Neural Network and CRNN (Recurrent Convolutional Neural Networks).

The triggers that open the NLP model backdoors are hidden in the adversarial samples

Deep neural networks (DNNS) have been proven to be vulnerable to adversarial attacks. But the adversarial perturbations are generated for specific input samples, and the perturbations of one sample cannot be applied to other samples. In this paper, we propose a method to search for the backdoor of the natural language processing (NLP) model under the black-box condition, and we find that the universal attack triggers exist in the adversarial samples. The method includes two steps. The first step is to extract aggressive words in the adversarial sample to form the adversarial knowledge base under the black-box condition. The second step is to generate universal attack triggers by minimizing the target prediction results of a batch of samples. When we add the generated trigger to any benign input, the prediction accuracy of the DNNS model can be reduced to close to zero. The experimental results show that our method can achieve a high attack success rate with a short trigger (e.g., more than 90% using only a trigger of length 3 when attacking BiLSTM on SST-2). In addition, experiments show that our method has higher transferability. Finally, for the backdoor vulnerabilities in the NLP models, we did two defense experiments: abnormal word detection and word frequency analysis, which improve the NLP model’s ability of resisting backdoor attacks.

A moving target defence approach for detecting deception attacks on cyber-physical systems

Cyber-physical systems have posed new challenges, affecting their users. The information security of these systems against cyber-attacks is considered one of the most complex issues in a wide range of defences. The present study proposes a new approach for detecting deception attacks based on moving target defence (MTD). The existing MTD algorithms can only recognize few attacks, including integrity attacks on the cyber-physical system, by limiting the adversary's knowledge of the original system. However, the proposed approach detects replay attacks as well. Moreover, a robust defence system is developed to mislead the adversary and make the attack identifiable from the defender's perspective by introducing a chaotic, hybrid, and extended MTD. Thus, three primary deception attacks, including False Data Injection (FDI), covert, and replay attacks, are detectable based on the proposed approach. The effectiveness of the proposed approach is validated by comparing it with continuous watermarking defence in a cyber-physical testbed.

Quantification of a Facility Radiological Security Risk Index With a Graphical User Interface Tool.

Healthcare facilities around the world routinely use radioactive sources to diagnose and treat illness. To effectively manage the security of radioactive sources, these facilities need to understand the risk, which is comprised of threat, vulnerability, and consequences. The threat component of risk requires knowledge of potential adversaries and understanding their capabilities and intentions. To help articulate the multiple layers of threat and support better informed decisions, the research developed a risk-based methodology to evaluate radiological security at the facility level. The methodology is applied to a radiological dispersal device (RDD) incident from three radionuclides of concern: 137Cs, 60Co, and 192Ir. The results of the research have led to the creation of a potential facility risk index (PFRI). The PFRI is mathematically represented as the exponential product of the maximum expected utility among the threat groups, the sum of geographic vulnerability and cultural vulnerability, and net consequences. The PFRI is a novel risk index that quantifies the facility risk on a scale of 1 to 10, 1 being "very low risk" and 10 being "very high risk." A MATLAB-based graphical user interface (GUI) tool was also developed to enable the radiological facility (i.e., healthcare facility) staff to conduct self-assessments and manage their most valuable assets. The PFRI methodology is a useful starting point for any healthcare facility risk assessment and is a valuable input for decision makers considering potential investments in security upgrades.

An Adversarial Approach to Protocol Analysis and Selection in Local Differential Privacy

Local Differential Privacy (LDP) is a popular standard for privacy-preserving data collection. Numerous LDP protocols have been proposed in the literature which differ in how they provide higher utility in different settings. Yet, few have engaged in analyzing the privacy relationships of these protocols under varying settings, and consequently, it is non-trivial to select which LDP protocol is best to use in a newly emerging application. In this paper, we present an adversarial approach to protocol analysis and selection and make three original contributions. First, we introduce a Bayesian adversary to analyze the privacy relationships of LDP protocols under varying settings. We show that different protocols have substantially different responses to the attack effectiveness of the Bayesian adversary, measured in terms of Adversarial Success Rate (ASR). Second, we provide a formal and empirical analysis on a set of privacy and utility-critical factors, including encoding parameters, privacy budget, data domain, adversarial knowledge, and statistical distribution. We show that different settings of these factors have significant effects on the ASRs of LDP protocols, and no protocol provides consistently low ASR across all settings. Third, we design and develop LDPLens, a prototype implementation of our proposed framework. Given a data collection scenario with various factors and constraints, LDPLens enables optimized selection of a desirable LDP protocol for the given scenario. We evaluate the effectiveness of LDPLens using three case studies with real-world datasets. Results show that LDPLens recommends a different protocol in each case study, and the protocol recommended by LDPLens can yield up to 1.5–2 fold reduction in utility loss, ASR or privacy budget compared to a randomly selected protocol.