Adversarial Scenarios Research Articles

Anonymous Quantum Conference Key Agreement

Conference Key Agreement (CKA) is a cryptographic effort of multiple parties to establish a shared secret key. In future quantum networks, generating secret keys in an anonymous way is of tremendous importance for parties that want to keep their shared key secret and at the same time protect their own identity. We provide a definition of anonymity for general protocols and present a CKA protocol that is provably anonymous under realistic adversarial scenarios. We base our protocol on shared Greenberger-Horne-Zeilinger states, which have been proposed as more efficient resources for CKA protocols, compared to bipartite entangled resources. The existence of secure and anonymous protocols based on multipartite entangled states provides a new insight on their potential as resources and paves the way for further applications.

A MILP Model for a Byzantine Fault Tolerant Blockchain Consensus

Mixed-integer mathematical programming has been widely used to model and solve challenging optimization problems. One interesting feature of this technique is the ability to prove the optimality of the achieved solution, for many practical scenarios where a linear programming model can be devised. This paper explores its use to model very strong Byzantine adversaries, in the context of distributed consensus systems. In particular, we apply the proposed technique to find challenging adversarial conditions on a state-of-the-art blockchain consensus: the Neo dBFT. Neo Blockchain has been using the dBFT algorithm since its foundation, but, due to the complexity of the algorithm, it is challenging to devise definitive algebraic proofs that guarantee safety/liveness of the system (and adjust for every change proposed by the community). Core developers have to manually devise and explore possible adversarial attacks scenarios as an exhaustive task. The proposed multi-objective model is intended to assist the search of possible faulty scenario, which includes three objective functions that can be combined as a maximization problem for testing one-block finality or a minimization problem for ensuring liveness. Automated graphics help developers to visually observe attack conditions and to quickly find a solution. This paper proposes an exact adversarial model that explores current limits for practical blockchain consensus applications such as dBFT, with ideas that can also be extended to other decentralized ledger technologies.

Effects of Cyber Attacks on AC and High-Voltage DC Interconnected Power Systems with Emulated Inertia

The high penetration of renewable energy resources and power electronic-based components has led to a low-inertia power grid which would bring challenges to system operations. The new model of load frequency control (LFC) must be able to handle the modern scenario where controlled areas are interconnected by parallel AC/HVDC links and storage devices are added to provide virtual inertia. Notably, vulnerabilities within the communication channels for wide-area data exchange in LFC loops may make them exposed to various cyber attacks, while it still remains largely unexplored how the new LFC in the AC/HVDC interconnected system with emulated inertia would be affected under malicious intrusions. Thus, in this article, we are motivated to explore possible effects of the major types of data availability and integrity attacks—Denial of Service (DoS) and false data injection (FDI) attacks—on such a new LFC system. By using a system-theoretic approach, we explore the optimal strategies that attackers can exploit to launch DoS or FDI attacks to corrupt the system stability. Besides, a comparison study is performed to learn the impact of these two types of attacks on LFC models of power systems with or without HVDC link and emulated inertia. The simulation results on the the exemplary two-area system illustrate that both DoS and FDI attacks can cause large frequency deviations or even make the system unstable; moreover, the LFC system with AC/HVDC interconnections and emulated inertia could be more vulnerable to these two types of attacks in many adversarial scenarios.

A survey of machine learning techniques in adversarial image forensics

Image forensic plays a crucial role in both criminal investigations (e.g., dissemination of fake images to spread racial hate or false narratives about specific ethnicity groups) and civil litigation (e.g., defamation). Increasingly, machine learning approaches are also utilized in image forensics. However, there are also a number of limitations and vulnerabilities associated with machine learning-based approaches, for example how to detect adversarial (image) examples, with real-world consequences (e.g., inadmissible evidence, or wrongful conviction). Therefore, with a focus on image forensics, this paper surveys techniques that can be used to enhance the robustness of machine learning-based binary manipulation detectors in various adversarial scenarios.

Quantum adversarial machine learning

Adversarial machine learning is an emerging field that focuses on studying vulnerabilities of machine learning approaches in adversarial settings and developing techniques accordingly to make learning robust to adversarial manipulations. It plays a vital role in various machine learning applications and has attracted tremendous attention across different communities recently. In this paper, we explore different adversarial scenarios in the context of quantum machine learning. We find that, similar to traditional classifiers based on classical neural networks, quantum learning systems are likewise vulnerable to crafted adversarial examples, independent of whether the input data is classical or quantum. In particular, we find that a quantum classifier that achieves nearly the state-of-the-art accuracy can be conclusively deceived by adversarial examples obtained via adding imperceptible perturbations to the original legitimate samples. This is explicitly demonstrated with quantum adversarial learning in different scenarios, including classifying real-life images (e.g., handwritten digit images in the dataset MNIST), learning phases of matter (such as, ferromagnetic/paramagnetic orders and symmetry protected topological phases), and classifying quantum data. Furthermore, we show that based on the information of the adversarial examples at hand, practical defense strategies can be designed to fight against a number of different attacks. Our results uncover the notable vulnerability of quantum machine learning systems to adversarial perturbations, which not only reveals a novel perspective in bridging machine learning and quantum physics in theory but also provides valuable guidance for practical applications of quantum classifiers based on both near-term and future quantum technologies.

Nuclear security system effectiveness assessment using bounded analysis approach: a case study of different adversary scenarios

ABSTRACT A security issue on nuclear facilities will give a significant effect on national security, economy, public health, or the combination among them. The evaluation of physical security effectiveness is done by considering it as a probability that the system will defeat an adversary at a particular time. This research examined the effectiveness of Physical Protection Systems (PPS) of a nuclear research reactor in Indonesia, as a role model facility. Three possible adversary scenarios were assessed. Estimation of Adversary Sequence Interruption model was implemented to assess the physical security system. The estimation of the effectiveness of physical-cybersecurity combination was done by Bounding Analysis Approach. The effectiveness of security system with cyber system working properly was assessed and compared with the system receiving cyber attacked. The results showed that the system effectiveness is high (0.839, 0.949, and 0.837, respectively). The results showed that the assessment method is effective to examine whether a security element is working effectively or not in such facility. This work will be useful as base guidelines for nuclear regulatory body and licensee for the application and evaluation of PPS and provision of countermeasure strategies in RSG GAS, and for further benefit, it can be used by nuclear facilities.

Quantum gate verification and its application in property testing

To guarantee the normal functioning of quantum devices in different scenarios, appropriate benchmarking tool kits are quite significant. Inspired by the recent progress on quantum state verification, here we establish a general framework of verifying a target unitary gate. In both the non-adversarial and adversarial scenarios, we provide efficient methods to evaluate the performance of verification strategies for any qudit unitary gate. Furthermore, we figure out the optimal strategy and its realization with local operations. Specifically, for the commonly-used quantum gates like single qubit and qudit gates, multi-qubit Clifford gates, and multi-qubit Controlled-Z(X) gates, we provide efficient verification protocols. Besides, we discuss the application of gate verification for the detection of entanglement-preserving property of quantum channels and further quantify the robustness measure of them. We believe that the gate verification is a promising way to benchmark a large-scale quantum circuit as well as to test its property.

Optimal Verification of Greenberger-Horne-Zeilinger States

We construct optimal protocols for verifying qubit and qudit GHZ states using local projective measurements. When the local dimension is a prime, an optimal protocol is constructed from Pauli measurements only. Our protocols provide a highly efficient way for estimating the fidelity and certifying genuine multipartite entanglement. In particular, they enable the certification of genuine multipartite entanglement using only one test when the local dimension is sufficiently large. By virtue of adaptive local projective measurements, we then construct protocols for verifying GHZ-like states that are optimal over all protocols based on one-way communication. The efficiency can be improved further if additional communications are allowed. Finally, we construct optimal protocols for verifying GHZ states and nearly optimal protocols for GHZ-like states in the adversarial scenario.

Toward a blockchain-based framework for challenge-based collaborative intrusion detection

Network intrusions are a big threat to network and system assets, which have become more complex to date. To enhance the detection performance, collaborative intrusion detection networks (CIDNs) are adopted by many organizations to protect their resources. However, such detection systems or networks are typically vulnerable to insider attacks, so that there is a need to implement suitable trust mechanisms. In the literature, challenge-based trust mechanisms are able to measure the trustworthiness of a node by evaluating the relationship between the sent challenges and the received responses. In practice, challenge-based CIDNs have shown to be robust against common insider attacks, whereas it may still be susceptible to advanced insider attacks. How to enhance the robustness of such challenge-based CIDNs remains an issue. Motivated by the recent development of blockchains, in this work, our purpose is to design a blockchained challenge-based CIDN framework that aims to combine blockchains with challenge-based trust mechanism. Our evaluation demonstrates that blockchain technology has the potential to enhance the robustness of challenge-based CIDNs in the aspects of trust management (i.e., enhancing the detection of insider nodes) and alarm aggregation (i.e., identifying untruthful inputs) under adversary scenarios.

Security Properties of Light Clients on the Ethereum Blockchain

Ethereum is a decentralized blockchain, known as being the second most popular public blockchain after Bitcoin. Since Ethereum is decentralised the canonical state is determined by the Ethereum network participants via a consensus mechanism without a centralized coordinator. The network participants are required to evaluate every transaction starting from the genesis block, which requires a large amount of network, computing, and storage resources. This is impractical for many devices with either limited computing resources or intermittent network connectivity. To overcome this drawback Ethereum defines a light client protocol where the light client fetches the blockchain state from a node operating as a light protocol server. Light clients are unable to maintain blockchain state internally, and as a consequence can only perform partial validation on blocks. Thus they rely on the light server for full block validation and to provide the updated blockchain state. Light clients connect to multiple light servers to mitigate the risk of relying on a single potentially dishonest server. Ethereum light clients are known to suffer from a probabilistic security model, but they are widely assumed to be secure under normal operating conditions. In fact, the implicit security assumptions of light clients have not been formally characterised in the literature. We present and analyse the probabilistic security guarantees under three different adversarial scenarios. The results show that for any adversary that is able to manipulate the network, the security assurances provided by the light protocol are severely impacted, and in some cases entirely lost. These results clearly demonstrate that the assumption of normal operating conditions is insufficient to justify the security assumptions of light clients. Our work also provides insight to the security of light clients under different security parameters, allowing light client implementers to more accurately understand the potential security trade-offs.

Security Promises and Vulnerabilities in Emerging Reconfigurable Nanotechnology-Based Circuits

Emerging nanotechnologies based reconfigurable field-effect transistors (RFETs) allow switching between p-type and n-type behavior at runtime upon application of different bias potentials. While prior works have mainly focused on security schemes using RFETs, here we revisit the underlying security promises, and furthermore, showcase certain circuit vulnerabilities which can lead to adversarial scenarios. More specifically, first, we explore how transistor-level reconfigurability can be leveraged for logic locking and split manufacturing in the pretext of RFET-based modeling of the ITC-99 circuits. We find that with only 30% reconfigurable logic gates, we can induce 100% output error rate (OER) and 31% Hamming distance(HD) on split-manufacturing schemes. Second, which is arguably more disruptive, we explore how the very reconfigurability can be exploited to induce either short-circuit or open-circuit configurations, essentially destroying the reliability, electrical, or functional characteristics of the chip. By means of fault modeling for such exploitation, 100% OER (across all benchmarks) as well as 36.5%, 40.02%, and 6.34% HD, respectively, can be expected for the MCNC, EPFL, and ITC-99 benchmarks. The novelty and severity of such disruptive scenario lies in the fact that they can be readily realized in an actual on-field RFET-based chip, either as an adversarial or a fail-safe measure.

General framework for verifying pure quantum states in the adversarial scenario

Bipartite and multipartite entangled states are of central interest in quantum information processing and foundational studies. Efficient verification of these states, especially in the adversarial scenario, is a key to various applications, including quantum computation, quantum simulation, and quantum networks. However, little is known about this topic in the adversarial scenario. Here we initiate a systematic study of pure-state verification in the adversarial scenario. In particular, we introduce a general method for determining the minimal number of tests required by a given strategy to achieve a given precision. In the case of homogeneous strategies, we can even derive an analytical formula. Furthermore, we propose a general recipe to verifying pure quantum states in the adversarial scenario by virtue of protocols for the nonadversarial scenario. Thanks to this recipe, the resource cost for verifying an arbitrary pure state in the adversarial scenario is comparable to the counterpart for the nonadversarial scenario, and the overhead is at most three times for high-precision verification. Our recipe can readily be applied to efficiently verify bipartite pure states, stabilizer states, hypergraph states, weighted graph states, and Dicke states in the adversarial scenario, even if only local projective measurements are accessible. This paper is an extended version of the companion paper Zhu and Hayashi, Phys. Rev. Lett. 123, 260504 (2019).

Efficient Verification of Pure Quantum States in the Adversarial Scenario.

Efficient verification of pure quantum states in the adversarial scenario is crucial to many applications in quantum information processing, such as blind measurement-based quantum computation and quantum networks. However, little is known about this topic so far. Here, we establish a general framework for verifying pure quantum states in the adversarial scenario and clarify the resource cost. Moreover, we propose a simple and general recipe to constructing efficient verification protocols for the adversarial scenario from protocols for the nonadversarial scenario. With this recipe, arbitrary pure states can be verified in the adversarial scenario with almost the same efficiency as in the nonadversarial scenario. Many important quantum states can be verified in the adversarial scenario using local projective measurements with unprecedented high efficiencies.

Efficient Verification of Hypergraph States

Graph states and hypergraph states are of wide interest in quantum information processing and foundational studies. Efficient verification of these states is a key to various applications. Here we propose a simple method for verifying hypergraph states which requires only two distinct Pauli measurements for each party, yet its efficiency is comparable to the best strategy based on entangling measurements. For a given state, the overhead is bounded by the chromatic number and degree of the underlying hypergraph. Our protocol is dramatically more efficient than all previous protocols based on local measurements, including tomography and direct fidelity estimation. It enables the verification of hypergraph states and genuine multipartite entanglement of thousands of qubits. The protocol can also be generalized to the adversarial scenario, while achieving almost the same efficiency. This merit is particularly appealing to demonstrating blind measurement-based quantum computation and quantum supremacy.

Framework for Analysis and Quantification of Wide-Area Control Resilience for Power Systems

Wide-area control helps damp the inter-area oscillations in modern power systems. However, compared to traditional local control, it is more vulnerable to cyber attacks due to its dependence on remote real-time-communicated measurements. In this paper, a formal mathematical framework for analyzing resilience of various wide-area controllers under adversarial scenarios is proposed. A novel resilience index based on the $\mathcal {H}_2$ norm of the post-attack closed-loop system is defined. Computation of such an index is, in general, challenging due to the combinatorial nature of the attack profiles and the bilinear nature of the problem. To address these difficulties, a gradient-based path-following-like solution is developed in this paper to solve a relaxed version of the original non-convex problem. Case studies on IEEE 39-bus system are performed to demonstrate the usage of the proposed framework.

Efficient verification of bipartite pure states

We propose practical and efficient protocols for verifying bipartite pure states for any finite dimension, which can also be applied to fidelity estimation. Our protocols are based on adaptive local projective measurements with either one-way or two-way communications, which are very easy to implement in practice. They can extract the key information much more efficiently than known protocols based on tomography or direct fidelity estimation, and their efficiencies are comparable to the best protocols based on entangling measurements. These protocols highlight the significance of mutually unbiased bases (MUBs) and complex projective 2-designs in quantum state verification and fidelity estimation. Moreover, our protocols can be applied to the adversarial scenario, in which states are controlled by a malicious adversary. In this case, surprisingly, one of our protocols based on local measurements is even optimal among protocols without locality restriction.

Thwarting Replication Attack Against Memristor-Based Neuromorphic Computing System

Neuromorphic architectures are widely used in many applications for advanced data processing and often implement proprietary algorithms. However, in an adversarial scenario, such systems may face elaborate security attacks including learning attack. In this article, we prevent an attacker with physical access from learning the proprietary algorithm implemented by the neuromorphic hardware. For this purpose, we leverage the obsolescence effect in memristors to judiciously reduce the accuracy of outputs for any unauthorized user. For a legitimate user, we regulate the obsolescence effect, thereby maintaining the accuracy of outputs in a suitable range. We extensively examine the feasibility of our proposed method with four datasets. We experiment under different settings, such as activation functions and constraints such as process variations, and estimate the calibration overhead. The security versus cost and performance versus resistance range tradeoffs for different applications are also analyzed. We then prove that the defense is still valid even if the attacker has the prior knowledge of the defense mechanism. Overall, our methodology is compatible with mainstream classification applications, memristor devices, and security and performance constraints.

Re-Evaluating ADEM: A Deeper Look at Scoring Dialogue Responses

Automatically evaluating the quality of dialogue responses for unstructured domains is a challenging problem. ADEM (Lowe et al. 2017) formulated the automatic evaluation of dialogue systems as a learning problem and showed that such a model was able to predict responses which correlate significantly with human judgements, both at utterance and system level. Their system was shown to have beaten word-overlap metrics such as BLEU with large margins. We start with the question of whether an adversary can game the ADEM model. We design a battery of targeted attacks at the neural network based ADEM evaluation system and show that automatic evaluation of dialogue systems still has a long way to go. ADEM can get confused with a variation as simple as reversing the word order in the text! We report experiments on several such adversarial scenarios that draw out counterintuitive scores on the dialogue responses. We take a systematic look at the scoring function proposed by ADEM and connect it to linear system theory to predict the shortcomings evident in the system. We also devise an attack that can fool such a system to rate a response generation system as favorable. Finally, we allude to future research directions of using the adversarial attacks to design a truly automated dialogue evaluation system.

When Not to Classify: Anomaly Detection of Attacks (ADA) on DNN Classifiers at Test Time.

A significant threat to the recent, wide deployment of machine learning-based systems, including deep neural networks (DNNs), is adversarial learning attacks. The main focus here is on evasion attacks against DNN-based classifiers at test time. While much work has focused on devising attacks that make small perturbations to a test pattern (e.g., an image) that induce a change in the classifier's decision, until recently there has been a relative paucity of work defending against such attacks. Some works robustify the classifier to make correct decisions on perturbed patterns. This is an important objective for some applications and for natural adversary scenarios. However, we analyze the possible digital evasion attack mechanisms and show that in some important cases, when the pattern (image) has been attacked, correctly classifying it has no utility---when the image to be attacked is (even arbitrarily) selected from the attacker's cache and when the sole recipient of the classifier's decision is the attacker. Moreover, in some application domains and scenarios, it is highly actionable to detect the attack irrespective of correctly classifying in the face of it (with classification still performed if no attack is detected). We hypothesize that adversarial perturbations are machine detectable even if they are small. We propose a purely unsupervised anomaly detector (AD) that, unlike previous works, (1) models the joint density of a deep layer using highly suitable null hypothesis density models (matched in particular to the nonnegative support for rectified linear unit (ReLU) layers); (2) exploits multiple DNN layers; and (3) leverages a source and destination class concept, source class uncertainty, the class confusion matrix, and DNN weight information in constructing a novel decision statistic grounded in the Kullback-Leibler divergence. Tested on MNIST and CIFAR image databases under three prominent attack strategies, our approach outperforms previous detection methods, achieving strong receiver operating characteristic area under the curve detection accuracy on two attacks and better accuracy than recently reported for a variety of methods on the strongest (CW) attack. We also evaluate a fully white box attack on our system and demonstrate that our method can be leveraged to strong effect in detecting reverse engineering attacks. Finally, we evaluate other important performance measures such as classification accuracy versus true detection rate and multiple measures versus attack strength.

Optimal verification and fidelity estimation of maximally entangled states

We study the verification of maximally entangled states by virtue of the simplest measurement settings: local projective measurements without adaption. We show that optimal protocols are in one-to-one correspondence with complex projective 2-designs constructed from orthonormal bases. Optimal protocols with minimal measurement settings are in one-to-one correspondence with complete sets of mutually unbiased bases. Based on this observation, optimal protocols are constructed explicitly for any local dimension, which can also be applied to estimating the fidelity with the target state and to detecting entanglement. In addition, we show that incomplete sets of mutually unbiased bases are optimal for verifying maximally entangled states when the number of measurement settings is restricted. Moreover, we construct optimal protocols for the adversarial scenario in which state preparation is not trusted. The number of tests has the same scaling behavior as the counterpart for the nonadversarial scenario; the overhead is no more than three times. We also show that the entanglement of the maximally entangled state can be certified with any given significance level using only one test as long as the local dimension is large enough.