Verification Of Neural Networks Research Articles

AI-Powered Neural Network Verification: System Verilog Methodologies for Machine Learning in Hardware

This research focuses on verifying neural network models using System Verilog, with two primary applications: visual edge detection and neuron behavior modeling. In modern chip design, hardware verification plays a crucial role in ensuring that complex neural models perform as expected. A neuron model based on Hubel and Wiesel’s feed-forward network architecture was proposed and tested using integrator and threshold modules implemented in Verilog. The proposed verification methodology employs self-checking test benches, supported by functional coverage and simulation, for comprehensive validation. The results demonstrate efficient verification with high coverage, paving the way for future advancements in hardware neural networks.

Neural networks in closed-loop systems: Verification using interval arithmetic and formal prover

Machine Learning approaches have been successfully used for the creation of high-performance control components of cyber–physical systems, where the control dynamics result from the combination of many subsystems. However, these approaches may lack the trustworthiness required to guarantee their reliable application in a safety-critical context. In this paper, we propose a combination of interval arithmetic and theorem-proving verification techniques to analyze safety properties in closed-loop systems that embed neural network components. We show the application of the proposed approach to a model-predictive controller for autonomous driving comparing the neural network verification performance with other existing tools. The results show that open-loop neural network verification through interval arithmetic can outperform existing approaches proving properties with a smaller time overhead. Furthermore, we demonstrate the capability of combining the two approaches to construct a formal model of the network in higher-order logic of the controlled system in a closed-loop.

NeVer2: learning and verification of neural networks

AbstractNeVer2 is an open-source, cross-platform tool aimed at designing, training, and verifying neural networks. It seamlessly integrates popular learning libraries with our verification backend, offering their functionalities via a graphical interface. Users can design the structure of a neural network by intuitively arranging blocks on a canvas. Subsequently, network training involves specifying dataset sources and hyperparameters through dialog boxes. After training, the verification process entails two steps: (i) incorporating input preconditions and output postconditions via dedicated blocks, and (ii) initiating verification with a simple “push-button” action. To our knowledge, there is currently no other publicly available tool that encompasses all these features. In this paper, we present a comprehensive description of NeVer2, illustrating its complete integration of design, training, and verification through examples. Additionally, we conduct experimental analyses on various verification benchmarks to illustrate the trade-off between completeness and computability using different algorithms. We also include a comparison with state-of-the-art tools such as $$\alpha $$ α ,$$\beta $$ β -CROWN and NNV for reference.

A hybrid quantum–classical algorithm for mixed-integer optimization in power systems

Mixed Integer Linear Programming (MILP) can be considered the backbone of the modern power system optimization process, with a large application spectrum, from Unit Commitment and Optimal Transmission Switching to verifying Neural Networks for power system applications. The main issue of these formulations is the computational complexity of the solution algorithms, as they are considered NP-Hard problems. Quantum computing has been tested as a potential solution towards reducing the computational burden imposed by these problems, providing promising results, motivating the can be used to speedup the solution of MILPs. In this work, we present a general framework for solving power system optimization problems with a Quantum Computer (QC), which leverages mathematical tools and QCs’ sampling ability to provide accelerated solutions. Our guiding applications are the optimal transmission switching and the verification of neural networks trained to solve a DC Optimal Power Flow. Specifically, using an accelerated version of Benders Decomposition , we split a given MILP into an Integer Master Problem and a linear Subproblem and solve it through a hybrid “quantum–classical” approach, getting the best of both worlds. We provide 2 use cases, and benchmark the developed framework against other classical and hybrid methodologies, to demonstrate the opportunities and challenges of hybrid quantum–classical algorithms for power system mixed integer optimization problems.

A Parallel Optimization Method for Robustness Verification of Deep Neural Networks

Deep neural networks (DNNs) have gained considerable attention for their expressive capabilities, but unfortunately they have serious robustness risks. Formal verification is an important technique to ensure network reliability. However, current verification techniques are unsatisfactory in time performance, which hinders the practical applications. To address this issue, we propose an efficient optimization method based on parallel acceleration with more computing resources. The method involves the speedup configuration of a partition-based verification aligned with the structures and robustness formal specifications of DNNs. A parallel verification framework is designed specifically for neural network verification systems, which integrates various auxiliary modules and accommodates diverse verification modes. The efficient parallel scheduling of verification queries within the framework enhances resource utilization and enables the system to process a substantial volume of verification tasks. We conduct extensive experiments on multiple commonly used verification benchmarks to demonstrate the rationality and effectiveness of the proposed method. The results show that higher efficiency is achieved after parallel optimization integration.

A Review of Abstraction Methods Toward Verifying Neural Networks

Neural networks as a machine learning technique are increasingly deployed in various domains. Despite their performance and their continuous improvement, the deployment of neural networks in safety-critical systems, in particular for autonomous mobility, remains restricted. This is mainly due to the lack of (formal) specifications and verification methods and tools that allow for having sufficient confidence in the behavior of the neural-network-based functions. Recent years have seen neural network verification getting more attention; many verification methods were proposed, yet the practical applicability of these methods to real-world neural network models remains limited. The main challenge of neural network verification methods is related to the computational complexity and the large size of neural networks pertaining to complex functions. As a consequence, applying abstraction methods for neural network verification purposes is seen as a promising mean to cope with such issues. The aim of abstraction is to build an abstract model by omitting some irrelevant details or some details that are not highly impacting w.r.t some considered features. Thus, the verification process is made faster and easier while preserving, to some extent, the relevant behavior regarding the properties to be examined on the original model. In this article, we review both the abstraction techniques for activation functions and model size reduction approaches, with a particular focus on the latter. The review primarily discusses the application of abstraction techniques on feed-forward neural networks and explores the potential for applying abstraction to other types of neural networks. Throughout the article, we present the main idea of each approach and then discuss its respective advantages and limitations in detail. Finally, we provide some insights and guidelines to improve the discussed methods.

Abstraction and Refinement: Towards Scalable and Exact Verification of Neural Networks

As a new programming paradigm, deep neural networks (DNNs) have been increasingly deployed in practice, but the lack of robustness hinders their applications in safety-critical domains. While there are techniques for verifying DNNs with formal guarantees, they are limited in scalability and accuracy. In this article, we present a novel counterexample-guided abstraction refinement (CEGAR) approach for scalable and exact verification of DNNs. Specifically, we propose a novel abstraction to break down the size of DNNs by over-approximation. The result of verifying the abstract DNN is conclusive if no spurious counterexample is reported. To eliminate each spurious counterexample introduced by abstraction, we propose a novel counterexample-guided refinement that refines the abstract DNN to exclude the spurious counterexample while still over-approximating the original one, leading to a sound, complete yet efficient CEGAR approach. Our approach is orthogonal to and can be integrated with many existing verification techniques. For demonstration, we implement our approach using two promising tools, Marabou and Planet , as the underlying verification engines, and evaluate on widely used benchmarks for three datasets ACAS , Xu , MNIST , and CIFAR-10 . The results show that our approach can boost their performance by solving more problems in the same time limit, reducing on average 13.4%–86.3% verification time of Marabou on almost all the verification tasks, and reducing on average 8.3%–78.0% verification time of Planet on all the verification tasks. Compared to the most relevant CEGAR-based approach, our approach is 11.6–26.6 times faster.

Efficient verification of neural networks based on neuron branching and LP abstraction

With the rapid development and wide application of neural networks in various domains including safety-critical systems, it is more and more important to investigate formal methods to provide strict guarantees on their behavior. As formal verification of a neural network has high computational complexity, caused by the nonlinear nature of activation functions such as ReLU, it is vital to improve the efficiency to solve verification problems. In this work, we propose a complete verification method for neural networks by means of neuron branching and linear programming (LP) abstraction. Specifically, the approach of branch and bound is adopted and a branching strategy is developed to divide a complex verification problem into sub-problems with smaller search space. And for each sub-problem, a part of ReLU neurons are abstracted with LP constraints and the others are accurately encoded with mixed integer linear programming (MILP) constraints, and an abstraction strategy is deployed to guide such mixed LP/MILP encoding so that the problem can be solved with reduced complexity. The method is implemented as a tool named BAVerify (Branching and Abstraction based Verification of neural networks), which is evaluated through experiments on benchmark network models and their verification problems. Experimental results show that BAVerify achieves a 43.13% advantage in verification efficiency compared with state-of-the-art complete methods for formal verification of neural networks.

A Dual Relaxation Method for Neural Network Verification

In the robustness verification of neural networks, formal methods have been used to give deterministic guarantees for neural networks. However, recent studies have found that the verification method of single-neuron relaxation in this field has an inherent convex barrier that affects its verification capability. To address this problem, we propose a new verification method by combining dual-neuron relaxation and linear programming. This method captures the dependencies between different neurons in the same hidden layer by adding a two-neuron joint constraint to the linear programming model, thus overcoming the convex barrier problem caused by relaxation for only a single neuron. Our method avoids the combination of exponential inequality constraints and can be computed in polynomial time. Experimental results show that we can obtain tighter bounds and achieve more accurate verification than single-neuron relaxation methods.

Verification of Neural Networks’ Global Robustness

Neural networks are successful in various applications but are also susceptible to adversarial attacks. To show the safety of network classifiers, many verifiers have been introduced to reason about the local robustness of a given input to a given perturbation. While successful, local robustness cannot generalize to unseen inputs. Several works analyze global robustness properties, however, neither can provide a precise guarantee about the cases where a network classifier does not change its classification. In this work, we propose a new global robustness property for classifiers aiming at finding the minimal globally robust bound, which naturally extends the popular local robustness property for classifiers. We introduce VHAGaR, an anytime verifier for computing this bound. VHAGaR relies on three main ideas: encoding the problem as a mixed-integer programming and pruning the search space by identifying dependencies stemming from the perturbation or the network's computation and generalizing adversarial attacks to unknown inputs. We evaluate VHAGaR on several datasets and classifiers and show that, given a three hour timeout, the average gap between the lower and upper bound on the minimal globally robust bound computed by VHAGaR is 1.9, while the gap of an existing global robustness verifier is 154.7. Moreover, VHAGaR is 130.6x faster than this verifier. Our results further indicate that leveraging dependencies and adversarial attacks makes VHAGaR 78.6x faster.

Exponent Relaxation of Polynomial Zonotopes and Its Applications in Formal Neural Network Verification

Formal verification of neural networks is a challenging problem due to the complexity and nonlinearity of neural networks. It has been shown that polynomial zonotopes can tightly enclose the output set of a neural network. Unfortunately, the tight enclosure comes with additional complexity in the set representation, thus, rendering subsequent operations expensive to compute, such as computing interval bounds and intersection checking. To address this issue, we present a novel approach to restructure a polynomial zonotope to tightly enclose the original polynomial zonotope while drastically reducing its complexity. The restructuring is achieved by relaxing the exponents of the dependent factors of polynomial zonotopes and finding an appropriate approximation error. We demonstrate the applicability of our approach on output sets of neural networks, where we obtain tighter results in various subsequent operations, such as order reduction, zonotope enclosure, and range bounding.

ReLU Hull Approximation

Convex hulls are commonly used to tackle the non-linearity of activation functions in the verification of neural networks. Computing the exact convex hull is a costly task though. In this work, we propose a fast and precise approach to over-approximating the convex hull of the ReLU function (referred to as the ReLU hull), one of the most used activation functions. Our key insight is to formulate a convex polytope that ”wraps” the ReLU hull, by reusing the linear pieces of the ReLU function as the lower faces and constructing upper faces that are adjacent to the lower faces. The upper faces can be efficiently constructed based on the edges and vertices of the lower faces, given that an n -dimensional (or simply n d hereafter) hyperplane can be determined by an ( n −1)d hyperplane and a point outside of it. We implement our approach as WraLU, and evaluate its performance in terms of precision, efficiency, constraint complexity, and scalability. WraLU outperforms existing advanced methods by generating fewer constraints to achieve tighter approximation in less time. It exhibits versatility by effectively addressing arbitrary input polytopes and higher-dimensional cases, which are beyond the capabilities of existing methods. We integrate WraLU into PRIMA, a state-of-the-art neural network verifier, and apply it to verify large-scale ReLU-based neural networks. Our experimental results demonstrate that WraLU achieves a high efficiency without compromising precision. It reduces the number of constraints that need to be solved by the linear programming solver by up to half, while delivering comparable or even superior results compared to the state-of-the-art verifiers.

Effective inclusion methods for verification of ReLU neural networks

Effective inclusion methods for verification of ReLU neural networks

INNAbstract: An INN-Based Abstraction Method for Large-Scale Neural Network Verification.

Neural networks (NNs) have witnessed widespread deployment across various domains, including some safety-critical applications. In this regard, the demand for verifying means of such artificial intelligence techniques is more and more pressing. Nowadays, the development of evaluation approaches for NNs is a hot topic that is attracting considerable interest, and a number of verification methods have been proposed. Yet, a challenging issue for NN verification is pertaining to the scalability when some NNs of practical interest have to be evaluated. This work aims to present INNAbstract, an abstraction method to reduce the size of NNs, which leads to improving the scalability of NN verification and reachability analysis methods. This is achieved by merging neurons while ensuring that the obtained model (i.e., abstract model) overapproximates the original one. INNAbstract supports networks with numerous activation functions. In addition, we propose a heuristic for nodes' selection to build more precise abstract models, in the sense that the outputs are closer to those of the original network. The experimental results illustrate the efficiency of the proposed approach compared to the existing relevant abstraction techniques. Furthermore, they demonstrate that INNAbstract can help the existing verification tools to be applied on larger networks while considering various activation functions.

Provable observation noise robustness for neural network control systems

Abstract Neural networks are vulnerable to adversarial perturbations: slight changes to inputs that can result in unexpected outputs. In neural network control systems, these inputs are often noisy sensor readings. In such settings, natural sensor noise – or an adversary who can manipulate them – may cause the system to fail. In this paper, we introduce the first technique to provably compute the minimum magnitude of sensor noise that can cause a neural network control system to violate a safety property from a given initial state. Our algorithm constructs a tree of possible successors with increasing noise until a specification is violated. We build on open-loop neural network verification methods to determine the least amount of noise that could change actions at each step of a closed-loop execution. We prove that this method identifies the unsafe trajectory with the least noise that leads to a safety violation. We evaluate our method on four systems: the Cart Pole and LunarLander environments from OpenAI gym, an aircraft collision avoidance system based on a neural network compression of ACAS Xu, and the SafeRL Aircraft Rejoin scenario. Our analysis produces unsafe trajectories where deviations under $1{\rm{\% }}$ of the sensor noise range make the systems behave erroneously.

Bridging formal methods and machine learning with model checking and global optimisation

Formal methods and machine learning are two research fields with drastically different foundations and philosophies. Formal methods utilise mathematically rigorous techniques for software and hardware systems' specification, development and verification. Machine learning focuses on pragmatic approaches to gradually improve a parameterised model by observing a training data set. While historically, the two fields lack communication, this trend has changed in the past few years with an outburst of research interest in the robustness verification of neural networks. This paper will briefly review these works, and focus on the urgent need for broader and more in-depth communication between the two fields, with the ultimate goal of developing learning-enabled systems with excellent performance and acceptable safety and security. We present a specification language, MLS2, and show that it can express a set of known safety and security properties, including generalisation, uncertainty, robustness, data poisoning, backdoor, model stealing, membership inference, model inversion, interpretability, and fairness. To verify MLS2 properties, we promote the global optimisation-based methods, which have provable guarantees on the convergence to the optimal solution. Many of them have theoretical bounds on the gap between current solutions and the optimal solution.

On applying residual reasoning within neural network verification

On applying residual reasoning within neural network verification

Extending Neural Network Verification to a Larger Family of Piece-wise Linear Activation Functions

Extending Neural Network Verification to a Larger Family of Piece-wise Linear Activation Functions

Neuron importance based verification of neural networks via divide and conquer

Neural networks are widely used in various applications such as image classification, speech recognition and natural language processing. As intelligent systems often rely on neural networks to make critical decisions, it is crucial to ensure that the behavior of a neural network is trustworthy. Along with the increase in the scale and complexity of neural networks, it becomes a challenging problem to verify their safety, since the large-scale state space may make the verification of safety properties infeasible or unsolvable. In order to achieve efficient safety verification of neural networks, this paper proposes a divide-and-conquer verification method based on neuron importance analysis. The method converts the safety constraints to be verified into an optimization problem composed of output neurons, and recursively measures the influence of neurons in the network on the final objective function to calculate their importance. The neurons with the greatest importance are split to refine the constraints, so that the original verification problem is transformed into a set of sub-problems for solution. The method is implemented as a tool named NIAVerify, which is evaluated through experiments of property verification on ReLU-based fully-connected benchmark networks trained on two classic datasets. Experimental results show that NIAVerify can achieve significant state space reduction and the verification efficiency is 44.80% higher than the existing verification method.

Mean field derivation and validity verification of neural networks coupled by Izhikevich neurons

Mean field derivation and validity verification of neural networks coupled by Izhikevich neurons