Fault Tolerance Of Neural Networks Research Articles

Finite-time sliding mode fault-tolerant neural network control for nonstrict-feedback nonlinear systems

Finite-time sliding mode fault-tolerant neural network control for nonstrict-feedback nonlinear systems

Fault-Tolerant Neural Network Accelerators With Selective TMR

This article presents a tool that analyzes sensitive computations in the neural network and triplicates them to increase the functional safety of the neural network accelerator. —Fei Su, Intel Corporation

Node co-activations as a means of error detection—Towards fault-tolerant neural networks

Machine learning has proved an efficient tool, but the systems need tools to mitigate risks during runtime. One approach is fault tolerance: detecting and handling errors before they cause harm. This paper investigates whether rare co-activations – pairs of usually segregated nodes activating together – are indicative of problems in neural networks (NN). These could be used to detect concept drift and flagging untrustworthy predictions. We trained four NNs. For each, we studied how often each pair of nodes activates together. In a separate test set, we counted how many rare co-activations occurred with each input, and grouped the inputs based on whether its classification was correct, incorrect, or whether its class was absent during training. Rare co-activations are much more common in inputs from a class that was absent during training. Incorrectly classified inputs averaged a larger number of rare co-activations than correctly classified inputs, but the difference was smaller. As rare co-activations are more common in unprecedented inputs, they show potential for detecting concept drift. There is also some potential in detecting single inputs from untrained classes. The small difference between correctly and incorrectly predicted inputs is less promising and needs further research.

Modularized Information Fusion Design of Urban Garden Landscape in Big Data Background

Traditional information fusion model has the problem of low efficiency in urban landscape design. In addition, using the current method to design urban commercial landscape public facilities, there are problems of large regional space occupation and unsatisfactory design effect. This paper designs a new modular information fusion model for urban landscape design process in view of genetic back propagation. On the basis of preprocessing sensor images, a digital elevation model is created using an ordered numerical sequence. Then, the stereo orthophoto image pair is obtained through the artificial parallax assistance mechanism, and the 3D garden landscape is generated by combining with the ant colony algorithm. The positive feedback mechanism of the ant colony algorithm is used to make the processing process converge continuously, and the optimal 3D garden landscape is finally generated by obtaining stereo orthophoto pairs through the artificial parallax-assisted mechanism. At the same time, the strong robustness and fault tolerance of neural network and parallel processing mechanism are utilized for fast information fusion. The scale and resources of garden design are described by the process dimension and the context dimension, and a modular garden landscape with distinct main body is built. Finally, the initial weight is optimized in the genetic real number coding algorithm, and the appropriate learning factor is selected to train the neural network so as to make the information fusion task. Experimental results show that the above model fusion process has good stability and low energy consumption for information fusion, which can promote the efficient construction of garden landscapes.

Fault-tolerant neural network training framework based on client-server

In order to realize low power consumption and real-time inference, AIoT devices have been applied in many fields of deep learning in recent years. However, some manufacturing processes cause some soft errors on AIOT devices in inference. For a neural network accelerator with a large amount of computation, it may lead to a large amount of computing error and a huge loss of prediction accuracy, which is intolerable for precision-sensitive applications such as autonomous drones. However, conventional fault tolerance techniques such as triple modular redundancy can incur considerable power consumption and performance penalty. In this paper, a client-server collaborative fault-tolerant neural network training framework is proposed. In the training, an AIoT processor with soft errors is used as the client, and the server learns the on-site computing errors with the application data of AIoT processor. Several representative neural network models were selected in the experiment. Compared with the off-line training model, the model trained by this method increases the top5 accuracy of the neural network by an average of 2.8%.

Resilience of neural networks for locomotion.

Locomotion is an essential behaviour for the survival of all animals. The neural circuitry underlying locomotion is therefore highly robust to a wide variety of perturbations, including injury and abrupt changes in the environment. In the short term, fault tolerance in neural networks allows locomotion to persist immediately after mild to moderate injury. In the longer term, in many invertebrates and vertebrates, neural reorganization including anatomical regeneration can restore locomotion after severe perturbations that initially caused paralysis. Despite decades of research, very little is known about the mechanisms underlying locomotor resilience at the level of the underlying neural circuits and coordination of central pattern generators (CPGs). Undulatory locomotion is an ideal behaviour for exploring principles of circuit organization, neural control and resilience of locomotion, offering a number of unique advantages including experimental accessibility and modelling tractability. In comparing three well-characterized undulatory swimmers, lampreys, larval zebrafish and Caenorhabditis elegans, we find similarities in the manifestation of locomotor resilience. To advance our understanding, we propose a comparative approach, integrating experimental and modelling studies, that will allow the field to begin identifying shared and distinct solutions for overcoming perturbations to persist in orchestrating this essential behaviour.

Fault tolerance of neural networks in adversarial settings

Artificial Intelligence systems require a through assessment of different pillars of trust, namely, fairness, interpretability, data and model privacy, reliability (safety) and robustness against against adversarial attacks. While these research problems have been extensively studied in isolation, an understanding of the trade-off between different pillars of trust is lacking. To this extent, the trade-off between fault tolerance, privacy and adversarial robustness is evaluated for the specific case of Deep Neural Networks, by considering two adversarial settings under a security and a privacy threat model. Specifically, this work studies the impact of the fault tolerance of the Neural Network on training the model by adding noise to the input (Adversarial Robustness) and noise to the gradients (Differential Privacy). While training models with noise to inputs, gradients or weights enhances fault tolerance, it is observed that adversarial robustness and fault tolerance are at odds with each other. On the other hand, ($\epsilon,\delta$)-Differentially Private models enhance the fault tolerance, measured using generalisation error, theoretically has an upper bound of $e^{\epsilon} - 1 + \delta$. This novel study of the trade-off between different elements of trust is pivotal for training a model which satisfies the requirements for different pillars of trust simultaneously.

Fault and Error Tolerance in Neural Networks: A Review

Beyond energy, the growing number of defects in physical substrates is becoming another major constraint that affects the design of computing devices and systems. As the underlying semiconductor technologies are getting less and less reliable, the probability that some components of computing devices fail also increases, preventing designers from realizing the full potential benefits of on-chip exascale integration derived from near atomic scale feature dimensions. As the quest for performance confronts permanent and transient faults, device variation, and thermal issues, major breakthroughs in computing efficiency are expected to benefit from unconventional and new models of computation, such as brain-inspired computing. The challenge is then to find not only high-performance and energy-efficient, but also fault-tolerant computing solutions. Neural computing principles remain elusive, yet as source of a promising fault-tolerant computing paradigm. In the quest to fault tolerance can be translated into scalable and reliable computing systems, hardware design itself and/or to use circuits even with faults has further motivated research on neural networks, which are potentially capable of absorbing some degrees of vulnerability based on their natural properties. This paper presents a survey on fault tolerance in neural networks manly focusing on well-established passive techniques to exploit and improve, by design, such potential but limited intrinsic property in neural models, particularly for feedforward neural networks. First, fundamental concepts and background on fault tolerance are introduced. Then, we review fault types, models, and measures used to evaluate performance and provide a taxonomy of the main techniques to enhance the intrinsic properties of some neural models, based on the principles and mechanisms that they exploit to achieve fault tolerance passively. For completeness, we briefly review some representative works on active fault tolerance in neural networks. We present some key challenges that remain to be overcome and conclude with an outlook for this field.

Distributed adaptive neural network control for a class of heterogeneous nonlinear multi-agent systems subject to actuation failures

ABSTRACTIn this paper, the cooperative adaptive consensus tracking problem for heterogeneous nonlinear multi-agent systems on directed graph is addressed. Each follower is modelled as a general nonlinear system with the unknown and nonidentical nonlinear dynamics, disturbances and actuator failures. Cooperative fault tolerant neural network tracking controllers with online adaptive learning features are proposed to guarantee that all agents synchronise to the trajectory of one leader with bounded adjustable synchronisation errors. With the help of linear quadratic regulator-based optimal design, a graph-dependent Lyapunov proof provides error bounds that depend on the graph topology, one virtual matrix and some design parameters. Of particular interest is that if the control gain is selected appropriately, the proposed control scheme can be implemented in a unified framework no matter whether there are faults or not. Furthermore, the fault detection and isolation are not needed to implement. Finally, a simulation is given to verify the effectiveness of the proposed method.

Fault tolerant control based on neural network interval type-2 fuzzy sliding mode controller for octorotor UAV

In this paper, a robust controller for a six degrees of freedom (6 DOF) octorotor helicopter control is proposed in presence of actuator and sensor faults. Neural networks (NN), interval type-2 fuzzy logic control (IT2FLC) approach and sliding mode control (SMC) technique are used to design a controller, named fault tolerant neural network interval type-2 fuzzy sliding mode controller (FTNNIT2FSMC), for each subsystem of the octorotor helicopter. The proposed control scheme allows avoiding difficult modeling, attenuating the chattering effect of the SMC, reducing the number of rules for the fuzzy controller, and guaranteeing the stability and the robustness of the system. The simulation results show that the FTNNIT2FSMC can greatly alleviate the chattering effect, tracking well in presence of actuator and sensor faults.

Numerical Simulation of Neural Network Components of Controlling and Measuring Systems

The article deals with the problem of calculating the fault tolerance of neural network components of industrial controlling and measuring systems used in mechanical engineering. We have formulated a general approach to developing methods for quantitative determination of the level of the fault tolerance of artificial neural networks with any structure and function. We have studied the fault tolerance of four artificial feedforward neural networks as well as the correlation between the result of determining the fault tolerance level and a selected performance parameter of artificial neural networks.

Parameter estimation of the exponentially damped sinusoids signal using a specific neural network

The problem of estimating the parameters of exponentially damped sinusoids (EDSs) signal has received very much attention in many fields. In this paper, a specific neural network termed EDSNN for parameter estimation of the EDSs has been proposed. Aiming at effectively evaluating the parameters of the EDSs signal, we construct a specific topology of EDSNN strictly following the mathematic formulation of EDSs signal. Then, what should be further done is how to train EDSNN using the data-set sampled from the EDSs signal. For this purpose, a modified Levenberg–Marquardt algorithm is derived for iteratively solving the weights of EDSNN by optimizing the pre-defined objective function. Profiting from good performance in fault tolerance of neural network, the proposed algorithm possesses a good performance in resistance to noise. Several computer simulations have been conducted to apply this method to some EDSs signal models. The results substantiate that the proposed EDSNN can synchronously obtain a higher precision for the damped factors, frequencies, also amplitudes and initial phases of all the EDSs than the state-of-the-art algorithm for noise free or noise case.

Concept Optimization Model with Multilevel Hierarchy Based on Fuzzy Neural Network

To implement optimization for mechanical concepts acquired by function analysis more effectively, BP neural network is adopted to structure multilevel evaluation model, which capitalizes on the features of nonlinearity, self-organization, and fault tolerance of neural network. By using appropriate data sets to train the neural network, expertise is acquired and expressed using a trained weight and threshold matrix. Once evaluation objectives of each candidate are fuzzily quantified, converted into evaluation attribute value, and fed into the trained network model, the optimal concept can be obtained. During the process, neural network is used to solve the bottle-neck problem of knowledge acquisition and expression and can be viewed as knowledge base and reasoning engine for the optimization. Hence the proposed evaluation model can effectively deal with concept evaluation and optimization problem with multilevel objective system.

Experimental Demonstration of Hopfield Neural Network using DNA molecules

ABSTRACTDNA Computing is a rapidly-developing interdisciplinary area which could benefit from more experimental results to solve problems with the current biological tools. In this study, we have integrated microelectronics and molecular biology techniques for showing the feasibility of Hopfield Neural Network using DNA molecules. Adleman’s seminal paper in 1994 showed that DNA strands using specific molecular reactions can be used to solve the Hamiltonian Path Problem. This accomplishment opened the way for possibilities of massively parallel processing power, remarkable energy efficiency and compact data storage ability with DNA. However, in various studies, small departures from the ideal selectivity of DNA hybridization lead to significant undesired pairings of strands and that leads to difficulties in schemes for implementing large Boolean functions using DNA. Therefore, these error prone reactions in the Boolean architecture of the first DNA computers will benefit from fault tolerance or error correction methods and these methods would be essential for large scale applications. In this study, we demonstrate the operation of six dimensional Hopfield associative memory storing various memories as an archetype fault tolerant neural network implemented using DNA molecular reactions. The response of the network suggests that the protocols could be scaled to a network of significantly larger dimensions. In addition the results are read on a Silicon CMOS platform exploiting the semiconductor processing knowledge for fast and accurate hybridization rates.

Empirical Study of Least Sensitive FFANN for Weight-Stuck-at Zero Fault

ABSTRACT An important consideration for neural hardware is its sensitivity to input and weight errors. In this paper, an empirical study is performed to analyze the sensitivity of feedforward neural networks for Gaussian noise to input and weight. 30 numbers of FFANN is taken for four different classification tasks. Least sensitive network for input and weight error is chosen for further study of fault tolerant behavior of FFANN. Weight stuck-at zero fault is selected to study error metrics of fault tolerance. Empirical results for a WSZ fault is demonstrated in this paper. configuration of the network. General Terms Artificial Neural Network Keywords Artificial Neural Network, Fault models, Sensitivity analysis. 1. INTRODUCTION Sensitivity analysis provides ability to analyze faulty behavior of neural hardware [3]. Error/fault in feedforward artificial neural network FFANN is classified as [20]: 1) node error/fault (a) Output node error/fault (b) Hidden layer node error/fault 2) weight error/fault 3) Input error/fault Study of fault tolerance and robustness in neural networks needs following three areas to be explored [18]: 1) The capability of neural networks to performs in case of failure of internal components (node/weight) 2) Tolerance capability of FFANN to input noise 3) Tolerance capability of FFANN to noise in weight Design of fault tolerant neural network for failure of an internal component is studied in [19], where redundancy technique has been proposed. Another technique to design fault tolerant neural networks is given in [21], where magnitude of weight is initially kept low, so that any error due to weight will be negligible. While fault tolerance networks for the noise perturbed in input data and weight requires understanding the effects due to small error in input and weight. Sensitivity analysis is required to study the effect of noise in input and weight [1][4][22]. In [10], Alippi

Convergence and Objective Functions of Some Fault/Noise-Injection-Based Online Learning Algorithms for RBF Networks

In the last two decades, many online fault/noise injection algorithms have been developed to attain a fault tolerant neural network. However, not much theoretical works related to their convergence and objective functions have been reported. This paper studies six common fault/noise-injection-based online learning algorithms for radial basis function (RBF) networks, namely 1) injecting additive input noise, 2) injecting additive/multiplicative weight noise, 3) injecting multiplicative node noise, 4) injecting multiweight fault (random disconnection of weights), 5) injecting multinode fault during training, and 6) weight decay with injecting multinode fault. Based on the Gladyshev theorem, we show that the convergence of these six online algorithms is almost sure. Moreover, their true objective functions being minimized are derived. For injecting additive input noise during training, the objective function is identical to that of the Tikhonov regularizer approach. For injecting additive/multiplicative weight noise during training, the objective function is the simple mean square training error. Thus, injecting additive/multiplicative weight noise during training cannot improve the fault tolerance of an RBF network. Similar to injective additive input noise, the objective functions of other fault/noise-injection-based online algorithms contain a mean square error term and a specialized regularization term.

Prediction error of a fault tolerant neural network

Prediction error is a powerful tool that measures the performance of a neural network. In this paper, we extend the technique to a kind of fault tolerant neural networks. Considering a neural network with multiple-node fault, we derive its generalized prediction error. Hence, the effective number of parameters of such a fault tolerant neural network is obtained. The difficulty in obtaining the mean prediction error is discussed. Finally, a simple procedure for estimation of the prediction error is empirically suggested.

A GENERALIZED ABFT TECHNIQUE USING A FAULT TOLERANT NEURAL NETWORK

In this paper, a measure of sensitivity is defined to evaluate fault tolerance of neural networks and we show that the sensitivity of a link is closely related to the amount of information passed through it. Based on this assumption, we prove that the distribution of output error caused by s-a-0 (stuck at 0) faults in an MLP network has a Gaussian distribution function. UDBP (Uniformly Distributed Back Propagation) algorithm is then introduced to minimize mean and variance of the output error. Then an MLP neural network trained with UDBP, contributes in an Algorithm-Based Fault Tolerant (ABFT) scheme to protect a nonlinear data process block. A systematic real convolution code guarantees that faults representing errors in the processed data will result in notable nonzero values in syndrome sequence. A majority logic decoder can now easily detect and correct single faults by observing the syndrome sequence. Simulation results demonstrating the error detection and correction behavior against random s-a-0 faults are presented too.

Investigating the Fault Tolerance of Neural Networks

Particular levels of partial fault tolerance (PFT) in feedforward artificial neural networks of a given size can be obtained by redundancy (replicating a smaller normally trained network), by design (training specifically to increase PFT), and by a combination of the two (replicating a smaller PFT-trained network). This letter investigates the method of achieving the highest PFT per network size (total number of units and connections) for classification problems. It concludes that for non-toy problems, there exists a normally trained network of optimal size that produces the smallest fully fault-tolerant network when replicated. In addition, it shows that for particular network sizes, the best level of PFT is achieved by training a network of that size for fault tolerance. The results and discussion demonstrate how the outcome depends on the levels of saturation of the network nodes when classifying data points. With simple training tasks, where the complexity of the problem and the size of the network are well within the ability of the training method, the hidden-layer nodes operate close to their saturation points, and classification is clean. Under such circumstances, replicating the smallest normally trained correct network yields the highest PFT for any given network size. For hard training tasks (difficult classification problems or network sizes close to the minimum), normal training obtains networks that do not operate close to their saturation points, and outputs are not as close to their targets. In this case, training a larger network for fault tolerance yields better PFT than replicating a smaller, normally trained network. However, since fault-tolerant training on its own produces networks that operate closer to their linear areas than normal training, replicating normally trained networks ultimately leads to better PFT than replicating fault-tolerant networks of the same initial size.

Application of fault-tolerant neural networks in the synthesis of microwave devices

A novel artificial neural network (ANN) suitable for computationally intensive problems is described in this paper. The usefulness of this ANN is demonstrated for the synthesis of a microstrip line. This ANN uses a standard neural network architecture consisting of a hetero-associative memory and exploits a fault-tolerant number representation, which gives significant insight into a new method of fault-tolerant computing. In addition this ANN provides an efficient method for synthesizing geometrical parameters of microwave devices, when stochastic features are incorporated in the synthesis process. Further research is required to investigate the potential of this new paradigm.