Accuracy Of Deep Neural Network Research Articles

Deep learning as Ricci flow.

Deep neural networks (DNNs) are powerful tools for approximating the distribution of complex data. It is known that data passing through a trained DNN classifier undergoes a series of geometric and topological simplifications. While some progress has been made toward understanding these transformations in neural networks with smooth activation functions, an understanding in the more general setting of non-smooth activation functions, such as the rectified linear unit (ReLU), which tend to perform better, is required. Here we propose that the geometric transformations performed by DNNs during classification tasks have parallels to those expected under Hamilton's Ricci flow-a tool from differential geometry that evolves a manifold by smoothing its curvature, in order to identify its topology. To illustrate this idea, we present a computational framework to quantify the geometric changes that occur as data passes through successive layers of a DNN, and use this framework to motivate a notion of 'global Ricci network flow' that can be used to assess a DNN's ability to disentangle complex data geometries to solve classification problems. By training more than 1500 DNN classifiers of different widths and depths on synthetic and real-world data, we show that the strength of global Ricci network flow-like behaviour correlates with accuracy for well-trained DNNs, independently of depth, width and data set. Our findings motivate the use of tools from differential and discrete geometry to the problem of explainability in deep learning.

NINNs: Nudging induced neural networks

Nudging induced neural networks (NINNs) algorithms are introduced to control and improve the accuracy of deep neural networks (DNNs). The NINNs framework can be applied to almost all pre-existing DNNs, with forward propagation, with costs comparable to existing DNNs. NINNs work by adding a feedback control term to the forward propagation of the network. The feedback term nudges the neural network towards a desired quantity of interest. NINNs offer multiple advantages, for instance, they lead to higher accuracy when compared with existing data assimilation algorithms such as nudging. Rigorous convergence analysis is established for NINNs. The algorithmic and theoretical findings are illustrated on examples from data assimilation and chemically reacting flows.

Partial density of states representation for accurate deep neural network predictions of X-ray spectra.

The performance of a machine learning (ML) algorithm for chemistry is highly contingent upon the architect's choice of input representation. This work introduces the partial density of states (p-DOS) descriptor: a novel, quantum-inspired structural representation which encodes relevant electronic information for machine learning models seeking to simulate X-ray spectroscopy. p-DOS uses a minimal basis set in conjunction with a guess (non-optimised) electronic configuration to extract and then discretise the density of states (DOS) of the absorbing atom to form the input vector. We demonstrate that while the electronically-focused p-DOS performs well in isolation, optimal performance is achieved when supplemented with nuclear structural information imparted via a geometric representation. p-DOS provides a description of the key electronic properties of a system which is not only concise and computationally efficient, but also independent of molecular size or choice of basis set. It can be rapidly generated, facilitating its application with large training sets. Its performance is demonstrated using a wide variety of examples at the sulphur K-edge, including the prediction of ultrafast X-ray spectroscopic signal associated with photoexcited 2(5H)-thiophenone. These results highlight the potential for ML models developed using p-DOS to contribute to the interpretation and prediction of experimental results e.g. in operando measurements of batteries and/or catalysts and femtosecond time-resolved studies, especially those made possible by emergent cutting-edge technologies, especially X-ray free electron lasers.

Benchmarking Test-Time DNN Adaptation at Edge with Compute-In-Memory

The prediction accuracy of deep neural networks (DNNs) deployed at the edge can deteriorate over time due to shifts in the data distribution. For heightened robustness, it’s crucial for DNNs to continually refine and improve their predictive capabilities. However, adaptation in resource-limited edge environments is fraught with challenges: (i) new labeled data might be unavailable; (ii) on-device adaptation is a necessity as cloud connections may be inaccessible; and (iii) the adaptation procedure should prioritize speed, memory efficiency, and energy conservation. Compute-In-Memory (CIM) has recently garnered attention for its computational efficacy and superior operational bandwidth. Additionally, emerging lightweight unsupervised DNN adaptation techniques during test-time have showcased promising results in enhancing model accuracy for data with noise. This article pioneers a holistic benchmarking exploration of these methods, assessing their performance and energy efficacy across diverse CIM architectures in edge and autonomous systems. Our findings reveal that the proposed adaptation strategies can adapt to both environment shifts and inherent hardware noise. Engaging in a thorough cross-layer algorithm-hardware-technology co-design space exploration, we highlight pivotal trade-offs among accuracy, performance, and energy for various DNN adaptation techniques and CIM configurations.

A depth analysis of recent innovations in non-invasive techniques using artificial intelligence approach for cancer prediction.

The fight against cancer, a relentless global health crisis, emphasizes the urgency for efficient and automated early detection methods. To address this critical need, this review assesses recent advances in non-invasive cancer prediction techniques, comparing conventional machine learning (CML) and deep neural networks (DNNs). Focusing on these seven major cancers, we analyze 310 publications spanning the years 2018 to 2024, focusing on detection accuracy as the key metric to identify the most effective predictive models, highlighting critical gaps in current methodologies, and suggesting directions for future research. We further delved into factors like datasets, features, and modalities to gain a comprehensive understanding of each approach's performance. Separate review tables for each cancer type and approach facilitated comparisons between top performers (accuracy exceeding 99%) and low performers (65.83 to 85.8%). Our exploration of public databases and commonly used classifiers revealed that optimal combinations of features, datasets, and models can achieve up to 100% accuracy for both CML and DNN. However, significant variations in accuracy (up to 35%) were observed, particularly when optimization was lacking. Notably, colorectal cancer exhibited the lowest accuracy (DNN 69%, CML 65.83%). A five-point comparative analysis (best/worst models, performance gap, average accuracy, and research trends) revealed that while DNN research is gaining momentum, CML approaches remain competitive, even outperforming DNN in some cases. This study presents an in-depth comparative analysis of CML and DNN techniques for cancer detection. This knowledge can inform future research directions and contribute to the development of increasingly accurate and reliable cancer detection tools.

On the choice of physical constraints in artificial neural networks for predicting flow fields

The application of Artificial Neural Networks (ANNs) has been extensively investigated for fluid dynamic problems. A specific form of ANNs are Physics-Informed Neural Networks (PINNs). They incorporate physical laws in the training and have increasingly been explored in the last few years. In this work, the prediction accuracy of PINNs is compared with that of conventional Deep Neural Networks (DNNs). The accuracy of a DNN depends on the amount of data provided for training. The change in prediction accuracy of PINNs and DNNs is assessed using a varying amount of training data. To ensure the correctness of the training data, they are obtained from analytical and numerical solutions of classical problems in fluid mechanics. The objective of this work is to quantify the fraction of training data relative to the maximum number of data points available in the computational domain, such that the accuracy gained with PINNs justifies the increased computational cost. Furthermore, the effects of the location of sampling points in the computational domain and noise in training data are analyzed. In the considered problems, it is found that PINNs outperform DNNs when the sampling points are positioned in the Regions of Interest. PINNs for predicting potential flow around a Rankine oval have shown a better robustness against noise in training data compared to DNNs. Both models show higher prediction accuracy when sampling points are randomly positioned in the flow domain as compared to a prescribed distribution of sampling points. The findings reveal new insights on the strategies to massively improve the prediction capabilities of PINNs with respect to DNNs.

Thermal Effects on Monolithic 3D Ferroelectric Transistors for Deep Neural Networks Performance

Monolithic three‐dimensional (M3D) integration advances integrated circuits by enhancing density and energy efficiency. Ferroelectric thin‐film transistors (Fe‐TFTs) attract attention for neuromorphic computing and back‐end‐of‐the‐line (BEOL) compatibility. However, M3D faces challenges like increased runtime temperatures due to limited heat dissipation, impacting system reliability. This work demonstrates the effect of temperature impact on single‐gate (SG) Fe‐TFT reliability. SG Fe‐TFTs have limitations such as read‐disturbance and small memory windows, constraining their use. To mitigate these, dual‐gate (DG) Fe‐TFTs are modeled using technology computer‐aided design, comparing their performance. Compute‐in‐memory (CIM) architectures with SG and DG Fe‐TFTs are investigated for deep neural networks (DNN) accelerators, revealing heat's detrimental effect on reliability and inference accuracy. DG Fe‐TFTs exhibit about 4.6x higher throughput than SG Fe‐TFTs. Additionally, thermal effects within the simulated M3D architecture are analyzed, noting reduced DNN accuracy to 81.11% and 67.85% for SG and DG Fe‐TFTs, respectively. Furthermore, various cooling methods and their impact on CIM system temperature are demonstrated, offering insights for efficient thermal management strategies.

Projection-based reduced order modeling of multi-species mixing and combustion

High-fidelity simulations of mixing and combustion processes are computationally demanding and time-consuming, hindering their wide application in industrial design and optimization. This study proposes projection-based reduced order models (ROMs) to predict spatial distributions of physical fields for multi-species mixing and combustion problems in a fast and accurate manner. The developed ROMs explore the suitability of various regression methods, including kriging, multivariate polynomial regression (MPR), k-nearest neighbors (KNN), deep neural network (DNN), and support vector regression (SVR), for the functional mapping between input parameters and reduced model coefficients of mixing and combustion problems. The ROMs are systematically examined using two distinct configurations: steam-diluted hydrogen-enriched oxy-combustion from a triple-coaxial nozzle and fuel-flexible combustion in a practical gas-turbine combustor. The projected low-dimensional manifolds are capable of capturing important combustion physics, and the response surfaces of reduced model coefficients present pronounced nonlinear characteristics of the flowfields with varying input parameters. The ROMs with kriging present a superior performance of establishing the input–output mapping to predict almost all physical fields, such as temperature, velocity magnitude, and combustion products for both test problems. The accuracy of DNN is less encouraging owing to the stringent requirement on the size of training database. KNN performs well in the region near the design points but its effectiveness diminishes when the test points are distant from the sampling points, whereas SVR and MPR exhibit large prediction errors. For the spatial prediction at unseen design points, the ROMs achieve a prediction time of up to eight orders of magnitude faster than conventional numerical simulations, rendering an efficient tool for the fast prediction of mixing and combustion fields and potentially an alternative of a full-order numerical solver.

Physics-driven deep neural networks for damage identification using guided wave damage interaction coefficients

In this paper, recent advancements in the development of a guided wave-based damage identification approach using wave damage interaction coefficients (WDICs) and deep neural networks (DNNs) are presented. These WDICs uniquely describe the complex scattering of guided waves around possible damages and depend on the properties of the damage itself. Hence, they are utilized as physics-based and highly sensitive damage features herein. It is demonstrated, that DNNs can effectively learn intricate relationships between damage characteristics and complex-shaped WDIC patterns from a compact sized training dataset. In this study, two training datasets are created by numerical finite element simulations and experimental scanning laser Doppler vibrometer measurements using a pseudo-damage approach. Therefore, the orientation and thickness of surface-bonded artificial damages are varied to generate the training data of 12 selected damage scenarios. The generalization capabilities of the fully trained DNNs allow to accurately predict WDICs even for damage scenarios unseen during training. The presented damage identification method leverages this powerful ability to characterize properties of unknown damages. Once trained, the accurate DNN predictions become available promptly and can be compared with measured WDICs from an unknown damage for selected sensor positions. The performance of both simulation-based and experiment-based structural health monitoring approaches are assessed, while also addressing current limitations and possible enhancements. For example, the great potential of incorporating the phase information of the complex-valued WDICs in the identification procedure is discussed. Hence, this study highlights the latest advancements in the development of the presented damage identification method with promising results and provides valuable insights in the application of WDICs as physics-based features for DNNs.

Introducing innovative deep neural networks to predict natural frequency of the nanocomposites reinforced concrete structures under external loading

The dynamic response of concrete structures reinforced with nanocomposites to supersonic airflow represents a critical aspect in aerospace and defense applications, necessitating accurate predictive models for enhanced structural integrity and performance. In this study, we introduce innovative deep neural networks (DNNs) as a novel approach to predict the dynamic behavior of such structures under supersonic airflow conditions. Traditional modeling techniques often face challenges in capturing the intricate interactions between material properties, structural geometry, and airflow dynamics, particularly in the presence of nanocomposite reinforcements. DNNs offer a promising solution by leveraging their ability to learn complex patterns and nonlinear relationships from extensive datasets. This paper presents a comprehensive framework for developing and deploying DNN-based models for dynamic prediction, encompassing network architecture design, training strategies, and data preprocessing techniques tailored to the unique characteristics of nanocomposite-reinforced concrete structures. Through a series of case studies and comparative analyses, we demonstrate the effectiveness and accuracy of DNNs in capturing the dynamic response of such structures to supersonic airflow, including phenomena such as vibration, flutter, and aerodynamic instability. Furthermore, we discuss the potential advantages and challenges associated with the adoption of DNNs in aerospace and defense applications, including model interpretability, computational efficiency, and data requirements. Finally, we outline future research directions and opportunities for further advancing the application of DNNs in addressing dynamic challenges in aerospace engineering and beyond.

A roulette wheel-based pruning method to simplify cumbersome deep neural networks

Deep neural networks (DNNs) have been applied in many pattern recognition or object detection applications. DNNs generally consist of millions or even billions of parameters. These demanding computational storage and requirements impede deployments of DNNs in resource-limited devices, such as mobile devices, micro-controllers. Simplification techniques such as pruning have commonly been used to slim DNN sizes. Pruning approaches generally quantify the importance of each component such as network weight. Weight values or weight gradients in training are commonly used as the importance metric. Small weights are pruned and large weights are kept. However, small weights are possible to be connected with significant weights which have impact to DNN outputs. DNN accuracy can be degraded significantly after the pruning process. This paper proposes a roulette wheel-like pruning algorithm, in order to simplify a trained DNN while keeping the DNN accuracy. The proposed algorithm generates a branch of pruned DNNs which are generated by a roulette wheel operator. Similar to the roulette wheel selection in genetic algorithms, small weights are more likely to be pruned but they can be kept; large weights are more likely to be kept but they can be pruned. The slimmest DNN with the best accuracy is selected from the branch. The performance of the proposed pruning algorithm is evaluated by two deterministic datasets and four non-deterministic datasets. Experimental results show that the proposed pruning algorithm generates simpler DNNs while DNN accuracy can be kept, compared to several existing pruning approaches.

AraDQ: an automated digital phenotyping software for quantifying disease symptoms of flood-inoculated Arabidopsis seedlings

BackgroundPlant scientists have largely relied on pathogen growth assays and/or transcript analysis of stress-responsive genes for quantification of disease severity and susceptibility. These methods are destructive to plants, labor-intensive, and time-consuming, thereby limiting their application in real-time, large-scale studies. Image-based plant phenotyping is an alternative approach that enables automated measurement of various symptoms. However, most of the currently available plant image analysis tools require specific hardware platform and vendor specific software packages, and thus, are not suited for researchers who are not primarily focused on plant phenotyping. In this study, we aimed to develop a digital phenotyping tool to enhance the speed, accuracy, and reliability of disease quantification in Arabidopsis.ResultsHere, we present the Arabidopsis Disease Quantification (AraDQ) image analysis tool for examination of flood-inoculated Arabidopsis seedlings grown on plates containing plant growth media. It is a cross-platform application program with a user-friendly graphical interface that contains highly accurate deep neural networks for object detection and segmentation. The only prerequisite is that the input image should contain a fixed-sized 24-color balance card placed next to the objects of interest on a white background to ensure reliable and reproducible results, regardless of the image acquisition method. The image processing pipeline automatically calculates 10 different colors and morphological parameters for individual seedlings in the given image, and disease-associated phenotypic changes can be easily assessed by comparing plant images captured before and after infection. We conducted two case studies involving bacterial and plant mutants with reduced virulence and disease resistance capabilities, respectively, and thereby demonstrated that AraDQ can capture subtle changes in plant color and morphology with a high level of sensitivity.ConclusionsAraDQ offers a simple, fast, and accurate approach for image-based quantification of plant disease symptoms using various parameters. Its fully automated pipeline neither requires prior image processing nor costly hardware setups, allowing easy implementation of the software by researchers interested in digital phenotyping of diseased plants.

Separating hard clean samples from noisy samples with samples’ learning risk for DNN when learning with noisy labels

Learning with Noisy Labels (LNL) methods aim to improve the accuracy of Deep Neural Networks (DNNs) when the training set contains samples with noisy or incorrect labels, and have become popular in recent years. Existing popular LNL methods frequently regard samples with high learning difficulty (high-loss and low prediction probability) as noisy samples; however, irregular feature patterns from hard clean samples can also cause high learning difficulty, which can lead to the misclassification of hard clean samples as noisy samples. To address this insufficiency, we propose the Samples’ Learning Risk-based Learning with Noisy Labels (SLRLNL) method. Specifically, we propose to separate noisy samples from hard clean samples using samples’ learning risk, which represents samples’ influence on DNN’s accuracy . We show that samples’ learning risk is comprehensively determined by samples’ learning difficulty as well as samples’ feature similarity to other samples, and thus, compared to existing LNL methods that solely rely on the learning difficulty, our method can better separate hard clean samples from noisy samples, since the former frequently possess irregular feature patterns. Moreover, to extract more useful information from samples with irregular feature patterns (i.e., hard samples), we further propose the Relabeling-based Label Augmentation (RLA) process to prevent the memorization of hard noisy samples and better learn the hard clean samples, thus enhancing the learning for hard samples. Empirical studies show that samples’ learning risk can identify noisy samples more accurately, and the RLA process can enhance the learning for hard samples. To evaluate the effectiveness of our method, we compare it with popular existing LNL methods on CIFAR-10, CIFAR-100, Animal-10N, Clothing1M, and Docred. The experimental results indicate that our method outperforms other existing methods. The source code for SLRLNL can be found at https://github.com/yangbo1973/SLRLNL.

CardioDefense: Defending against adversarial attack in ECG classification with adversarial distillation training

In clinics, doctors rely on electrocardiograms (ECGs) to assess severe cardiac disorders. Owing to the development of technology and the increase in health awareness, ECG signals are currently obtained by using medical and commercial devices. Deep neural networks (DNNs) can be used to analyze these signals because of their high accuracy rate. However, researchers have found that adversarial attacks can significantly reduce the accuracy of DNNs. Studies have been conducted to defend ECG-based DNNs against traditional adversarial attacks, such as projected gradient descent (PGD), and smooth adversarial perturbation (SAP) which targets ECG classification; however, to the best of our knowledge, no study has completely explored the defense against adversarial attacks targeting ECG classification. Thus, we did different experiments to explore the effects of defense methods against white-box adversarial attack and black-box adversarial attack targeting ECG classification, and we found that some common defense methods performed well against these attacks. Besides, we proposed a new defense method based on adversarial distillation training (named CardioDefense) which comes from defensive distillation and can effectively improve the generalization performance of DNNs. The results show that our method performed more effectively against adversarial attacks targeting on ECG classification than the other baseline methods, namely, adversarial training, defensive distillation, Jacob regularization, and noise-to-signal ratio regularization. Furthermore, we found that our method performed better against PGD attacks with low noise levels, which means that our method has stronger robustness.

Neural Network Compression Based on Tensor Ring Decomposition.

Deep neural networks (DNNs) have made great breakthroughs and seen applications in many domains. However, the incomparable accuracy of DNNs is achieved with the cost of considerable memory consumption and high computational complexity, which restricts their deployment on conventional desktops and portable devices. To address this issue, low-rank factorization, which decomposes the neural network parameters into smaller sized matrices or tensors, has emerged as a promising technique for network compression. In this article, we propose leveraging the emerging tensor ring (TR) factorization to compress the neural network. We investigate the impact of both parameter tensor reshaping and TR decomposition (TRD) on the total number of compressed parameters. To achieve the maximal parameter compression, we propose an algorithm based on prime factorization that simultaneously identifies the optimal tensor reshaping and TRD. In addition, we discover that different execution orders of the core tensors result in varying computational complexities. To identify the optimal execution order, we construct a novel tree structure. Based on this structure, we propose a top-to-bottom splitting algorithm to schedule the execution of core tensors, thereby minimizing computational complexity. We have performed extensive experiments using three kinds of neural networks with three different datasets. The experimental results demonstrate that, compared with the three state-of-the-art algorithms for low-rank factorization, our algorithm can achieve better performance with much lower memory consumption and lower computational complexity.

Model Predictive Control of Deep Neural Network Model with ReLU Structure using Mixed Integer Programming

This study proposes a model predictive control framework using a mixed integer programming for the deep neural network (DNN) model with rectified linear unit (ReLU). The accurate DNN model with ReLU using the past input time series for the diesel engine is constructed. In this method, the ReLU is represented by inequalities including 0-1 variables (binaries). The model predictive control problem is formulated as a mixed integer programming problem and is applied to the control problem of the diesel engine. Dynamical DNN model of a diesel engine is constructed using the experimental data. This model can represent complicate engine phenomena such as strong nonlinearity and delay. Simulation results of the worldwide harmonized transient cycle (WHTC) verify the effectiveness of the proposed method. The difficulties of non-linearity, delay, and interference in the control problem of the diesel engine are solved by using a systematic model predictive control of the DNN model.

Joint Protection Scheme for Deep Neural Network Hardware Accelerators and Models

Deep neural networks (DNNs) are utilized in numerous image processing, object detection, and video analysis tasks and need to be implemented using hardware accelerators to achieve practical speed. Logic locking is one of the most popular methods for preventing chip counterfeiting. Nevertheless, existing logic-locking schemes need to sacrifice the number of input patterns leading to wrong output under incorrect keys to resist the powerful satisfiability (SAT)-attack. Furthermore, DNN model inference is fault-tolerant. Hence, using a wrong key for those SAT-resistant logic-locking schemes may not affect the accuracy of DNNs. This makes the previous SAT-resistant logic-locking scheme ineffective on protecting DNN accelerators. Besides, to prevent DNN models from being illegally used, the models need to be obfuscated by the designers before they are provided to end-users. Previous obfuscation methods either require long time to retrain the model or leak information about the model. This paper proposes a joint protection scheme for DNN hardware accelerators and models. The DNN accelerator is modified using a hardware key (Hkey) and a model key (Mkey). Different from previous logic locking, the Hkey, which is used to protect the accelerator, does not affect the output when it is wrong. As a result, the SAT attack can be effectively resisted. On the other hand, a wrong Hkey leads to substantial increase in memory accesses, inference time, and energy consumption and makes the accelerator unusable. A correct Mkey can recover the DNN model that is obfuscated by the proposed method. Compared to previous model obfuscation schemes, our proposed method avoids model retraining and does not leak model information.

A Bayesian deep learning framework for reliable fault diagnosis in wind turbine gearboxes under various operating conditions

Vibration-based fault diagnostics combined with deep learning approaches has promising applications in detecting and diagnosing faults in wind turbine gearboxes. Specifically when time series vibration data is transformed to a 2-dimensional cyclic spectral coherence maps, the accuracy of deep neural networks in classifying faults increases. Nevertheless, standard deep learning techniques are vulnerable to inaccurate predictions when tested with new data originating from unseen faults or unusual operating conditions. To address some of these shortcomings in the context of wind turbine gearboxes, this paper investigates fault diagnostics using Bayesian convolutional neural network which provide accurate results with uncertainty bounds reducing wrong overconfident classifications. The performance of Bayesian and standard neural networks is compared using a simulation-based dataset of acceleration signals generated from a multibody dynamic model of a 5 MW wind turbine. The framework proposed in this paper has relevance to fault detection and diagnosis in other rotating machinery applications.

A White-Box Testing for Deep Neural Networks Based on Neuron Coverage.

With the introduction of neuron coverage as a testing criterion for deep neural networks (DNNs), covering more neurons to detect more internal logic of DNNs became the main goal of many research studies. While some works had made progress, some new challenges for testing methods based on neuron coverage had been proposed, mainly as establishing better neuron selection and activation strategies influenced not only obtaining higher neuron coverage, but also more testing efficiency, validating testing results automatically, labeling generated test cases to extricate manual work, and so on. In this article, we put forward Test4Deep, an effective white-box testing DNN approach based on neuron coverage. It is based on a differential testing framework to automatically verify inconsistent DNNs' behavior. We designed a strategy that can track inactive neurons and constantly triggered them in each iteration to maximize neuron coverage. Furthermore, we devised an optimization function that guided the DNN under testing to deviate predictions between the original input and generated test data and dominated unobservable generation perturbations to avoid manually checking test oracles. We conducted comparative experiments with two state-of-the-art white-box testing methods DLFuzz and DeepXplore. Empirical results on three popular datasets with nine DNNs demonstrated that compared to DLFuzz and DeepXplore, Test4Deep, on average, exceeded by 32.87% and 35.69% in neuron coverage, while reducing 58.37% and 53.24% testing time, respectively. In the meantime, Test4Deep also produced 58.37% and 53.24% more test cases with 23.81% and 98.40% fewer perturbations. Even compared with the two highest neuron coverage strategies of DLFuzz, Test4Deep still enhanced neuron coverage by 4.34% and 23.23% and achieved 94.48% and 85.67% higher generation time efficiency. Furthermore, Test4Deep could improve the accuracy and robustness of DNNs by merging generated test cases and retraining.

CRAFT: Criticality-Aware Fault-Tolerance Enhancement Techniques for Emerging Memories-Based Deep Neural Networks

Deep Neural Networks (DNNs) have emerged as the most effective programming paradigm for computer vision and natural language processing applications. With the rapid development of DNNs, efficient hardware architectures for deploying DNN-based applications on edge devices have been extensively studied. Emerging Non-Volatile Memories (NVMs), with their better scalability, non-volatility and good read performance, are found to be promising candidates for deploying DNNs. However, despite the promise, emerging NVMs often suffer from reliability issues such as stuck-at faults, which decrease the chip yield/memory lifetime and severely impact the accuracy of DNNs. A stuck-at cell can be read but not reprogrammed, thus, stuck-at faults in NVMs may or may not result in errors depending on the data to be stored. By reducing the number of errors caused by stuck-at faults, the reliability of a DNN-based system can be enhanced. This paper proposes CRAFT, i.e., Criticality-Aware Fault-Tolerance Enhancement Techniques to enhance the reliability of NVM-based DNNs in the presence of stuck-at faults. A data block remapping technique is used to reduce the impact of stuck-at faults on DNNs accuracy. Additionally, by performing bit-level criticality analysis on various DNNs, the critical-bit positions in network parameters that can significantly impact the accuracy are identified. Based on this analysis, we propose an encoding method which effectively swaps the critical bit positions with that of non-critical bits when more errors (due to stuck-at faults) are present in the critical bits.