Accuracy Degradation Research Articles

Evaluation of Accuracy Degradation Resulting from Concept Drift in a Fake News Detection System Using Emotional Expression

Fake news on social media has become a social problem. Fake news refers to false information that is deliberately intended to deceive people. Several studies have been conducted on automatic detection systems that reduce the damage caused by fake news. However, most studies address the improvements made in detection accuracy, and real-world operations are rarely discussed. As the contents and expressions of fake news change over time, a model with a high detection accuracy loses accuracy after a few years. This phenomenon is called concept drift. As most conventional methods employ word representations, these methods exhibit accuracy degradation resulting from changes in word fads and usage. However, methods using the sentiment information of words can identify inflammatory sentences, which is a characteristic of fake news, and may suppress performance degradation caused by concept drift. In this study, a model using vector representations obtained from an emotion dictionary was compared with a model using conventional word embedding. Subsequently, we verified the resistance of the model to performance degradation. The results revealed the method using sentiment representation is less susceptible to concept drift. Models and learning methods that can achieve both detection accuracy and resistance to accuracy degradation can enable further development of fake news detection systems.

Dual-feature Fusion Attention Network for Small Object Segmentation

Accurate segmentation of medical images is an important step during radiotherapy planning and clinical diagnosis. However, manually marking organ or lesion boundaries is tedious, time-consuming, and prone to error due to subjective variability of radiologist. Automatic segmentation remains a challenging task owing to the variation (in shape and size) across subjects. Moreover, existing convolutional neural networks based methods perform poorly in small medical objects segmentation due to class imbalance and boundary ambiguity. In this paper, we propose a dual feature fusion attention network (DFF-Net) to improve the segmentation accuracy of small objects. It mainly includes two core modules: the dual-branch feature fusion module (DFFM) and the reverse attention context module (RACM). We first extract multi-resolution features by multi-scale feature extractor, then construct DFFM to aggregate the global and local contextual information to achieve information complementarity among features, which provides sufficient guidance for accurate small objects segmentation. Moreover, to alleviate the degradation of segmentation accuracy caused by blurred medical image boundaries, we propose RACM to enhance the edge texture of features. Experimental results on datasets NPC, ACDC, and Polyp demonstrate that our proposed method has fewer parameters, faster inference, and lower model complexity, and achieves better accuracy than more state-of-the-art methods.

Sparse hydrophone array synthesis for frequency‐invariant and multi‐angle beampatterns using block‐structured Bayesian compressive sensing

AbstractTo solve the synthesised accuracy degradation of traditional Bayesian compressive sensing (BCS) in wideband multi‐beam hydrophone array synthesis, a novel sparse array synthesis method based on block‐structure Bayesian compressive sensing (BS‐BCS) is proposed for frequency‐invariant and multi‐angle beampatterns. The multi‐angle beampatterns at different frequencies are synthesised in block structure using the shared position strategy which effectively reduces the number of array elements. In order to meet the minimum spacing requirement of adjacent array elements in practical engineering, the constraint converted off‐grid method is proposed to improve the array synthesis accuracy. The grid scaling is introduced to obtain the possible array element positions. The array positions and weights can be solved by the BCS solver and corrected by Taylor perturbation. To validate the effectiveness and feasibility of the proposed method, simulations and actual experiments are carried out. The synthesised errors and peak sidelobe levels (PSLLs) of multi‐angle beampatterns obtained by the constraint converted off‐grid method outperforms those of other methods. The beamforming results of actual experiments in marine environment show that the PSLLs of the beampatterns obtained by BS‐BCS is lower at least 6.3 dB than that of uniform array, which demonstrates the effectiveness of the proposed method.

X-NVDLA: Runtime Accuracy Configurable NVDLA Based on Applying Voltage Overscaling to Computing and Memory Units

This paper investigates a runtime accuracy reconfigurable implementation of an energy efficient deep learning accelerator. It is based on voltage overscaling (VOS) technique which provides dynamic adjustment of approximation level as well as improving lifetime/reliability of the accelerator. The technique is applied to both computing and memory units where based on the minimum required accuracy, the applied voltage is adjusted during the runtime. The implementation of the network is performed using NVDLA which is an open-source CNN (Convolutional Neural Network) accelerator. The approximation is applied to both the accelerator MAC array employed for the required network computations and to the accelerator SRAM memory utilized for storing the inputs (images), weights, and activation data. To control the accuracy degradation of the approximate accelerator (called X-NVDLA), the reduced voltage is applied only to LSB bits of the MAC array and SRAM unit. To assess the efficacy of the proposed energy efficient accelerator, the energy-accuracy characteristics of X-NVDLA when running LeNet-5 and ResNet-50 networks with 8-bit (integer) precision are investigated. In addition, the characteristic of bias temperature instability (BTI), as one of the lifetime deteriorating phenomena is determined. The study includes energy improvement versus accuracy degradation as a function of overscaled voltages and number of approximate the least significant bits using a 15nm FinFET technology.

Network Pruning for Bit-Serial Accelerators

Bit-serial architectures (BSAs) are becoming increasingly popular in low-power neural network processor (NNP) designs for edge scenarios. However, the performance and energy efficiency of state-of-the-art BSA NNPs heavily depends on both the proportion and distribution of ineffectual weight bits in neural networks (NNs). To boost the performance of typical BSA accelerators, we present Bit-Pruner, a software approach to learn BSA-favored NNs without resorting to hardware modifications. Bit-Pruner not only progressively prunes but also restructures the non-zero bits in weights so that the number of non-zero bits in the model can be reduced and the corresponding computing can be load-balanced to suit the target BSA accelerators. On top of Bit-Pruner, we further propose a Pareto frontier optimization algorithm to adjust the bit-pruning rate across network layers and fulfill diverse NN processing requirements in terms of performance and accuracy for various edge scenarios. However, an aggressive Bit-Pruner can lead to non-trivial accuracy loss especially for lightweight NNs and complex tasks. To this end, alternating direction method of multipliers (ADMM) is adapted to the retraining phase in Bit-Pruner to smooth the abrupt disturbance due to bit-pruning and enhance the resulting model accuracy. According to the experiments, Bit-Pruner increases the bit-sparsity up to 94.4% with negligible accuracy degradation and achieves optimized trade-off between NN accuracy and energy efficiency even under very-aggressive performance constraints. When pruned models are deployed onto typical BSA accelerators, the average performance is 2.1X and 1.6X higher than the baseline networks without pruning and those with classical weight pruning, respectively.

GMM-Based Adaptive Extended Kalman Filter Design for Satellite Attitude Estimation under Thruster-Induced Disturbances.

Star images from star trackers are usually defocused to capture stars over an exposure time for better centroid measurements. While a satellite is maneuvering, the star point on the screen of the camera is affected by the satellite, which results in the degradation of centroid measurement accuracy. Additionally, this could result in a worse star vector outcome. For geostationary satellites, onboard thrusters are used to maintain or change orbit parameters under orbit disturbances. Since there is misalignment in the thruster and torque is generated by an impulsive shape signal from the torque command, it is difficult to generate target torque; in addition, it also impacts the star image because the impulsive torque creates a sudden change in the angular velocity in the satellite dynamics. This makes the noise of the star image non-Gaussian, which may require introducing a method for dealing with non-Gaussian measurement noise. To meet this goal, in this study, an adaptive extended Kalman filter is implemented to predict measurement vectors with predicted states. The GMM (Gaussian mixture model) is connected in this sequence, giving weighting parameters to each Gaussian density and resulting in the better prediction of measurement vectors. Simulation results show that the GMM-EKF exhibits a better performance than the EKF for attitude estimation, with 30% improvement in performance. Therefore, the GMM-EKF could be a more attractive approach for use with geostationary satellites during station-keeping maneuvers.

Learning Bilateral Clipping Parametric Activation for Low-Bit Neural Networks

Among various network compression methods, network quantization has developed rapidly due to its superior compression performance. However, trivial activation quantization schemes limit the compression performance of network quantization. Most conventional activation quantization methods directly utilize the rectified activation functions to quantize models, yet their unbounded outputs generally yield drastic accuracy degradation. To tackle this problem, we propose a comprehensive activation quantization technique namely Bilateral Clipping Parametric Rectified Linear Unit (BCPReLU) as a generalized version of all rectified activation functions, which limits the quantization range more flexibly during training. Specifically, trainable slopes and thresholds are introduced for both positive and negative inputs to find more flexible quantization scales. We theoretically demonstrate that BCPReLU has approximately the same expressive power as the corresponding unbounded version and establish its convergence in low-bit quantization networks. Extensive experiments on a variety of datasets and network architectures demonstrate the effectiveness of our trainable clipping activation function.

Biofidelic image registration for head and neck region utilizing an in-silico articulated skeleton as a transformation model

Objective. We propose an integration scheme for a biomechanical motion model into a deformable image registration. We demonstrate its accuracy and reproducibility for adaptive radiation therapy in the head and neck region. Approach. The novel registration scheme for the bony structures in the head and neck regions is based on a previously developed articulated kinematic skeleton model. The realized iterative single-bone optimization process directly triggers posture changes of the articulated skeleton, exchanging the transformation model within the deformable image registration process. Accuracy in terms of target registration errors in the bones is evaluated for 18 vector fields of three patients between each planning CT and six fraction CT scans distributed along the treatment course. Main results. The median of target registration error distribution of the landmark pairs is 1.4 ± 0.3 mm. This is sufficient accuracy for adaptive radiation therapy. The registration performs equally well for all three patients and no degradation of the registration accuracy can be observed throughout the treatment. Significance. Deformable image registration, despite its known residual uncertainties, is until now the tool of choice towards online re-planning automation. By introducing a biofidelic motion model into the optimization, we provide a viable way towards an in-build quality assurance.

Numerical geometric acoustics: An eikonal-based approach for modeling sound propagation in 3D environments

We present algorithms for solving high-frequency acoustic scattering problems in complex domains. The eikonal and transport partial differential equations from the WKB/geometric optic approximation of the Helmholtz equation are solved recursively to generate boundary conditions for a tree of eikonal/transport equation pairs, describing the phase and amplitude of a geometric optic wave propagating in a complicated domain, including reflection and diffraction. Edge diffraction is modeled using the uniform theory of diffraction. For simplicity, we limit our attention to domains with piecewise linear boundaries and a constant speed of sound. The domain is discretized into a conforming tetrahedron mesh. For the eikonal equation, we extend the jet marching method to tetrahedron meshes. Hermite interpolation enables second order accuracy for the eikonal and its gradient and first order accuracy for its Hessian, computed using cell averaging. To march the eikonal on an unstructured mesh, we introduce a new method of rejecting unphysical updates by considering Lagrange multipliers and local visibility. To handle accuracy degradation near caustics, we introduce several fast Lagrangian initialization algorithms. We store the dynamic programming plan uncovered by the marcher in order to propagate auxiliary quantities along characteristics. We introduce an approximate origin function which is computed using the dynamic programming plan, and whose 1/2-level set approximates the geometric optic shadow and reflection boundaries. We also use it to propagate geometric spreading factors and unit tangent vector fields needed to compute the amplitude and evaluate the high-frequency edge diffraction coefficient. We conduct numerical tests on a semi-infinite planar wedge to evaluate the accuracy of our method. We also show an example with a more realistic building model with challenging architectural features. Finally, we demonstrate a simple approach to extending the method to handle nonconstant speeds of sound by modifying the semi-Lagrangian updates to account for a varying speed.

Impact of the Covid-19 pandemic on the performance of machine learning algorithms for predicting perioperative mortality

BackgroundMachine-learning models are susceptible to external influences which can result in performance deterioration. The aim of our study was to elucidate the impact of a sudden shift in covariates, like the one caused by the Covid-19 pandemic, on model performance.MethodsAfter ethical approval and registration in Clinical Trials (NCT04092933, initial release 17/09/2019), we developed different models for the prediction of perioperative mortality based on preoperative data: one for the pre-pandemic data period until March 2020, one including data before the pandemic and from the first wave until May 2020, and one that covers the complete period before and during the pandemic until October 2021. We applied XGBoost as well as a Deep Learning neural network (DL). Performance metrics of each model during the different pandemic phases were determined, and XGBoost models were analysed for changes in feature importance.ResultsXGBoost and DL provided similar performance on the pre-pandemic data with respect to area under receiver operating characteristic (AUROC, 0.951 vs. 0.942) and area under precision-recall curve (AUPR, 0.144 vs. 0.187). Validation in patient cohorts of the different pandemic waves showed high fluctuations in performance from both AUROC and AUPR for DL, whereas the XGBoost models seemed more stable. Change in variable frequencies with onset of the pandemic were visible in age, ASA score, and the higher proportion of emergency operations, among others. Age consistently showed the highest information gain. Models based on pre-pandemic data performed worse during the first pandemic wave (AUROC 0.914 for XGBoost and DL) whereas models augmented with data from the first wave lacked performance after the first wave (AUROC 0.907 for XGBoost and 0.747 for DL). The deterioration was also visible in AUPR, which worsened by over 50% in both XGBoost and DL in the first phase after re-training.ConclusionsA sudden shift in data impacts model performance. Re-training the model with updated data may cause degradation in predictive accuracy if the changes are only transient. Too early re-training should therefore be avoided, and close model surveillance is necessary.

RescueSNN: enabling reliable executions on spiking neural network accelerators under permanent faults.

To maximize the performance and energy efficiency of Spiking Neural Network (SNN) processing on resource-constrained embedded systems, specialized hardware accelerators/chips are employed. However, these SNN chips may suffer from permanent faults which can affect the functionality of weight memory and neuron behavior, thereby causing potentially significant accuracy degradation and system malfunctioning. Such permanent faults may come from manufacturing defects during the fabrication process, and/or from device/transistor damages (e.g., due to wear out) during the run-time operation. However, the impact of permanent faults in SNN chips and the respective mitigation techniques have not been thoroughly investigated yet. Toward this, we propose RescueSNN, a novel methodology to mitigate permanent faults in the compute engine of SNN chips without requiring additional retraining, thereby significantly cutting down the design time and retraining costs, while maintaining the throughput and quality. The key ideas of our RescueSNN methodology are (1) analyzing the characteristics of SNN under permanent faults; (2) leveraging this analysis to improve the SNN fault-tolerance through effective fault-aware mapping (FAM); and (3) devising lightweight hardware enhancements to support FAM. Our FAM technique leverages the fault map of SNN compute engine for (i) minimizing weight corruption when mapping weight bits on the faulty memory cells, and (ii) selectively employing faulty neurons that do not cause significant accuracy degradation to maintain accuracy and throughput, while considering the SNN operations and processing dataflow. The experimental results show that our RescueSNN improves accuracy by up to 80% while maintaining the throughput reduction below 25% in high fault rate (e.g., 0.5 of the potential fault locations), as compared to running SNNs on the faulty chip without mitigation. In this manner, the embedded systems that employ RescueSNN-enhanced chips can efficiently ensure reliable executions against permanent faults during their operational lifetime.

GAN-Driven Data Poisoning Attacks and Their Mitigation in Federated Learning Systems

Federated learning (FL) is an emerging machine learning technique where machine learning models are trained in a decentralized manner. The main advantage of this approach is the data privacy it provides because the data are not processed in a centralized device. Moreover, the local client models are aggregated on a server, resulting in a global model that has accumulated knowledge from all the different clients. This approach, however, is vulnerable to attacks because clients can be malicious or malicious actors may interfere within the network. In the first case, these types of attacks may refer to data or model poisoning attacks where the data or model parameters, respectively, may be altered. In this paper, we investigate the data poisoning attacks and, more specifically, the label-flipping case within a federated learning system. For an image classification task, we introduce two variants of data poisoning attacks, namely model degradation and targeted label attacks. These attacks are based on synthetic images generated by a generative adversarial network (GAN). This network is trained jointly by the malicious clients using a concatenated malicious dataset. Due to dataset sample limitations, the architecture and learning procedure of the GAN are adjusted accordingly. Through the experiments, we demonstrate that these types of attacks are effective in achieving their task and managing to fool common federated defenses (stealth). We also propose a mechanism to mitigate these attacks based on clean label training on the server side. In more detail, we see that the model degradation attack causes an accuracy degradation of up to 25%, while common defenses can only alleviate this for a percentage of ∼5%. Similarly, the targeted label attack results in a misclassification of 56% compared to 2.5% when no attack takes place. Moreover, our proposed defense mechanism is able to mitigate these attacks.

An INS/DVL integrated navigation filtering method against complex underwater environment

In this paper, a filtering method against complex underwater environment is proposed for the inertial navigation system/doppler velocity log (INS/DVL) of the unmanned underwater vehicle (UUV). The method has two stages. The first stage is the pre-cruise. In this stage, the martingale distance is constructed. And according to chi-square test, the two thresholds can be obtained, which are used to determine the complexity of the environment. Then, these thresholds are used in cruise of UUV. During cruise, the martingale distance is similarly constructed. The environment is judged in real time according to chi-square test. When the martingale distance exceeds the first threshold, the environment is considered to be of high complexity. The robust method based on covariance matching is used. When the martingale distance lies between the first and second thresholds, the UUV is considered to be in a low complex environment, and the adaptive method is used. Otherwise, the conventional nonlinear filter is performed. Finally, the results of field experiment show that the method can avoid the over compensation of parameters in stable case from conventional methods, and comprehensively solve the problem of degradation of accuracy due to different environment.

FPGA Hardware Acceleration Design for Deep Learning

A type of artificial neural network called a convolutional neural network (CNN) can learn characteristics from a huge amount of data and performs very well in the field of large-scale image processing. CNN simulates the behavior of a biological optic nerve. In recent years, with the development of deep neural network algorithms and hardware technology, the current "CPU+GPU" model servers cannot meet the neural network structure in various fields, so a large amount of deep CNN accelerators based on the FPGA platform have gradually emerged. FPGA is beginning to be used in the fields of image recognition and natural language processing because of its programmability, high performance, high stability, high security, and low power consumption. Though FPGA has proven to have better performance, there is still room for optimization at the design level. Yolov3, as a classical algorithm, still consumes a lot of time and computational resources in actual operations. To address this problem, this experiment partially optimizes the Yolov3 algorithm by introducing the CBAM attention mechanism in the Yolov3 model and pruning the embedded system with different proportions using the Network slimming method. Finally, it is verified on a TX2 embedded device developed by Nvidia using the COCO dataset. The experiment finds that the precision, mAP, and the number of parameters of the optimized Yolov3 algorithm under different optimization strategies. It is shown that the Yolov3 algorithm still has more optimization strategies that can reduce the time required for computation and the memory occupied more effectively without any degradation in accuracy.

FedKL: Tackling Data Heterogeneity in Federated Reinforcement Learning by Penalizing KL Divergence

One of the fundamental issues for Federated Learning (FL) is data heterogeneity, which causes accuracy degradation, slow convergence, and the communication bottleneck issue. Although the impact of data heterogeneity on supervised FL has been widely studied, the related investigation for Federated Reinforcement Learning (FRL) is still in its infancy. In this paper, we first define the type and level of data heterogeneity for FRL systems. By inspecting the connection between the global and local objective functions, we prove that local training can benefit the global objective, if the local update is properly penalized by the total variation (TV) distance between the local and global policies. A necessary condition for the global policy to be learn-able from the local environments is also derived, which is directly related to the heterogeneity level. Based on the theoretical result, a Kullback-Leibler (KL) divergence based penalty is proposed to directly constrain the model outputs in the distribution space and the convergence proof of the proposed algorithm is also provided. By jointly penalizing the divergence of the local policy from the global policy with a global penalty and penalizing each iteration of the local training with a local penalty, the proposed method achieves a better trade-off between training speed (step size) and convergence. Experiment results on two popular Reinforcement Learning (RL) experiment platforms demonstrate the advantage of the proposed algorithm over existing methods in accelerating and stabilizing the training process with heterogeneous data.

A Novel Robust Adaptive Nonsingular Fast Integral Terminal Sliding Mode Controller for Permanent Magnet Linear Synchronous Motors

To realize the high-precision tracking control of permanent magnet linear synchronous motor (PMLSM) in complex working conditions, a control system with strong robustness must be equipped for PMLSM, so this article proposes a novel robust adaptive nonsingular fast integral terminal sliding mode control (ANFITSMC) strategy. First, a dynamical mathematical model of PMLSM is established to describe the thrust and unknown uncertainty in the motion process. Second, considering that the uncertain factors, such as external disturbances and high-frequency noises in the system, may cause the degradation of position tracking accuracy, a nonsingular fast integral terminal sliding mode control (NFITSMC) method that only requires position and velocity information to estimate the lumped uncertainty of the system is designed. Meanwhile, the adaptive control algorithm is used to estimate the lumped uncertainty dynamically in real time, thus weakening the chattering and improving the tracking accuracy. Also, the stability of this control system is proven by utilizing the Lyapunov stability theory. Finally, the high-precision positioning experimental platform is used to carry out experimental tests with different position trajectory commands. The experimental results show that the proposed ANFITSMC method not only enables the system to have higher control accuracy but also ensures a more robust performance.

A Dense ResNet Model with RGB Input Mapping for Cross-Domain Mechanical Fault Diagnosis

In actual engineering applications, the mechanical machine is exposed to uncertain conditions such as noise interference and various loads. The commonly used fault diagnosis models suffer degradation in the prediction accuracy in such complex industrial environments where the available label samples are insufficient and the conditions are varied. To combat this challenge, a cross-domain mechanical fault diagnosis method based on the deep-learning networks is proposed. It utilizes small samples, i.e., 10% of the total, and operates on the time-series signal collected from the mechanical equipment. It provides a classification accuracy of more than 97% on the dataset from Case Western Reserve University (CWRU) under variable conditions and 97.56% with the noise interference of 0 dB. The one-dimensional vibration signal is first converted into an image through RGB mapping. Then, the derived RGB image is capable of the time dependent and spatial properties of the time sequence signal and can be directly used as the input of the deep-learning networks. The deep-learning networks model, i.e., the ResNet, is adopted for the fault feature extraction and additional dense connections are added among the residual blocks to supplement the insufficient labeled samples within the networks. Then, an RGB-DResNet is constructed, capable of retaining the robust features for the classification of the mechanical faults in different working conditions. Finally, through retraining the model by use of transfer learning, the derived RGB-TDResNet model gives a fine adaption to the feature distribution with a small amount of target domain information. The performance of the proposed fault diagnosis model was validated on the dataset from CWRU. The results show that it provides a high identification accuracy and strong robustness in variable operating conditions as well as the noise environment. It is a rather promising approach for dealing with the cross-domain tasks of mechanical fault diagnosis.

Radiation-Tolerant Deep Learning Processor Unit (DPU)-Based Platform Using Xilinx 20-nm Kintex UltraScale FPGA

This paper presents a platform and design approach for enabling radiation-tolerant deep learning acceleration on SRAM-based 20nm Kintex UltraScale™ FPGAs, for terrestrial and high-radiation environments. The presented platform is suitable for deep neural network (DNN) implementations with an emphasis on image classification and includes solutions to mitigate both radiation-induced Single Event Functional Interrupts (SEFIs) and network datapath corruptions. The radiation-tolerant deep learning platform combines Xilinx’s Deep Learning Processing Unit (DPU) IP, Triple Modular Redundancy (TMR) MicroBlaze soft processor IP and Soft Error Mitigation (SEM)-IP to mitigate SEFIs. Furthermore, a technique known as Fault Aware Training (FAT) was applied to effectively mitigate single event effects in the datapath. Test results from a high-energy proton beam (> 60 MeV) experiment using the ResNet-18 Convolutional Neural Network (CNN) for image classification are presented. The Single Event Upset (SEU) rate, system-level SEFI rate and neural network classification/datapath performance are compared between the radiation-tolerant platform and a standard, non-mitigated approach. Results show that datapath classification errors dominate the system response (90%) vs. SEFIs (10%). When compared to standard non-mitigated training techniques, the radiation-tolerant platform using fault aware training methods shows dramatic improvements in overall system response: the overall single event cross-section was reduced by half and 40% reduction in misclassification errors were observed. Also, datapath events with classification accuracy degradation larger than 5% were completely mitigated. The SEFI rate was reduced by 100X with implemented solutions and can be further reduced by optimizing the physical separation between TMR modules.

Experimental Assessment of Multilevel RRAM-Based Vector-Matrix Multiplication Operations for In-Memory Computing

Resistive random access memory (RRAM)-based hardware accelerators are playing an important role in the implementation of in-memory computing (IMC) systems for artificial intelligence applications. The latter heavily rely on vector-matrix multiplication (VMM) operations that can be efficiently boosted by RRAM devices. However, the stochastic nature of the RRAM technology is still challenging real hardware implementations. To study the accuracy degradation of consecutive VMM operations, in this work we programed two RRAM subarrays composed of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$8\times $ </tex-math></inline-formula> 8 one-transistor-one-resistor (1T1R) cells following two different distributions of conductive levels. We analyze their robustness against 1000 identical consecutive VMM operations and monitor the inherent devices’ nonidealities along the test. We finally quantize the accuracy loss of the operations in the digital domain and consider the trade-offs between linearly distributing the resistive states of the RRAM cells and their robustness against nonidealities for future implementation of IMC hardware systems.

Hybrid Precision in Resistive Memory-Based Convolutional Kernel for Fault-Resilient Neuromorphic Systems

As simple convolution computation is intensively and iteratively performed to extract features from input images, cross-point arrays with resistive random access memory (RRAM) serving as a kernel weight can accelerate the relevant mathematical operations in hardware. However, considering actual RRAM characteristics, either variability or unexpected permanent failure from the filamentary switching mechanism is observed, degrading recognition performance. This study investigates the impact of fault in a conventional kernel structure, where two adjacent columns in the array represent a single weight, on feature extraction using MATLAB. First, the fault types of HfO <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$_{\textit{x}}$</tex-math> </inline-formula> -based multilevel RRAM is categorized. The results reveal that the unidirectional fault of RRAM primarily worsens the accuracy of image recognition. This is because the subtraction of negative weights from positive ones is crucial for identifying the edges of images through convolution operations. Therefore, we exploit a kernel structure, in which a single column dedicated to negative weights is located next to a matrix of positive weights. In addition, we reduce the weight precision for negative weights, while quantizing positive weights to higher bits. By mitigating the subtraction errors achieved by the kernel structure with hybrid precision, we improved fault tolerance, minimizing accuracy degradation.