Nonlinear Weight Quantification for Mitigating Stress Induced Disturb Effect on Multilevel RRAM-Based Neural Network Accelerator

Lindong Wu,Qingyu Chen,Yabo Qin,Yimao Cai,Ru Huang,Zhizhen Yu

doi:10.1109/jeds.2021.3110877

Abstract

The RRAM-based array is one of the most promising core functional primitives to accelerate the inference process of neural networks. However, the stress-induced disturbance can cause a significant accuracy drop during inference process where input vectors with different voltage levels are fed to the device. This kind of disturb can hardly be avoided by optimizing the fabrication process. Here, we investigate this phenomenon based on TaOx-based devices with different electrodes. The results indicate that the stress-induced disturb mainly appears in the intermediate resistance states when the voltage is applied on the device. The simulation result by COMSOL reveals the relationship between read disturb and electric field. Therefore, we propose a nonlinear weight quantification method to mitigate read disturb effect on inference accuracy by reducing the number of devices in intermediate resistance states. The simulation results based on the fully-connected networks for MNIST recognition indicate that stress disturb phenomenon can be well suppressed by nonlinear weight quantification compared with the conventional linear quantification method, which will advance the application of the RRAM-based accelerator.

Highlights

NOWADAYS, resistive random access memory (RRAM) is considered as one of the most promising candidates for the construction of neuromorphic computing system to break the bottleneck of Von-Neumann architecture [1]-[8]
We extend our previous work presented in [21], where we investigate the read disturb effect on resistance drift of multilevel RRAM with a structure of TiN/TaOx/Pt and proposed a nonlinear weight quantification (NWQ) method to reduce the number of devices in intermediate-resistance states (IRS) to mitigate the read disturb effect on inference accuracy of deep neural networks (DNN)
These results demonstrate that NWQ-based network possesses better resilience than linear weight quantification (LWQ)-based network and NWQ method can effectively mitigate the read disturb effect on multilevel RRAM-based neural network accelerator, which will advance the application of RRAM-based accelerator

Summary

Introduction

NOWADAYS, resistive random access memory (RRAM) is considered as one of the most promising candidates for the construction of neuromorphic computing system to break the bottleneck of Von-Neumann architecture [1]-[8]. The TaOx-based RRAM exhibits great electrical characteristic, including excellent endurance, high uniformity, and good linearity, and has drawn lots of research attention [9]-[12]. The multilevel RRAM-based array has been adopted to accelerate inference process of deep neural networks (DNN) and behaves well in various tasks, such as image classification and object detection [13]-[15]. It is inevitable that read disturb phenomenon reflected in the resistance drift will happen during inference process when a read voltage is applied on the device [16]-[18].

Methods

Results

Conclusion