Pruning of Deep Neural Networks for Fault-Tolerant Memristor-based Accelerators

Abstract

Hardware-level reliability is a major concern when deep neural network (DNN) models are mapped to neuromorphic accelerators such as memristor-based crossbars. Manufacturing defects and variations lead to hardware faults in the crossbar. Although memristor-based DNNs are inherently tolerant to these faults and many faults are benign for a given inferencing application, there is still a non-negligible number of critical faults (CFs) in the memristor crossbars that can lead to misclassification. It is therefore important to efficiently identify these CFs so that fault-tolerance solutions can focus on them. In this paper, we present an efficient technique based on machine learning to identify these CFs; CFs can be identified with over 98% accuracy and at a rate that is 20 times faster than a baseline using random fault injection. We next present a fault-tolerance technique that iteratively prunes a DNN by targeting weights that are mapped to CFs in the memristor crossbars. Our results for the CIFAR-10 data set and several benchmark DNNs show that the proposed pruning technique eliminates up to 95% of the CFs with less than 1% DNN inferencing accuracy loss. This reduction in the total number of CFs leads to a 99% savings in the hardware redundancy required for fault tolerance.