Reducing the Impact of Phase-Change Memory Conductance Drift on the Inference of large-scale Hardware Neural Networks

Abstract

We extensively analyze the impact of conductance "drift" - inherent to Phase Change Memory (PCM) devices - on the inference of pre-trained Deep Neural Networks (DNNs). Starting from large-array experimental characterization of PCM drift in partial-SET states, we build a statistical model of conductance drift capturing the cycle-to-cycle variability in drift-coefficient ν observed empirically. Using a variety of small and large DNNs (few-layer perceptrons using MNIST dataset, ResNet-(10,18,34) using CIFAR-10, and a 2-layer LSTM using "Alice in Wonderland"), we characterize how the combination of drift and ν-variability can induce long-term degradations in DNN accuracy. We then introduce several techniques to suppress these effects, including "slope correction" and modifications to the DNN architecture (choice of squashing function, the number of hidden units, hidden layers, or convolution filters). Finally, we show how noise sources such as 1/f and Random Telegraph Noise (RTN), and device-to-device variability in heater diameter, can complicate the accurate measurement of PCM drift.