Lightweight memristive gated recurrent unit networks

Abstract

Memristor arrays have been widely used to accelerate neural network algorithms for edge intelligence. Although previous study has revealed the potential of memristive Long Short-Term Memory (LSTM) neural network for sequential information processing, it is difficult to map its large weights to existing memristor arrays due to the limitation of the fabrication. Here we demonstrate several gate-variants that further simplify the original Gated Recurrent Unit (GRU) neural network, a simplified version of LSTM, by removing its specific weight matrix, which is used to lower the requirement of weights mapping in memristor arrays. We then performed software validation and offline inference device validation of these new networks using a variety of different datasets based on Mixed National Institute of Standards and Technology (MNIST) dataset segmentation, testing their effectiveness for classifying sequential datasets of different sizes and their robustness for device imperfection. Ultimately, we illustrate that these gate-variants of GRU networks can maintain the performance of the original network in different segmented MNIST datasets, while reducing the memristor array size for more than 50% on most datasets. This work will further advance the edge application of memristive Recurrent Neural Networks (RNN) with limited resources.