A Comparison of Quantized Convolutional and LSTM Recurrent Neural Network Models Using MNIST

Abstract

In this paper, a software comparative analysis of two neural network models is presented, namely, Convolutional Neural Network (CNN), and Long Short Term Memory (LSTM) neural network. The evaluation is performed using the famous deep learning database the “MNIST” to check the accuracy, model size, speed and complexity of the two models for future digital realization on reconfigurable hardware. In addition to that, we optimize the size of the two models by quantizing the weights width to 8-bits instead of 32-bits. The results show an extensive reduction in the size of each model (by 10X) with a slight drop in the accuracy. The results also show that the CNN is more accurate and much faster than LSTMs making it the best model to be implemented on reconfigurable hardware.