Deep Learning Frameworks for Convolutional Neural Networks—A Benchmark Test

Abstract

Deep neural networks are state of the art for many machine-learning problems. The architecture of deep neural networks is inspired by the hierarchical structure of the brain. Deep neural networks feature a hierarchical, layer-wise arrangement of nonlinear activation functions (neurons) fed by inputs scaled by linear weights (synapses). Deep Learning frameworks simplify model development and training by providing high-level primitives for complex and error-prone mathematical transformations, like gradient descent, back-propagation, and inference. The main goal of this study is to compare the performance of two trending frameworks, Caffe and TensorFlow for Deep Machine Learning. As benchmark for the comparison with other approaches in deep learning was a well-analyzed database for handwritten cipher classification chosen. These two frameworks (Caffe and TensorFlow) were chosen out of nine different frameworks, after checking which ones better fits the proposed selection criteria. Caffe and TensorFlow frameworks were selected after nine different frameworks were analyzed using the following selection criteria: programming language, functionality level, algorithms and network architecture, pretrained models and community activity. As the performance is heavily affected by the number of parameters in the networks, four well-known convolutional neural network (CNN) architectures (LeNet, AlexNet, VGG19 and GoogLeNet) were trained and tested using both frameworks. Using architectures with different deepness will allow investigating how the number of hidden layers improves the accuracy of the CNN. As a CNN requires high computational effort, two computers equipped with different NVIDIA GPU were used to ease the effort and they were used to investigate how the hardware improves the performance of the CNN and if it is worthy to invest on it. As the CNN are widely used for image classification, it was defined that the used architecture was going to be used for classification of handwritten numbers, because this example of classification is very well analyzed and can serve as a benchmark for comparison. Due to this and considering that the training of these networks requires a huge amount of data, MNIST database was set for training and testing them. All the architectures were adapted to the3 MNIST database and they were developed for Caffe and TensorFlow frameworks and analyzed on the named architectures. The biggest differences on our hardware we have got in training the VVG19 and the GoogLeNet architectures in TensorFlow and Caffe.