Performance Analysis of Deep Learning Libraries: TensorFlow and PyTorch

Felipe Florencio,Edward David Moreno,Thiago Valen�,Methanias Cola�O Junior

doi:10.3844/jcssp.2019.785.799

Abstract

Through the increase in deep learning study and use, in the last years there was a development of specific libraries for Deep Neural Network (DNN). Each one of these libraries has different performance results and applies different techniques to optimize the implementation of algorithms. Therefore, even though implementing the same algorithm and using different libraries, the performance of their executions may have a considerable variation. For this reason, developers and scientists that work with deep learning need scientific experimental studies that examine the performance of those libraries. Therefore, this paper has the aim of evaluating and comparing these two libraries: TensorFlow and PyTorch. We have used three parameters: Hardware utilization, hardware temperature and execution time in a context of heterogeneous platforms with CPU and GPU. We used the MNIST database for training and testing the LeNet Convolutional Neural Network (CNN). We performed a scientific experiment following the Goal Question Metrics (GQM) methodology, data were validated through statistical tests. After data analysis, we show that PyTorch library presented a better performance, even though the TensorFlow library presented a greater GPU utilization rate.

Highlights

Deep learning is an artificial intelligence field that allows computers to learn with experience and understand the world in terms of a concept hierarchy, with each concept defined by its relationship with simpler concepts
The analysis of the related works demonstrated the absence of performance analysis of PyTorch library, in the analysis process was verified that the selected experiments do not take in consideration the utilization rate of hardware components and that, in general, the authors do not use statistical tests for validating the data extracted from the experiment
Deep Neural Network tools are a recurrent subject in the academia and industry with several researches related to the creation of new libraries, library performance evaluation, execution optimization techniques and the creation of dedicated devices

Summary

Introduction

Deep learning is an artificial intelligence field that allows computers to learn with experience and understand the world in terms of a concept hierarchy, with each concept defined by its relationship with simpler concepts. If we draw a graph showing how these concepts are built on top of each other, the graph is deep, with many layers. For this reason, it is called deep learning (Goodfellow et al, 2016). The deep learning idea is to train an Artificial Neural Network (ANN) of multiple layers in a set of data in order to allow it to deal with real world tasks. There are many deep learning libraries, such as TensorFlow, Theano, CNTK, Caffe, Torch, Neon and PyTorch. Each one of these libraries has different features of performance and applies different techniques to optimize the implementation of algorithms. Even though implementing the same algorithm in different structures, the performance of these different implementations may have a considerable variation (Bahrampour et al, 2015; Shatnawi et al, 2018)

Objectives

Methods

Results

Conclusion