Regularized Convolutional Neural Network for Highly Effective Parallel Processing

Abstract

Convolutional neural network (CNN) has been adopted in various areas. Using graphics processing unit (GPU), speed improvement can be achieved on CNN, and many studies have proposed such acceleration methods. However, parallelizing the CNN on GPU is not straightforward because there are irregular characteristics in generating output feature maps in typical CNN models. In this paper, we propose a method that maximizes the utilization of GPU by modifying convolution combinations of a well-known CNN network, LeNet-5. Our regularized implementation on a heterogeneous system has achieved an improvement of up to 37.26 times in convolution and sub-sampling layers. Further, an energy consumption reduction of up to 26.40 times is achieved.