A Fine-grained Optimization to Winograd Convolution Based on Micro-architectural Features of CPU

Abstract

Deep Learning has become an infrastructure for a variety of AI applications. As a basic component of deep neural networks (DNNs), the convolution layer is compute-intensive and imposes a critical impact on the overall performance of deep learning. Accordingly, a large number of works pursue to accelerate the convolution layer by GPUs. We argue that the CPUs are able to supply comparable computing power with that of GPUs, and should be explored to expedite the deep learning as well. In this work, we focus on the widely-adopted Winograd convolution and propose a novel implementation based on the high performance CPUs equipped with AVX instructions.The Winograd convolution is well-known to achieve high performance by reducing the costly multiplication operations. However, our experiments demonstrate that the obtained performance is far from the theoretical analysis. In this paper, we analyze the state-of-the-art implementation of Winograd convolution on CPUs, investigating its three performance limitations including register block, data layout and cache block, and accordingly propose three measures, i.e., register block fixing, data layout adjustment, cache block searching, to improve the Winograd convolution in Intel DNNL library. The experiments show that our design is able to improve the original implementation by as many as 1.39×.