A Multi-objective Evolutionary Approach for Efficient Kernel Size and Shape for CNN

Abstract

In order to improve the classification accuracy, state-of-the-art CNNs usually involve large and complex convolutional layers. However, for certain applications, e.g. Internet of Things (IoT), where such CNNs are to be implemented on resource-constrained platforms, the CNN architectures have to be small and efficient. To deal with this problem, reducing the resource consumption in convolutional layers has become one of the most significant efforts when designing CNNs. In this work, a multi-objective optimisation approach is proposed to trade-off between the amount of computation and network accuracy by using Multi-Objective Evolutionary Algorithms (MOEAs). The size and number of convolution kernels used are proportional to computational resource consumption of CNNs. Therefore, this paper considers reducing the computational resource consumption by reducing the size of kernels in convolutional layers. In addition to kernel size unconventional kernel shapes are investigated and results show these clearly improve performance over using traditionally square convolution kernels alone. The main contributions of this paper are therefore a methodology to significantly reduce computational cost of CNNs, based on unconventional kernel shapes, and provide different trade-offs for specific use cases. The experimental results further demonstrate that the proposed method achieves large improvements in resource consumption with no significant reduction in network performance. Compared with the benchmark CNN, the best trade-off architecture shows a reduction in multiplications of up to 4.06X and with slight increase in classification accuracy on CIFAR-10 dataset.