Optimizing Convolutional Neural Network Hyperparameters by Enhanced Swarm Intelligence Metaheuristics

Nebojsa Bacanin,Milan Tuba,Timea Bezdan,Eva Tuba,Ivana Strumberger

doi:10.3390/a13030067

Abstract

Computer vision is one of the most frontier technologies in computer science. It is used to build artificial systems to extract valuable information from images and has a broad range of applications in various areas such as agriculture, business, and healthcare. Convolutional neural networks represent the key algorithms in computer vision, and in recent years, they have attained notable advances in many real-world problems. The accuracy of the network for a particular task profoundly relies on the hyperparameters’ configuration. Obtaining the right set of hyperparameters is a time-consuming process and requires expertise. To approach this concern, we propose an automatic method for hyperparameters’ optimization and structure design by implementing enhanced metaheuristic algorithms. The aim of this paper is twofold. First, we propose enhanced versions of the tree growth and firefly algorithms that improve the original implementations. Second, we adopt the proposed enhanced algorithms for hyperparameters’ optimization. First, the modified metaheuristics are evaluated on standard unconstrained benchmark functions and compared to the original algorithms. Afterward, the improved algorithms are employed for the network design. The experiments are carried out on the famous image classification benchmark dataset, the MNIST dataset, and comparative analysis with other outstanding approaches that were tested on the same problem is conducted. The experimental results show that both proposed improved methods establish higher performance than the other existing techniques in terms of classification accuracy and the use of computational resources.

Highlights

Convolutional neural networks (ConvNets or CNN) are biologically-inspired structures and represent a special type of neural network
When a modified method is created, its effectiveness should first be validated against a wider set of benchmarks, and the performance should be compared with the original method
We proposed two enhanced versions of tree growth algorithm (TGA) and firefly algorithm (FA) swarm intelligence metaheuristics for optimizing the CNN’s hyperparameters

Summary

Introduction

Convolutional neural networks (ConvNets or CNN) are biologically-inspired structures and represent a special type of neural network. CNN is most commonly used to process 2D signals for digital image processing, but it can be used to process 1D signals. ConvNets are very successful in different visual tasks, such as classifying images, detecting objects in pictures [1], segmenting images [2], generating image descriptions [3], recognizing faces [4], pose estimation [5], and others. It was successfully applied to digit recognition. AlexNet [7] was the first architecture that achieved significant results, popularized CNN, and brought a revolution in computer vision. AlexNet has a very similar architecture to LeNet, but it is much deeper. Some of the other well-known architectures are VGG [8], and more modern networks are GoogleNet [9], ResNet [10], DenseNet [11], and SENet [12]

Objectives

Results

Conclusion