MODE-CNN: A fast converging multi-objective optimization algorithm for CNN-based models

Abstract

Convolutional neural networks (CNNs) have been used to solve many problems in computer science with a high level of success, and have been applied in many fields in recent years. However, most of the designs of these models are still tuned manually; obtaining the highest performing CNN model is therefore very time-consuming, and is sometimes not achievable. Recently, researchers have started using optimization algorithms for the automatic adjustment of the hyper-parameters of CNNs. In particular, single-objective optimization algorithms have been used to achieve the highest network accuracy for the design of a CNN. When these studies are examined, it can be seen that the most significant problem in the optimization of the parameters of a CNN is that a great deal of time is required for tuning. Hence, optimization algorithms with high convergence rates are needed for the parameter optimization of deep networks. In this study, we first develop an algorithm called MODE-CNN, based on the multi-objective differential evolution (MODE) algorithm for parameter optimization of CNN or CNN-based methods. MODE-CNN is then compared with four different multi-objective optimization algorithms. This comparison is carried out using 16 benchmark functions and four different metrics, with 100 independent runs. It is observed that the algorithm is robust and competitive compared to alternative approaches, in terms of its accuracy and convergence. Secondly, the MODE-CNN algorithm is used in the parameter optimization of a CNN-based method, developed previously by the authors, for the segmentation and classification of medical images. In this method, there are three parameters that influence the test time and accuracy: the general stride (GS), neighbour distance (ND), and patch accuracy (PA). These parameters need to be optimized to give the highest possible accuracy and lowest possible test time. With the MODE-CNN algorithm, the most appropriate GS, ND, and PA values are obtained for the test time and accuracy. As a result, it is observed that the MODE-CNN algorithm is successful, both in comparison with multi-objective algorithms and in the parameter optimization of a CNN-based method.