Investigation of training performance of convolutional neural networks evolved by genetic algorithms using an activity function

Abstract

This article presents a study on the training performance of convolutional neural networks (CNN) evolved by genetic algorithms (GA) using an activity function for image recognition. Globally, GA has been considered as one of the most robust search optimization methods in machine learning and artificial intelligent systems. Currently, when CNN is used in 2D image recognition, the ReLU activity function is mostly applied with back propagation (BP) for signal processing and image recognition, because the sigmoid function has a gradient disappearance problem. Although the sigmoid function is good for three-layered neural networks, its performance degrades for multilayer neural networks, especially in BP training. In this study, we also focus on the training performance of an activity function with CNN evolved by GA, especially when the intermediate convolution layers are used. We also evaluate the training accuracy of various activity functions for image recognition with CNN for an automatic driving application using the GA training method.