Shallow and deep neural network training by water wave optimization

Abstract

It is well known that the performance of artificial neural networks (ANNs) is significantly affected by their structure design and parameter selection, for which traditional training methods have drawbacks such as long training times, over-fitting, and premature convergence. Evolutionary algorithms (EAs) have provided an effective tool for ANN parameter optimization. However, simultaneously optimizing ANN structures and parameters remains a difficult problem. In this study, we adapt water wave optimization (WWO), a relatively new EA, for optimizing both the parameters and structures of ANNs, including classical shallow ANNs and deep neural networks (DNNs). We use a variable-dimensional solution encoding to represent both the structure and parameters of an ANN, and adapt WWO propagation, refraction, and breaking operators to efficiently evolve variable-dimensional solutions to solve the complex network optimization problems. Computational experiments on a variety of benchmark datasets show that the WWO algorithm achieves a very competitive performance compared to other popular gradient-based algorithms and EAs.