An enhanced artificial neural network with a shuffled complex evolutionary global optimization with principal component analysis

Tiantian Yang,Ata Akabri Asanjan,Mohammad Faridzad,Negin Hayatbini,Xiaogang Gao,Soroosh Sorooshian

doi:10.1016/j.ins.2017.08.003

Tiantian Yang, Ata Akabri Asanjan + Show 4 more

Open Access

https://doi.org/10.1016/j.ins.2017.08.003

Copy DOI

Journal: Information sciences	Publication Date: Aug 4, 2017
Citations: 87	License type: publisher-specific-oa

Affiliation: Center for Hydrometeorology and Remote Sensing

Abstract

The classical Back-Propagation (BP) scheme with gradient-based optimization in training Artificial Neural Networks (ANNs) suffers from many drawbacks, such as the premature convergence, and the tendency of being trapped in local optimums. Therefore, as an alternative for the BP and gradient-based optimization schemes, various Evolutionary Algorithms (EAs), i.e., Particle Swarm Optimization (PSO), Genetic Algorithm (GA), Simulated Annealing (SA), and Differential Evolution (DE), have gained popularity in the field of ANN weight training. This study applied a new efficient and effective Shuffled Complex Evolutionary Global Optimization Algorithm with Principal Component Analysis – University of California Irvine (SP-UCI) to the weight training process of a three-layer feed-forward ANN. A large-scale numerical comparison is conducted among the SP-UCI-, PSO-, GA-, SA-, and DE-based ANNs on 17 benchmark, complex, and real-world datasets. Results show that SP-UCI-based ANN outperforms other EA-based ANNs in the context of convergence and generalization. Results suggest that the SP-UCI algorithm possesses good potential in support of the weight training of ANN in real-word problems. In addition, the suitability of different kinds of EAs on training ANN is discussed. The large-scale comparison experiments conducted in this paper are fundamental references for selecting proper ANN weight training algorithms in practice.

Highlights

The Artificial Neural Network (ANN) is a powerful, nonlinear, and adaptive mathematical predictive model that was inspired by the neurological structure of the human brain
Yang et al / Information Sciences 418–419 (2017) 302–316 the output-layer errors were purposely propagated into hidden-layers, and the optimal weights in the complete ANN were derived with gradient descent optimization
Rumelhart et al [35] demonstrated that the BP scheme worked far faster than earlier approaches for training ANNs, and made it possible to use neural networks to solve problems that had been unsolvable in many fields

Summary

Introduction

The Artificial Neural Network (ANN) is a powerful, nonlinear, and adaptive mathematical predictive model that was inspired by the neurological structure of the human brain. In Rumelhart et al [35], several neural networks were tested with the BP scheme, in which. T. Yang et al / Information Sciences 418–419 (2017) 302–316 the output-layer errors were purposely propagated into hidden-layers, and the optimal weights in the complete ANN were derived with gradient descent optimization. Rumelhart et al [35] demonstrated that the BP scheme worked far faster than earlier approaches for training ANNs, and made it possible to use neural networks to solve problems that had been unsolvable in many fields. The use of gradient-based optimizations is skeptical [40,44]. BP and gradient-based optimization schemes are extremely sensitive to initial conditions [18] and the prediction accuracy will dramatically decrease as the number of hidden neurons increases when using BP and gradient-based optimization schemes [14]

Methods

Results

Discussion

Conclusion