Comparing Imperialist Competitive Algorithm With Backpropagation Algorithms For Training Feedforward Neural Network

Abstract

Artificial Neural Networks (ANN) and evolutionary algorithms are two relatively young research areas that were subject to a steadily growing interest during the past years. The use of ANN has been proved to be a cost-effective technique. It is very important to choose a suitable algorithm for training a neural network. Mostly Back Propagation (BP) algorithm is a gradient descent algorithm (a first-order optimization algorithm) on the error space, which most likely gets trapped into a local minimum and has very slow convergence. This shortcoming can be removed by global searching ability of the evolutionary algorithms such as Imperialist Competitive Algorithm (ICA) which is a new evolutionary algorithm based on the human's socio-political evolution. This investigation provides a comparison between training a neural network with BP algorithms used for training Feed-forward Neural Networks (FNN) and ICA. Among the BP algorithms, Gradient descent, Levenberg–Marquardt, Conjugate gradient descent, Resilient, BFGS Quasi-newton, and One-step secant algorithm are tested then the obtained results will be compared with the results of training the neural network with ICA. Also, Accuracy and Mean Squared Error (MSE) are the main measures selected to assess both models. Also the MSE was used as a criterion to specify optimum number of neurons in the hidden layer. The results showed that ICA approach outperforms the BP for training neural network models.