Multi-Layer Perceptron Training Optimization Using Nature Inspired Computing

Abstract

Although the multi-layer perceptron (MLP) neural networks provide a lot of flexibility and have proven useful and reliable in a wide range of classification and regression problems, they still have limitations. One of the most common is associated with the optimization algorithm used to train them. The most commonly used training method is stochastic gradient descent with backpropagation (or backpropagation for short) because it is mathematically tractable (given that the activation functions are differentiable). However, backpropagation is not guaranteed to find the globally optimal set of weights and biases. As a result, the MLP is often incapable of obtaining a desirable solution to the problem. Clonal selection algorithms (CSA) are optimization procedures that effectively explore a complex and large space to find values near the global optimum. Consequently, CSA can be used to solve the problem of training MLP networks. This paper presents a novel implementation of CSA for training MLP architectures to solve real-world problems such as breast cancer diagnosis, active sonar target classification, wheat classification, and flower classification. The CSA is used to find the optimal weights and biases that will significantly increase the classification accuracy of the MLP. The performance of our proposed approach is compared with other popular training methods: genetic algorithm (GA), ant colony optimization (ACO), particle swarm optimization (PSO), Harris hawks optimization (HHO), moth-flame optimization (MFO), flower pollination algorithm (FPA), and backpropagation (BP). The comparison is benchmarked using five classification datasets: Iris Flower, Sonar, Wheat Seeds, Breast Cancer Wisconsin, and Haberman’s Survival. Comparative study results illustrate the improvements in MLP performance gained by using CSA over other training methods, and hence it can be considered a competitive approach to training MLP networks when solving real-world applications in various disciplines.