Automatic selection of hidden neurons and weights in neural networks for data classification using hybrid particle swarm optimization, multi-verse optimization based on Lévy flight

Abstract

Within neural networks, it is considered a difficult task to find optimal values for the number of hidden neurons and connection weights simultaneously. This is because altering the hidden neurons significantly affects a neural network’s structure and increases the difficulty of the training process which needs special considerations. Particle swarm optimization (PSO) is one of the most important metaheuristic algorithms due to its convergence speed and its simplicity of implementation. Multi-verse optimization (MVO) based on Levy flight is a recent and fast algorithm and can avoid premature convergence and can achieve a better balance between exploration and exploitation. This paper presents a new training method based on hybrid particle swarm optimization with Multi-verse optimization based on Levy flight (PLMVO) to optimize the number of hidden neurons and connection weights simultaneously in feedforward neural networks (FFNN). The hybrid algorithm is utilized to search better in solution space which proves its efficiency in reducing the problems of trapping in local minima. To evaluate the proposed algorithm we used three experimental series. In the first one, the proposed PLMVO algorithm is compared with the MVO and PSO algorithms to solve a set of 15 benchmark functions to find the global solution. Meanwhile, in the second experiment, the performance of the proposed approach was compared with five evolutionary techniques and the standard momentum backpropagation and adaptive learning rate. The comparison was benchmarked and evaluated using nine bio-medical datasets. The results of the comparative study show that PLMVO outperformed other training methods in most datasets and can be an alternative to other training methods. In the third experiment, the proposed PLMVO-MLP is used to predict malicious executable Linux files. The implemented model achieved very promising results with very high accuracy of 1.0, and an average f-measure of 1.0.