Metaheuristic-based extreme learning machines: a review of design formulations and applications

Abstract

Extreme learning machine (ELM) is a novel and recent machine learning algorithm which was first proposed by Huang et al. (Proceedings of 2004 IEEE international joint conference on, pp 985–990, 2004). Over the last decade, ELM has gained a remarkable research interest with tremendous audiences from different domains in a short period of time due to its impressive characteristics over other single hidden-layer feedforward neural networks. Although ELM enjoys powerful advantages, it still has some potential weaknesses like performance sensitivity to the initial condition of the input weights, number of hidden neurons, and the selection of activation functions. In order to overcome the limitations of classical ELM, many metaheuristic algorithms including the evolutionary algorithms, swarm intelligence, memetic and trajectory algorithms have been proposed for optimizing the different components of ELM by researchers aiming to improve the generalization performance of ELM networks for different types of complex problems and applications. Therefore our review paper intent to conduct a deep study of the important aspects of applying metaheuristic algorithms for optimizing ELM networks. Three main streams of research lines are identified: the optimization of input weights and hidden biases, selection of hidden neurons, and optimization of activation functions. Furthermore, this paper will discuss a wide spectrum of applications of metaheuristic-based ELM models. We will highlight the strengths of these models and the improvements that are suggested in the literature to overcome their weaknesses. We touch upon several interesting and challenging open issues in optimizing ELM using metaheuristics.