An Improved Double Hidden-Layer Variable Length Incremental Extreme Learning Machine Based on Particle Swarm Optimization

Abstract

Extreme learning machine (ELM) has been widely used in diverse domains. With the development of deep learning, integrating ELM with some deep learning method has become a new perspective method for extracting and classifications. However, it may require a large number of hidden nodes and lead to the ill-condition problem for its random generation. In this paper, an effective hybrid approach based on Variable-length Incremental ELM and Particle Swarm Optimization (PSO) algorithm (PSO-VIELM) is proposed which can be used to regulate weights and extract features. In the new method, we build two hidden layers to establish a structure which is compact with a better generalization performance. In the first hidden layer named extraction layer, we make the feature learning to the raw data, and make dynamic updates for hidden layer nodes, and use the fitting error as the fitness function to update the weights corresponding to the hidden nodes with the method of PSO. In the second hidden layer named classification layer, we make a classification for the processed data from extraction layer and use cross-entropy as the fitness function to update the weights in the net. In order to find the appropriate number of hidden layer nodes, all hidden nodes will no longer grow in the case of a rebound in the fitness function on the validation set. The result in some datasets shows that PSO-VIELM has a better generalization performance than other constructive ELMs.