A hierarchical stacking extreme learning machine for multi-classification

Abstract

Extreme learning machine (ELM) is an emerging single hidden layer feedforward neural network learning, whose hidden node parameters are randomly generated, and the output weights are computed by linear regression algorithms. This paper proposes a hierarchical stacking framework for ELM (HS-ELM), which is characterized by encoding the input features layer by layer with ELM, and the encoded features are passed forwards to realize the stacking of features. The network is divided into two parts, a stacking autoencoder and a final classifier. Stacking autoencoder allows inheritance of the original features without attenuating as the number of hidden layers increases. In the stacking autoencoder, a sparse training method based on LI norm constraints is used to make the network more compact. In addition, the bias and activation functions of the stacking autoencoder are appropriately streamlined to further reduce the training time of the network. The extracted features are finally classified using an ELM parallel structure with input to output layer connection. Experiments on three widely used classification datasets show that the proposed learning framework has a universal learning ability and obtains similar or higher accuracy than the existing state-of-the-art hierarchical ELM method.