A generalized pruning algorithm for extreme learning machine

Abstract

This paper presents a generalized pruning extreme learning machine (GP-ELM) algorithm which can generate a compact single-hidden-layer neural network (SLNN) by automatically pruning the number of hidden nodes iteratively while keep high accuracy. The proposed GP-ELM algorithm initializes a SLNN by using extreme learning algorithm (ELM) algorithm given superfluous number of hidden nodes. The following of GP-ELM algorithm consists of two iterative processes. First, the significance of each hidden nodes is estimated and the insignificant nodes are removed. Secondly, the existing hidden nodes are updated by using ELM algorithm. These two processes continue until the stop condition is satisfied such that a reasonably compact network is achieved. Compared against other state-of-the-art algorithms, GP-ELM algorithm has mainly two improvements. First, a supervised training process is integrated into the pruning process such that significance of each hidden node can be estimated more precisely in the next iteration. Secondly, the pruning threshold is set based on the input data dimension such that the threshold can accommodate any data type. Experimental results have shown that the GP-ELM algorithm can automatically achieve a reasonable compact network structure while keep comparable or much higher accuracy in classification and regression.