An improved extreme learning algorithm based on truncated singular value decomposition

Abstract

With respect to the ill-posed problem when calculating output weights of the ELM (Extreme Learning Machine), an improved ELM algorithm based on TSVD (Truncated Singular Value Decomposition) is proposed in this paper. The degree of ill-condition is severe if the hidden layer output matrix has a large condition number. In such case, the output weights computed by general SVD (Singular Value Decomposition) method will be large and unevenly distributed, which would result in a worsened stability and anti-interference ability. Also, the over-fitting phenomenon presented easily. TSVD is an effective regularization method. It can eliminate the influence caused by small singular values and enhance the generalization ability of the model. As for selecting truncation parameter, it is determined by minimizing the GCV (Generalized Cross-Validation) function with the relationship between TSVD and Tikhnovo Regularization. Simulation results illustrate that TSVD-ELM performs higher prediction accuracy than original ELM on data with noise and increases the model's robustness. Finally, the proposed method is used to build a soft-sensor model to predict the quality of iron ore pellet and gets an acceptable error rate.