Pruning Extreme Learning Machines Using the Successive Projections Algorithm

Abstract

Extreme Learning Machine (ELM) is a recently proposed machine learning method with successful applications in many domains. The key strengths of ELM are its simple formulation and the reduced number of hyper-parameters. Among these hyper-parameters, the number of hidden nodes has significant impact on ELM performance since too few/many hidden nodes may lead to underfitting/overfitting. In this work, we propose a pruning strategy for ELM using the Successive Projections Algorithm (SPA) as an approach to automatically find the number of hidden nodes. SPA was originally proposed for variable selection. In this work, it was adapted in order to be used to prune ELMs. The proposed method was compared to the Optimally Pruned Extreme Learning Machine algorithm (OP-ELM), which is considered as a state of the art method. Real world datasets were used to assess the performance of the proposed method for regression and classification problems. The application of the proposed model resulted in much simpler models with similar performance compared to the OP-ELM. For some classification instances, the performance of the proposed method outperformed the OP-ELM method.