Pruning method in the architecture of extreme learning machines based on partial least squares regression

Abstract

The extreme learning machines-ELM are useful models for performing classification and regression of data, as well as being an alternative methodology to techniques that use back-propagation to determine values of parameters used in hidden layers of the learning model. A problem that ELM may face when performing data mining techniques is to become a very generalist model with no accurate or time-consuming processing due to a high number of neurons in the hidden layers of its architecture, if a model very restricted the characteristics of the sample that was used for the training, losing its generalization capacity, As the number of neurons in the hidden layer increases, information unnecessary to the model can be included in the operations performed by the model, impairing the accuracy end of the ELM when sorting or performing regression data. To solve this problem, a pruning method based on partial least squares regression was proposed to act together with the neurons of the hidden layer of an ELM. By obtaining a ranking vector with regression values and between variables in the hidden layer of ELM, the selection of the neurons that have the best predictive capacity in the responses between the data is used. Data classification tests were performed using bases commonly used in work related to machine learning and compared to other classifier models. Lastly, it is verified that an ELM with fewer neurons in the hidden layer, where the selected neurons are the ones that contribute better to the classification of the model, improve the final accuracy of the model in comparison to an architecture with a much larger number of neurons and is statistically similar to other pruning methods of the neurons in the inner layer of the ELM, allowing gain of model performance without damaging the accuracy in the final results, without losing its ability to perform classification tasks correctly.