Robust Extreme Learning Machines with Different Loss Functions

Abstract

Extreme learning machine (ELM) has demonstrated great potential in machine learning owing to its simplicity, rapidity and good generalization performance. However, the traditional ELM is sensitive to noise and outliers due to using traditional least square loss function. In this paper, we present a new mixed loss function from a combination of pinball loss and least square loss. Then three robust ELM frameworks are proposed based on rescaled hinge loss function, pinball loss function and mixed loss function respectively to enhance noise robustness. To train the proposed ELM with rescaled hinge loss, the half-quadratic optimization algorithm is used to handle nonconvexity, and we demonstrate the convergence of the resulting algorithm. Furthermore, the proposed methods are applied to various datasets including classification data and regression data, with different types of noises such as feature noise and target noise. Compared with traditional methods, experiment results on UCI benchmark datasets show that the proposed methods are less sensitive to noises and achieve better performance in classification and regression applications.