Kernel Risk-Sensitive Loss based Hyper-graph Regularized Robust Extreme Learning Machine and Its Semi-supervised Extension for Classification

Abstract

Kernel Risk-Sensitive Loss (KRSL) is a nonlinear similarity measure defined in kernel space, which enable the gradient based method to achieve higher accuracy while effectively weakening the negative effects caused by noise and outliers. Defined as a kernel function expectation between two random variables, KRSL has been successfully applied in robust machine learning and signal processing. Extreme Learning Machine, as one of the most popular methods of machine learning, has attracted great attention in supervised learning and semi-supervised learning. However, when the data contains noise and outliers, the manifold structure of the data is not considered or the neural network structure is too complex, the performance of traditional ELM methods will decline. Therefore, based on KRSL, hyper-graph regularization and L2,1-norm, we first propose a more robust ELM method named Kernel Risk-Sensitive Loss Based Hyper-graph Regularized Robust Extreme Learning Machine (KRSL-HRELM). In KRSL-HRELM, KRSL is introduced into ELM to enhance its ability to handle noise and outliers. Moreover, the hyper-graph regularization is integrated into the method to learn the higher-order geometric structure information between the data. In addition, the L2,1-norm is introduced to constrain the output weight matrix to obtain a sparse network model. Inspired by other semi-supervised ELM methods, we extend KRSL-HRELM to semi-supervised learning and propose its semi-supervised version semi-supervised KRSL-HRELM (SS- KRSL-HRELM). Empirical studies on a large number of real-world datasets show that the proposed methods are competitive with other advanced supervised or semi-supervised learning methods in terms of robustness and efficiency.