Optimizing area under the ROC curve via extreme learning machines

Abstract

Recently, Extreme learning machine (ELM), an efficient training algorithm for single-hidden-layer feedforward neural networks (SLFN), has gained increasing popularity in machine learning communities. In this paper the ELM based Area Under the ROC Curve (AUC) optimization algorithms are studied so as to further improve the performance of ELM for imbalanced datasets. For binary class problems, a novel ELM algorithm is proposed based on an efficient least square method. For multi-class problems, the following works are done in this paper: First of all, theoretical comparison analysis is proposed for the potential multi-class extensions of AUC; Secondly, a unified objective function for multi-class AUC optimization is proposed following the theoretical analysis; Subsequently, two ELM based multi-class AUC optimization algorithms called ELMMAUC and ELMmacroAUC respectively are proposed followed with complexity analyses; Finally, the generalization analysis is established for ELMMAUC in search of theoretical supports. Empirical study on a variety of real-world datasets show the effectiveness of our proposed algorithms.