Proximal Stochastic AUC Maximization

Abstract

This work considers a stochastic optimization problem for maximizing the AUC (area under the ROC curve). The AUC metric has proven to be a reliable performance measure for evaluating a model learned on imbalanced data. The batch pairwise learning methods (e.g., rankSVM) can achieve a quadratic convergence to the optimal solution. However, the batch learning paradigm hinders the scalability of these methods. Recently different online and stochastic AUC maximization algorithms are developed. While these can scale well for large-scale data, they either cannot generalize as good as the batch AUC methods or suffer from slow convergence, which minimizes their scalability. A recent stochastic pairwise learning algorithm for AUC maximization suggests to schedule both the regularization and the averaging steps to improve the generalization capability and the convergence speed. Building on this algorithm, we develop a simple proximal stochastic AUC maximization algorithm. The proposed algorithm uses a proximal operator of the pairwise hinge loss function, which encourages small update steps. Averaging these adjacent weights has a significant improvement on the converges rate of the final model. Experiments on several benchmark data sets show that the proposed algorithm can achieve AUC classification accuracy on par with that of the batch method while being considerably efficient. The proposed algorithm also outperforms state-of-the-art online and stochastic algorithms in terms of generalization performance and convergence rate.