Robust pairwise learning with Huber loss

Abstract

Pairwise learning naturally arises from machine learning tasks such as AUC maximization, ranking, and metric learning. In this paper we propose a new pairwise learning algorithm based on the additive noise regression model, which adopts the pairwise Huber loss and applies effectively even to the situation where the noise only satisfies a weak moment condition. Owing to the robustness of Huber loss function, this new method is resistant to heavy-tailed errors or outliers in the response variable. We establish a comparison theorem to characterize the gap between the excess generalization error and the prediction error. We derive the error bounds and convergence rates under appropriate conditions. It is worth mentioning that all the results are established under the (1+ϵ)-th moment condition of the noise variable. It is rather weak particularly in the case of ϵ<1, which means the noise variable does not even admit a finite variance.