Cost-sensitive support vector machines

Abstract

Many machine learning applications involve imbalance class prior probabilities, multi-class classification with many classes (often addressed by one-versus-rest strategy), or “cost-sensitive” classification. In such domains, each class (or in some cases, each sample) requires special treatment. In this paper, we use a constructive procedure to extend SVM’s standard loss function to optimize the classifier with respect to class imbalance or class costs. By drawing connections between risk minimization and probability elicitation, we show that the resulting classifier guarantees Bayes consistency. We further analyze the primal and the dual objective functions and derive the objective function in a regularized risk minimization framework. Finally, we extend the classifier to the with cost-sensitive learning with example dependent costs. We perform experimental analysis on class imbalance, cost-sensitive learning with given class and example costs and show that proposed algorithm provides superior generalization performance, compared to conventional methods.