A Robust Probability Classifier Based on the Modifiedχ2-Distance

Yongzhi Wang,Jining Yi,Yuli Zhang,Honggang Qu,Jinli Miu

doi:10.1155/2014/621314

Abstract

We propose a robust probability classifier model to address classification problems with data uncertainty. A class-conditional probability distributional set is constructed based on the modifiedχ2-distance. Based on a “linear combination assumption” for the posterior class-conditional probabilities, we consider a classification criterion using the weighted sum of the posterior probabilities. An optimal robust minimax classifier is defined as the one with the minimal worst-case absolute error loss function value over all possible distributions belonging to the constructed distributional set. Based on the conic duality theorem, we show that the resulted optimization problem can be reformulated into a second order cone programming problem which can be efficiently solved by interior algorithms. The robustness of the proposed model can avoid the “overlearning” phenomenon on training sets and thus keep a comparable accuracy on test sets. Numerical experiments validate the effectiveness of the proposed model and further show that it also provides promising results on multiple classification problems.

Highlights

Statistics classification has been extensively studied in the field of machine learning and statistics
Unlike the “conditional independence assumption” in naive Bayes classifiers (NBC), we introduce a “linear combination assumption” for the posterior class-conditional probabilities, and the proposed classifier takes a linear combination form of these probabilities based on different features and it will assign the sample to the class with the maximal posterior probability
From the tested real-life application, we conclude that the proposed robust probability classifier (RPC) has the robustness to provide better performance for both binary and multiple classification problems compared with regularized SVM (RSVM) and NBC

Summary

Introduction

Statistics classification has been extensively studied in the field of machine learning and statistics. They make no assumption on the class-conditional distributions, but only the mean and covariance matrix of each class are assumed to be known Under this assumption, the designed classifier is determined by minimizing the worst-case probability of misclassification under all possible choices of classconditional distributions with the given mean and covariance matrix. Hoi and Lyu [16] study a quadratic classifier with positive definite covariance matrices and further consider the problem of finding a convex set to cover known sampled data in one class while minimizing the worst-case misclassification probability.

Classifier Models

Solution Methods for RPC

Numerical Experiments on Real-World Applications

Findings

Conclusion