New one-versus-all [formula omitted] solving intra–inter class imbalance with extended manifold regularization and localized relative maximum margin

Abstract

The optimal separating hyperplane with maximum margin plays a central role in the support vector machine (SVM) classifiers, but is exclusively determined by the support vectors that does not take any structure information into account, especially in the one-versus-all (OVA) SVM for the multi-class classification application where the positive samples and negative ones are usually unbalanced. To address this intra–inter class imbalance problem, a new OVA SVM method is proposed by extending manifold regularization and enhancing the relative maximum margin. It aims to minimize the scatter of nearby intra-class point pairs while penalize that of nearby inter-class point pairs not being arbitrarily large through the constraints of marginal separation and manifold regularization. This objective is transformed into a constrained optimization problem that overcomes both the classical SVM's weakness of ignoring the data underlying structure, and the Laplacian SVM's weakness of abandoning class label information and class separation of manifolds. It also takes the positive and negative samples into different consideration according to their unbalanced distribution. Additionally, risk bounds are derived for the proposed formulation based on the theory of Rademacher complexity and the improvement on the bounds of standard SVM is proven. The comparative experiments for multi-class classification, face recognition, and natural image segmentation on several synthetic and benchmark data sets validate the effectiveness of the proposed method and indicate the consideration of both preserving intrinsic within-class manifold structure and bounding the local relative margin between class is helpful to improve the OVA SVM classification performance.