A new adaptive weighted imbalanced data classifier via improved support vector machines with high-dimension nature

Abstract

The standard support vector machine (SVM) models are widely used in various fields, but we show that they are not rationally defined from the perspective of geometric point, which is likely to degrade the models’ performances theoretically, especially under the high-dimensional cases. In this paper, we consider a composite penalty and propose an elastic net support vector machine (ENSVM). Unlike the doubly regularized support vector machine (DrSVM, Wang et al. (2006)), we impose the penalty to the slack variables rather than the normal vectors of the hyperplane. Then, we prove that ENSVM is more rationally defined than standard SVM and DrSVM (in section 3.2.1). Moreover, the ENSVM demonstrates a more stable and high-dimension nature inherently, while the simulation results cogently support these merits. Besides, we also combine fused weights with ENSVM and propose an adaptive weighted elastic net support vector machine (AWENSVM), to make the primal model more adaptive and robust to the imbalanced data. Compared with the other popular SVMs, the AWENSVM model proposed in this paper performs better obviously.