Analysis of the Distance Between Two Classes for Tuning SVM Hyperparameters

Abstract

An important step in the construction of a support vector machine (SVM) is to select optimal hyperparameters. This paper proposes a novel method for tuning the hyperparameters by maximizing the distance between two classes (DBTC) in the feature space. With a normalized kernel function, we find that DBTC can be used as a class separability criterion since the between-class separation and the within-class data distribution are implicitly taken into account. Employing DBTC as an objective function, we develop a gradient-based algorithm to search the optimal kernel parameter. On the basis of the geometric analysis and simulation results, we find that the optimal algorithm and the initialization problem become very simple. Experimental results on the synthetic and real-world data show that the proposed method consistently outperforms other existing hyperparameter tuning methods.