Minimum class variance multiple kernel learning

Abstract

The purpose of multiple kernel learning (MKL) is to learn an appropriate kernel from a set of predefined base kernels. Most of the MKL methods follow the basic idea of support vector machine (SVM) to learn the optimal weights of base kernels and build the used classifier. However, SVM is a local method and ignores the structure information of the data in that its solution is exclusively determined by the so-called support vectors. In the paper, we propose an improved SVM-based MKL method called minimum class variance multiple kernel learning (MCVMKL). The key characteristic of MCVMKL is that it exploits the ellipsoidal structure of the data during learning the optimal weights and building the classifier. Besides, its formulation is invariant to scalings of the weights of base kernels. We develop two optimization strategies to handle the optimization model of MCVMKL. Further, we derive a rough upper bound for the objective function of MCVMKL and propose a variant called trace-constrained multiple kernel learning (TCMKL) by using the trace of the within-class scatter matrix. TCMKL enlarges the margin between different classes and simultaneously shrinks the region covering the data as much as possible. Moreover, it can automatically tune the regularization parameter and so saves the training time due to avoiding using the time-consuming cross-validation technique to select an appropriate regularization parameter. Finally, the comprehensive experiments are conducted and the results demonstrate that the proposed methods are effective and can achieve better performance over the competing methods.