Multiple Kernel SVM Based on Two-Stage Learning

Xingrui Gong,Qingxin Luo,Yan Yang,Jie Xu,Yuze Duan,Bin Zou

doi:10.1109/access.2020.2998772

Xingrui Gong, Qingxin Luo + Show 4 more

Open Access

https://doi.org/10.1109/access.2020.2998772

Copy DOI

Abstract

In this paper we introduce the idea of two-stage learning for multiple kernel SVM (MKSVM) and present a new MKSVM algorithm based on two-stage learning (MKSVM-TSL). The first stage is the pre-learning and its aim is to obtain the information of data such that the “important” samples for classification can be generated in the formal learning stage and these samples are uniformly ergodic Markov chain (u.e.M.c.). To study comprehensively the proposed MKSVM-TSL algorithm, we estimate the generalization bound of MKSVM based on u.e.M.c. samples and obtain its fast learning rate. And in order to show the performance of the proposed MKSVM-TSL algorithm for better, we also perform the numerical experiments on various publicly available datasets. From the experimental results, we can find that compared to three classical multiple kernel learning (MKL) algorithms, the proposed MKSVM-TSL algorithm has better performance in three aspects of the total time of sampling and training, the accuracy and the sparsity of classifiers, respectively.

Highlights

Support Vector Machine (SVM) is one of the most widely applied machine learning methods for pattern recognition problems [1]
NUMERICAL STUDIES In order to show the performance of MKSVM-TSL, we compare the proposed MKSVM-TSL algorithm with three multiple kernel learning (MKL) algorithms, the mean weighted MKSVM based on randomly independent samples [25], the multiple kernel least squares SVM (MKLSSVM) with semi-infinite programming (SIP) based on randomly independent samples [12] and the ratio weighted MKSVM (Rat-MKSVM) based on randomly independent samples [40]
In order to show the performance of MKSVM-TSL with k = 1 for better, we present Figs. 1-6 to compare 50-times experiments of MKSVM-TSL based on k = 1 with MKSVM, MKLSSVM and Rat-MKSVM

Summary

INTRODUCTION

Support Vector Machine (SVM) is one of the most widely applied machine learning methods for pattern recognition problems [1]. The algorithmic complexity of MKL combining multiple kernels is very high for the big size of training samples This implies that MKL method has good learning performance, MKL method is usually very time-consuming and even difficult to implement when the scale of training samples is larger. To solve this problem, we present the idea of two-stage learning for multiple kernel SVM (MKSVM) algorithm in this paper. The aim of two-stage learning is to generate the ‘‘important’’ samples from the given training set in the stage of formal learning by making use of the information obtained in the stage of pre-learning To our knowledge, this is the first algorithm of MKL method.

MULTIPLE KERNEL SVM FORMULATION

MKSVM-TSL ALGORITHM

NUMERICAL STUDIES

CONCLUSIONS