Approximate Parameter Tuning of Support Vector Machines

Abstract

Parameter Tuning is an indispensable step to guarantee generalization of support vector machines (SVM). Previous methods can be reduced to a nested two-layer framework, where the inner layer solves a convex optimization problem, and the outer layer selects the hyper-parameters by minimizing either cross validation error or other error bounds. In this paper, we propose a novel efficient parameter tuning approach via kernel matrix approximation, focusing on the efficiency improvement of SVM training in the inner layer. We first develop a kernel matrix approximation algorithm MoCIC. Then, we apply MoCIC to compute a low-rank approximation of the kernel matrix, and then use the approximate matrix to approximately solve the quadratic programming of SVM, and finally select the optimal candidate parameters through the approximate cross validation error (ACVE). We verify the feasibility and the efficiency of parameter tuning approach based on MoCIC on 5 benchmark datasets. Experimental results show that our approach can dramatically reduce time consumption of parameter tuning and meanwhile guarantee the effectiveness of the selected parameters.