A comparative study for error approximation of some kernel functions in Smooth Support Vector Machines

Abstract

Support Vector Machine (SVM) as one of the most popular machine learning methods is playing a significant role in statistical learning theory. Smooth Support Vector Machine (SSVM) is one of new formulation to improve the SVM. In SSVM, smoothing method is used to optimize the unconstrained model. Smoothing function can be used to replace plus function in SVM. In this paper we evaluate eight smoothing functions including quadratic polynomial, fourth order polynomial, piecewise polynomial, spline function, sixth order polynomial, advanced fourth order polynomial function, quadratic Bezier function, third order Bezier function, and fourth order Bezier function. Some of those functions have been studied previously in order to find better accuracy in SVM. In this research, we evaluate and analyze the performance of all those eight smoothing functions based on their infinity-norm values. We compare the error approximation between smoothing function and plus function (as the standard kernel function in SVM) where the best smoothing function has the minimum error of its infinity-norm. Based on theoretical analysis and numerical approximation, our results show that piecewise polynomial function is better than quadratic polynomial function, fourth polynomial function, and third order spline function. While the advanced fourth order polynomial function and sixth order polynomial function have error approximation value almost the same as that of plus function. However, since the piecewise polynomial function has less control parameters than those of the advanced fourth order polynomial function and the sixth order polynomial function, we could not conclude which one is the best. Furthermore, based on values of infinity-norm of all the smoothing functions, we found that the quadratic Bezier and quadratic polynomial show the same error values. Meanwhile the forth order Bezier function shows the smallest error approximation value among the other functions which have been tested. In conclusion, based on our results in this study we found that the forth order Bezier function is the best choice among the eight smoothing function for SSVM.