PAC-Bayes bounds for twin support vector machines

Abstract

Twin support vector machines are regarded as a milestone in the development of support vector machines. Compared to standard support vector machines, they learn two nonparallel hyperplanes rather than one as in standard support vector machines for binary classification, and work faster and sometimes perform better than support vector machines. One of the reasons that support vector machines are widely used is that they are supported by strong statistical learning theory. However, relatively little is known about the theoretical analysis of twin support vector machines. As recent tightest bounds for practical applications, PAC-Bayes bound and prior PAC-Bayes bound are based on a prior and posterior over the distribution of classifiers. In this paper, we study twin support vector machines from a theoretical perspective and use the PAC-Bayes bound and prior PAC-Bayes bound to measure the generalization error bound of twin support vector machines. Experimental results on real-world datasets show better predictive capabilities of the PAC-Bayes bound and prior PAC-Bayes bound for twin support vector machines compared to the PAC-Bayes bound and the prior PAC-Bayes bound for support vector machines.