Structured Large Margin Machine Ensemble

Abstract

Large margin classifiers have been widely applied in solving supervised learning problems. One representative model in large margin learning is the support vector machine (SVM). SVM is an unstructured classifier since the data structure information is underutilized and the decision hyperplane calculation relies exclusively on the support vectors. To incorporate the data covariance information into the large margin learning, structured large margin machine (SLMM) is recently proposed and show better performance than classical SVM in some applications. Instead of utilizing the data structures straightly like SLMM, SVM ensemble (SVMe) improves the generalization ability of SVM in another way by combining the outputs of a series of SVMs. Inspired by SVMe, we are going to explore the ensemble counterpart for SLMM, i.e., SLMMe, and validate the effectiveness of multiple SLMM system. Experimental results on benchmark datasets demonstrate that SLMMe improves SLMM by reducing its variance, and SLMMe outperforms SVMe in most cases in terms of both classification accuracy and variance.