A novel regularization method for decorrelation learning of non-parallel hyperplanes

Abstract

Non-parallel hyperplane classifiers have attracted many attention due to their wider applicability than parallel hyperplane classifiers, and it accelerates the training speed through separately training of each hyperplane. However, it does not consider the correlation relationship among the non-parallel hyperplanes. In addition, since these non-parallel hyperplanes are built on the same training data, there may exist correlation among them due to redundancy of the data. In this paper, we investigate the decorrelation problem of non-parallel hyperplane classifiers. We integrate non-parallel hyperplanes into a unified model and explore this relationship through a joint classifier learning approach. Taking the twin support vector machine as an example, a novel decorrelation regularization term is added into the joint problem. Additionally, an effective alternating optimization algorithm has been introduced to address this nonconvex problem. To evaluate the proposed method, a series of experiments are conducted on binary and multi-class classification datasets from the benchmark database repository. Experimental results compared to several parallel and non-parallel hyperplane based classifiers demonstrate the effectiveness of the proposed method.