Balanced binary search tree multiclass decomposition method with possible non-outliers

Abstract

Multiclass decomposition algorithms are the means by which binary classification algorithms like support vector machine are used for multiclass classification problems. The popular multiclass decomposition algorithms like one against one (OAO), one against all (OAA), etc., perform the decomposition in a naive manner. This paper presents a novel heuristic-based decomposition algorithm that takes the Hausdorff distance between two classes to decide the decomposition. During the decomposition, rules are made to ensure a balanced binary search tree structure. To model the uncertainty and class noise present in the data, an unsupervised outlier detection technique has been used so that only possible non-outliers take part in the decomposition process. The presented algorithm has been evaluated and compared against OAO and OAA methods across 6 datasets. While evaluating the decomposition algorithms, fuzzy support vector machine has been used to model the class noise during each binary classification. The comparison shows that presented method not only provides comparable performance, but also in all cases, can classify the test samples with fewer average number of support vectors, thus leading to faster test performance. The paper further observes that the proposed approach can provide statistically better performance when the decomposition structure is learned only using the possible non-outliers, as compared to the scenario where the decomposition structure is learned using all samples.