N-class Problem Research Articles

Universum Selection for Boosting the Performance of Multiclass Support Vector Machines Based on One-versus-One Strategy

We propose a novel framework to enhance the performance of the one-versus-one support vector machine by using Universum. For solving a multiclass classification problem, one-versus-one is one of the state-of-the-art algorithms, which constructs N(N−1)/2 binary classifiers for an N-class problem. Each binary classifier is originally learned by two classes of data as positive and negative classes while the other N−2 remaining classes are ignored, even if they might also represent a hidden concept of the application domain and can help to boost the performance of the classifier. Vapnik et al. [20, 21] introduced Universum binary support vector machines to enable the use of samples that do not belong to positive and negative classes and called these samples Universum samples. However, not all Universum samples can be helpful; moreover, improper selection of Universum samples can prevent the construction of an effective binary classifier. For the construction of a Universum binary classifier in the one-versus-one strategy, there are 2N−2 candidate subsets of classes of Universum data; a proper selection of them can be difficult, based on the number of classes. We design an algorithm to obtain a suitable subset of classes of Universum data by applying the proposed performance measure that reflects the properties of Universum data relative to labeled training data. This measure is based on the analysis of the projection of Universum data onto the normal direction vector of the standard binary SVM hyperplane. We demonstrate experimentally that our proposed strategy outperforms existing methods.

Towards the Automatic Classification of Avian Flight Calls for Bioacoustic Monitoring.

Automatic classification of animal vocalizations has great potential to enhance the monitoring of species movements and behaviors. This is particularly true for monitoring nocturnal bird migration, where automated classification of migrants’ flight calls could yield new biological insights and conservation applications for birds that vocalize during migration. In this paper we investigate the automatic classification of bird species from flight calls, and in particular the relationship between two different problem formulations commonly found in the literature: classifying a short clip containing one of a fixed set of known species (N-class problem) and the continuous monitoring problem, the latter of which is relevant to migration monitoring. We implemented a state-of-the-art audio classification model based on unsupervised feature learning and evaluated it on three novel datasets, one for studying the N-class problem including over 5000 flight calls from 43 different species, and two realistic datasets for studying the monitoring scenario comprising hundreds of thousands of audio clips that were compiled by means of remote acoustic sensors deployed in the field during two migration seasons. We show that the model achieves high accuracy when classifying a clip to one of N known species, even for a large number of species. In contrast, the model does not perform as well in the continuous monitoring case. Through a detailed error analysis (that included full expert review of false positives and negatives) we show the model is confounded by varying background noise conditions and previously unseen vocalizations. We also show that the model needs to be parameterized and benchmarked differently for the continuous monitoring scenario. Finally, we show that despite the reduced performance, given the right conditions the model can still characterize the migration pattern of a specific species. The paper concludes with directions for future research.

A hierarchical multiclass support vector machine incorporated with holistic triple learning units

This paper proposes a new hierarchical learning structure, namely the holistic triple learning (HTL), for extending the binary support vector machine (SVM) to multi-classification problems. For an N-class problem, a HTL constructs a decision tree up to a depth of $$\lceil N/3\rceil+1$$. A leaf node of the decision tree is allowed to be placed with a holistic triple learning unit whose generalisation abilities are assessed and approved. Meanwhile, the remaining nodes in the decision tree each accommodate a standard binary SVM classifier. The holistic triple classifier is a regression model trained on three classes, whose training algorithm is originated from a recently proposed implementation technique, namely the least-squares support vector machine (LS-SVM). A major novelty with the holistic triple classifier is the reduced number of support vectors in the solution. For the resultant HTL-SVM, an upper bound of the generalisation error can be obtained. The time complexity of training the HTL-SVM is analysed, and is shown to be comparable to that of training the one-versus-one (1-vs.-1) SVM, particularly on small-scale datasets. Empirical studies show that the proposed HTL-SVM achieves competitive classification accuracy with a reduced number of support vectors compared to the popular 1-vs-1 alternative.

Task decomposition and modular single-hidden-layer perceptron classifiers for multi-class learning problems

One of keys for multilayer perceptrons (MLPs) to solve the multi-class learning problems is how to make them get good convergence and generalization performances merely through learning small-scale subsets, i.e., a small part of the original larger-scale data sets. This paper first decomposes an n-class problem into n two-class problems, and then uses n class-modular MLPs to solve them one by one. A class-modular MLP is responsible for forming the decision boundaries of its represented class, and thus can be trained only by the samples from the represented class and some neighboring ones. When solving a two-class problem, an MLP has to face with such unfavorable situations as unbalanced training data, locally sparse and weak distribution regions, and open decision boundaries. One of solutions is that the samples from the minority classes or in the thin regions are virtually reinforced by suitable enlargement factors. And next, the effective range of an MLP is localized by a correction coefficient related to the distribution of its represented class. In brief, this paper focuses on the formation of economic learning subsets, the virtual balance of imbalanced training sets, and the localization of generalization regions of MLPs. The results for the letter and the extended handwritten digital recognitions show that the proposed methods are effective.

Architecture of Fuzzy Support Vector Machines for Pattern Classification

In conventional support vector machines (SVMs), an n-class problem is converted into n two-class problems. For the i-th two-class problem we determine the optimal decision function which separates class i from the remaining classes. In classification, a sample is classified into class i only when the value of the i-th decision function is positive. In this architecture, the sample is unclassifiable if the values of more than two decision functions are positive or all the values are negative. In this paper, to overcome this problem, we propose fuzzy support vector machines (FSVMs). Using the decision functions obtained by training the SVM, for each class, we define a truncated polyhedral pyramidal membership function. Since, for the data in the classifiable regions. the classification results are the same for the two methods, the generalization ability of the FSVM is the same with or better than that of the SVM. We evaluate our method for three benchmark data sets and demonstrate the superiority of the FSVM over the SVM.

The optimum class-selective rejection rule

Class-selective rejection is an extension of simple rejection. That is, when an input pattern cannot be reliably assigned to one of the N classes in an N-class problem, it is assigned to a subset of classes that are most likely to issue the pattern, instead of simply being rejected. By selecting more classes, the risk of making an error can be reduced, at the price of subsequently having a larger remaining number of classes. The optimality of class-selective rejection is therefore defined as the best trade-off between error rate and average number of selected classes. Formally, the trade-off study is embedded in the framework of decision theory. The average expected loss is expressed as a linear combination of error rate and average number of classes. The minimization of the average expected loss, therefore, provides the best trade-off. The complexity of the resulting optimum rule is reduced, via a discrete convex minimization, to be linear in the number of classes. Upper-bounds on error rate and average number of classes are derived. An example is provided to illustrate various aspects of the optimum decision rule. Finally, the implications of the new decision rule are discussed.

Nonparametric classifier design using greedy tree-structured vector quantization technique

In this paper, we propose a novel tree-structured vector quantization (TSVQ) design algorithm for the applications of nonparametric pattern recognition. The TSVQ design algorithm is used to reduce the large size of the design sets required by a nonparametric classifier. For an N-class problem, the TSVQ consists of N branches with one for each class. Using the design sets as training data, the algorithm splits the leaf nodes in a greedy manner to minimize the classification error rate for tree-growing. Simulation results show that the classifiers designed using this new algorithm require less classification time than that required by other design set reduction algorithms. In addition, in many cases, the new classifiers enjoy almost the same low error rate as that of traditional k-NN nonparametric classifiers.