Performance Of Individual Classifiers Research Articles

Sea Ice Classification Using Cryosat-2 Altimeter Data by Optimal Classifier–Feature Assembly

Sea ice type is one of the most sensitive variables in Arctic ice monitoring and detailed information about it is essential for ice situation evaluation, vessel navigation, and climate prediction. Many machine-learning methods including deep learning can be employed for ice-type detection, and most classifiers tend to prefer different feature combinations. In order to find the optimal classifier–feature assembly (OCF) for sea ice classification, it is necessary to assess their performance differences. The objective of this letter is to make a recommendation for the OCF for sea ice classification using Cryosat-2 (CS-2) data. Six classifiers including convolutional neural network (CNN), Bayesian, $K$ nearest-neighbor (KNN), support vector machine (SVM), random forest (RF), and back propagation neural network (BPNN) were studied. CS-2 altimeter data of November 2015 and May 2016 in the whole Arctic were used. The overall accuracy was estimated using multivalidation to evaluate the performances of individual classifiers with different feature combinations. Overall, RF achieved a mean accuracy of 89.15%, followed by Bayesian, SVM, and BPNN (~86%), outperforming the worst (CNN and KNN) by 7%. Trailing-edge width (TeW) and leading-edge width (LeW) were the most important features, and feature combination of TeW, LeW, Sigma0, maximum of the returned power waveform (MAX), and pulse peakiness (PP) was the best choice. RF with feature combination of TeW, LeW, Sigma0, MAX, and PP was finally selected as the OCF for sea ice classification and the results that demonstrated this method achieved a mean accuracy of 91.45%, which outperformed the other state-of-art methods by 9%.

A sentiment classification model based on multiple classifiers

With the widespread usage of social networks, forums and blogs, customer reviews emerged as a critical factor for the customers’ purchase decisions. Since the beginning of 2000s, researchers started to focus on these reviews to automatically categorize them into polarity levels such as positive, negative, and neutral. This research problem is known as sentiment classification. The objective of this study is to investigate the potential benefit of multiple classifier systems concept on Turkish sentiment classification problem and propose a novel classification technique. Vote algorithm has been used in conjunction with three classifiers, namely Naive Bayes, Support Vector Machine (SVM), and Bagging. Parameters of the SVM have been optimized when it was used as an individual classifier. Experimental results showed that multiple classifier systems increase the performance of individual classifiers on Turkish sentiment classification datasets and meta classifiers contribute to the power of these multiple classifier systems. The proposed approach achieved better performance than Naive Bayes, which was reported the best individual classifier for these datasets, and Support Vector Machines. Multiple classifier systems (MCS) is a good approach for sentiment classification, and parameter optimization of individual classifiers must be taken into account while developing MCS-based prediction systems.

Joint Binary Classifier Learning for ECOC-Based Multi-Class Classification

Error-correcting output coding (ECOC) is one of the most widely used strategies for dealing with multi-class problems by decomposing the original multi-class problem into a series of binary sub-problems. In traditional ECOC-based methods, binary classifiers corresponding to those sub-problems are usually trained separately without considering the relationships among these classifiers. However, as these classifiers are established on the same training data, there may be some inherent relationships among them. Exploiting such relationships can potentially improve the generalization performances of individual classifiers, and, thus, boost ECOC learning algorithms. In this paper, we explore to mine and utilize such relationship through a joint classifier learning method, by integrating the training of binary classifiers and the learning of the relationship among them into a unified objective function. We also develop an efficient alternating optimization algorithm to solve the objective function. To evaluate the proposed method, we perform a series of experiments on eleven datasets from the UCI machine learning repository as well as two datasets from real-world image recognition tasks. The experimental results demonstrate the efficacy of the proposed method, compared with state-of-the-art methods for ECOC-based multi-class classification.

Monitoring tool wear using classifier fusion

Real time monitoring of manufacturing processes using a single sensor often poses significant challenge. Sensor fusion has thus been extensively investigated in recent years for process monitoring with significant improvement in performance. This paper presents the results for a monitoring system based on the concept of classifier fusion, and class-weighted voting is investigated to further enhance the system performance. Classifier weights are based on the overall performances of individual classifiers, and majority voting is used in decision making. Acoustic emission monitoring of tool wear during the coroning process is used to illustrate the concept. A classification rate of 87.7% was obtained for classifier fusion with unity weighting. When weighting was based on overall performance of the respective classifiers, the classification rate improved to 95.6%. Further using state performance weighting resulted in a 98.5% classification. Finally, the classifier fusion performance further increased to 99.7% when a penalty vote was applied on the weighting factor.

IJARCCE - Computer and Communication Engineering

Concept drift is a major problem that the chemical sensor community is facing in their research and development. This arises when the predictive characteristic feature of the target variable in the sensor setup, changes due to some chemical or physical interaction of the environment elements with the surface of the sensor and other factors such as aging and poisoning of the surface. This is not a new problem but is having a wide scope of further developments. Recent research suggests an upcoming solution for this drift compensation as ensemble of classifiers with uniform weightage to all the participating classifiers or some non-uniform weightage according to performance of individual classifier. There arise some ill posed problems with this ensemble of classifiers. This paper introduces a new machine learning approach to solve this problem through regularized weighted ensemble of classifiers for overcoming the time dependent drift occurrence. The weights are chosen by regularized majority voting which are then associated with the individual classifiers to form the ensembles. This regularized drift compensation algorithm is applied to solve the gas discrimination problem for classification of 6 different volatile organic compounds over time series data set consisting of data recorded using an array of 16 metal oxides sensors for 3 years. Our experiment tries various different regularization approaches and finds best improvement in case of double regularization of Single Value Decomposition and L2 regularization. Results clearly indicate improved classification accuracy in presence of drift and better results than recent reporting. To the best of our knowledge, such a machine learning approach has not yet been applied for drift correction in sensor community.

Affect classification using genetic-optimized ensembles of fuzzy ARTMAPs

Training neural networks in distinguishing different emotions from physiological signals frequently involves fuzzy definitions of each affective state. In addition, manual design of classification tasks often uses sub-optimum classifier parameter settings, leading to average classification performance. In this study, an attempt to create a framework for multi-layered optimization of an ensemble of classifiers to maximize the system's ability to learn and classify affect, and to minimize human involvement in setting optimum parameters for the classification system is proposed. Using fuzzy adaptive resonance theory mapping (ARTMAP) as the classifier template, genetic algorithms (GAs) were employed to perform exhaustive search for the best combination of parameter settings for individual classifier performance. Speciation was implemented using subset selection of classification data attributes, as well as using an island model genetic algorithms method. Subsequently, the generated population of optimum classifier configurations was used as candidates to form an ensemble of classifiers. Another set of GAs were used to search for the combination of classifiers that would result in the best classification ensemble accuracy. The proposed methodology was tested using two affective data sets and was able to produce relatively small ensembles of fuzzy ARTMAPs with excellent affect recognition accuracy.

A New Approach to Classifier Fusion Based on Upper Integral

Fusing a number of classifiers can generally improve the performance of individual classifiers, and the fuzzy integral, which can clearly express the interaction among the individual classifiers, has been acknowledged as an effective tool of fusion. In order to make the best use of the individual classifiers and their combinations, we propose in this paper a new scheme of classifier fusion based on upper integrals, which differs from all the existing models. Instead of being a fusion operator, the upper integral is used to reasonably arrange the finite resources, and thus to maximize the classification efficiency. By solving an optimization problem of upper integrals, we obtain a scheme for assigning proportions of examples to different individual classifiers and their combinations. According to these proportions, new examples could be classified by different individual classifiers and their combinations, and the combination of classifiers that specific examples should be submitted to depends on their performance. The definition of upper integral guarantees such a conclusion that the classification efficiency of the fused classifier is not less than that of any individual classifier theoretically. Furthermore, numerical simulations demonstrate that most existing fusion methodologies, such as bagging and boosting, can be improved by our upper integral model.

A computational pipeline for the development of multi-marker bio-signature panels and ensemble classifiers

BackgroundBiomarker panels derived separately from genomic and proteomic data and with a variety of computational methods have demonstrated promising classification performance in various diseases. An open question is how to create effective proteo-genomic panels. The framework of ensemble classifiers has been applied successfully in various analytical domains to combine classifiers so that the performance of the ensemble exceeds the performance of individual classifiers. Using blood-based diagnosis of acute renal allograft rejection as a case study, we address the following question in this paper: Can acute rejection classification performance be improved by combining individual genomic and proteomic classifiers in an ensemble?ResultsThe first part of the paper presents a computational biomarker development pipeline for genomic and proteomic data. The pipeline begins with data acquisition (e.g., from bio-samples to microarray data), quality control, statistical analysis and mining of the data, and finally various forms of validation. The pipeline ensures that the various classifiers to be combined later in an ensemble are diverse and adequate for clinical use. Five mRNA genomic and five proteomic classifiers were developed independently using single time-point blood samples from 11 acute-rejection and 22 non-rejection renal transplant patients. The second part of the paper examines five ensembles ranging in size from two to 10 individual classifiers. Performance of ensembles is characterized by area under the curve (AUC), sensitivity, and specificity, as derived from the probability of acute rejection for individual classifiers in the ensemble in combination with one of two aggregation methods: (1) Average Probability or (2) Vote Threshold. One ensemble demonstrated superior performance and was able to improve sensitivity and AUC beyond the best values observed for any of the individual classifiers in the ensemble, while staying within the range of observed specificity. The Vote Threshold aggregation method achieved improved sensitivity for all 5 ensembles, but typically at the cost of decreased specificity.ConclusionProteo-genomic biomarker ensemble classifiers show promise in the diagnosis of acute renal allograft rejection and can improve classification performance beyond that of individual genomic or proteomic classifiers alone. Validation of our results in an international multicenter study is currently underway.

Classifiers selection in ensembles using genetic algorithms for bankruptcy prediction

Ensemble learning is a method to improve the performance of classification and prediction algorithms. Many studies have demonstrated that ensemble learning can decrease the generalization error and improve the performance of individual classifiers and predictors. However, its performance can be degraded due to multicollinearity problem where multiple classifiers of an ensemble are highly correlated with. This paper proposes a genetic algorithm-based coverage optimization technique in the purpose of resolving multicollinearity problem. Empirical results with bankruptcy prediction on Korea firms indicate that the proposed coverage optimization algorithm can help to design a diverse and highly accurate classification system.

A Kind of Combination Feature Division and Diversity Measure of Multi-Classifier Selective Ensemble Algorithm

In ensemble learning, in order to improve the performance of individual classifiers and the diversity of classifiers, from the classifiers generation and combination, this paper proposes a kind of combination feature division and diversity measure of multi-classifier selective ensemble algorithm. The algorithm firstly applied bagging method to create some feature subsets, Secondly using principal component analysis of feature extraction method on each feature subsets, then select classifiers with high-classification accuracy; finally before classifier combination we use classifier diversity measure method select diversity classifiers. Experimental results prove that classification accuracy of the algorithm is obviously higher than popular bagging algorithm.

Locally application of random subspace with simple Bayesian classifier

Naive Bayes algorithm captures the assumption that every attribute is independent from the rest of the attributes, given the state of the class attribute. In this study, we attempted to increase the prediction accuracy of the simple Bayes model. Because the concept of combining classifiers is proposed as a new direction for the improvement of the performance of individual classifiers, we propose a technique of localised multiple simple Bayes models. The ensemble consists of multiple simple Bayes models constructed locally by pseudorandomly selecting subsets of components of the feature vector, that is, simple Bayes models constructed in randomly chosen subspaces. Finally, we performed a large-scale comparison with other attempts that have tried to improve the accuracy of the naive Bayes algorithm as well as other state-of-the-art algorithms and ensembles on 26 standard benchmark datasets and the proposed method gave better accuracy in most cases.

Optimal ensemble construction via meta-evolutionary ensembles

In this paper, we propose a meta-evolutionary approach to improve on the performance of individual classifiers. In the proposed system, individual classifiers evolve, competing to correctly classify test points, and are given extra rewards for getting difficult points right. Ensembles consisting of multiple classifiers also compete for member classifiers, and are rewarded based on their predictive performance. In this way we aim to build small-sized optimal ensembles rather than form large-sized ensembles of individually-optimized classifiers. Experimental results on 15 data sets suggest that our algorithms can generate ensembles that are more effective than single classifiers and traditional ensemble methods.

Integration of multiple neural classifiers for heart beat recognition

PurposeThis paper presents new approach to the integration of neural classifiers. Typically only the best trained network is chosen, while the rest is discarded. However, combining the trained networks helps to integrate the knowledge acquired by the component classifiers and in this way improves the accuracy of the final classification. The aim of the research is to develop and compare the methods of combining neural classifiers of the heart beat recognition.Design/methodology/approachTwo methods of integration of the results of individual classifiers are proposed. One is based on the statistical reliability of post‐processing performance on the trained data and the second uses the least mean square method in adjusting the weights of the weighted voting integrating network.FindingsThe experimental results of the recognition of six types of arrhythmias and normal sinus rhythm have shown that the performance of individual classifiers could be improved significantly by the integration proposed in this paper.Practical implicationsThe presented application should be regarded as the first step in the direction of automatic recognition of the heart rhythms on the basis of the registered ECG waveforms.Originality/valueThe results mean that instead of designing one high performance classifier one can build a number of classifiers, each of not superb performance. The appropriate combination of them may produce a performance of much higher quality.

A theoretical and experimental analysis of linear combiners for multiple classifier systems

In this paper, a theoretical and experimental analysis of linear combiners for multiple classifier systems is presented. Although linear combiners are the most frequently used combining rules, many important issues related to their operation for pattern classification tasks lack a theoretical basis. After a critical review of the framework developed in works by Tumer and Ghosh on which our analysis is based, we focus on the simplest and most widely used implementation of linear combiners, which consists of assigning a nonnegative weight to each individual classifier. Moreover, we consider the ideal performance of this combining rule, i.e., that achievable when the optimal values of the weights are used. We do not consider the problem of weights estimation, which has been addressed in the literature. Our theoretical analysis shows how the performance of linear combiners, in terms of misclassification probability, depends on the performance of individual classifiers, and on the correlation between their outputs. In particular, we evaluate the ideal performance improvement that can be achieved using the weighted average over the simple average combining rule and investigate in what way it depends on the individual classifiers. Experimental results on real data sets show that the behavior of linear combiners agrees with the predictions of our analytical model. Finally, we discuss the contribution to the state of the art and the practical relevance of our theoretical and experimental analysis of linear combiners for multiple classifier systems.

Physical field models for pattern classification

Recent findings in pattern recognition show that dramatic improvement of the recognition rate can be obtained by application of fusion systems utilizing many different and diverse classifiers for the same task. Apart from a good individual performance of individual classifiers the most important factor is the useful diversity they exhibit. In this work we present an example of a novel, well performing non-parametric classifier design, which shows a substantial level of diversity with respect to other commonly used classifiers. Inspired by the mechanics of omnipresent physical fields like gravitational or electrostatic, we considered the data as particles carrying elementary units of charge. The charge has been presented as a source of the potential triggering attracting interaction among the data. This interaction has been reformulated as data matching procedure and developed into original classification technique where the unlabelled testing data are captured by the labelled training data and share their labels. In the extended model apart from the spatial data distribution we also exploit topology of class labels to devise repelling force as field action between differently labelled data samples. As we show introduction of the repelling force clearly smoothes the decision boundaries and improves performance while still preserving attractive diversity properties of the classification model. The paper covers extensive examples and visual interpretations of the presented techniques supported by the experimental work with established datasets and classifiers.

Robust Combining of Disparate Classifiers through Order Statistics

Integrating the outputs of multiple classifiers via combiners or meta-learners has led to substantial improvements in several difficult pattern recognition problems. In this article, we investigate a family of combiners based on order statistics, for robust handling of situations where there are large discrepancies in performance of individual classifiers. Based on a mathematical modelling of how the decision boundaries are affected by order statistic combiners, we derive expressions for the reductions in error expected when simple output combination methods based on the median, the maximum and in general, the ith order statistic, are used. Furthermore, we analyse the trim and spread combiners, both based on linear combinations of the ordered classifier outputs, and show that in the presence of uneven classifier performance, they often provide substantial gains over both linear and simple order statistics combiners. Experimental results on both real world data and standard public domain data sets corroborate these findings.

Methods of combining multiple classifiers and their applications to handwriting recognition

Possible solutions to the problem of combining classifiers can be divided into three categories according to the levels of information available from the various classifiers. Four approaches based on different methodologies are proposed for solving this problem. One is suitable for combining individual classifiers such as Bayesian, k-nearest-neighbor, and various distance classifiers. The other three could be used for combining any kind of individual classifiers. On applying these methods to combine several classifiers for recognizing totally unconstrained handwritten numerals, the experimental results show that the performance of individual classifiers can be improved significantly. For example, on the US zipcode database, 98.9% recognition with 0.90% substitution and 0.2% rejection can be obtained, as well as high reliability with 95% recognition, 0% substitution, and 5% rejection. >