Dynamic Classifier Selection Research Articles

An efficient slap image matching system based on dynamic classifier selection and aggregation

The performance of any existing slap fingerprint image based matching systems is limited because it uses a fixed classifier, a fixed fusion rule and considers all fingerprints in the slap image without giving any importance on the quality of the images. In this paper, fingerprints present in slap images are matched by selecting and aggregating several classifiers based on the quality of fingerprints. The proposed system avoids the matching of poor quality fingerprints and fuses matching scores of only moderate quality fingerprints using average or quality based fusion. However, for verification, poor quality fingerprints are also considered if there does not exist enough number of moderate quality fingerprints. For poor quality fingerprint matching, density score based fusion technique which fuses the matching scores obtained from a global feature matcher and a local feature matcher has been used. The proposed system has been tested on a database of size 10,800 slap images collected from 1800 different hands.

Dynamic Selection of Classifiers in Bug Prediction: An Adaptive Method

In the last decades, the research community has devoted a lot of effort in the definition of approaches able to predict the defect proneness of source code files. Such approaches exploit several predictors (e.g., product or process metrics) and use machine learning classifiers to predict classes into buggy or not buggy, or provide the likelihood that a class will exhibit a fault in the near future. The empirical evaluation of all these approaches indicated that there is no machine learning classifier providing the best accuracy in any context, highlighting interesting complementarity among them. For these reasons ensemble methods have been proposed to estimate the bug-proneness of a class by combining the predictions of different classifiers. Following this line of research, in this paper we propose an adaptive method, named ASCI (Adaptive Selection of Classifiers in bug prediction), able to dynamically select among a set of machine learning classifiers the one which better predicts the bug-proneness of a class based on its characteristics. An empirical study conducted on 30 software systems indicates that ASCI exhibits higher performances than five different classifiers used independently and combined with the majority voting ensemble method.

Segmentation of brain MR images using a proper combination of DCS based method with MRF

Manual segmentation of Magnetic Resonance Images (MRI) is a time-consuming process, thus automatic segmentation of brain MR images has attracted more attention in recent years. In this paper, we introduce Dynamic Classifier Selection Markov Random Field (DCSMRF) algorithm for supervised segmentation of brain MR images into three main tissues such as White Matter (WM), Gray Matter (GM) and Cerebrospinal Fluid (CSF). DCSMRF combines a novel ensemble method with the Markov Random Field (MRF) algorithm and tries to obtain the advantages of both algorithms. For the ensemble part of DCSMRF, we propose an ensemble method called Dynamic Classifier System-Weighted Local Accuracy (DCS-WLA) which is a type of Combination of Multiple Classifier (CMC) algorithm. Later, the MRF algorithm is utilized for incorporating spatial, contextual and textural information in this paper. For the MRF section, an energy function based on the output of the DCS-WLA algorithm is proposed, then maximum value for Maximum A Posterior (MAP) criterion is searched to obtain optimal segmentation. The MRF algorithm applies similar to a post processing step in which only a subset of pixels is selected for optimization step. Hence, a vast amount of search space is pruned. Consequently, the computational burden of the proposed algorithm is more tolerable than the conventional MRF-based methods. Moreover, by employing ensemble algorithms, the accuracy and reliability of final results are enhanced compared to the individual methods.

Prototype selection for dynamic classifier and ensemble selection

In dynamic ensemble selection (DES) techniques, only the most competent classifiers, for the classification of a specific test sample, are selected to predict the sample’s class labels. The key in DES techniques is estimating the competence of the base classifiers for the classification of each specific test sample. The classifiers’ competence is usually estimated according to a given criterion, which is computed over the neighborhood of the test sample defined on the validation data, called the region of competence. A problem arises when there is a high degree of noise in the validation data, causing the samples belonging to the region of competence to not represent the query sample. In such cases, the dynamic selection technique might select the base classifier that overfitted the local region rather than the one with the best generalization performance. In this paper, we propose two modifications in order to improve the generalization performance of any DES technique. First, a prototype selection technique is applied over the validation data to reduce the amount of overlap between the classes, producing smoother decision borders. During generalization, a local adaptive K-Nearest Neighbor algorithm is used to minimize the influence of noisy samples in the region of competence. Thus, DES techniques can better estimate the classifiers’ competence. Experiments are conducted using 10 state-of-the-art DES techniques over 30 classification problems. The results demonstrate that the proposed scheme significantly improves the classification accuracy of dynamic selection techniques.

Dynamic classifier selection for one-class classification

One-class classification is among the most difficult areas of the contemporary machine learning. The main problem lies in selecting the model for the data, as we do not have any access to counterexamples, and cannot use standard methods for estimating the classifier quality. Therefore ensemble methods that can use more than one model, are a highly attractive solution. With an ensemble approach, we prevent the situation of choosing the weakest model and usually improve the robustness of our recognition system. However, one cannot assume that all classifiers available in the pool are in general accurate – they may have local competence areas in which they should be employed. In this work, we present a dynamic classifier selection method for constructing efficient one-class ensembles. We propose to calculate the competencies of all classifiers for a given validation example and use them to estimate their competencies over the entire decision space with the Gaussian potential function. We introduce three measures of classifier’s competence designed specifically for one-class problems. Comprehensive experimental analysis, carried on a number of benchmark data and backed-up with a thorough statistical analysis prove the usefulness of the proposed approach.

A new hyperspectral band selection and classification framework based on combining multiple classifiers

Due to the high-dimension characteristics of hyperspectral data, dimensionality reduction is becoming an important problem in hyperspectral image classification. Band selection can retain the information which is capable of keeping the original physical meaning of the spectral channels, and thus it has attracted more research interests. This paper tackles the band selection problem from the perspective of multiple classifiers combination, which can obtain higher classification accuracy. In the newly formulated framework of band selection and classification based on combination of multiple classifiers (BS_CMC), stochastic algorithms are firstly employed to generate several groups of initial band subset, on which a pool of classifiers is constructed. Then, improved classifier selection algorithm based on error diversity is proposed to select several member classifiers from the initial classifier pool. And finally the classification is performed through dynamic classifier selection based on local classification accuracy. The experimental results on two benchmark data sets show that the proposed approach can select those bands with more discriminative information and improve the classification accuracy effectively.

Aggregation of classifiers ensemble using local discriminatory power and quantiles

The paper presents a new approach to the dynamic classifier selection in an ensemble by applying the best suited classifier for the particular testing sample. It is based on the area under curve (AUC) of the receiver operating characteristic (ROC) of each classifier. To allow application of different types of classifiers in an ensemble and to reduce the influence of outliers, the quantile representation of the signals is used. The quantiles divide the ordered data into essentially equal-sized data subsets providing approximately uniform distribution of [0–1] support for each data point. In this way the recognition problem is less sensitive to the outliers, scales and noise contained in the input attributes. The numerical results presented for the chosen benchmark data-mining sets and for the data-set of images representing melanoma and non-melanoma skin lesions have shown high efficiency of the proposed approach and superiority to the existing methods.

Dynamic selection of the best base classifier in One versus One

Class binarization strategies decompose the original multi-class problem into several binary sub-problems. One versus One (OVO) is one of the most popular class binarization techniques, which considers every pair of classes as a different sub-problem. Usually, the same classifier is applied to every sub-problem and then all the outputs are combined by some voting scheme. In this paper we present a novel idea where for each test instance we try to assign the best classifier in each sub-problem of OVO. To do so, we have used two simple Dynamic Classifier Selection (DCS) strategies that have not been yet used in this context. The two DCS strategies use K-NN to obtain the local region of the test-instance, and the classifier that performs the best for those instances in the local region, is selected to classify the new test instance. The difference between the two DCS strategies remains in the weight of the instance. In this paper we have also proposed a novel approach in those DCS strategies. We propose to use the K-Nearest Neighbor Equality (K-NNE) method to obtain the local accuracy. K-NNE is an extension of K-NN in which all the classes are treated independently: the K nearest neighbors belonging to each class are selected. In this way all the classes take part in the final decision. We have carried out an empirical study over several UCI databases, which shows the robustness of our proposal.

Local Accuracy Measurement for Face Recognition System using Numerous Classifier (PCA, GA and ANN)

the various biometric methods, Face Recognition has become one of the most burning topic tasks in the pattern recognition field during the past decades. In This Work a Face Recognition System has been developed By applying different multiple classifier selection schemes on the output of three different classification methods namely Artificial Neural Network, Genetic Algorithm And Euclidean distance measure based on Principal Component Analysis dimensionality reduction technique. Dynamic classifier selection technique using classifier of local accuracy measurement is to look up the individual correctness of individual classifier and select the best one from them. Here it is proposed a classifier local accuracy measurement technique which is used to dynamic classifier selection algorithm. From the result and performance analysis it can be said that numerous classifier selection schemes give better performance than single classifier and dynamic classifier selection algorithm using proposed classifier local accuracy measurement technique gives stable high and better performance.

Forest species recognition based on dynamic classifier selection and dissimilarity feature vector representation

Multiple classifiers on the dissimilarity space are proposed to address the problem of forest species recognition from microscopic images. To that end, classical texture-based features such as Gabor filters, local binary patterns (LBP) and local phase quantization (LPQ), as well as two keypoint-based features, the scale-invariant feature transform (SIFT) and the speeded up robust features (SURF), are used to generate a pool of diverse classifiers on the dissimilarity space. A comprehensive set of experiments on a database composed of 2,240 microscopic images from 112 different forest species was used to evaluate the performance of each individual classifier of the generated pool, the combination of all classifiers, and different dynamic selection of classifiers (DSC) methods. The best result (93.03 %) was observed by incorporating probabilistic information in a DSC method based on multiple classifier behavior.

Dynamic classifier selection using spectral-spatial information for hyperspectral image classification

This paper presents a dynamic classifier selection approach for hyperspectral image classification, in which both spatial and spectral information are used to determine a pixel’s label once the remaining classified pixels’ neighborhood meets the threshold. For volumetric texture feature extraction, a volumetric gray level co-occurrence matrix is used; for spectral feature extraction, a minimum estimated abundance covariance-based band selection is used. Two hyperspectral remote sensing datasets, HYDICE Washington DC Mall and AVIRIS Indian Pines, are employed to evaluate the performance of the developed method. The classification accuracies of the two datasets are improved by 1.13% and 4.47%, respectively, compared with the traditional algorithms using spectral information. The experimental results demonstrate that the integration of spectral information with volumetric textural features can improve the classification performance for hyperspectral images.

Dynamic selection of classifiers—A comprehensive review

This work presents a literature review of multiple classifier systems based on the dynamic selection of classifiers. First, it briefly reviews some basic concepts and definitions related to such a classification approach and then it presents the state of the art organized according to a proposed taxonomy. In addition, a two-step analysis is applied to the results of the main methods reported in the literature, considering different classification problems. The first step is based on statistical analyses of the significance of these results. The idea is to figure out the problems for which a significant contribution can be observed in terms of classification performance by using a dynamic selection approach. The second step, based on data complexity measures, is used to investigate whether or not a relation exists between the possible performance contribution and the complexity of the classification problem. From this comprehensive study, we observed that, for some classification problems, the performance contribution of the dynamic selection approach is statistically significant when compared to that of a single-based classifier. In addition, we found evidence of a relation between the observed performance contribution and the complexity of the classification problem. These observations allow us to suggest, from the classification problem complexity, that further work should be done to predict whether or not to use a dynamic selection approach.

A Novel Dynamic Classifier Selection for Transmission Line Fault Location

A Novel Dynamic Classifier Selection for Transmission Line Fault Location

Dynamic classifier selection for One-vs-One strategy: Avoiding non-competent classifiers

The One-vs-One strategy is one of the most commonly used decomposition technique to overcome multi-class classification problems; this way, multi-class problems are divided into easier-to-solve binary classification problems considering pairs of classes from the original problem, which are then learned by independent base classifiers.The way of performing the division produces the so-called non-competence. This problem occurs whenever an instance is classified, since it is submitted to all the base classifiers although the outputs of some of them are not meaningful (they were not trained using the instances from the class of the instance to be classified). This issue may lead to erroneous classifications, because in spite of their incompetence, all classifiers' decisions are usually considered in the aggregation phase.In this paper, we propose a dynamic classifier selection strategy for One-vs-One scheme that tries to avoid the non-competent classifiers when their output is probably not of interest. We consider the neighborhood of each instance to decide whether a classifier may be competent or not. In order to verify the validity of the proposed method, we will carry out a thorough experimental study considering different base classifiers and comparing our proposal with the best performer state-of-the-art aggregation within each base classifier from the five Machine Learning paradigms selected. The findings drawn from the empirical analysis are supported by the appropriate statistical analysis.

A Novel Remote Sensing Image Classification Scheme Based on Data Fusion, Multiple Features and Ensemble Learning

In this paper, we propose a novel scheme to improve the accuracy of remote sensing image classification by integrating data fusion, multiple feature combination and ensemble learning. Intensity-Hue-Saturation (IHS), Gram-Schmidt (GS), Brovey and wavelet fusion methods are first performed to obtain the optimal fusion images of high resolution and multispectral images. Support Vector Machine (SVM) classifier is then adopted to classify the fused image with different feature sets, and ensemble learning algorithm based on dynamic classifier selection (DCS) is finally used to integrate multiple classification maps. The proposed classification scheme is implemented with three remote sensing data sets, obtaining the highest overall accuracy and kappa coefficient in all cases (92.63% and 0.8917 for BJ-1 data set, 81.89% and 0.7513 for Landsat TM and SPOT4 data set, 92.21% and 0.8838 for ALOS data set respectively). The experimental results show that the integration of data fusion, feature combination and ensemble learning improves the classification performance obviously and has great potential in practical uses.

Dynamic classifier ensemble model for customer classification with imbalanced class distribution

Customer classification is widely used in customer relationship management including churn prediction, credit scoring, cross-selling and so on. In customer classification, an important yet challenging problem is the imbalance of data distribution. In this paper, we combine ensemble learning with cost-sensitive learning, and propose a dynamic classifier ensemble method for imbalanced data (DCEID). For each test customer, it can adaptively select out the more appropriate one from the two kinds of dynamic ensemble approach: dynamic classifier selection (DCS) and dynamic ensemble selection (DES). Meanwhile, new cost-sensitive selection criteria for DCS and DES are constructed respectively to improve the classification ability for imbalanced data. We apply this method to a credit scoring dataset in UCI and a real churn prediction dataset from a telecommunication company. The experimental results show that the classification performance of DCEID is not only better than some static ensemble methods such as weighted random forests and improved balanced random forests, but also better than the existing DCS and DES strategies.

A probabilistic model of classifier competence for dynamic ensemble selection

The concept of a classifier competence is fundamental to multiple classifier systems (MCSs). In this study, a method for calculating the classifier competence is developed using a probabilistic model. In the method, first a randomised reference classifier (RRC) whose class supports are realisations of the random variables with beta probability distributions is constructed. The parameters of the distributions are chosen in such a way that, for each feature vector in a validation set, the expected values of the class supports produced by the RRC and the class supports produced by a modelled classifier are equal. This allows for using the probability of correct classification of the RRC as the competence of the modelled classifier. The competences calculated for a validation set are then generalised to an entire feature space by constructing a competence function based on a potential function model or regression. Three systems based on a dynamic classifier selection and a dynamic ensemble selection (DES) were constructed using the method developed. The DES based system had statistically significant higher average rank than the ones of eight benchmark MCSs for 22 data sets and a heterogeneous ensemble. The results obtained indicate that the full vector of class supports should be used for evaluating the classifier competence as this potentially improves performance of MCSs.

The build of a dynamic classifier selection ICBP system and its application to pattern recognition

In neural network ensemble, the diversity of its constitutive component networks is a crucial factor to boost its generalization performance. In terms of how each ensemble system solves the problem, we can roughly categorize the existing ensemble mechanism into two groups: data-driven and model-driven ensembles. The former engenders diversity to ensemble members by manipulating the data, while the latter realizes ensemble diversity by manipulating the component models themselves. Within a neural network ensemble, standard back-propagation (BP) networks are usually used as a base component. However, in this article, we will use our previously designed improved circular back-propagation (ICBP) neural network to establish such an ensemble. ICBP differentiates from BP network not only because an extra anisotropic input node is added, but also more importantly, because of the introduction of the extra node, it possesses an interesting property apart from the BP network, i.e., just through directly assigning different sets of values 1 and −1 to the weights connecting the extra node to all the hidden nodes, we can construct a set of heterogeneous ICBP networks with different hidden layer activation functions, among which we select four typical heterogeneous ICBPs to build a dynamic classifier selection ICBP system (DCS-ICBP). The system falls into the category of model-driven ensemble. The aim of this article is to explore the relationship between the explicitly constructed ensemble and the diversity scale, and further to verify feasibility and effectiveness of the system on classification problems through empirical study. Experimental results on seven benchmark classification tasks show that our DCS-ICBP outperforms each individual ICBP classifier and surpasses the performance of combination of ICBP using the majority voting technique, i.e. majority voting ICBP system (MVICBP). The successful simulation results validate that in DCS-ICBP we provide a new constructive method for diversity enforcement for ICBP ensemble systems.

A dynamic overproduce-and-choose strategy for the selection of classifier ensembles

The overproduce-and-choose strategy, which is divided into the overproduction and selection phases, has traditionally focused on finding the most accurate subset of classifiers at the selection phase, and using it to predict the class of all the samples in the test data set. It is therefore, a static classifier ensemble selection strategy. In this paper, we propose a dynamic overproduce-and-choose strategy which combines optimization and dynamic selection in a two-level selection phase to allow the selection of the most confident subset of classifiers to label each test sample individually. The optimization level is intended to generate a population of highly accurate candidate classifier ensembles, while the dynamic selection level applies measures of confidence to reveal the candidate ensemble with the highest degree of confidence in the current decision. Experimental results conducted to compare the proposed method to a static overproduce-and-choose strategy and a classical dynamic classifier selection approach demonstrate that our method outperforms both these selection-based methods, and is also more efficient in terms of performance than combining the decisions of all classifiers in the initial pool.

From dynamic classifier selection to dynamic ensemble selection

In handwritten pattern recognition, the multiple classifier system has been shown to be useful for improving recognition rates. One of the most important tasks in optimizing a multiple classifier system is to select a group of adequate classifiers, known as an Ensemble of Classifiers (EoC), from a pool of classifiers. Static selection schemes select an EoC for all test patterns, and dynamic selection schemes select different classifiers for different test patterns. Nevertheless, it has been shown that traditional dynamic selection performs no better than static selection. We propose four new dynamic selection schemes which explore the properties of the oracle concept. Our results suggest that the proposed schemes, using the majority voting rule for combining classifiers, perform better than the static selection method.