Combination Of Individual Classifiers Research Articles

MEGA: Predicting the best classifier combination using meta-learning and a genetic algorithm

Classifier combination through ensemble systems is one of the most effective approaches to improve the accuracy of classification systems. Ensemble systems are generally used to combine classifiers; However, selecting the best combination of individual classifiers is a challenging task. In this paper, we propose an efficient assembling method that employs both meta-learning and a genetic algorithm for the selection of the best classifiers. Our method is called MEGA, standing for using MEta-learning and a Genetic Algorithm for algorithm recommendation. MEGA has three main components: Training, Model Interpretation and Testing. The Training component extracts meta-features of each training dataset and uses a genetic algorithm to discover the best classifier combination. The Model Interpretation component interprets the relationships between meta-features and classifiers using a priori and multi-label decision tree algorithms. Finally, the Testing component uses a weighted k-nearest-neighbors algorithm to predict the best combination of classifiers for unseen datasets. We present extensive experimental results that demonstrate the performance of MEGA. MEGA achieves superior results in a comparison of three other methods and, most importantly, is able to find novel interpretable rules that can be used to select the best combination of classifiers for an unseen dataset.

Using Diversity for Classifier Ensemble Pruning: An Empirical Investigation

The concept of `diversity' has been one of the main open issues in the field of multiple classifier systems. In this paper we address a facet of diversity related to its effectiveness for ensemble construction, namely, explicitly using diversity measures for ensemble construction techniques based on the kind of overproduce and choose strategy known as ensemble pruning. Such a strategy consists of selecting the (hopefully) more accurate subset of classifiers out of an original, larger ensemble. Whereas several existing pruning methods use some combination of individual classifiers' accuracy and diversity, it is still unclear whether such an evaluation function is better than the bare estimate of ensemble accuracy. We empirically investigate this issue by comparing two evaluation functions in the context of ensemble pruning: the estimate of ensemble accuracy, and its linear combination with several well-known diversity measures. This can also be viewed as using diversity as a regularizer, as suggested by some authors. To this aim we use a pruning method based on forward selection, since it allows a direct comparison between different evaluation functions. Experiments on thirty-seven benchmark data sets, four diversity measures and three base classifiers provide evidence that using diversity measures for ensemble pruning can be advantageous over using only ensemble accuracy, and that diversity measures can act as regularizers in this context.

Comparative Analysis of Classifier Fusers

There are many methods of decision making by an ensemble of classifiers. The most popular are methods that have their origin in voting method, where the decision of the common classifier is a combination of individual classifiers’ outputs. This work presents comparative analysis of some classifier fusion methods based on weighted voting of classifiers’ responses and combination of classifiers’ discriminant functions. We discus different methods of producing combined classifiers based on weights. We show that it is not possible to obtain classifier better than an abstract model of committee known as an Oracle if it is based only on weighted voting but models based on discriminant function or classifier using feature values and class numbers could outperform the Oracle as well. Delivered conclusions are confirmed by the results of computer experiments carried out on benchmark and computer generated data.

An empirical study of binary classifier fusion methods for multiclass classification

One of the most important topics in information fusion is the combination of individual classifiers in multi-classifier systems. We have two different tasks in this area: one is the training and construction of ensembles of classifiers, with each one being able to solve the multiclass problem; the other task is the fusion of binary classifiers, with each one solving a different two-class problem to construct a multiclass classifier. This paper is devoted to the study of several aspects on the fusion process of binary classifiers to obtain a multiclass classifier. In the general case of a classification problem with more than two classes, we are faced with the issue that many algorithms either work better with two-class problems or are specifically designed for two-class problems. In such cases, a binarization method that maps the multiclass problem into several two-class problems must be used. In this task, information fusion plays a central role because of the combination of the prediction of the different binary classifiers into a multiclass classifier. Several issues regarding the way binary learners are trained and combined are raised by this task. Issues such as individual accuracy, diversity, and independence are common to other information fusion tasks such as the construction of ensembles of classifiers. This paper presents a study of the different class binarization methods for the various standard multiclass classification problems that have been proposed while addressing aspects not considered in previous works. We are especially concerned with many of the general assumptions in the field that have not been fully assessed by experimentation. We test the different methods in a large set of real-world problems from the UCI Machine Learning Repository, and we use six different base learners. Our results corroborate some of the previous results present in the literature. Furthermore, we present new results regarding the influence of the base learner on the performance of each method. We also show new results on the behavior of binary testing error and the independence of binary classifiers depending on the coding strategy. Finally, we study the behavior of the methods when the number of classes is high and in the presence of noise.

A hybrid approach for efficient ensembles

An ensemble of classifiers, or a systematic combination of individual classifiers, often results in better classifications in comparison to a single classifier. However, the question regarding what classifiers should be chosen for a given situation to construct an optimal ensemble has often been debated. In addition, ensembles are often computationally expensive since they require the execution of multiple classifiers for a single classification task. To address these problems, we propose a hybrid approach for selecting and combining data mining models to construct ensembles by integrating Data Envelopment Analysis and stacking. Experimental results show the efficiency and effectiveness of the proposed approach.

An adaptive strategy for the classification of g-protein coupled receptors

One of the major problems in computational biology is the inability of existing classification models to incorporate expanding and new domain knowledge. The problem of static classification models is addressed in this paper by the introduction of incremental learning for problems in boinformatics. Many machine learning tools have been applied to this problem using static machine learning structures such as neural networks or support vector machines that are unable to accommodate new information into their existing models. We utilize the fuzzy ARTMAP as an alternate machine learning system that has the ability of incrementally learning new data as it becomes available. The fuzzy ARTMAP is found to be comparable to many of the widespread machine learning systems. The use of an evolutionary strategy in the selection and combination of individual classifiers into an ensemble system, coupled with the incremental learning ability of the fuzzy ARTMAP is proven to be suitable as a pattern classifier. The algorithm presented is tested using data from the G-Coupled Protein Receptors Database and shows good accuracy of 83%. The system presented is also generally applicable, and can be used in problems in genomics and proteomics.