A genetic-based approach to features selection for ensembles using a hybrid and adaptive fitness function

Abstract

Recent researches on feature selection have been conducted in an attempt to find efficient methods for automatic selection of relevant features. The idea is to select a subset of attributes which are as representative as possible of the original data. Committees of classifiers, also known as ensemble systems, are composed of individual classifiers, organized in a parallel way and their output are combined in a combination method, which provides the final output of the system. In the context of these systems, feature selection methods can be used to provide different subsets of attributes for the individual classifiers, aiming to reduce redundancy among the attributes of a pattern and to increase the diversity in such systems. There are several methods to select features in ensembles systems and genetic algorithms (GA) is one of the most used methods. The main problem of using GA is the choice of the fitness function since the use of the ensemble accuracy means a complex and time consuming process and filter approaches may not reflect the real meaning of the solution. In this paper, we use feature selection via genetic algorithm to generate different subsets for the individual classifiers. In our proposal, we will used a hybrid and adaptive fitness function, in which we consider both approaches, filter and wrapper. In order to evaluate our proposal, experiments were conducted involving 10 different types of machine learning algorithms on 14 datasets. We will analyse the performance results of the proposed model compared with a genetic algorithm using a filter approach as well as the standard Bagging algorithm without feature selection.