Extracting salient information from discarded features via attribute selection and pruning

Abstract

This paper introduces a novel two-step proposal for attribute subset selection. The first step is able to extract handy information from non-selected features after an initial feature subset selection to be added to the preliminary subset selected; for this step the feature subset selector relies on the Correlation-based Feature Selection (CFS). The second step tries to prune the solution given that in the first step some extra attributes out of initially non-selected features have been added to the relevant features; the ultimate goal of this pruning is to optimise the attribute subset prior to conduct a classification task. In accordance with the experimental results, the simplification of solutions is effective since an improvement takes place in most of the cases where the new method does determine some attributes to be removed. Some performance measures are reported within a test bed composed of a good number of binary and multi-class classification problems. Comparisons with other attribute selection procedures such as CFS, Fast Correlation-based Feature Selection and Gain Ratio revealed that the new approach is very competitive and may be taken into account by the data preparation community.