Complexity Measures Effectiveness in Feature Selection

Abstract

Feature selection is an important pre-processing step usually mandatory in data analysis by Machine Learning techniques. Its objective is to reduce data dimensionality by removing irrelevant and redundant features from a dataset. In this work we investigate how the presence of irrelevant features in a dataset affects the complexity of a classification problem solution. This is performed by monitoring the values of some complexity measures extracted from the original and preprocessed datasets. These descriptors allow estimating the intrinsic difficulty of a classification problem. Some of these measures are then used in feature ranking. The results are promising and reveal that the complexity measures are indeed suitable for estimating feature importance in classification datasets.