Weighted-Features Construction as a Bi-level Problem

Abstract

Feature selection and construction are important pre-processing techniques in machine learning and data mining. They may allow not only dimensionality reduction but also classifier accuracy and efficiency improvement. Feature selection aims at selecting relevant features from the original feature set, which could be less informative to achieve good performance. Feature construction may work well as it creates new highlevel features, but these features do not have the same degree of importance, which makes the use of weighted-features construction a very challenging topic. In this paper, we propose a bi-level evolutionary approach for efficient feature selection and simultaneous feature construction and feature weighting, called Bi-level Weighted-Features Construction (BWFC). The basic idea of our BWFC is to exploit the bi-level model for performing feature selection and weighted-features construction with the aim of finding an optimal subset of features combinations. Our approach has been assessed on six high-dimensional datasets and compared against three existing approaches, using three different classifiers for accuracy evaluation. Experimental results show that our proposed algorithm gives competitive and better results with respect to the state-of-the-art algorithms.