Supervised feature selection by constituting a basis for the original space of features and matrix factorization

Abstract

Most of existing research works in the field of feature selection via matrix factorization techniques have been employed for unsupervised learning problems. This paper introduces a new framework for the supervised feature selection, called supervised feature selection by constituting a basis for the original space of features and matrix factorization (SFS-BMF). To this end, SFS-BMF is a guided search to find a basis for the original space of features that inherently contains linearly independent features and can be replaced with the original space. For finding the best subset of features regarding the class attribute, information gain is utilized for the process of constructing a basis. In fact, a basis for the original features is constructed according to the most informative features in terms of the information gain. Then, this basis is decomposed through a matrix factorization form in order to select a subset of features. Our proposed method guarantees the maximum relevancy of selected features to the output by using the information gain while simultaneously secures the minimum redundancy among them based on the linear independence property. Several experiments on high-dimensional microarray datasets are conducted for illustrating the efficiency of SFS-BMF. The experimental results show that the proposed SFS-BMF method outperforms some state-of-the-art feature selection methods with respect to classification performance and also according to the computational complexity.