Ordinal preserving matrix factorization for unsupervised feature selection

Abstract

Feature selection aims to remove the irrelevant and redundant features to reduce the dimensionality of data and increase the efficiency of learning algorithms. Specifically, unsupervised feature selection without any label information has become a challenging and significant task in machine learning applications. In this paper, a novel algorithm called Ordinal Preserving Matrix Factorization (OPMF), which incorporates matrix factorization, ordinal locality structure preserving and inner-product regularization into a unified framework, is proposed for feature selection. The advantages of our algorithm are three-fold. First, the ordinal locality property of original data is preserved by introducing a triplet-based loss function to the selected features, which is of great importance for distance-based classification and clustering tasks. Second, an inner product regularization term is incorporated into the proposed framework, so that the selected features obtained by our OPMF can be sparse and low redundant. Third, a simple and efficient iteratively updating algorithm is derived to solve the objective function of the proposed algorithm. Extensive experimental results on six datasets demonstrate that the proposed OPMF can obtain competitive performance compared to the existing state-of-the-art unsupervised feature selection approaches.