Logistic local hyperplane-Relief: A feature weighting method for classification

Abstract

Relief-based algorithms have been widely used for feature selection because of their low computational cost and high accuracy. However, the available Relief-based algorithms have their limitations. To improve the performance of Relief-based methods further, we propose a novel feature selection algorithm based on the logistic iterative-Relief (LI-Relief) and local hyperplane-Relief (LH-Relief) methods, called logistic local hyperplane-based Relief (LLH-Relief). LLH-Relief uses local learning to find neighbor representations for given samples and learns feature weights by solving the optimization problem with logistic regression and the ℓ1-norm regularization terms. To demonstrate the validity and the effectiveness of LLH-Relief for feature selection in supervised learning, we perform extensive experiments on toy and real-world datasets. Experimental results indicate that LLH-Relief is very promising.