Feature selection via max-independent ratio and min-redundant ratio based on adaptive weighted kernel density estimation

Abstract

Feature selection based on entropy structure can be roughly divided into two types according to whether they are related to kernel density estimation (KDE). The first type is feature selection based on non-KDE entropy. The second type is feature selection based on KDE entropy. Compared with the first type, the second type avoids discretization and obtains more accurate mutual information when handling continuous data. However, existing feature selection methods based on KDE entropy neglect the fact that samples with noise have negative impacts on KDE. Besides, the feature evaluation functions don’t effectively assess relevance and redundancy of features. Thus, a feature selection method via maximizing independent and minimizing redundant classification information ratio is constructed based on adaptive weighted kernel density estimation. Firstly, an adaptive weighted kernel density estimation model is designed. Secondly, the entropy structure is defined by the adaptive weighted kernel density estimation model, and their theoretical properties are explored. Thirdly, a feature selection algorithm via maximizing independent and minimizing redundant classification information ratio is designed from the viewpoint of adaptive weighted kernel density estimation. Finally, comprehensive experiments are performed. The results illustrate that our approach has certain robustness, validity and superiority compared with other representative feature selection approaches.