Ensemble selector mixed with pareto optimality to feature reduction

Abstract

Among various tasks of feature reduction, how to search qualified features and then construct the corresponding reduct which satisfies a given constraint is a hot topic. Constraint is the pre-defined condition in feature reduction, it rules two crucial aspects: 1) when the process of searching can be terminated to construct reduct; 2) whether a feature/set of features should be added into reduct pool or not. Obviously, such a task of feature reduction is constraint-dependent. However, some limitations may naturally emerge: 1) constraint-independent features are ignored; 2) fixed constraint hinders the diverse evaluations of features; 3) selected features based on single constraint are powerless to data perturbation. Therefore, an Ensembler Mixed with Pareto Optimality (EmPo) is developed in this study. Firstly, the principle of Restricted Pareto Optimality is proposed to identify constraint-independent features before performing constraint related searching, which indicates a two-stages strategy to feature reduction. Naturally, diverse evaluations of features are achieved in such a process. Secondly, a data perturbation w.r.t. either sample or feature aspect is employed to obtain multiple reductions of features. The objective is to further improve the stability of the classification results based on those multiple reducts. It should also be emphasized that EmPo is a general framework, and most existing approaches to feature reduction can be embedded into it to further improve their performances. Finally, the effectiveness of our EmPo is validated over 20 datasets within 4 different ratios of noisy label.