Information-theoretic partially labeled heterogeneous feature selection based on neighborhood rough sets

Abstract

With the rapid increase of large-scale, real-world datasets, it becomes critical to address the problem of partially labeled heterogeneous feature selection (i.e., some samples, which own numerical and categorical features, have no labels). Existing solutions typically adopt linear correlations between features. In this paper, three different monotonic uncertainty measures are defined on equivalence classes and neighborhood classes to study the partially labeled heterogeneous feature selection by exploring the nonlinear correlations. First, consistent entropy and monotonic neighborhood entropy, based on classical rough set theory and neighborhood rough set theory, are proposed to construct a uniform measure for feature selection in heterogeneous datasets. Furthermore, a maximal neighborhood entropy strategy is developed by considering the inconsistency of neighborhood classes described by the features and partial labels. Finally, two feature selection algorithms are presented by three novel monotonic uncertainty measures. The comparative experiments demonstrate the effectiveness and superiority of the newly proposed feature selection measures.