An improved normalized mutual information variable selection algorithm for neural networks

Abstract

In this paper, normalized mutual information feature selection (NMIFS) and tabu search (TS) are integrated to develop a new variable selection algorithm for neural networks. NMIFS is applied to select influential variables that contribute to the output variable, and avoids selecting redundant variables by calculating mutual information. TS based strategy is designed to prevent NMIFS from falling into local optimal situation. The proposed algorithm performs the variable selection by combining entropy information, mutual information, and validating error information of neural networks. Therefore, the algorithm proposed in this paper has more advantages than previous mutual information based variable selection algorithms. Several simulation datasets with different scale, correlation and noise are implemented to compare the performance of the proposed algorithm with several classical algorithms. The experiments show that developed algorithm present better model accuracy with fewer variables selected, compared to other state-of-art methods.