A novel variable selection algorithm for multi-layer perceptron with elastic net

Abstract

The paper proposed a new iterative variable selection algorithm for multi-layer perceptron (MLP) to predict significant variables in complex industrial processes. The proposed algorithm starts from training an ordinary MLP with the original dataset. Following that, the elastic net (EN) algorithm is applied to perform the input weights shrinkage for the well-trained MLP, in which the optimal shrinkage parameters are calculated by grid search and cross-validation. On this basis, a local search strategy is combined to further optimize the input variable selection. The irrelative input variables will be deleted and a new dataset is constructed with remaining variables. This process is repeated until the prescribed termination rules are met. Benchmark datasets with different size and correlation and a practical air separation process are applied and comparisons with other algorithms are made to illustrate its performance. Statistical results show that the proposed algorithm can build more accurate model with fewer variables than other algorithms.