A nonlinear mixed–integer programming approach for variable selection in linear regression model

Abstract

Modern statistical studies often encounter regression models with high dimensions in which the number of features p is greater than the sample size n. Although the theory of linear models is well–established for the traditional assumption p < n, making valid statistical inference in high dimensional cases is a considerable challenge. With recent advances in technologies, the problem appears in many biological, medical, social, industrial, and economic studies. As known, the LASSO method is a popular technique for variable selection/estimation in high dimensional sparse linear models. Here, we show that the prediction performance of the LASSO method can be improved by eliminating the structured noises through a mixed–integer programming approach. As a result of our analysis, a modified variable selection/estimation scheme is proposed for a high dimensional regression model which can be considered as an alternative of the LASSO method. Some numerical experiments are made on the classical riboflavin production and some simulated data sets to shed light on the practical performance of the suggested method.