Regression with adaptive lasso and correlation based penalty

Abstract

Feature selection for high-dimensional data is an important issue in machine learning, pattern recognition and bioinformatics fields. Feature selection algorithms are proposed to select the relevant feature subset from the original features. To adaptively identify the important highly correlated features from high-dimensional data which often beneficial to improve classification accuracy is a challenge. In this paper, we propose a regularized logistic regression with adaptive Lasso and correlation based penalty model to select informative highly correlated features adaptively. To incorporate significance of features into regression model, we first measure significance of each feature based on mutual information, and propose an adaptive weight construction strategy. Based on the adaptive weight construction strategy, the proposed adaptive logistic regression can impose a large amount of penalty on irrelevant features, and thus noise features are easily removed from the model and remain the informative features. The experimental results on the simulation and real-world datasets demonstrate the effectiveness and the superiority the proposed model by comparing it to existing competing regularized logistic regression models.