Variable and model selection method in linear regression for analysis

Abstract

When making decisions, probabilistic reasoning is used to utilize the known information to predict or determine those unobserved factors and variables that are crucial for the outcome. To find the relationship between outcome and one or more predictors, linear regression is commonly used in statistics and machine learning algorithms. In this article, the basic concept of linear regression and hypothesis testing are reviewed. The common modern methods for variable and model selection including Stepwise selection, Akaikes information criterion, Bayesian information criterion, and Mallows C_p are discussed and reviewed. Each method and criterion have its own uniqueness and limitation depending on the dataset and the purpose of analysis. Through discussing this, this paper aims to inform and explain each different method for variable and model selection in linear regression and provide information to help choose the most suitable methods to predict or find the relationship between response variables and the independent variables for analysis.