On some variable selection procedures based on data for regression models

Abstract

We discuss variable selection procedures in regression models based on large data sets. Our purpose is to discover the pattern of the association between the dependent variable and the independent variables. The real-valued variable Y is the response (dependent) variable. The variables in the vector X ̲ are the predictor (independent) variables, which may be either ordered or categorical. A function d ( X ̲ ) is defined on the measure space χ taking on real values. In regression models, predictors have been constructed using a parametric approach under the assumption that E ( Y | X ̲ = x ̲ ) = d ( x ̲ , θ ̲ ) , where d has known functional form depending on x ̲ and a finite set of parameters θ ̲ = ( θ 1 , θ 2 , … , θ m ) . Then θ ̲ is estimated by the least squares method. In practical applications, however, the functional form of d is usually unknown. In such a situation, it is difficult to determine the regression function and we use Classification And Regression Trees (CART) methods that integrate the data and the model. We propose selection procedures to select important predictor variables in the regression model based on data. Some criteria for selecting the important variables are discussed. An empirical study based on an annual survey of inbound visitors in Taiwan is provided to illustrate the implementation of our multiple decision procedure.