Comparison of Forward Selection, Backward Elimination, and Generalized Simulated Annealing for Variable Selection

Abstract

In order to obtain quantitative information, it is often necessary for a chemist to employ regression methods. An algorithm is described for determining the optimal subset of variables which gives the best prediction in regression analysis. The procedure is based on the generalized simulated annealing method (GSA) of optimization. From a comparison study with standard methods of variable subset selection by forward selection and backward elimination, GSA is found to perform better. Three data sets are used for distinction purposes. Two of the data sets are relatively small, allowing comparison to the global variable subset obtained by computing all possible variable combinations.