The Choice of Variables in Multiple Regression

Abstract

Summary This paper is concerned with the analysis of data from a multiple regression of a single variable, y, on a set of independent variables, x 1,x 2,...,xr. It is argued that the form of the analysis should depend on the use that is to be made of the regression, and that therefore an approach employing ideas from decision theory may be worth while. Two situations are analysed in this way: in the first it is desired to predict a future value of y; in the second we wish to control y at a preassigned value. The two analyses are found to be different: in particular, the standard errors of the regression coefficients are found to be irrelevant in the prediction problem, but not in the control problem. In the former it is shown that, under rather special assumptions on the multiple regression experiment, the analysis is similar to that recommended by other writers. If the costs of control do not depend on the values at which the control takes place, a similar analysis holds for the second problem. The approach throughout is Bayesian: there is no discussion of this point, I merely ask the non-Bayesian reader to examine the results and consider whether they provide sensible and practical answers.