Biases in prediction by regression for certain incompletely specified models

Abstract

The technique of regression analysis has been in general use for many years. Its wide acceptance has led people in many fields to adjust the technique in various ways to answer questions which are outside its field of applicability. Thus, an engineer might feel that the tensile strength of cement depends on the ratio of cement to aggregate, amount of water used, temperature and humidity of surrounding air during hardening, pressure applied, coarseness of the aggregate used, etc. To estimate the relationship of strength to the other variables mentioned he might very well assume a linear regression model, perform an experiment or series of experiments, and use the usual least squares technique to estimate the unknown coefficients in the model. At some point in this procedure he may decide that all the independent variables or predictors he has included in his model are not really necessary to predict tensile strength 'accurately'. To decide the point objectively he tests the hypothesis (or hypotheses) that the coefficients of the doubtful predictors are equal to zero. If he accepts the hypothesis, he simply deletes the corresponding predictors from his model and, if he rejects the hypothesis, he retains the doubtful variables in his model. Similarly, a chemist might use the same type of procedure in deciding which of a group of predictors is necessary to predict accurately the purity of a chemical resulting from a given reaction. A psychologist might use the same procedure to decide which of a battery of test scores are necessary to predict an individual's 'success' in a given job. Many examples could be quoted of possible applications of the above-mentioned adjustment of the usual regression technique. Similarly, many different types of adjustments (that is, adapting regression techniques to answer different types of questions) could also be mentioned. (See, for example, Fireman & Wadleigh, 1951; Hollingsworth, 1959; Peperzak, 1956; Summerfield & Lubin, 1951.) Since this paper is concerned only with the particular technique already mentioned, no reference will be made to other possible adaptations. Bancroft (1944) was one of the first to consider the impact of preliminary tests of significance on subsequent estimation. His work was concerned with the bias introduced into an estimate of a regression coefficient, if a second coefficient is tested to be zero. Mosteller (1948) investigated the problem of pooling two means in estimating a population mean, using a preliminary test of significance to make the decision. Kitagawa (1951) obtained the distribution function and moments of the estimator studied by Bancroft and of a pooled estimator of a mean. He later (1959) studied the effect of certain preliminary tests of significance, to decide how many predictors should be included in a predictand, on the