The Impact of Random Predictors on Comparisons of Coefficients Between Models: Comment on Clogg, Petkova, and Haritou

Abstract

As Clogg, Petkova, and Haritou (1995) correctly observe, regression analysts are often intensely concerned with what happens to the coefficient of a predictor variable when additional variables are introduced into a regression model. Unfortunately, this concern is typically expressed in comparisons that lack any measure of statistical reliability. To remedy this deficiency, Clogg, Petkova, and Haritou (hereafter CPH) propose a set of methods for testing whether the change in a regression coefficient (or set of coefficients) is statistically significant. These methods have the virtues of simplicity and applicability to a wide class of generalized linear regression models. CPH deserve credit for identifying an important but overlooked problem, and their solutions are both clever and elegant. Nevertheless, I believe that their proposed methods suffer from a fundamental flaw: they make unrealistic assumptions about the sampling properties of the predictor variables. Not surprisingly, this strategy leads to a substantial simplification of their methodology. But in doing so, it exposes the analyst to the risk of highly misleading conclusions. My main objective is to explain why I think their assumptions are problematic and to suggest what may go wrong as a consequence. I shall focus primarily on the three-variable linear model since it embodies all the critical issues without the distracting complications of the multivariable and nonlinear cases. And if the methods are deficient in the threevariable case, there is little point in generalizing them to more complicated situations. I shall also suggest some alternative methods for both three-variable and multivariable linear models.