Impact of simultaneous omission of a variable and an observation on a linear regression equation

Abstract

The impact of omitting a variable on a regression equation has a venerable history. One of the earliest references is Cochran (1938). The study of the influence of an observation on a regression equation is of more recent origin but has been extensively studied; see e.g., Cook (1977), Andrews and Pregibon (1978), Hoaglin and Welsch (1978), Belsley, Kuh, and Welsch (1980), Draper and John (1981), Velleman and Welsch (1981), and Cook and Weisberg (1982). In this article we attempt to study the joint impact on a regression equation of simultaneous omission of a variable and an observation. Each observation affects the fitted regression equation differently and has a different influence on each variable. We elucidate this interrelationship among variables and observations and examine its impact on the fitted regression equation. Closed form analytical expressions for the impact of simultaneous omission of a variable and an observation on the residual sum of squares, the fitted values, and the ith predicted value are derived. We also give a statistic for testing the significance of the jth variable when the ith observation is omitted. Our analysis shows explicity the direction and magnitude of the impact of deleting the ith observation on the jth regression coefficient. The method of analysis we present has considerable significance for variable selection problems and can unearth situations where a single observation is instrumental in retaining (removing) a variable in (from) a multiple regression equation. An example using a real-life data set is used to illustrate the new methodology.