Multiple Regressions

Abstract

AbstractThe purpose of the simple regression is to study the relationship between one explanatory variable and one dependent variable. The purpose of a multiple regression (the term was first used by Pearson and Lee (1908)) is to learn about the relationships between several explanatory variables and a dependent variable. The extension of the model from one explanatory variable into several explanatory variables introduces several complications. For example, in a multiple regression setting one has to consider the effects of the relationships among the explanatory variables on the estimates. On the other hand, an advantage is that one can mix the regression methodologies used (i.e., apply different regression methodologies to different explanatory variables). In this chapter we will be mainly interested in methods of multiple regressions that are based on the simple regression coefficients. By “based on” we mean not only that the multiple regression coefficients are derived by the same principle that is used to derive the simple regression coefficients but also that the simple regression coefficients are used as the building blocks of the multiple regression coefficients. As such, one can learn about their properties from the properties of the simple coefficients. In particular, we have shown in Chap. 7 that the Ordinary Least Squares (OLS) and semi-parametric Gini regression estimators can be interpreted as the slopes of the linear approximations to a regression curve, because they are based on weighted averages of slopes defined between adjacent observations. In other words, the linearity assumption on the regression curve is not used in the estimation stage. This property continues to hold in our extension into the multiple regression case. However, we do introduce some kind of a linearity requirement. The linearity requirement differs from the linearity assumption on the model because it is imposed on the set of equations that are used to derive the multiple regression coefficients, as will be seen below.KeywordsExplanatory VariableOrdinary Little SquareInstrumental VariableQuantile RegressionRegression CurveThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.