Multiple linear regression with correlated explanatory variables and responses

Abstract

Different from the traditional linear regression model that captures only the errors of dependent variables (responses), this contribution presents a new multiple linear regression model where, besides the errors of responses, the errors of explanatory variables and their correlations with response errors are rigorously taken into account. The new regression model is typically a non-linear errors-in-variables (EIV) model, which is referred to as the error-affected and correlated linear regression (ECLR) in this paper. Considering the fact that only part of elements in design matrix A of the regression model are random, the authors express error matrix EA of A as a function of EX consists of all non-zero random errors. Then, the authors can easily formulate the stochastic model without the effect of non-random elements in A. An iterative solution is derived based on the Euler–Lagrange minimisation problem for ECLR. The authors further show that ECLR is very general and some of the existing linear regression methods, the ordinary least squares (OLS), the total least squares (TLS) and the weighted total least squares (WTLS), are the special cases. The experiments show that the ECLR method generally has a better performance than the OLS, TLS and WTLS methods in terms of the difference between the solution and the true values when the explanatory variables and responses are significantly correlated.