Linear Regression with Non-Constant, Unknown Error Variances: Sampling Experiments with Least Squares, Weighted Least Squares and Maximum Likelihood Estimators

Abstract

The theory of simple linear regression is extended to the case of non-uniform error variances for the situation in which replicates are available at each sample point in the domain of the independent variable. The performance of least squares (LS), weighted least squares (WLS), and maximum likelihood (ML) estimators of the regression parameters was compared in sampling experiments in which the error variance varied by as much as 16-fold over the domain of the independent variable and for both normal and log-normal distributions for the error. The LS estimators performed significantly better than the others when few replicates were available at each of many sampling points. The WLS estimators improved as the number of replicates increased and were clearly superior when the number of replicates per point exceeded the number of points in the independent variable. The ML estimates were usually close to the WLS estimates but the WLS estimates were more often the better of the two. A first order approximation for the estimators of variances of the WLS estimators is derived as well as an iterative method for finding the ML estimators.