Least-Squares Analysis of Data with Uncertainty in y and x: Algorithms in Excel and KaleidaGraph

Abstract

For the least-squares analysis of data having multiple uncertain variables, the generally accepted best solution comes from minimizing the sum of weighted squared residuals over all uncertain variables, with, for example, weights in xi taken as inversely proportional to the variance σxi2. A complication in this method, here called “total variance” (TV), is the need to calculate residuals δi in every uncertain variable. In x–y problems, that means adjustments must be obtained for x as well as for the customary y. However, for the straight-line fit model, there is a simpler procedure, a version of effective variance (EV) methods, that requires only the residuals in y and agrees exactly with the TV method. Furthermore, Monte Carlo calculations have shown that this EV2 method is statistically comparable to the TV method for many common nonlinear fit models. This method is easy to code for computation in Excel and programs like KaleidaGraph, as is illustrated here for several examples, including implicit nonli...