Linearity of calibration curves: use and misuse of the correlation coefficient

Abstract

The correlation coefficient is commonly used to evaluate the degree of linear association between two variables. However, it can be shown that a correlation coefficient very close to one might also be obtained for a clear curved relationship. Other statistical tests, like the Lack-of-fit and Mandel’s fitting test thus appear more suitable for the validation of the linear calibration model. A number of cadmium calibration curves from atomic absorption spectroscopy were assessed for their linearity. All the investigated calibration curves were characterized by a high correlation coefficient (r >0.997) and low quality coefficient (QC <5%), but the straight-line model was systematically rejected at the 95% confidence level on the basis of the Lack-of-fit and Mandel’s fitting test. Furthermore, significantly different results were achieved between a linear regression model (LRM) and a quadratic regression (QRM) model in forecasting values for mid-scale calibration standards. The results obtained with the QRM did not differ significantly from the theoretically expected value, while those obtained with the LRM were systematically biased. It was concluded that a straight-line model with a high correlation coefficient, but with a lack-of-fit, yields significantly less accurate results than its curvilinear alternative.