Bland and Altman agreement method: to plot differences against means or differences against standard? An endless tale?

Abstract

In the Bland and Altman analysis of agreement studies, there is some controversy whether "to plot the differences between the Standard/actual measurement method and the test/new measurement method against their mean" or "to plot the differences against the standard method". Of course, this is not just a "graphic dispute" as a regression model is inherent in the graphical choice to test the proportional and systematic biases. We revised two relevant papers claiming to plot the differences against the standard and outlined their pitfalls taking into account the underlying statistical methodology. Furthermore, we have considered the conditions (correlation between the two measurement methods and ratio of their variances) leading correlation coefficient and regression slope between differences and means or differences and standard different from zero. We have shown the situations in which the regression slope and the correlation coefficient calculated from the differences and means according to Bland and Altman approach or calculated from the differences and standard are closer to zero giving the minimum possible value of spurious proportional error between the two methods. We highlighted how the calculation of the expected values of the correlation coefficients and, above all, of the regression slope can be very useful for choosing thestatistical model in the context of an agreement study between two measurement methods. Finally, we outlined some recommendations for understanding the real possibility of carrying out agreement or calibration studies.