Bayesian analysis of homogeneity studies in the production of reference materials

Abstract

For almost two decades, the batch homogeneity in the production of reference materials has been evaluated using analysis of variance (ANOVA) to determine the between-bottle standard deviation. This approach replaced at that time the use of the F-test in ANOVA to determine whether the ratio of the mean squares $${MS}_{\mathrm {between}}/{MS}_{\mathrm {within}}$$ is statistically significant. Problems arise when $${MS}_{\mathrm {between}} < {MS}_{\mathrm {within}}$$ , because classical ANOVA provides a negative between-bottle variance, which is then often set to zero. By using a Bayesian hierarchical model, based on the same assumptions as traditional ANOVA, we show that even if $${MS}_{\mathrm {between}} < {MS}_{\mathrm {within}}$$ , there can be a relevant level of between-bottle inhomogeneity to account for. The Bayesian analysis produces a nonzero value for the between-bottle standard deviation, which dismisses the practice of setting this standard deviation to 0. At the same time, it dismisses the current guidance given in ISO Guide 35 under these circumstances. Finally, it is shown that traditional ANOVA, meta-analysis methods and Bayesian analysis give very similar answers as long as $${MS}_{\mathrm {between}} > {MS}_{\mathrm {within}}$$ , so there is no need to discourage using these methods in favour of a Bayesian analysis, provided that the repeatability of the measurement method used to conduct the between-bottle homogeneity study is sufficient to characterise the dispersion across the bottles (items) in a batch of a reference material.