Hierarchical Bayesian analysis of true discrimination rates in replicated triangle tests

Abstract

Several methods for testing the null hypothesis in replicated triangle tests are known and frequently used. Deriving further reliable information about the distribution of discrimination rates can be more difficult due to possible assessor heterogeneity, though. In contrast to unreplicated tests, it cannot be assumed that an assessor has truly discriminated either never or always. To overcome this problem, we use a Bayesian hierarchical model to estimate the distribution of the true discrimination rates. For the calculations, we apply a Monte Carlo Markov Chain (MCMC) sampler. An example employing 30 assessors with 3–10 replications each indicates that roughly about 1/3 of the panelists is not able to differentiate between the products, while another third has most likely a noteworthy discrimination probability. The last third is somewhere in between, with some chance that they can hardly discriminate, but also some chance that they indeed have a positive discrimination rate between 0.1 and 0.5. The distribution of these probabilities as well as of the mean discrimination rate can be estimated from the model. In our example, the average true discrimination rate is estimated as 0.32 with a corresponding 95% confidence interval of [0.23; 0.41]. This suggests that the average probability for the difference between the products being truly perceived lies between 23% and 41%. General results indicate that at least 5 or 6 replications are needed to reasonably approximate individual panelists’ behaviour in triangle tests.