Equivalence and non‐inferiority tests using replicated discrimination and preference data

Abstract

AbstractUsing replications in discrimination tests is becoming more common in times of strict budgetary and time constraints. For the proof of differences, it is well‐known that the standard binomial test can be used. However, it is no longer applicable if the objective is to show equivalence or non‐inferiority, as potential differences among assessors (assessor heterogeneity/overdispersion) might invalidate the binomial test. We reapply ideas described earlier for the development of a confidence interval to derive a direct asymptotic test for equivalence or non‐inferiority using replicated discrimination and preference data, both for the cases of equal and unequal numbers of replications among assessors. The suggested test is largely model‐free, that is, does not require any assumptions that cannot be easily warranted by the test design and execution. At the same time, implementation is surprisingly easy by using the R code provided or any simple spreadsheet editor, or even manually.Practical ApplicationsShowing equivalence in perception between stimuli becomes increasingly important in applications, for example, in cost‐savings or product‐matching. The suggested approach is statistically valid without potentially doubtful model assumptions, yet at the same time simple and easy to use. Tables with critical values and R code for the evaluations further ease adoption, as illustrated by three small examples. The power assessments indicate that the loss in power is only moderate as long as the number of replications is not excessive, making replicate evaluations in discrimination tests a viable option for showing equivalence. The common approach of concluding for equivalence when a test for differences does not turn out to be statistically significant is heavily flawed; given that a valid yet simple approach to establish equivalence from replicated discrimination and preference data is provided here, such practice should be abandoned.