Panel performance checks on check-all-that-apply sensory evaluation, a focus on product discrimination

Abstract

Check-All-That-Applies (CATA) is a method commonly used for sensory and consumer tests. Within FrieslandCampina(FC), CATA is used to supplement quantitative descriptive methods for products’ characterization. Typically for quantitative descriptive methods, ANOVA is commonly used for panel performance checks to ensure the good quality of data (ISO 2012). However, data generated from CATA experiments are binary and the assumptions for ANOVA are violated, meaning alternatives should be considered. However, most studies in literature for the analysis of CATA data involve consumers, and we investigate whether the statistical treatment recommended could also be used for panel performance checks.4 separate datasets on dairy products profiled by the Singapore panel in FC were used. Each dataset had at least 9 panellists with one duplicate. The panel was evaluated on an overall level for the ability to discriminate between products using 3 different approaches; Non-parametric analysis using Cochran’s Q Test, Generalized Linear Mixed Effect Models(GLMEM) using ANOVA, and GLMEM using Logistic Regression.The p-values and the test statistics were compared. Cochran’s Q test provides good results but is sensitive to low elicitation rates. ANOVA is stable in results and interaction effects penalizes p-value. Logistic Regression has convergence issues and provides unstable results.