Rate-all-that-apply (RATA) with semi-trained assessors: An investigation of the method reproducibility at assessor-, attribute- and panel-level

Abstract

Rate-all-that-apply (RATA) is a variant of check-all-that-apply (CATA) questions that allows assessor to rate the intensity of selected attributes. Compared to CATA, RATA has the potential to improve sample description and discrimination, and might be more useful when only a small number of assessors are available. Before advocating its use with confidence, investigations on the method validity and reproducibility are necessary.Within this context, this short paper examined the reproducibility of results obtained by RATA within a test–retest paradigm, drawing on data from a case study involving sensory assessment of common defects in chocolate production. Criteria considered were within-assessors reproducibility, attribute stability, and configurational agreement between samples spaces obtained across replicated evaluations. The results showed that although within-assessors reproducibility was moderate, RATA exhibited a very good reproducibility at panel level, as indicated by the high configurational agreement between product maps obtained from individual replicates. The method showed a good reproducibility also at the level of individual attributes. Indications were obtained that the reproducibility of RATA with semi-trained subjects might be similar to that of a simple checklist, in spite of the addition of the intensity rating step.Overall, the work presented in this short paper supports the validity of RATA as a sensory profiling tool, and suggests that its application with semi-trained assessors may be particularly advantageous for industrial applications.