Evaluation of a rating-based variant of check-all-that-apply questions: Rate-all-that-apply (RATA)

Abstract

The current research explored the possibility of using attribute ratings as a variant (RATA: rate-all-that-apply) to CATA questions (check-all-that-apply), in order to improve sample description and discrimination and to engage participants in greater cognitive processing. The RATA question variant was implemented by asking participants, for the terms they ticked as “apply”, to indicate intensity (using a 3-pt scale with anchors ‘low’, ‘medium’ or ‘high’) or rate applicability (using a 5-pt scale anchored at ‘slightly applicable’ and ‘very applicable’). A total of four studies with 328 consumers were conducted. Studies 1–3 involved the consumption of products (milk desserts, bread and gummy lollies), whereas Study 4 was performed with yogurt labels. A between-subjects design was used in all studies to compare product characterizations from CATA and RATA questions. Across the four studies, compared to the simple CATA questions, the RATA variant led to an increase in the total number of selected terms and a small increase in the percentage of terms for which significant differences among samples were identified. Although the stability of sample and term configurations from CATA and RATA questions, calculated using a bootstrapping re-sampling approach, were similar, for two of the four studies RATA questions provided more stable sample and term configurations. Results from the present work reveal the potential of intensity-based CATA variants with consumers for sensory product characterization, but also suggest that these may be study and sample specific.