Controlled product testing in the home - implications for sample size

Abstract

Consumer product testing in the laboratory or using a Central Location Test (CLT) is a common approach to collect consumer responses to multiple products. The Covid-19 pandemic has challenged companies to adapt, with both sensory and consumer testing in home becoming a common way of working. Moving to the home for controlled product tests brings with it both practical and statistical considerations.To investigate the question, ‘What are the sample size implications of moving from traditional CLTs to Controlled Home Tests (CHT)?’, 245 datasets from 16 organisations and covering 19 countries were combined to undertake a meta-analysis. Consumer tests were mainly in the food category, included an overall liking question and covered both monadic and sequential monadic tests.Test noise variation was examined looking at the impact of test type, number of products and number of respondents. The CHT was significantly less variable than the traditional CLT (Lab or Hall) with a greater than 9% drop in noise variation. It was also found that noise variation increases with number of products tested in sequential monadic testing. As a result, recommendations on appropriate consumer sample sizes for different scenarios were calculated.