Proficiency testing for sensory ranking panels: measuring panel performance

Abstract

Proficiency testing in sensory analysis is an important step towards demonstrating that results from one sensory panel are consistent with the results of other sensory panels. The uniqueness of sensory analysis poses some specific problems for measuring the proficiency of the human instrument (panel). As part of an EU supported project, ProfiSens, 14 panels undertook ranking of sweetness on five samples of apple juice. Four panels were designated ‘validation’ panels, whose data were used firstly to establish the expected ranking results, and secondly to set the performance criteria that a trained sensory panel would be expected to achieve. Four key measures of a panel’s performance were investigated: the ability to rank the samples in the correct order; the significance level associated with differences between samples; the number of pairs of samples that a panel found to be different at a specified level of significance; and the degree of agreement between assessors within a panel. For each of these criteria an ‘expected result’ was considered, as well as an overall measure of performance. The data from the remaining panels were analysed, and the level of performance was recorded for each of the stated criteria. Results indicated different levels of performance, and the research also revealed the importance of choice of validation panels and the screening of samples prior to testing. A simpler performance scheme was proposed to address issues relating to attaching arbitrary weightings to each of the performance criteria, and to address potential problems associated with combining different measurement criteria into a single performance score. While there is still potential for further development and refinement, ProfiSens has made a significant contribution to proficiency testing for sensory analysis.