Validation of multivariate classification methods using analytical fingerprints – concept and case study on organic feed for laying hens

Abstract

Multivariate classification methods based on analytical fingerprints have found many applications in the food and feed area, but practical applications are still scarce due to a lack of a generally accepted validation procedure. This paper proposes a new approach for validation of this type of methods. A part of the validation procedure requires a description of qualitative aspects: the method’s goal and purpose and adequateness of the sample sets used. The required quantitative performance is assessed from probabilistic data. Probability distributions are generalized using kernel density estimates, which allow meaningful interpolation and direct comparison and combination of different distributions. We propose inclusion of a permutation test, and provide suggestions for the assessment of the analytical repeatability in the method’s probabilistic units. The latter can serve as a quality control measure. For assessment of the method’s overall performance, we propose to apply the combined cross validation and external validation set probability distributions in order to obtain the best estimate for the method’s performance on future samples. Qualitative and quantitative aspects are to be combined into a validation dossier stating performance for a well-defined purpose and scope. The proposed validation approach is applied to a case study: a binary classification discriminating organic from conventional laying hen feed based on fatty acid profiling that is essential to ensure the organic status of eggs for human consumption. For this case study, an expected accuracy for organic feed recognition of 96% is obtained for an explicitly defined scope.