Model selection challenges with application to multivariate calibration updating methods

Abstract

AbstractAn important issue in multivariate calibration including model updating methods with multiple tuning parameters is selection of final models. Model updating is an adaption process where models are updated from predicting in primary sample and measurement conditions to predict the analyte in new secondary conditions. A single process to select models (tuning parameter values) with satisfactory bias–variance trade‐offs across multiple data sets and modeling methods is challenging. This paper reports on evaluating the consistency of a collection of model quality measures to select models across five near‐infrared (NIR) data sets for three calibration updating approaches. The goal is to formulate a reliable model selection process that is nearly data and model updating method independent. Two of the three model updating approaches require primary and secondary analyte reference values, and the third only needs primary reference values (unlabeled relative to secondary). However, all model selection methods considered do require secondary samples with reference values. It is found that which evaluated model quality measure to use depends on the degree of spectral similarity between primary and secondary spectra as characterized by the indicator of spectral uniqueness measure developed in this paper. From the results presented, more work is needed to better characterize model selection dependency on model quality measures, number of samples and respective inherent compositions (data set‐dependent matrix effects), and tuning parameter ranges.