Cost-benefit analysis of calibration model maintenance strategies for process monitoring

Abstract

The long-term prediction performance of spectroscopic calibration models is a critical factor to monitor or control many production processes. Over time, new variations may emerge that deteriorate prediction performance. Therefore, models have to be maintained to retain or improve their prediction performance through time, requiring considerable resources and data. Maintenance should improve relevant predictions but also needs to be resource and cost efficient. Current approaches do not consider these trade-offs. We propose a new method to quantify the effectiveness and cost of model maintenance strategies based on historical data. Model performance over time for past, imminent and future samples is evaluated as these may react differently to maintenance. The model performance and required updating resources are translated into relative cost and benefit to compare strategies and determine optimal maintenance parameters. We used this method to evaluate a maintenance strategy that combines adding incoming samples to the calibration data with re-optimization of spectral preprocessing and modelling parameters. Continuously adding samples to the calibration data is shown to improve prediction performance and leads to more robust and generic models for emerging variations in all investigated data streams. Selectively adding incoming sample variations showed a reduced prediction performance but saves considerably in resources. Comparing model performance on the different sampling windows can also be used to determine an optimal updating frequency. This novel strategy to evaluate the expected performance and determine an optimal maintenance strategy is generally applicable and should lead to robust and consistently high prospective and/or retrospective model performance through time, which can be crucial for optimal operation and fault detection in industrial processes.