Automated tuning of denoising algorithms for noise removal in chromatograms

Abstract

Different algorithms, such as the Savitzky-Golay filter and Whittaker smoother, have been proposed to improve the quality of experimental chromatograms. These approaches avoid excessive noise from hampering data analysis and as such allow an accurate detection and quantification of analytes. These algorithms require fine-tuning of their hyperparameters to regulate their roughness and flexibility. Traditionally, this fine-tuning is done manually until a signal is obtained that removes the noise while conserving valuable peak information. More objective and automated approaches are available, but these are usually method specific and/or require previous knowledge.In this work, the L-and V-curve, k-fold cross-validation, autocorrelation function and residual variance estimation approach are introduced as alternative automated and generally applicable parameter tuning methods. These methods do not require any previous information and are compatible with a multitude of denoising methods. Additionally, for k-fold cross-validation, autocorrelation function and residual variance estimation, a novel implementation based on median estimators is proposed to handle the specific shape of chromatograms, typically composed of alternating flat baselines and sharp peaks. These tuning methods are investigated in combination with four denoising methods; the Savitsky-Golay filter, Whittaker smoother, sparsity assisted signal smoother and baseline estimation and denoising using sparsity approach.It is demonstrated that the median estimators approach significantly improves the denoising and information conservation performance of relevant smoother-tuner combinations up to a factor 4 for simulated datasets and even up to a factor 10 for an experimental chromatogram. Moreover, the parameter tuning methods relying on residual variance estimation, k-fold cross-validation and autocorrelation function lead to similar small root-mean squared errors on the different simulated datasets and experimental chromatograms. The sparsity assisted signal smoother and baseline estimation and denoising using sparsity approach, which both rely on the use of sparsity, systematically outperform the two other methods and are hence most appropriate for chromatograms.