Nonlinear error compensation algorithm for signal resolution of chromatic confocal measurements

Abstract

The mechanisms of the nonlinear error, primarily stemming from the deviation of the polynomial model fitting the wavelength-distance mapping relationship in chromatic confocal measurement, have been elucidated. A compensation method is proposed for the nonlinear error, based on a regression tree model. Through calibration experiments using our chromatic confocal sensor, an impressive improvement of over 40 % in precision could be achieved by our proposed compensation method, which advances from ±1 μm to ±0.6 μm within a 4 mm measurement range.