Implementation of regularization for separable nonlinear least squares problems

Abstract

This paper considers a class of separable nonlinear least squares problems in which a model can be represented as a linear combination of nonlinear functions. A regularized nonlinear parameter optimization approach is presented for coping with the potential ill-conditioned problem of parameter divergence. Together with a regularization parameter detection technique, Tikhonov regularization and truncated singular value decomposition are utilized in the estimation of the linear parameters if the nonlinear parameters are changed during the parameter optimization process, which centers on a nonlinear parameter search using the Levenberg-Marquardt algorithm. Benefiting from the regularization in parameter optimization, the potential ill-conditioned issue can be avoided, and the multi-step-ahead forecasting accuracy of the estimated model may be largely improved. The usefulness of this approach is illustrated by means of a chaotic time-series prediction and nonlinear industrial process modeling.