An Iterative Implementation of Variable Projection for Separable Nonlinear Optimization Problems

Abstract

The separable nonlinear least-squares (SNLLS) problems considered in this article frequently appear in a wide range of research fields, such as machine learning, computer vision, system identification, and signal processing. The variable projection algorithm proposed by Golub and Pereyra, which reduces the dimension of the parameters by projecting the linear parameters out of the problem, is quite valuable in solving SNLLS problems. Previous implementations of the variable projection algorithm are based on matrix factorization. In this article, we propose an iterative implementation of the variable projection algorithm. Compared with previous implementations based on matrix decomposition, the proposed method can effectively avoid suffering from large condition number of the matrix or even matrix decomposition failure when dealing with ill-posed SNLLS problems. Numerical experiments on real-world data and synthetic data show the efficiency and robustness of the proposed iterative variable projection algorithm.