Gradient based Identification Algorithm for Separable Nonlinear Models

Abstract

Separable nonlinear models are a class of models that can separately represent the linear and nonlinear parameters, which have a wide range of applications in many fields. This paper attempts to explore the performance of gradient based algorithms in identifying separable nonlinear models. We choose the RBF-AR model as the research objection, which is a classical separable nonlinear model and has a wide range of applications in time series modelling. Three different gradient based optimization strategies are used to estimate the parameters of the RBF-AR model, including the classical gradient descent (GD) method, the alternative gradient descent (GD-L) method, and the gradient based variable projection(GD-VP) method. The experimental results show that the gradient descent method is sensitive to the learning rate and is prone to gradient explosion problems, so the convergence speed is slow; while the VP algorithm can alleviate the impact of the learning rate, and improve the efficiency and stability of the gradient descent alaorithm.