Extended Stochastic Gradient Research Articles

Convergence of the auxiliary model-based multi-innovation generalized extended stochastic gradient algorithm for Box–Jenkins systems

This paper focuses on the parameter estimation problem of Box–Jenkins systems. Using the multi-innovation identification theory, an auxiliary model-based multi-innovation generalized extended stochastic gradient algorithm is derived. The convergence of the proposed algorithm is analyzed based on the stochastic martingale theory. It is proved that the parameter estimation errors converge to zero under persistent excitation conditions. Two simulation examples are provided to confirm the convergence results.

A multi-innovation generalized extended stochastic gradient algorithm for output nonlinear autoregressive moving average systems

This paper proposes a generalized extended stochastic gradient (GESG) algorithm for estimating the parameters of a class of Wiener nonlinear autoregressive moving average systems using the gradient search. In order to improve the convergence rates of the GESG algorithm, a multi-innovation GESG algorithm is derived. The simulation results indicate that the proposed algorithms can effectively estimate the parameters of a class of output nonlinear systems.

Consistency of the extended gradient identification algorithm for multi-input multi-output systems with moving average noises

The consistency of identification algorithms for systems with colored noises is a main topic in system identification. This paper focuses on the extended stochastic gradient (ESG) identification algorithm for the multivariable linear systems with moving average noises. By integrating the noise regression terms and the noise model parameters into the information matrix and the parameter vector, and based on the gradient search principle, the ESG algorithm is presented. The unknown noise terms in the information matrix are replaced with their estimates. The convergence analysis shows that the parameter estimation error converges to zero under a persistent excitation condition. Two simulation examples are given to illustrate the effectiveness of the algorithm.

Gradient-based parameter estimation for input nonlinear systems with ARMA noises based on the auxiliary model

This paper presents a gradient-based iterative identification algorithm and an auxiliary-model-based multi-innovation generalized extended stochastic gradient algorithm for input nonlinear systems with autoregressive moving average (ARMA) noises, i.e., the input nonlinear Box–Jenkins (IN–BJ) systems. The estimation errors given by the gradient-based iterative algorithm are smaller than the generalized extended stochastic gradient algorithm under same data lengths. A simulation example is provided.

Multi-innovation Extended Stochastic Gradient Algorithm and Its Performance Analysis

This paper derives the multi-innovation extended stochastic gradient algorithm for controlled autoregressive moving average models by expanding the scalar innovation to an innovation vector and analyzes its performance in detail. Four convergence theorems are given for the multi-innovation extended stochastic gradient algorithm to show that the parameter estimates converge to their true values under the weak persistent excitation condition. The simulation results show that the proposed algorithm can produce more accurate parameter estimates than the traditional extended stochastic gradient algorithm.

The residual-based ESG algorithm and its performance analysis

The performance of the residual-based extended stochastic gradient (ESG) algorithms for identifying CARMA models with disturbances is analyzed under weaker conditions on statistical properties of the noise. The paper derives the conditions under which the parameter estimation errors converge to zero. Three examples are given to show the advantages of the proposed algorithm.

The residual based interactive stochastic gradient algorithms for controlled moving average models

This paper derives a residual based interactive stochastic gradient (ISG) parameter estimation algorithm for controlled moving average (CMA) models and studied the performance of the residual based ISG algorithm under weaker conditions on statistical properties of the noise. Compared with the residual based extended stochastic gradient algorithm for identifying CMA models, the proposed ISG algorithm can give highly accurate parameter estimates by the simulation example.