Extended Stochastic Gradient Algorithm Research Articles

Moving data window-based partially-coupled estimation approach for modeling a dynamical system involving unmeasurable states

The simultaneous parameter and state estimation for a multi-input multi-output (MIMO) state space system from a set of measurement data is taken into account in this paper. Firstly, in line with the number of the system outputs, the considered MIMO system is transformed to some subsystems, which lessens the dimensions and the number of the parameters to be estimated. Secondly, by designing the moving data window that contains the latest batch of collected data, we develop a moving data window-based partially-coupled average extended stochastic gradient algorithm for parameter estimation. Thirdly, once the parameter estimates are obtained, a new state filter is designed to produce the estimates of the unmeasurable states by means of the Kalman filtering principle. Then we propose a combined state filtering and moving data window-based partially-coupled average extended stochastic gradient (CSF-MDW-PC-A-ESG) algorithm to produce the estimates of the parameters and states simultaneously. To reveal the superiority of the CSF-MDW-PC-A-ESG algorithm, a combined state filtering and partially-coupled average extended stochastic gradient (CSF-PC-A-ESG) algorithm is given to make a comparison. Finally, the effectiveness and superiority of the proposed CSF-MDW-PC-A-ESG algorithm are proved in a simulation example. The results from the illustrative example show that the CSF-MDW-PC-A-ESG algorithm is effective to produce the estimates of the parameters and states and that the CSF-MDW-PC-A-ESG algorithm has the higher efficient data utilization, the more accurate parameter estimation capability and the better model fitting ability than the CSF-PC-A-ESG algorithm.

Multi‐innovation gradient parameter estimation for multivariable systems based on the maximum likelihood principle

AbstractThis article considers the parameter estimation problems of linear multivariable systems with unknown disturbances. For the parameter matrices in the multivariable systems, the model decomposition technique is used to reduce the computational complexity by decomposing the multivariable system into several subsystems with only the parameter vectors. By means of the negative gradient search, a decomposition‐based maximum likelihood recursive extended stochastic gradient algorithm is derived. In order to improve the parameter estimation accuracy, by introducing the multi‐innovation identification theory, a decomposition‐based maximum likelihood multi‐innovation extended stochastic gradient algorithm is proposed. The simulation results illustrate the effectiveness of the proposed algorithms.

Parameter estimation for a class of radial basis function-based nonlinear time-series models with moving average noises

This paper focuses on the parameter estimation for radial basis function-based state-dependent autoregressive models with moving average noises (RBF-ARMA models). An extended projection algorithm is derived based on the negative gradient search. In order to reduce the sensitivity of the algorithm to noise and reduce the fluctuations of the parameter estimation errors, a modified extended stochastic gradient algorithm is proposed. By introducing a moving data window, a modified moving data window-based extended stochastic gradient algorithm is further developed to improve the parameter estimation accuracy. The simulation results show that the proposed algorithms can effectively estimate the parameters of the RBF-ARMA models.

Filtering-Based Parameter Identification Methods for Multivariable Stochastic Systems

This paper presents an adaptive filtering-based maximum likelihood multi-innovation extended stochastic gradient algorithm to identify multivariable equation-error systems with colored noises. The data filtering and model decomposition techniques are used to simplify the structure of the considered system, in which a predefined filter is utilized to filter the observed data, and the multivariable system is turned into several subsystems whose parameters appear in the vectors. By introducing the multi-innovation identification theory to the stochastic gradient method, this study produces improved performances. The simulation numerical results indicate that the proposed algorithm can generate more accurate parameter estimates than the filtering-based maximum likelihood recursive extended stochastic gradient algorithm.

Recursive coupled projection algorithms for multivariable output‐error‐like systems with coloured noises

By combining the coupling identification concept with the gradient search, this study develops a partially coupled generalised extended projection algorithm and a partially coupled generalised extended stochastic gradient algorithm to estimate the parameters of a multivariable output-error-like system with autoregressive moving average noise from input–output data. The key is to divide the identification model into several submodels based on the hierarchical identification principle and to establish the parameter estimation algorithm by using the coupled relationship between these submodels. The simulation test results indicate that the proposed algorithms are effective.

Maximum likelihood‐based gradient estimation for multivariable nonlinear systems using the multiinnovation identification theory

SummaryThis article considers the identification problems of multivariable input nonlinear systems with unmeasured disturbances. For the identification difficulty caused by the crossproducts between the parameters of the linear block and the nonlinear block, the key term separation technique is adopted to separate the parameters of the nonlinear block from the parameters of the linear block. By combining the model decomposition technique and the hierarchical identification principle, a key term separation‐based maximum likelihood recursive extended stochastic gradient algorithm with reduced computational complexity is presented to estimate all the parameters directly. By introducing the multiinnovation identification theory, a key term separation‐based maximum likelihood multiinnovation extended stochastic gradient algorithm is proposed to improve the parameter estimation accuracy. The simulation results illustrate the effectiveness of the proposed methods.

Hierarchical multi‐innovation generalised extended stochastic gradient methods for multivariable equation‐error autoregressive moving average systems

This study presents the modelling technology of multivariable equation-error autoregressive moving average (EEARMA) systems through observational data of systems. Aiming to develop a simplified identification algorithm, the original multivariable EEARMA model to be identified is separated into two sub-identification models. After the model decomposition, a two-stage generalised extended stochastic gradient (GESG) algorithm is presented in accordance with these two separated submodels. By adding more observations to the recursive computation, the corresponding two-stage multi-innovation GESG (MI-GESG) algorithm, namely, hierarchical multi-innovation generalised extended stochastic gradient algorithm, is derived for the multivariable EEARMA systems through expanding the innovation vector to the innovation matrices. The simulation example verifies that the performance about the computational accuracy of the two-stage MI-GESG algorithm is improved compared with the two-stage GESG algorithm.

The filtering based auxiliary model generalized extended stochastic gradient identification for a multivariate output-error system with autoregressive moving average noise using the multi-innovation theory

This paper studies the parameter estimation algorithms of multivariate output-error autoregressive moving average (M-OEARMA) systems. By means of the filtering technique and the auxiliary model identification idea, this paper gives an auxiliary model generalized extended stochastic gradient (AM-GESG) algorithm for identifying the M-OEARMA system as a comparison. In order to enhance the performance of the AM-GESG algorithm, a modified filtering based AM-GESG algorithm and a filtering based auxiliary model multi-innovation generalized extended stochastic gradient algorithm are proposed. Compared with the AM-GESG algorithm, the proposed two algorithms can generate highly accurate parameter estimates. The simulation examples demonstrate that the proposed algorithms are effective for identifying the M-OEARMA systems.

Recursive search‐based identification algorithms for the exponential autoregressive time series model with coloured noise

This study focuses on the recursive parameter estimation problems for the non-linear exponential autoregressive model with moving average noise (the ExpARMA model for short). By means of the gradient search, an extended stochastic gradient (ESG) algorithm is derived. Considering the difficulty of determining the step-size in the ESG algorithm, a numerical approach is proposed to obtain the optimal step-size. In order to improve the parameter estimation accuracy, the authors employ the multi-innovation identification theory to develop a multi-innovation ESG (MI-ESG) algorithm for the ExpARMA model. Introducing a forgetting factor into the MI-ESG algorithm, the parameter estimation accuracy can be further improved. With an appropriate innovation length and forgetting factor, the variant of the MI-ESG algorithm is effective to identify all the unknown parameters of the ExpARMA model. A simulation example is provided to test the proposed algorithms.

Parameter Estimation of Neuro-Fuzzy Wiener Model With Colored Noise Using Separable Signals

This paper considers a neuro-fuzzy based identification problem for Wiener model with controlled autoregressive moving average noise. The separable signal is applied to decouple the dynamic linear part and the static nonlinear part, and the correlation analysis method is adopted to estimate the parameters of the linear part. To improve the convergence rate of generalized extended stochastic gradient (GESG) algorithm, a generalized extended stochastic gradient algorithm with a forgetting factor is derived for estimating the parameters of the nonlinear part and the parameters of noise model. Examples results verify the effectiveness of the proposed method.

Maximum likelihood gradient‐based iterative estimation for multivariable systems

This study concerns the parameter identification issues for a class of multivariable systems with moving average noise. The main contributions are to transform a multivariable system into several subsystems for reducing the computational complexity by means of the decomposition technique and to deal with the coloured noise for improving the estimation accuracy by using the maximum likelihood principle and the iterative estimation theory. As the maximum likelihood gradient-based iterative algorithm makes sufficient use of all the observed data at each iteration, the parameter estimation accuracy can be enhanced. The numerical simulation results demonstrate that the proposed algorithm has faster convergence rates and better tracking performance than the compared multivariable extended stochastic gradient algorithm.

Partially coupled gradient estimation algorithm for multivariable equation‐error autoregressive moving average systems using the data filtering technique

System identification provides many convenient and useful methods for engineering modelling. This study targets the parameter identification problems for multivariable equation-error autoregressive moving average systems. To reduce the influence of the coloured noises on the parameter estimation, the data filtering technique is adopted to filter the input and output data, and to transform the original system into a filtered system with white noises. Then the filtered system is decomposed into several subsystems and a filtering-based partially-coupled generalised extended stochastic gradient algorithm is developed via the coupling concept. In contrast to the multivariable generalised extended stochastic gradient algorithm, the proposed algorithm can give more accurate parameter estimates. Finally, the effectiveness of the proposed algorithm is well demonstrated by simulation examples.

Parameter estimation algorithm for multivariable controlled autoregressive autoregressive moving average systems

This paper investigates parameter estimation problems for multivariable controlled autoregressive autoregressive moving average (M-CARARMA) systems. In order to improve the performance of the standard multivariable generalized extended stochastic gradient (M-GESG) algorithm, we derive a partially coupled generalized extended stochastic gradient algorithm by using the auxiliary model. In particular, we divide the identification model into several subsystems based on the hierarchical identification principle and estimate the parameters using the coupled relationship between these subsystems. The simulation results show that the new algorithm can give more accurate parameter estimates of the M-CARARMA system than the M-GESG algorithm.

Maximum Likelihood Multi-innovation Stochastic Gradient Estimation for Multivariate Equation-error Systems

This paper focuses on the parameter estimation problems of multivariate equation-error systems. A multi-innovation generalized extended stochastic gradient algorithm is presented as a comparison. Based on the maximum likelihood principle and the coupling identification concept, the multivariate equation-error system is decomposed into several regressive identification subsystems, each of which has only a parameter vector, and a coupled subsystem maximum likelihood multi-innovation stochastic gradient identification algorithm is developed for estimating the parameter vectors of these subsystems. The simulation results show that the coupled subsystem maximum likelihood multi-innovation stochastic gradient algorithm can generate more accurate parameter estimates and has faster convergence rates compared with the multi-innovation generalized extended stochastic gradient algorithm.

A filtering based multi-innovation extended stochastic gradient algorithm for multivariable control systems

For a multivariable system with moving average noise (i.e., a multivariable controlled autoregressive moving average system), this paper proposes a filtering based extended stochastic gradient (ESG) algorithm and a filtering based multi-innovation ESG algorithm for improving the parameter estimation accuracy. The key is using the filtering technique and the multi-innovation identification theory. The proposed algorithms can identify the parameters of the system model and the noise model. The filtering based multi-innovation ESG algorithm can give more accurate parameter estimates. The numerical simulation results demonstrate that the proposed algorithms work well.

Recursive least squares algorithm and gradient algorithm for Hammerstein–Wiener systems using the data filtering

This paper considers the parameter estimation problems of Hammerstein–Wiener systems by using the data filtering technique. In order to improve the estimation accuracy, the data filtering-based recursive generalized extended least squares algorithm is derived. In order to improve the computational efficiency, the data filtering-based generalized extended stochastic gradient algorithm is derived for estimating the system parameters. Finally, the computational efficiency of the proposed algorithms is analyzed and compared. The simulation results indicate that the proposed algorithms can effectively estimate the parameters of Hammerstein–Wiener systems.

Convergence of the recursive identification algorithms for multivariate pseudo‐linear regressive systems

SummaryThe performance analysis of the recursive algorithms for the multivariate systems with an autoregressive moving average noise process is still open. This paper analyzes the convergence of two recursive identification algorithms, the multivariate recursive generalized extended least squares algorithm and the multivariate generalized extended stochastic gradient algorithm, for pseudo‐linear multivariate systems and proves that the parameter estimation errors consistently converge to zero under persistent excitation conditions. The simulation results show that the proposed algorithms work well. Copyright © 2015 John Wiley & Sons, Ltd.

Convergence of the auxiliary model-based multi-innovation generalized extended stochastic gradient algorithm for Box–Jenkins systems

This paper focuses on the parameter estimation problem of Box–Jenkins systems. Using the multi-innovation identification theory, an auxiliary model-based multi-innovation generalized extended stochastic gradient algorithm is derived. The convergence of the proposed algorithm is analyzed based on the stochastic martingale theory. It is proved that the parameter estimation errors converge to zero under persistent excitation conditions. Two simulation examples are provided to confirm the convergence results.

A multi-innovation generalized extended stochastic gradient algorithm for output nonlinear autoregressive moving average systems

This paper proposes a generalized extended stochastic gradient (GESG) algorithm for estimating the parameters of a class of Wiener nonlinear autoregressive moving average systems using the gradient search. In order to improve the convergence rates of the GESG algorithm, a multi-innovation GESG algorithm is derived. The simulation results indicate that the proposed algorithms can effectively estimate the parameters of a class of output nonlinear systems.

Consistency of the extended gradient identification algorithm for multi-input multi-output systems with moving average noises

The consistency of identification algorithms for systems with colored noises is a main topic in system identification. This paper focuses on the extended stochastic gradient (ESG) identification algorithm for the multivariable linear systems with moving average noises. By integrating the noise regression terms and the noise model parameters into the information matrix and the parameter vector, and based on the gradient search principle, the ESG algorithm is presented. The unknown noise terms in the information matrix are replaced with their estimates. The convergence analysis shows that the parameter estimation error converges to zero under a persistent excitation condition. Two simulation examples are given to illustrate the effectiveness of the algorithm.