Multi-innovation Stochastic Gradient Algorithm Research Articles

Parameter identification algorithm for ship manoeuvrability and wave peak model based multi-innovation stochastic gradient algorithm use data filtering technique

This paper addresses the issue of identifying ship motion parameters and wave peak frequency. Utilising the Euler discretisation principle, we establish a discrete-time auto-regressive moving average model with exogenous input (ARMAX) for the ship-wave system. Furthermore, we develop a filtering-based stochastic gradient algorithm for the system by applying filtering techniques and auxiliary model identification idea. A filtering-based multi-innovation stochastic gradient algorithm, utilizing the multi-innovation identification theory, was developed to enhance the convergence rate and accuracy of parameter identification. This approach was found to be more effective than the filtering-based stochastic gradient algorithm. Simulation results validate the efficacy of the proposed algorithm in parameter identification.

Identification of Multi-Innovation Stochastic Gradients with Maximum Likelihood Algorithm Based on Ship Maneuverability and Wave Peak Models

This paper investigates the problem of real-time parameter identification for ship maneuvering parameters and wave peak frequency in an ocean environment. Based on the idea of Euler discretion, a combined model of ship maneuvering and wave peak frequency (ship–wave) is made a discretion, and a discrete-time auto-regressive moving-average model with exogenous input (ARMAX) is derived for parameter identification. Based on the ideas of stochastic gradient identification and multi-innovation theory, a multi-innovation stochastic gradient (MI-SG) algorithm is derived for parameter identification of the ship–wave discretion model. Maximum likelihood theory is introduced to propose a maximum likelihood-based multi-innovation stochastic gradient (ML-MI-SG) algorithm. Compared to the MI-SG algorithm, the ML-MI-SG algorithm shows improvements in both parameter identification accuracy and identification convergence speed. Simulation results verify the effectiveness of the proposed algorithm.

Auxiliary model-based hierarchical stochastic gradient methods for multiple-input multiple-output systems

This paper presents a hierarchical identification model for multiple-input multiple-output (MIMO) systems. An auxiliary model-based hierarchical stochastic gradient (AM-HSG) algorithm is derived by means of the auxiliary model identification idea and the hierarchical identification principle. Furthermore, an auxiliary model-based hierarchical multi-innovation stochastic gradient (AM-HMISG) algorithm is derived by utilizing the multi-innovation identification theory. In order to compare the computational efficiency of the AM-HSG and AM-HMISG algorithms for identifying MIMO systems, this paper gives the existing traditional gradient algorithms and discusses the complexity of these algorithms in detail. Finally, the simulation example tests the effectiveness of all four algorithms.

Hierarchical multi-innovation stochastic gradient identification algorithm for estimating a bilinear state-space model with moving average noise

This paper considers the combined parameter and state estimation problem of a bilinear state space system with moving average noise. There are product terms of state variables and control variables in bilinear systems, which brings difficulties to parameter and state estimation. By designing a bilinear state estimator based on Kalman filter and using input–output data to estimate the state, a hierarchical multi-innovation stochastic gradient (i.e., H-MISG) algorithm based on the state estimator is proposed to jointly estimate unknown states and parameters. In addition, compared with the hierarchical stochastic gradient algorithm, H-MISG algorithm introduces the innovation length parameter, makes full use of the system input and output data information, and improves the accuracy of parameter estimation. Numerical simulation examples verify the effectiveness of the proposed algorithm.

Identification of a nonlinear rational model based on bias compensated multi-innovation stochastic gradient algorithm

The nonlinear rational model is a generalized nonlinear model and has been gradually applied in modelling many dynamic processes. The parameter identification of a class of nonlinear rational models is studied in this paper. This identification problem is very challenging because of the complexity of the rational model and the coupling between model inputs and outputs. To identify the nonlinear model, a bias compensated multi-innovation stochastic gradient algorithm is presented. The multi-innovation technique replacing the scalar innovation with an information vector is adopted to accelerate the traditional stochastic gradient algorithm. However, the estimate obtained by the accelerated algorithm is biased because of the correlation between the information vector and the noise. To overcome this difficulty, a bias compensation strategy is used. The bias is calculated and compensated to get an unbiased estimate. Theoretical analysis shows that the proposed algorithm can give biased estimates with linear complexity. The proposed algorithm is validated by a numerical experiment and the modelling of the propylene catalytic oxidation.

Auxiliary Model-Based Multi-Innovation Fractional Stochastic Gradient Algorithm for Hammerstein Output-Error Systems

This paper focuses on the nonlinear system identification problem, which is a basic premise of control and fault diagnosis. For Hammerstein output-error nonlinear systems, we propose an auxiliary model-based multi-innovation fractional stochastic gradient method. The scalar innovation is extended to the innovation vector for increasing the data use based on the multi-innovation identification theory. By establishing appropriate auxiliary models, the unknown variables are estimated and the improvement in the performance of parameter estimation is achieved owing to the fractional-order calculus theory. Compared with the conventional multi-innovation stochastic gradient algorithm, the proposed method is validated to obtain better estimation accuracy by the simulation results.

Novel identification algorithms for Hammerstein systems in ill-conditioned situations

Two algorithms named Recursive Least Square algorithm for Ill-Conditioned situations (RLS-IC) and Multi Innovation Stochastic Gradient algorithm for Ill-Conditioned situations (MISG-IC) are proposed based on multi-innovation identification theory, least squares, stochastic gradient and singular value decomposition to identify Hammerstein systems in ill-conditioned situations. In ill-conditioned situations, identification algorithms based on classic least-squares and stochastic gradient lead to uncertain estimates and may be ineffective. Simulation examples show that the proposed algorithms are suitable for parameter estimation in Hammerstein nonlinear systems with ill-conditioned situations.

Parameter estimation algorithms of linear systems with time-delays based on the frequency responses and harmonic balances under the multi-frequency sinusoidal signal excitation

In this paper, we study the parameter estimation problem of a class of linear time-delay systems. Based on the frequency responses and harmonic balance methods and by means of the gradient search, a two-stage stochastic gradient and gradient-based iterative algorithm is developed by using the collected data under the sinusoidal excitation. To improve the estimation accuracy, a two-stage multi-innovation stochastic gradient and gradient-based iterative algorithm is presented. Two simulation examples are provided to test the proposed algorithms.

Correlation Analysis-based Stochastic Gradient and Least Squares Identification Methods for Errors-in-variables Systems Using the Multiinnovation

This paper deals with the identification problem of discrete-time linear time-invariant errors-in-variables systems for the case of the colored output noise. Based on the correlation analysis, the multi-innovation theory is introduced to the errors-in-variables systems where both input and output data are noisy. A correlation analysis-based multi-innovation stochastic gradient algorithm and a correlation analysis-based multi-innovation least squares algorithm are proposed by means of the multi-innovation theory in order to improve the parameter accuracy. The simulation results confirm that these two algorithms are effective.

Multi-Innovation Stochastic Gradient Parameter and State Estimation Algorithm for Dual-Rate State-Space Systems with d-Step Time Delay

This paper presents a multi-innovation stochastic gradient parameter estimation algorithm for dual-rate sampled state-space systems with d-step time delay by the multi-innovation identification theory. Considering the stochastic disturbance in industrial process and using the gradient search, a multi-innovation stochastic gradient algorithm is proposed through expanding the scalar innovation into an innovation vector in order to obtain more accurate parameter estimates. The difficulty of identification is that the information vector in the identification model contains the unknown states. The proposed algorithm uses the state estimates of the observer instead of the state variables to realize the parameter estimation. The simulation results indicate that the proposed algorithm works well.

Acceleration Harmonic Estimation for Hydraulic Servo Shaking Table Based on Multi-Innovation Stochastic Gradient Algorithm

As one of the critical test equipment, hydraulic servo shaking table is widely used in shaking environment simulation of structural components and systems. However, inherent nonlinear factors of a hydraulic servo shaking table can cause amplitude attenuation and phase lag when corresponding to a sinusoidal acceleration signal, which leads to serious harmonic distortion. In order to improve waveform reproduction performance of sinusoidal signals, the amplitude and phase of harmonic should be estimated accurately. In this paper, the multi-innovation stochastic gradient (MISG) algorithm is presented for dynamically estimating the harmonic information. Simulation and experiment results demonstrate that the proposed algorithm has high estimation precision and good convergence performance.

Output Delay Sliding Mode Tracking Control of SRM based on Multi-innovation Model Identification

The drive system of switched reluctance motor (SRM) is a complex nonlinear system that is composed of many links. The delay in the measurement of the speed and position signal of SRM is caused by the factors that affect the measuring sensor. To effectively improve the influence of the SRM rotor position and speed signal delay on the system performance, a sliding mode position tracking method based on output delay observation was proposed in this study. First, the model was discretized according to the structure and characteristics of SRM and the mathematical parameters of the system were identified using a multi-innovation stochastic gradient (MISG) algorithm. Second, a delay state observer was constructed on the basis of an SRM system model with output delay. Then, the sliding mode tracking control method based on the delay state observation compensation was proposed and combined with sliding mode control theory. Lastly, the effectiveness of the designed model parameter identification, delay state observation, and output delay control methods were compared through numerical simulation. Results show that when uncertain factors, such as noise, are present in the system, the MISG identification method can rapidly and accurately identify the parameters of the SRM model compared with the stochastic gradient identification method; the identification accuracy of the former is four times higher than that of the latter. Similarly, the sliding mode position tracking control method based on output delay observer can rapidly and accurately track the position and speed within 0.5 s. However, its position (0.2 rad) and velocity (0.233 rad/s) tracking exhibit large steady-state errors when no delay observation compensation is present. The proposed method not only demonstrates high position tracking accuracy, but also possesses strong robustness to output delay.

Two-stage multi-innovation stochastic gradient algorithm for multivariate output-error ARMA systems based on the auxiliary model

ABSTRACTThis paper investigates the parameter estimation problem for multivariate output-error systems perturbed by autoregressive moving average noises. Since the identification model has two different kinds of parameters, a vector and a matrix, the gradient algorithm cannot be used directly. Therefore, we decompose the original system model into two sub-models and proceed the identification problem by the collaboration between the two sub-models. By employing the gradient search and determining the optimal step-sizes, we present an auxiliary model based two-stage projection algorithm. However, in order to alleviate the sensitivity to the noise, we reselect the step-sizes and derive the auxiliary model based two-stage stochastic gradient (AM-2S-SG) algorithm. Based on the AM-2S-SG algorithm, an auxiliary model based two-stage multi-innovation stochastic gradient algorithm is proposed to generate more accurate estimates. Finally, numerical simulations are provided to demonstrate the effectiveness of the proposed algorithms.

The innovation algorithms for multivariable state‐space models

SummaryThis paper derives the input‐output representation of the dynamical system described by a linear multivariable state‐space model and the corresponding multivariate linear regressive model (ie, multivariate equation‐error model). A projection identification algorithm, a multivariate stochastic gradient identification algorithm, and a multi‐innovation stochastic gradient (MISG) identification algorithm are proposed for multivariate equation‐error systems by using the negative gradient search and the multi‐innovation identification theory. The convergence analysis of the MISG algorithm indicates that the parameter estimation errors converge to zero under the persistent excitation condition. Finally, a numerical example illustrates the effectiveness of the proposed algorithms.

Weighted Parameter Estimation for Hammerstein Nonlinear ARX Systems

This paper proposes parameter estimation algorithms for Hammerstein nonlinear ARX systems. By making full use of the current and previous input–output data of the system, a weighted multi-innovation stochastic gradient algorithm is presented to improve the convergence rate of identification. The innovation term in the traditional identification algorithms can be treated as a particle in the particle-filtering technique, and the weight of each innovation then can be computed according to their importance. The simulation results indicate that the algorithm can improve the accuracy of parameter estimation.

Multi‐innovation gradient estimation algorithms for multivariate equation‐error autoregressive moving average systems based on the filtering technique

This study concentrates on the parameter estimation of multivariate pseudo-linear autoregressive moving average systems by means of the multi-innovation identification theory and data filtering technique. A multi-innovation stochastic gradient algorithm is derived by introducing the innovation length in the stochastic gradient algorithm. Then, the original system is transformed into two subsystems by using a filter. A filtering-based multi-innovation stochastic gradient algorithm is presented, whose parameter estimation accuracy is higher than the multi-innovation stochastic gradient algorithm. The simulation results confirm that these two algorithms are effective.

Decomposition-based Gradient Estimation Algorithms for Multivariable Equation-error Systems

This paper concerns the parameter identification methods of multivariable equation-error systems. By means of the decomposition technique, the multivariable identification model is transformed into two sub-identification models and a decomposition-based stochastic gradient (D-SG) algorithm is presented for estimating the parameters of these two submodels. In order to further improve the convergence rate and the parameter estimation accuracy, we expand the innovation vectors to the innovation matrices and develop a decomposition-based multi-innovation stochastic gradient (D-MISG) algorithm. The simulation results confirm that the D-MISG algorithm can provide more accurate parameter estimates than the D-SG algorithm.

Parameter Identification of Nonlinear Systems with Time-delay Based on the Multi-innovation Stochastic Gradient Algorithm

This paper considers the parameter identification problem of block-oriented Hammerstein nonlinear systems with time-delay. Firstly, we adopt the data filtering technique to transform the identification model so that all the parameters will be separated in the resulting identification model which has no redundant parameters. Secondly, a multi-innovation stochastic gradient algorithm is used to estimate the system parameters. The proposed method has high computational efficiency and good accuracy. Simulation results are presented to demonstrate the effectiveness of the proposed algorithm.

The filtering based parameter identification for bilinear-in-parameter systems

This paper discusses the parameter estimation for a class of bilinear-in-parameter systems with colored noise. By utilizing the filtering technique, we derive the relationship between the filtered output and the measurement output and obtain two linear regressive sub-models. A filtering based multi-innovation stochastic gradient algorithm is derived for interactively identifying each sub-model. The proposed algorithm avoids the estimation of correlated noise and improves the parameter estimation accuracy by making full use of the measurement data. The numerical simulation results indicate that the proposed algorithm has higher estimation accuracy than the hierarchical multi-innovation stochastic gradient algorithm.

Multi-innovation Stochastic Gradient Algorithms for Input Nonlinear Time-Varying Systems Based on the Line Search Strategy

Block-oriented nonlinear systems have attracted a considerable attention for their flexible structure and practicability. This study proposes a novel multi-innovation stochastic gradient (MISG) algorithm to address the identification problem in input nonlinear systems. This involves applying the inexact line search strategy to determine an appropriate convergence factor at each recursive step. The proposed algorithm tracks the nonlinear system dynamics faster than the conventional MISG algorithm. It is therefore suitable for online identification and can be applied to nonlinear time-varying systems. The concept of auxiliary model identification is also adopted for dealing with unmeasurable variables. The effectiveness of the proposed algorithm is verified through simulated examples.