Multi-innovation Identification Theory Research Articles

Parameter identification algorithm for ship manoeuvrability and wave peak model based multi-innovation stochastic gradient algorithm use data filtering technique

This paper addresses the issue of identifying ship motion parameters and wave peak frequency. Utilising the Euler discretisation principle, we establish a discrete-time auto-regressive moving average model with exogenous input (ARMAX) for the ship-wave system. Furthermore, we develop a filtering-based stochastic gradient algorithm for the system by applying filtering techniques and auxiliary model identification idea. A filtering-based multi-innovation stochastic gradient algorithm, utilizing the multi-innovation identification theory, was developed to enhance the convergence rate and accuracy of parameter identification. This approach was found to be more effective than the filtering-based stochastic gradient algorithm. Simulation results validate the efficacy of the proposed algorithm in parameter identification.

Auxiliary model-based hierarchical stochastic gradient methods for multiple-input multiple-output systems

This paper presents a hierarchical identification model for multiple-input multiple-output (MIMO) systems. An auxiliary model-based hierarchical stochastic gradient (AM-HSG) algorithm is derived by means of the auxiliary model identification idea and the hierarchical identification principle. Furthermore, an auxiliary model-based hierarchical multi-innovation stochastic gradient (AM-HMISG) algorithm is derived by utilizing the multi-innovation identification theory. In order to compare the computational efficiency of the AM-HSG and AM-HMISG algorithms for identifying MIMO systems, this paper gives the existing traditional gradient algorithms and discusses the complexity of these algorithms in detail. Finally, the simulation example tests the effectiveness of all four algorithms.

Least squares parameter estimation and multi-innovation least squares methods for linear fitting problems from noisy data

Least squares is an important method for solving linear fitting problems and quadratic optimization problems. This paper explores the properties of the least squares methods and the multi-innovation least squares methods. It demonstrates lemmas and theorems about the least squares and multi-innovation least squares parameter estimation algorithms after reviewing and surveying some important contributions in the area of system identification, such as the auxiliary model identification idea, the multi-innovation identification theory, the hierarchical identification principle, the coupling identification concept and the filtering identification idea. The results of the least squares and multi-innovation least squares algorithms for linear regressive systems with white noises can be extended to other systems with colored noises.

A novel multi-innovation gradient support vector machine regression method

For the regression problem of support vector machine, the solution processes of the most existing methods use offline datasets, which cannot be realized online. For this problem, this paper presents a new online approach to identify these unknown parameters contained in the support vector machine. A new cost function is constructed by substituting the error term into the standard cost function, which is different from the standard support vector machine, and the gradient descent approach is then used to minimize the newly created loss function, thus proposing a stochastic gradient support vector machine algorithm to estimate the unknown parameters based on the recursive identification methods. Furthermore, to advance the property of the stochastic gradient support vector machine algorithm, a moving data window is used to widen the scalar information into a fixed-length innovation vector, thereby increasing the amount of information used in the parameter estimation based on the multi-innovation identification theory. In addition, the forgetting factor is brought into the proposed algorithms, and the corresponding forgetting factor recursive algorithms are derived. These methods are recursive identification methods, which may be implemented online and are more efficient in terms of computing. Finally, utilizing the MatLab platform, the validity and usefulness of the explored methodologies are proven using several numerical simulation examples.

Parameter estimation for nonlinear Volterra systems by using the multi-innovation identification theory and tensor decomposition

The Volterra model can represent a wide range of nonlinear dynamical systems. However, its practical use in nonlinear system identification is limited due to the exponentially growing number of Volterra kernel coefficients as the degree increases. This paper considers the identification issue of discrete-time nonlinear Volterra systems and uses a tensorial decomposition called PARAFAC to represent the Volterra kernels which can provide a significant parametric reduction compared with the conventional Volterra model. Applying the multi-innovation identification theory, the recursive algorithm by combining the l2-norm is proposed for the PARAFAC-Volterra models with the Gaussian noises. In addition, the multi-innovation algorithm combining with the logarithmic p-norms is investigated for the nonlinear Volterra systems with the non-Gaussian noises. Finally, some simulation results illustrate the effectiveness of the proposed identification methods.

Auxiliary Model-Based Multi-Innovation Fractional Stochastic Gradient Algorithm for Hammerstein Output-Error Systems

This paper focuses on the nonlinear system identification problem, which is a basic premise of control and fault diagnosis. For Hammerstein output-error nonlinear systems, we propose an auxiliary model-based multi-innovation fractional stochastic gradient method. The scalar innovation is extended to the innovation vector for increasing the data use based on the multi-innovation identification theory. By establishing appropriate auxiliary models, the unknown variables are estimated and the improvement in the performance of parameter estimation is achieved owing to the fractional-order calculus theory. Compared with the conventional multi-innovation stochastic gradient algorithm, the proposed method is validated to obtain better estimation accuracy by the simulation results.

Research and Analysis of Maximum Likelihood Parameter Identification Based on The Recursion of Longdongnan Economic Model

By establishing the regional economic model of southeast Gansu Economic Model as the maximum likelihood parameter identification,the maximum likelihood identification is an estimation method based on probability and statistics theory,by constructing the likelihood function with independent variables and maximizing the likelihood function,the model has good statistical properties and is widely used in many fields.However, the research of maximum likelihood identification method is still limited. Therefore, it is of theoretical significance and academic value to study the maximum likelihood identification method for linear systems and Hammerstein nonlinear systems by combining data filtering technique, hierarchical identification principle and multi-innovation identification theory. In order to improve the identification accuracy, maximum likelihood gradient iteration and maximum likelihood gradient iteration based on finite data window are proposed for output error moving average system.This paper holds that the pattern conforms to the requirements of economic development of Southeast Longnan and has a strong reference for other regions.The simulation results verify the feasibility and validity of the identification modeling method.

Novel identification algorithms for Hammerstein systems in ill-conditioned situations

Two algorithms named Recursive Least Square algorithm for Ill-Conditioned situations (RLS-IC) and Multi Innovation Stochastic Gradient algorithm for Ill-Conditioned situations (MISG-IC) are proposed based on multi-innovation identification theory, least squares, stochastic gradient and singular value decomposition to identify Hammerstein systems in ill-conditioned situations. In ill-conditioned situations, identification algorithms based on classic least-squares and stochastic gradient lead to uncertain estimates and may be ineffective. Simulation examples show that the proposed algorithms are suitable for parameter estimation in Hammerstein nonlinear systems with ill-conditioned situations.

Filtering-Based Parameter Identification Methods for Multivariable Stochastic Systems

This paper presents an adaptive filtering-based maximum likelihood multi-innovation extended stochastic gradient algorithm to identify multivariable equation-error systems with colored noises. The data filtering and model decomposition techniques are used to simplify the structure of the considered system, in which a predefined filter is utilized to filter the observed data, and the multivariable system is turned into several subsystems whose parameters appear in the vectors. By introducing the multi-innovation identification theory to the stochastic gradient method, this study produces improved performances. The simulation numerical results indicate that the proposed algorithm can generate more accurate parameter estimates than the filtering-based maximum likelihood recursive extended stochastic gradient algorithm.

Data filtering‐based recursive identification for an exponential autoregressive moving average model by using the multi‐innovation theory

This study employs the data filtering technique to investigate the recursive identification problems for a non-linear exponential autoregressive model with moving average noise, i.e. the ExpARMA model. Whitening the ExpARMA model by a linear filter, the original identification model is divided into a filtered identification model and a coloured noise model, then a filtering-based extended stochastic gradient algorithm is derived. In order to improve the parameter estimation accuracy, the multi-innovation identification theory is used to develop a filtering-based multi-innovation extended stochastic gradient algorithm for the ExpARMA model. A simulation example is given to demonstrate the superiority of the proposed filtering-based multi-innovation algorithm over the existing algorithms.

Multi-Innovation Stochastic Gradient Parameter and State Estimation Algorithm for Dual-Rate State-Space Systems with d-Step Time Delay

This paper presents a multi-innovation stochastic gradient parameter estimation algorithm for dual-rate sampled state-space systems with d-step time delay by the multi-innovation identification theory. Considering the stochastic disturbance in industrial process and using the gradient search, a multi-innovation stochastic gradient algorithm is proposed through expanding the scalar innovation into an innovation vector in order to obtain more accurate parameter estimates. The difficulty of identification is that the information vector in the identification model contains the unknown states. The proposed algorithm uses the state estimates of the observer instead of the state variables to realize the parameter estimation. The simulation results indicate that the proposed algorithm works well.

Maximum likelihood‐based gradient estimation for multivariable nonlinear systems using the multiinnovation identification theory

SummaryThis article considers the identification problems of multivariable input nonlinear systems with unmeasured disturbances. For the identification difficulty caused by the crossproducts between the parameters of the linear block and the nonlinear block, the key term separation technique is adopted to separate the parameters of the nonlinear block from the parameters of the linear block. By combining the model decomposition technique and the hierarchical identification principle, a key term separation‐based maximum likelihood recursive extended stochastic gradient algorithm with reduced computational complexity is presented to estimate all the parameters directly. By introducing the multiinnovation identification theory, a key term separation‐based maximum likelihood multiinnovation extended stochastic gradient algorithm is proposed to improve the parameter estimation accuracy. The simulation results illustrate the effectiveness of the proposed methods.

Hierarchical least squares parameter estimation algorithm for two-input Hammerstein finite impulse response systems

This paper considers the parameter estimation problems of two-input single-output Hammerstein finite impulse response systems. A hierarchical least squares algorithm is proposed for improving the computational efficiency through combining the hierarchical identification principle and the identification model decomposition, and a multi-innovation least squares algorithm is proposed for enhancing the parameter estimation accuracy based on the multi-innovation identification theory. The key is to derive two sub-identification models, each of which contains a set of merged parameter vectors. The proposed algorithm is effective and can generate highly accurate parameter estimates compared with the over-parametrization identification method, and can be easily extended to multi-input multi-output systems. Finally, an illustrative example is provided to verify the effectiveness of the proposed algorithm.

Maximum likelihood identification of dual‐rate Hammerstein output‐error moving average system

This study discusses the parameter estimation of the Hammerstein output-error moving average system using the dual-rate sampled data. The polynomial transformation technique is used to obtain the identification model of the discussed dual-rate sampled systems. The stochastic gradient optimisation method is an effective optimisation method. Compared with the Newton optimisation, it only needs to calculate the first derivative during the optimisation and the amount of calculation is relatively small. It is a good choice to use the stochastic gradient algorithm for the identification of Hammerstein dual-rate model after using the polynomial transformation technique. In order to improve the convergence speed, a maximum likelihood forgetting factor stochastic gradient identification algorithm is proposed by combining the maximum likelihood principle and the gradient search method. The convergence of the algorithm is analysed by using the stochastic process theory. Furthermore, in order to improve the estimation accuracy of the identification algorithm, a maximum likelihood multi-innovation forgetting factor stochastic gradient algorithm is proposed by using the multi-innovation identification theory. The effectiveness of the proposed algorithms is illustrated by a numerical simulation example and a water tank system.

Decomposition‐based multiinnovation gradient identification algorithms for a special bilinear system based on its input‐output representation

SummaryThis article considers the parameter estimation for a special bilinear system with colored noise. Its input‐output representation is derived by eliminating the state variables in the bilinear system. Based on the input‐output representation of the bilinear system, a multiinnovation generalized extended stochastic gradient (MI‐GESG) algorithm is proposed by using the multiinnovation identification theory. Furthermore, a decomposition‐based multiinnovation (ie, hierarchical multiinnovation) generalized extended stochastic gradient identification (H‐MI‐GESG) algorithm is derived to enhance the parameter estimation accuracy by using the hierarchical identification principle, and a GESG algorithm is presented for comparison. Compared with the existing identification algorithms for the bilinear system, the proposed MI‐GESG and H‐MI‐GESG algorithms can generate more accurate parameter estimation. Finally, a simulation example is provided to verify the effectiveness of the proposed algorithms.

Recursive search‐based identification algorithms for the exponential autoregressive time series model with coloured noise

This study focuses on the recursive parameter estimation problems for the non-linear exponential autoregressive model with moving average noise (the ExpARMA model for short). By means of the gradient search, an extended stochastic gradient (ESG) algorithm is derived. Considering the difficulty of determining the step-size in the ESG algorithm, a numerical approach is proposed to obtain the optimal step-size. In order to improve the parameter estimation accuracy, the authors employ the multi-innovation identification theory to develop a multi-innovation ESG (MI-ESG) algorithm for the ExpARMA model. Introducing a forgetting factor into the MI-ESG algorithm, the parameter estimation accuracy can be further improved. With an appropriate innovation length and forgetting factor, the variant of the MI-ESG algorithm is effective to identify all the unknown parameters of the ExpARMA model. A simulation example is provided to test the proposed algorithms.

Two-stage Gradient-based Iterative Estimation Methods for Controlled Autoregressive Systems Using the Measurement Data

This paper considers the parameter identification problems of controlled autoregressive systems using observation information. According to the hierarchical identification principle, we decompose the controlled autoregressive system into two subsystems by introducing two fictitious output variables. Then a two-stage gradient-based iterative algorithm is proposed by means of the iterative technique. In order to improve the performance of the tracking the time-varying parameters, we derive a two-stage multi-innovation gradient-based iterative algorithm based on the multi-innovation identification theory. Finally, an example is provided to illustrate the effectiveness of the proposed algorithms.

Multi‐innovation gradient estimation algorithms for multivariate equation‐error autoregressive moving average systems based on the filtering technique

This study concentrates on the parameter estimation of multivariate pseudo-linear autoregressive moving average systems by means of the multi-innovation identification theory and data filtering technique. A multi-innovation stochastic gradient algorithm is derived by introducing the innovation length in the stochastic gradient algorithm. Then, the original system is transformed into two subsystems by using a filter. A filtering-based multi-innovation stochastic gradient algorithm is presented, whose parameter estimation accuracy is higher than the multi-innovation stochastic gradient algorithm. The simulation results confirm that these two algorithms are effective.

Gradient-Based Iterative Parameter Estimation Algorithms for Dynamical Systems from Observation Data

It is well-known that mathematical models are the basis for system analysis and controller design. This paper considers the parameter identification problems of stochastic systems by the controlled autoregressive model. A gradient-based iterative algorithm is derived from observation data by using the gradient search. By using the multi-innovation identification theory, we propose a multi-innovation gradient-based iterative algorithm to improve the performance of the algorithm. Finally, a numerical simulation example is given to demonstrate the effectiveness of the proposed algorithms.

Kalman filtering based gradient estimation algorithms for observer canonical state-space systems with moving average noises

This paper focuses on the joint parameter and state estimation issue for observer canonical state-space systems with white noises in state equations and moving average noises in output equations. By means of the Kalman filtering and the gradient search, we derive a Kalman filtering based extended stochastic gradient algorithm. For purpose of achieving the higher parameter estimation accuracy, a Kalman filtering based multi-innovation extended stochastic gradient algorithm is proposed on the basis of the multi-innovation identification theory. Finally, the effectiveness of the proposed algorithms is validated through a numerical example.