Stochastic Convergence Analysis Research Articles

Auxiliary model‐based recursive least squares and stochastic gradient algorithms and convergence analysis for feedback nonlinear output‐error systems

SummaryThis paper deals with the problem of the parameter estimation for feedback nonlinear output‐error systems. The auxiliary model‐based recursive least squares algorithm and the auxiliary model‐based stochastic gradient algorithm are derived for parameter estimation. Based on the stochastic process theory, the convergence of the proposed algorithms are proved. The simulation results indicate that the proposed algorithms can estimate the parameters of feedback nonlinear output‐error systems effectively.

Sampling Points-Independent Identification of the Fractional Maxwell Model of Viscoelastic Materials Based on Stress Relaxation Experiment Data.

Considerable development has been observed in the area of applying fractional-order rheological models to describe the viscoelastic properties of miscellaneous materials in the last few decades together with the increasingly stronger adoption of fractional calculus. The fractional Maxwell model is the best-known non-integer-order rheological model. A weighted least-square approximation problem of the relaxation modulus by the fractional Maxwell model is considered when only the time measurements of the relaxation modulus corrupted by additive noises are accessible for identification. This study was dedicated to the determination of the model, optimal in the sense of the integral square weighted model quality index, which does not depend on the particular sampling points applied in the stress relaxation experiment. It is proved that even when the real description of the material relaxation modulus is entirely unknown, the optimal fractional Maxwell model parameters can be recovered from the relaxation modulus measurements recorded for sampling time points selected randomly according to respective randomization. The identified model is a strongly consistent estimate of the desired optimal model. The exponential convergence rate is demonstrated both by the stochastic convergence analysis and by the numerical studies. A simple scheme for the optimal model identification is given. Numerical studies are presented for the materials described by the short relaxation times of the unimodal Gauss-like relaxation spectrum and the long relaxation times of the Baumgaertel, Schausberger and Winter spectrum. These studies have shown that the appropriate randomization introduced in the selection of sampling points guarantees that the sequence of the optimal fractional Maxwell model parameters asymptotically converge to parameters independent of these sampling points. The robustness of the identified model to the measurement disturbances was demonstrated by analytical analysis and numerical studies.

Identification of Wiener Systems with Recursive Gauss-Seidel Algorithm

The Recursive Gauss-Seidel (RGS) algorithm is presented that is implemented in a one-step Gauss-Seidel iteration for the identification of Wiener output error systems. The RGS algorithm has lower processing intensity than the popular Recursive Least Squares (RLS) algorithm due to its implementation using one-step Gauss-Seidel iteration in a sampling interval. The noise-free output samples in the data vector used for implementation of the RGS algorithm are estimated using an auxiliary model. Also, a stochastic convergence analysis is presented, and it is shown that the presented auxiliary model-based RGS algorithm gives unbiased parameter estimates even if the measurement noise is coloured. Finally, the effectiveness of the RGS algorithm is verified and compared with the equivalent RLS algorithm by computer simulations.

Stochastic convergence analysis of US state economic freedom sub‐components: Evidence from unit root tests for bounded processes

AbstractThis study tests for convergence of the three main sub‐components of economic freedom (government spending, taxation, and labor market freedom) for the 50 US states from 1981 to 2019. In particular, we test for stochastic convergence using unit root tests that account for structural breaks and bounded processes. By controlling for these sources of parameter instability, we find limited evidence in favor of stochastic convergence across the three sub‐components of economic freedom. However, we find more evidence of stochastic convergence with respect to the sub‐components than in the case of the overall economic freedom at the US state level. Our results highlight the distinct nature of the institutional quality across states as represented by government spending, taxation, and labor market freedom.

Research on wettability of nickel coating changes induced in the electrodeposition process

• The sample-based stochastic convergence analysis were adopted to estimate the uncertainty. • Appling CCD method to design electrodeposition experiment selected five main electrodeposition parameters as variables. • Surrogate coupling models was established based on experimental results. • The concentration of ion and brightener have significant effects on the wettability of the coating. In this study, nickel coatings were electrodeposited on copper substrate. The effect of electrodeposition process parameters on surface morphology and wettability of nickel coatings were studied, such as ions source, anode activator, pH buffer, brightener and current density. According to the experiment plan established by Central Composite Design (CCD) with five parameters as mentioned above, nickel coatings have been fabricated on the copper sample using one-step electrodeposition method. Then, response surface methodology was utilized to establish surrogate models based on the static contact angle of nickel coatings which were measured by water liquids. 3D digital microscope, scanning electron microscope (SEM) and energy dispersive X-Ray spectroscopy (EDS) showed rough hierarchical structure, surface morphology and chemical composition on the coating surface, respectively. Subsequently, a sampling-based stochastic model was devoted to weigh the effects of uncertainty in each electrodeposition process parameter on the variability of water contact angle. According to the obtained results, the concentration of brightener as well as ions source greatly affects the wettability of nickel coating. The method of uncertainty analysis can be applied to study the uncertain influence factors in electrochemistry research.

Regional difference and convergence analysis of marine science and technology innovation efficiency in China

Based on panel data of 11 coastal provinces and cities in China during 2006–2015, stochastic frontier analysis(SFA) is used to calculate the efficiency of marine science and technology innovation in this study. Then, the regional differences and influencing factors of this efficiency are analyzed, and finally, a panel unit root test method is used to conduct a stochastic convergence analysis of the efficiency of marine science and technology innovation. According to the results, the overall marine science and technology innovation efficiency shows an upward trend during the statistical period and substantially differs among the 11 coastal provinces and cities. Among them, the 10-year average efficiency of marine science and technology innovation in Shandong province is the highest. In addition, there is a significant negative relation between the degree of government support and efficiency of marine science and technology innovation. The scale of marine scientific research institutions, structure of scientific research personnel, development degree of marine economy, and structure of marine industry have significant positive impacts on the marine scientific and technology innovation efficiency. In summary, there is no stochastic convergence among the Bohai Rim, Yangtze River Delta, and Pan-Pearl River Delta regions. Expanding the scale of marine scientific research institutions and promoting the development of senior personnel in the marine industry, development of marine secondary industries, and integration of marine science and technology innovation with economic development are expected to play positive roles in improving the marine science and technology innovation efficiency.

Study on the Influences of Multiple Parameters With Uncertainty in the Clamp-On Ultrasonic Flowmeter

For a clamp-on ultrasonic flowmeter based on transit-time differential method, the flow rate calculation model with flow velocity correction is established and verified by experiments carried out in a flow standard facility. Based on established flow rate calculation model, a systematic methodology incorporating the parametric uncertainty distribution is proposed to analyze the influences of multiple parameters with uncertainty on measurement error by establishing the sample-based stochastic model. Monte Carlo sampling (MCS) method is used to randomly select the value of each parameter from its prescribed Gaussian distribution and then combining them together as one sample, and the number of selected samples is determined by conducting stochastic convergence analysis of mean value and the standard deviation of input and output parameters. And finally the influence degrees of multiple parameters are analyzed and compared quantitatively on the same dimension to find out which parameters are influential and which are negligible. The results reveal the different influences of multiple parameters with uncertainty, which can be used as reference for performance improvement and measuring error analysis of the clamp-on ultrasonic flowmeter in industrial local measurement.

Finite volume method for solving the stochastic Helmholtz equation

In this paper, we consider the linear finite volume method (FVM) for the stochastic Helmholtz equation, driven by white noise perturbed forcing terms in one-dimension. We first deduce the linear FVM for the deterministic Helmholtz problem. The dispersion equation is presented, and the error between the numerical wavenumber and the exact wavenumber is then analyzed. Comparisons between the linear FVM and the linear finite element method (FEM) are also made. The theoretical analysis and practical calculation indicate that the error of the linear FVM is half of that of the linear FEM. For the stochastic Helmholtz equation, convergence analysis and error estimates are given for the numerical solutions. The effects of the noises on the accuracy of the approximations are illustrated. Numerical experiments are provided to examine our theoretical results.

Stochastic convergence analysis of cubature Kalman filter with intermittent observations

The stochastic convergence of the cubature Kalman filter with intermittent observations (CKFI) for general nonlinear stochastic systems is investigated. The Bernoulli distributed random variable is employed to describe the phenomenon of intermittent observations. According to the cubature sample principle, the estimation error and the error covariance matrix (ECM) of CKFI are derived by Taylor series expansion, respectively. Afterwards, it is theoretically proved that the ECM will be bounded if the observation arrival probability exceeds a critical minimum observation arrival probability. Meanwhile, under proper assumption corresponding with real engineering situations, the stochastic stability of the estimation error can be guaranteed when the initial estimation error and the stochastic noise terms are sufficiently small. The theoretical conclusions are verified by numerical simulations for two illustrative examples; also by evaluating the tracking performance of the optical-electric target tracking system implemented by CKFI and unscented Kalman filter with intermittent observations (UKFI) separately, it is demonstrated that the proposed CKFI slightly outperforms the UKFI with respect to tracking accuracy as well as real time performance.

Stochastic convergence analysis of recursive successive over-relaxation algorithm in adaptive filtering

A stochastic convergence analysis of the parameter vector estimation obtained by the recursive successive over-relaxation (RSOR) algorithm is performed in mean sense and mean-square sense. Also, excess of mean-square error and misadjustment analysis of the RSOR algorithm is presented. These results are verified by ensemble-averaged computer simulations. Furthermore, the performance of the RSOR algorithm is examined using a system identification example and compared with other widely used adaptive algorithms. Computer simulations show that the RSOR algorithm has better convergence rate than the widely used gradient-based algorithms and gives comparable results obtained by the recursive least-squares RLS algorithm.

A stochastic convergence analysis for Tikhonov regularization with sparsity constraints

In this paper we investigate convergence properties of Tikhonov regularization for linear ill-posed problems under a stochastic error model. Namely, we assume that we are given a finite amount of measurements, each contaminated by Gaussian noise with zero mean and known finite variance. Using Besov-space penalty terms to promote sparse solutions with respect to a preassigned wavelet basis, the Ky Fan metric allows us to lift deterministic convergence results into the stochastic setting. In particular, we formulate a general convergence theorem and propose a formula to directly calculate a suitable regularization parameter. This immediately leads to convergence rates. Numerical examples are presented to verify the theoretical results.

Euler-type schemes for weakly coupled forward-backward stochastic differential equations and optimal convergence analysis

We introduce a new Euler-type scheme and its iterative algorithm for solving weakly coupled forward-backward stochastic differential equations (FBSDEs). Although the schemes share some common features with the ones proposed by C. Bender and J. Zhang [Ann. Appl. Probab., 2008, 18: 143–177], less computational work is needed for our method. For both our schemes and the ones proposed by Bender and Zhang, we rigorously obtain first-order error estimates, which improve the half-order error estimates of Bender and Zhang. Moreover, numerical tests are given to demonstrate the first-order accuracy of the schemes.

Nonlinear Stochastic Convergence Analysis of Regional Unemployment Rates in Poland

This paper analyzes convergence of unemployment rates in Poland at NUTS4 level by testing nonlinear convergence, applying the modied KSS-CHLL for each pair of territorial units. The results suggest that actually the convergence is a rare phenomenon and occurs only in 1916 cases out of potential over 70 000 combinations. This paper inquires what systematic reasons contribute to this phenomenon. There are some circumstances under which unemployment convergence should be more awaited than in the others. These include sharing a higher level territorial authority, experiencing similar labour market hardship or sharing the same structural characteristics. For each of these three criteria we analyse the frequency of the dierential nonstationarity within groups (as evidence of convergence) and across groups (as evidence of catching up).

An augmented Lagrangian fish swarm based method for global optimization

This paper presents an augmented Lagrangian methodology with a stochastic population based algorithm for solving nonlinear constrained global optimization problems. The method approximately solves a sequence of simple bound global optimization subproblems using a fish swarm intelligent algorithm. A stochastic convergence analysis of the fish swarm iterative process is included. Numerical results with a benchmark set of problems are shown, including a comparison with other stochastic-type algorithms.

Continuous-Time Model Identification and State Estimation Using Non-Uniformly Sampled Data

This paper presents theory, algorithms and validation results for system identification of continuous-time state-space models from finite non-uniformly sampled input-output sequences. The algorithms developed are methods of model identification and stochastic realization adapted to the continuous-time model context using non-uniformly sampled input-output data. The resulting model can be decomposed into an input-output model and a stochastic innovations model. For state estimation dynamics, we have designed a procedure to provide separate continuous-time temporal update and error-feedback update based on non-uniformly sampled input-output data. Stochastic convergence analysis is provided.

Panel data stochastic convergence analysis of the Mexican regions

The stochastic convergence amongst Mexican Federal entities is analyzed in panel data framework. The joint consideration of cross-section dependence and multiple structural breaks is required to ensure that the statistical inference is based on statistics with good statistical properties. Once these features are accounted for, evidence in favour of stochastic convergence is found. Since stochastic convergence is a necessary, but not sufficient, condition for convergence as predicted by economic growth models, the paper also investigates whether β-convergence process has taken place. We found that the Mexican states have followed either heterogeneous convergence patterns or divergence process throughout the analyzed period.

Tekrarlamalı Gauss-Seidel Yardımcı Değişkenler Algoritması İle Transfer Fonksiyonu Parametrelerinin Yansız Tahmini

In this paper, a recursive algorithm based on the use of Gauss-Seidel iterations and instrumental variables together is introduced for unbiased estimation of transfer function parameters of linear time invariant discrete-time systems. Furthermore, a stochastic convergence analysis of the proposed algorithm is performed and it is shown that the proposed algorithm is an unbiased parameter estimator that gives optimum solution of normal equations even if the measurement noise is colored. The proposed algorithm is used for estimation of transfer function parameters of a sample second order system and compared with similar algorithms by a simulation study. According to the results obtained, it is shown that the proposed algorithm is a good alternative to the others by viewpoints of computational complexity and convergence rate.

Stochastic convergence analysis and parameter selection of the standard particle swarm optimization algorithm

This letter presents a formal stochastic convergence analysis of the standard particle swarm optimization (PSO) algorithm, which involves with randomness. By regarding each particle's position on each evolutionary step as a stochastic vector, the standard PSO algorithm determined by non-negative real parameter tuple { ω , c 1 , c 2 } is analyzed using stochastic process theory. The stochastic convergent condition of the particle swarm system and corresponding parameter selection guidelines are derived.

The Modified Oja-RLS Algorithm, Stochastic Convergence Analysis and Application for Image Compression

The results of theoretical analysis for stochastic convergence of the modified Oja-RLS learning rule are presented. The rule is used to find Karhunen Loeve Transform. Based on this algorithm, an image compression scheme is developed by combining approximated 2D KLT transform and JPEG standard quantization and entropy coding stages. Though 2D KLT transform is of higher complexity than 2D DCT, the resulting PSNR quality of reconstructed images is better even by 2[dB].

Stochastic convergence analysis of a partially adaptive two-layer perceptron using a system identification model

This paper studies the stationary points of the output layer of a two-layer perceptron which attempts to identify the parameters of a specific nonlinear system. The training sequence is modeled as the binary output of the nonlinear system when the input is comprised of an independent sequence of zero-mean Gaussian vectors with independent components. The training rule for the output layer weights is a modified version of Rosenblatt's algorithm. Equations are derived which define the stationary points of the algorithm for an arbitrary output nonlinearity g(x). For the subsequent analysis, the output nonlinearity is specialized to g( x) = sgn( x). The solutions to these equations show that the only stationary points occur when the hidden weights of the perceptron are constrained to lie on the plane spanned by the nonlinear system model. In this plane, the angles of the perceptron weights and of the nonlinear system model weights satisfy a pair of homogeneous linear equations with an infinity of solutions. However, there is a unique solution for algorithm convergence (i.e., zero error) such that the parameters of the two-layer perceptron must exactly match that of the nonlinear system.