Gradient Descent Estimation Research Articles

Filtering-based gradient joint identification algorithms for nonlinear fractional-order models with colored noises

The main focus is to realize the on-line parameter estimation of the nonlinear fractional-order model with colored noises in this paper. Firstly, an auxiliary model gradient descent algorithm is derived to synchronously produce the estimates of the parameters, including the fractional-order. In order to decrease the noise interference, the filtering-based gradient descent estimation framework by constructing a linear filter provides a feasible method. Furthermore, the forgetting factor is introduced to the proposed filtering-based algorithm for the sake of improving the convergence rate. Comparative simulation results demonstrate the proposed effectiveness and high approximation accuracy of the proposed algorithms.

A SIEVE STOCHASTIC GRADIENT DESCENT ESTIMATOR FOR ONLINE NONPARAMETRIC REGRESSION IN SOBOLEV ELLIPSOIDS.

The goal of regression is to recover an unknown underlying function that best links a set of predictors to an outcome from noisy observations. in nonparametric regression, one assumes that the regression function belongs to a pre-specified infinite-dimensional function space (the hypothesis space). in the online setting, when the observations come in a stream, it is computationally-preferable to iteratively update an estimate rather than refitting an entire model repeatedly. inspired by nonparametric sieve estimation and stochastic approximation methods, we propose a sieve stochastic gradient descent estimator (Sieve-SGD) when the hypothesis space is a Sobolev ellipsoid. We show that Sieve-SGD has rate-optimal mean squared error (MSE) under a set of simple and direct conditions. The proposed estimator can be constructed with a low computational (time and space) expense: We also formally show that Sieve-SGD requires almost minimal memory usage among all statistically rate-optimal estimators.

Frequency Estimators for SOGI FLL: Modeling, Design, and Equivalence for FLL Advancements

Different frequency estimators have been reported for frequency-locked loop (FLL) implementations. One example is the widely employed gradient descent estimator in second-order generalized integrator (SOGI) FLL. Whether this estimator differs from those in some existing FLLs is unknown. Therefore, in this article, gradient descent and two other estimators with amplitude normalization in the stationary and synchronous reference frames are developed for SOGI FLL implementation. By obtaining linear model of SOGI FLL with the three estimators, equivalence of their FLL is established. Thus, it is shown that when the estimator parameters are designed based on the established equivalence, the FLL dynamics is dictated by the SOGI stage. Thus, research involving orthogonal signal generation is crucial to FLL’s rapid advancements. Furthermore, harmonic rejection in SOGI FLL with the estimators is considered where it is shown that SOGI FLL with amplitude normalization in the synchronous reference frame is ideal for applications involving harmonics.

Frequency-Locked Loops in Electrical Power and Energy Systems: Equivalent or Different to Phase-Locked Loops?

In the electrical power and energy (EPE) area, the traditional concept in designing the synchronization system of power converters is using the phase-locked loop (PLL) concept. An alternative way is using the frequency-locked loop (FLL) concept, which is also known as the adaptive notch filter, adaptive noise cancelation technique, gradient descent estimator, adaptive observer, D-estimation method, and transformation angle detector, among others names. Along with the increased popularity and application of FLLs in EPE systems, the misconception about this design concept is growing. To be more exact, most EPE research communities introduce FLLs as synchronization systems of different properties compared to PLLs. This article aims to explain the main reasons behind this misconception and contribute to dispelling it.

Online drift estimation for jump-diffusion processes

We show the convergence of an online stochastic gradient descent estimator to obtain the drift parameter of a continuous-time jump-diffusion process. The stochastic gradient descent follows a stochastic path in the gradient direction of a function to find a minimum, which in our case determines the estimate of the unknown drift parameter. We decompose the deviation of the stochastic descent direction from the deterministic descent direction into four terms: the weak solution of the non-local Poisson equation, a Riemann integral, a stochastic integral, and a covariation term. This decomposition is employed to prove the convergence of the online estimator and we use simulations to illustrate the performance of the online estimator.

Mixtures of Dirichlet-Multinomial distributions for supervised and unsupervised classification of short text data

Topic detection in short textual data is a challenging task due to its representation as high-dimensional and extremely sparse document-term matrix. In this paper we focus on the problem of classifying textual data on the base of their (unique) topic. For unsupervised classification, a popular approach called Mixture of Unigrams consists in considering a mixture of multinomial distributions over the word counts, each component corresponding to a different topic. The multinomial distribution can be easily extended by a Dirichlet prior to the compound mixtures of Dirichlet-Multinomial distributions, which is preferable for sparse data. We propose a gradient descent estimation method for fitting the model, and investigate supervised and unsupervised classification performance on real empirical problems.

Online Drift Estimation for Jump-Diffusion Processes

We show the convergence of an online stochastic gradient descent estimator to obtain the drift parameter of a continuous-time jump-diffusion process. The stochastic gradient descent follows a stochastic path in the gradient direction of a function to find a minimum, which in our case determines the estimate of the unknown drift parameter. We decompose the deviation of the stochastic descent direction from the deterministic descent direction into four terms: the weak solution of the non-local Poisson equation, a Riemann integral, a stochastic integral, and a covariation term. This decomposition is employed to prove the convergence of the online estimator and we use simulations to illustrate the performance of the online estimator.

Stable Confident Rating Prediction in Collaborative Filtering

Stability of matrix factorization is estimated in terms of gradient descent estimation at each iteration which ultimately defines the stability of recommendations in collaborative filtering techniques. Also, stability is inversely proportional to total number of iterations used for estimating ratings. This gives rise to the need of the method which possesses better rating predictability within less number of iterations. The accuracy of rating prediction is found to be better when user to user trust score is estimated using similarity of individual rated items. The trust estimation is also prone to sparsity due to irregularity of ratings in large volume of data sets (big data). Based on the experimentation strategy and platform requirements, the method of trust evaluation is proposed in this positional paper. The Lyapunov stability solver functions can be used directly in obtaining solution for the trust score amongst users which can bring sufficient stability in learning stages of matrix factorization process and hence better performance in predicting the ratings of non-rated items. Here, the results obtained possess the sufficient gravity for consideration of predicted ratings which also keeps errors in prediction at lowest level for rated items. The papers are addressed from similar domain in related work section to compare proposed work in terms of novelty and performance. The results obtained are satisfactory, which are assessed in terms of mean absolute error (MAE).

A Family of Gradient Descent Grid Frequency Estimators for the SOGI Filter

This paper applies the adaptive gradient descent method to the second-order generalized integrator (SOGI) filter in order to find an online estimation algorithm for the grid frequency, which leads to the proposal of three possible estimators. One of them is identical to the frequency-locked loop algorithm reported in the literature, which proves that it should be understood as a gradient descent estimation algorithm and not as a “frequency locked loop.” The proposed gradient descent estimators are simple and suitable to be implemented into a digital processor with small computational burden. However, due to the SOGI characteristics, the estimators show to be especially sensitive to subharmonic and dc-offset voltage distortion. These problems are removed adopting a cascaded double SOGI approach, which strongly increases the rejection capability to harmonics and enhances the response to voltage sags. Simulation and experimental results are provided to validate the proposed contribution.

Pilot Stick Input Estimates from Attitude Commands as Indicator of Systemic Changes

Control-theoretic modeling of the human operator dynamic behavior in manual control tasks has a long and rich history. In the last two decades, there has been a renewed interest in modeling the human operator. There has also been significant work on techniques used to identify the pilot model of a given structure. The purpose of this research is to attempt to go beyond identification of a pilot model based on collected experimental data and to develop a predictor of pilot behavior. Two sets of experimental data were analyzed: 1) the effects of an adaptive controller on pilot compensation during control surface failures in cruise and 2) heads-up-display concepts during takeoff and landing. A general linear in-parameter model structure is used to represent a pilot. Three different estimation methods are explored—a gradient descent estimator, a least squares estimator with exponential forgetting, and a least squares estimator with bounded gain forgetting. This paper discusses the accuracy of each of the three methods to estimate pilot longitudinal and lateral stick input from the flight director’s commanded pitch and bank attitudes and the differences in piloting techniques during takeoff, cruise, and landing phases of flight.

ML and MAP channel estimation for distributed one-way relay networks with orthogonal training

In this letter, we investigate the individual channel estimation for the classical distributed-space-time-coding (DSTC) based one-way relay network (OWRN) under the superimposed training framework. Without resorting to the composite channel estimation, as did in traditional work, we directly estimate the individual channels from the maximum likelihood (ML) and the maximum a posteriori (MAP) estimators. We derive the closed-form ML estimators with the orthogonal training designing. Due to the complicated structure of the MAP in-channel estimator, we design an iterative gradient descent estimation process to find the optimal solutions. Numerical results are provided to corroborate our studies.

Non‐linear adaptive control for electromagnetic actuators

The authors study here the problem of adaptive ‘soft-landing’ control for electromagnetic actuators. The soft landing requires accurate control of the actuator's moving element between two desired positions. They propose a non-linear adaptive controller to solve the problem of robust trajectory tracking for the moving element, when considering model uncertainties with linear parametrisation. The controller is an integral input-to-state stability (iISS) backstepping controller, merged with gradient descent estimation filters to estimate model uncertainties with linear parametrisation. They show that it ensures bounded tracking errors for bounded estimation errors. Furthermore, iISS result allows us to represent the bound on tracking error as a decreasing function of the estimation error. They demonstrate the effectiveness of this controller with numerical tests.

Modelling and identification of coupled markov chain model with application

A parametrized Markov chain model is developed to model the action of a biological ion channel. The proposed model takes the form of a set of identical binary chains which are dependent according to a coupling parameter. The outputs of these chains combine additively to give a record of ion current buried in large amounts of noise versus time. When varied, the coupling parameter realizes a range of behaviours from tight coupling to complete independence. Other model parameters describe the intrinsic characteristics of the binary chains as well as their behaviour when fully coupled. An identification procedure for the model parameters is developed based on hidden Markov modelling ideas but incorporating a gradient descent estimation algorithm. In this way we cope with the non-linear form of the dependence of the model on the parameters. The model and identification methods are tested and verified on simulated data and real biological data