Newton-based Extremum Seeking Algorithm Research Articles

Stochastic Extremum Seeking for Dynamic Maps With Delays

We address a Newton-based extremum seeking algorithm for maximizing higher derivatives of unknown maps in the presence of known time delays. Different from all previous works on this topic, we employ stochastic instead of periodic perturbations, allow arbitrarily long output delays and consider a dynamic map to be optimized. We incorporate a novel predictor feedback for delay compensation and show local exponential stability along with convergence to a small neighborhood of the unknown extremum point. For the purpose of the proof, we apply a backstepping transformation and averaging theory in infinite dimensions for stochastic systems. Moreover, simulations highlight the effectiveness of the proposed predictor-feedback scheme.

Extremum Seeking for Maximizing Higher Derivatives of Unknown Maps in Cascade with Reaction-Advection-Diffusion PDEs

We present a generalization of the scalar Newton-based extremum seeking algorithm, which maximizes the map’s higher derivatives in the presence of dynamics described by Reaction-Advection-Diffusion (RAD) equations. Basically, the effects of the Partial Differential Equations (PDEs) in the additive dither signals are canceled out using the trajectory generation paradigm. Moreover, the inclusion of a boundary control for the RAD process stabilizes the closed-loop feedback system. By properly demodulating the map output corresponding to the manner in which it is perturbed, the extremum seeking algorithm maximizes the n-th derivative only through measurements of the own map. The Newton-based extremum seeking approach removes the dependence of the convergence rate on the unknown Hessian of the higher derivative, an effort to improve performance and remove limitations of standard gradient-based extremum seeking. In particular, our RAD compensator employs the same perturbation-based (averaging-based) estimate for the Hessian’s inverse of the function to be maximized provided by a differential Riccati equation applied in the previous publications free of PDEs. We prove local stability of the algorithm, maximization of the map sensitivity and convergence to a small neighborhood of the desired (unknown) extremum by means of backstepping transformation, Lyapunov functional and the theory of averaging in infinite dimensions.

Maximizing Map Sensitivity and Higher Derivatives Via Extremum Seeking

We present a generalization to the scalar Newton-based extremum seeking algorithm which, through perturbation-induced measurements of an unknown steady-state input-to-output map, maximizes the map's higher derivatives. As with other extremum seeking (ES) problems, the scheme relies on derivative estimators and learning dynamics operating in different time scales. We provide analysis for the typical deterministic (sinusoidal) perturbation of the optimal parameter estimate. Also, we alternatively include a stochastic method where the map is perturbed via the sinusoid of Brownian motion about the boundary of a circle. By properly demodulating the map output corresponding to the manner in which it is perturbed, the ES algorithm maximizes the $n{\rm th}$ derivative only through measurements of the map. The Newton-based ES approach removes the dependence of the convergence rate on the unknown Hessian of the higher derivative, an effort to improve performance over standard gradient-based ES. Our design stems from the existing multivariable Newton-based ES algorithm where a differential Riccati equation estimates the inverse Hessian of the function to be maximized. Algebraically computing a direct estimate of the inverse Hessian is susceptible to singularity, whereas employing the Riccati filter removes that potential. We prove local stability of the algorithm for general nonlinear equilibrium profiles of dynamic maps and compare the Newton-based method against the gradient-based method. We also extend this abstraction to multiplayer noncooperative games but limit our attention to a two-player game for notational simplicity and length considerations.

Deterministic and Stochastic Newton-based extremum seeking for higher derivatives of unknown maps with delays

We present a Newton-based extremum seeking algorithm for maximizing higher derivatives of unknown maps in the presence of time delays using deterministic perturbations. Different from previous works about extremum seeking for higher derivatives, arbitrarily long input-output delays are allowed. We incorporate a predictor feedback with a perturbation-based estimate for the Hessian’s inverse using a differential Riccati equation. As a bonus, the convergence rate of the real-time optimizer can be made user-assignable, rather than being dependent on the unknown Hessian of the higher-derivative map. Averaging method for arbitrary shaped derivatives under delays is presented. Exponential stability and convergence to a small neighbourhood of the unknown extremum point are achieved for locally quadratic derivatives by using a backstepping transformation and averaging theory in infinite dimensions. Furthermore, we give a brief introduction into stochastic Newton-based Extremum Seeking for constant output delays, where we show the differences and similarities with respect to the deterministic case. We also present illustrative numerical examples in order to highlight the effectiveness of the proposed predictor-based extremum seeking for time-delay compensation applying both deterministic and stochastic perturbations.

Multivariable Newton-based extremum seeking

We present a Newton-based extremum seeking algorithm for the multivariable case. The design extends the recent Newton-based extremum seeking algorithms for the scalar case and introduces a dynamic estimator of the inverse of the Hessian matrix that removes the difficulty with the possible singularity of a possible direct estimate of the Hessian matrix. The estimator of the inverse of the Hessian has the form of a differential Riccati equation. We prove local stability of the new algorithm for general nonlinear dynamic systems using averaging and singular perturbations. In comparison with the standard gradient-based multivariable extremum seeking, the proposed algorithm removes the dependence of the convergence rate on the unknown Hessian matrix and makes the convergence rate, of both the parameter estimates and of the estimates of the Hessian inverse, user-assignable. In particular, the new algorithm allows all the parameters to converge with the same speed, yielding straight trajectories to the extremum even with maps that have highly elongated level sets, in contrast to curved “steepest descent” trajectories of the gradient algorithm. Simulation results show the advantage of the proposed approach over gradient-based extremum seeking, by assigning equal, desired convergence rates to all the parameters using Newton’s approach.