Stochastic Adaptive Control Research Articles

Stochastic Adaptive Control for ARMAX Systems with Unknown Orders, Time-Delay and Coefficients

For multidimensional ARMAX systems with unknown orders, time-delay and coefficients the optimal adaptive controls are designed so that all unknown parameters contained in the system are consistently estimated and simultaneously, the quadratic loss functions are minimized. Convergence rates of both parameter estimate and loss function are also derived.

Adaptive Control of Time Varying Stochastic Systems

The stability and performance of a stochastic adaptive control algorithm applied to a randomly varying linear system is considered.The class of B-bounded stochastic processes is introduced. It is demonstrated that any i.i .d process whose square possesses a moment generating function is B-bounded, and that this class of stochastic processes is closed under finite dimensional linear transformations. Hence a stationary Gaussian process with rational spectral density is always Bounded.If the parameter and disturbance processes entering the system are B-bounded, and the parameter variation is small in a statistical sense, then closed loop stability in all L2 sense will be guaranteed.By strengthening the hypotheses on the disturbance and parameter processes a Markovian state process may be constructed, and in this case it is shown that loss functions on the joint input-output-parameter estimate process converge to their expectation with respect to an invariant probability at a geometric rate.

Convergence of adaptive control schemes using least-squares parameter estimates

The stability, convergence, asymptotic optimality, and self-tuning properties of stochastic adaptive control schemes based on least-squares estimates of the unknown parameters are examined. It is assumed that the additive noise is i.i.d. and Gaussian, and that the true system is of minimum phase. The Bayesian embedding technique is used to show that the recursive least-squares parameter estimates converge in general. The normal equations of least squares are used to establish that all stable control law designs used in a certainty-equivalent (i.e. indirect) procedure generally yield a stable adaptive control system. Four results are given to characterize the limiting behavior precisely. A certainty-equivalent self-tuning regulator is shown to yield strongly consistent parameter estimates when the delay is strictly greater than one, even without any excitation in the reference trajectory.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Mean Exit Times and Rates of Convergence for Stochastic Systems with Applications to Adaptive Control

A stability theory for stochastic systems based upon Lyapunov functions and the ergodic theory of Markov chains is presented. These results are applied to the analysis of a Kaiman filter based stochastic adaptive control algorithm

Stochastic adaptive control and Martingale limit theory

Recently, S.P. Meyn and P.E. Caines (ibid., vol.AC-32, p.220-6, 1987) have used ergodic theory for Markov processes to give the first asymptotic stability analysis of a nontrivial stochastic adaptive control problem. By nontrivial is meant a stochastic adaptive control problem whose parameter variation has finite nonzero power. They correctly observed that the stochastic Lyapunov function methods fail here, because there is no almost sure parameter convergence. It is shown here how Martingale asymptotics can be used to produce many results close to those of Meyn and Caines, as well as to supply some new observations. Strengths and weaknesses of both approaches are discussed. >

Decentralized stochastic adaptive control of interconnected systems

AbstractThis paper considers the stochastic adaptive control problem for a class of large‐scale systems formed by arbitrary interconnection of subsystems with unknown parameters and non‐linearities. For the estimation of the unknown parameters of the local controllers, stochastic approximation algorithms are used. Conditions sufficient for global stability of the overall system are established. It is shown that the overall tracking error is bounded by a quantity depending on the size of interconnections.

Certainty equivalence control with forcing: revisited

Certainty equivalence control with forcing has been shown to be optimal for several stochastic adaptive control problems with the average cost per unit time criterion. Recently researchers have started looking at stochastic adaptive control problems with a view to minimizing the rate of increase of the learning loss. This criterion is stronger than the average cost per unit time criterion. Certainty equivalence control with forcing does not usually suffice for the learning loss criterion and one has to develop fairly complicated schemes in order to achieve optimality. The objective of this paper is to see how well one might be able to do with a certainty-equivalence-control-with-forcing type of scheme. In particular we construct a class of such schemes whose learning loss is O(( log n) 1+δ) for δ > 0, whereas optimal schemes typically have a O( log n) learning loss.

Adaptive Stochastic Control of Liquid Level Regulators in the Nuclear Plant Production Equipment

The practical aspects of adaptive control problem of liquid level regulators in the nuclear plant production equipment are investigated in the paper. Under operation conditions a solution of this problem is based on use of adaptive proportional-differential control laws and the nonlinear Kalman filters. Two adaptive control algorithms are synthetized: a direct adaptive control algorithm using identification of controlled object transient response and algorithm based on the stochastic identification of regulating valve discharge characteristic. The filtering algorithms provide calculation of smoothed values of parameters observed and the velocities of their variations. An amplitude discrimination of input signals is entered into the filtering algorithms with the purpose to reject the abnormal measurements. A check of unit load variation velocity is used for identification of functioning mode. Under emergency conditions leading to an unloading of turboset, the for-ced control signals of significant interval are used. The significant interval of these signals is defined according to the results of identification of the functional link “turboset loads - regulating valve nominal position” which is occured in the process of loading. The charac-teristics of the synthetized algorithms are illustrated by transient process graphs.

Stochastic adaptive control of a class of non‐minimum‐phase systems

AbstractThis paper presents a computationally efficient pole‐zero placement adaptive controller for linear discrete stochastic systems with coloured noise. The systems are assumed to be open‐loop stable, but not necessarily minimum phase. It is shown that the algorithm asymptotically achieves the required properties of the closed‐loop system.

Convergence rates in stochastic adaptive tracking

Abstract For stochastic control systems described by the ARMAX model with unknown matrix coefficients, the stochastic adaptive control is designed so that the parameter estimates converge to the true values with a rate of convergence O((log n)(log log n) c /n a) with a>0, c>1 and the tracking error tends to its minimum value at a speed of O(n -1/2e) with e>0.

Comments on "An algorithm for a solution of a stochastic adaptive linear quadratic optimal control problem

It is shown that there is an inconsistency in the derivation of the adaptive estimation equations by R. Rishel and L. Harris (see ibid., vol.AC-31, p.1165-70, Dec. 1986). An extra condition that is required is noted. >

Robust recursive identification of multidimensional linear regression models

Stochastic adaptive estimation and control algorithms involving recursive prediction estimates have guaranteed convergence rates when the noise is not ‘too’ coloured, as when a positive-real condition on the noise mode is satisfied. Moreover, the whiter the noise environment the more robust are the algorithms. This paper shows that for linear regression signal models, the suitable introduction of while noise into the estimation algorithm can make it more robust without compromising on convergence rates. Indeed, there are guaranteed attractive convergence rates independent of the process noise colour. No positive-real condition is imposed on the noise model.

Stochastic Adaptive Control for Time-varying Systems Convergent at the Rate of 1/t

In this paper, it is shown that the standard adaptive control algorithms for time-invariant systems have an inherent robustness properties which makes them applicable. without modification, to time-varying systems whose parameters convergent at the rate of 1/t . it is proved that a system which is convergent at the rate of 1/t towords a strictly passive system is eventually strictly passive in a certain sense.Then we show the global convergence for the discrete time stochastic adaptive variance control when the system parameters are convergent at the rate of lit to values satisfying the conditions for globally convergent time-invariant system. Some properties of the systems on strong consistency are studied

Nonlinear Stochastic Adaptive Control Based on Dynamic Compensation and Optimization

For a class of nonlinear stochastic systems described by the generalised Hammerstein (GH) model, an adaptive control algorithm incorporating dynamic compensation and optimization is presented. The closed-loop stability as well as global convergence are proved under certain condition. When the dynamic compensators can't be prespecified off-line because the priori knowledge of the controlled systems is absent or the system parameters may be slow time-invariant, a stable and globally convergent stochastic adaptive control algorithm with the function to adjust the dynamic compensators on-line is further suggested. The algorithms can be applied to systems unstable or/and of ‘nonminimum-phase’ behavior.

Unification of Discrete and Continuous Time Stochastic Adaptive Control Algorithms

In this paper, we present a formulation of a continuous time stochastic adaptive control algorithm and explore the relationship with classical continuous time deterministic schemes and with discrete time stochastic adaptive control algorithms. We give a complete global stability theory and highlight the technical difficulties which arise in the continuous time stochastic case.

The Structural Robustness of Stochastic Systems

In stochastic system theory the analysis of a controlled system is typically carried out by assuming some idealized system model, and then establishing the desired conclusions for the ideal system. An underlying assumption is that the results obtained for the model will apply to the true system if the model is good enough. In this paper we apply the theory of Markov processes to attempt to justify this assumptionIt is assumed that a Markov chain Φ evolving on Euclidean space exists, and that the input and output processes appear as functions of Φ. Stochastic systems of this form are commonly found in stochastic adaptive control problems. For example, the controlled systems of [Meyn and Caines, 1987] and [Goodwin, Ramadge, and Caines, 1981] are of this form when the disturbance processes are independent and identically distributed (i.i.d.).It is demonstrated that invariant probabilities on the state process determine the asymptotic behavior of the overall system. The robustness questions of interest are then explored by introducing a notion of convergence for stochastic systems and investigating the behavior of the invariant probabilities corresponding to a convergent sequence of stochastic systemsThese general results are applied to the analysis of linear state space systems under nonlinear feedbackI am indebted to Peter Caines of the Department of Electrical Engineering at McGill University for suggesting that I apply the “Markov state space” theory of this paper and [Meyn and Caines, 1988] to the analysis of linear state space systems under nonlinear feedback. This application forms the second half of this paper

Theory-based inductive learning: An integration of symbolic and quantitative methods : Spencer Star Department of Computer Science, Carnegie-Mellon University, Pittsburgh, Pennsylvania 15213

The objective of this paper is to propose a method that will generate a causal explanation of observed events in an uncertain world and then make decisions based on that explanation. Feedback can cause the explanation and decisions to be modified. I call the method Theory-Based Inductive Learning (T-BIL). T-BIL integrates deductive learning, based on a technique called Explanation-Based Generalization (EBG) from the field of machine learning, with inductive learning methods from Bayesian decision theory. T-BIL takes as inputs (1) a decision problem involving a sequence of related decisions over time, (2) a training example of a solution to the decision problem in one period, and (3) the domain theory relevant to the decision problem. T-BIL uses these inputs to construct a probabilistic explanation of why the training example is an instance of a solution to one stage of the sequential decision problem. This explanation is then generalized to cover a more general class of instances and is used as the basis for making the next-stage decisions. As the outcomes of each decision are observed, the explanation is revised, which in turn affects the subsequent decisions. A detailed example is presented that uses T-BIL to solve a very general stochastic adaptive control problem for an autonomous mobile robot.

Robustness analysis of identification and adaptive control for stochastic systems

This paper gives a preliminary robustness analysis for stochastic systems with unmodelled dynamics. The extended least squares algorithm and stochastic adaptive control for tracking are defined on the basis of the modelled part of the system, and it is shown that the parameter estimate is closed to the true one and the tracking error is small if the unmodelled dynamics is small.

An Unified Approach to Tracking and Quadratic Cost for Stochastic Adaptive Control Systems

For the multidimensional CARMA system with unknown matrix coef ficients the optimal stochastic adaptive control for quadratic cost limsupn→∞1n∑i=0n-1{(yi−yi*)τQ1(yi−yi*)+u1τQ2ui} with {yj*} being a reference sequence and the strongly consistent estimates for unknown parameters on the basis of the extended least squares (ELS) algorithm and the stochastic gradient algorithm (SG) are given. It is also shown that, if Q1=I and Q2=εI as ε→0 the minimal value of the quadratic cost tends to the minimal tracking error.

Stochastic Stability and the Ergodic Theory of Markov Processes with Applications to Adaptive Control

The principal techniques used up to now for the analysis of stochastic adaptive control systems have been (i) super-martingale (often called stochastic Lyapunov) methods and (ii) methods relying upon the strong consistency of some parameter estimation scheme. Optimal stochastic control and filtering methods have also been employed. Although there have been some successes, the extension of these techniques to a broad class of adaptive control problems, including the case of time varying parameters, has been difficult. In this paper a new approach is adopted: if an underlying Markovian state space system for the controlled process is available, and if this process possesses stationary transition probabilities, then the powerful ergodic theory of Markov processes may be applied. Subject to technical conditions one may deduce (amongst other facts) (i) the existence of an invariant measure μ∞ for the process and (ii) the convergence almost surely of the sample averages of a function of the state process (and of its expectation) to its conditional expectation [μ∞] with respect to a sub-σ-field of invariant sets ΣI. The technique is illustrated by an application to a previously unsolved problem involving a linear system with unbounded random time varying parameters. Work supported by Canada NSERC Grant No.: 1329 and a UK SERC Visiting Research Fellowship.