Stochastic Adaptive Control Problem Research Articles

Soil acidity adaptive control problem

Problem of soil acidity regularization is modeled as stochastic adaptive control problem with a linear difference equation of the dynamics of a field pH level. Stochastic component in the equation represents an individual time variability of soil acidity of an elementary section. We use Bayesian approach to determine a posteriori probability density function of the unknown parameters of the stochastic transition process. The Kullback–Leibler information divergence is used as a measure of difference between true distribution and its estimation. Algorithm for the construction of an adaptive stabilizing control in such a linear control system is proposed in the paper. Numerical realization of the algorithm is represented for a problem of a field soil acidity control.

Stochastic adaptive air-fuel ratio control of spark ignition engines

This paper investigates the stochastic adaptive control problem of the air–fuel ratio in gasoline engines. A discrete-time dynamic model is introduced to represent the cyclic transient characteristics of the air mass and the fuel mass with consideration of residual gas variation. Then, using the model, a feedback adaptive control law is derived to guarantee the convergence of the regulation error of the air–fuel ratio in the sense of mean square stability, where the unknown perturbation in the cyclic fresh air charge is targeted by the adaptation parameter of the controller. To show its effectiveness, experiments are conducted on a V6 gasoline engine. © 2014 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

Adaptive boundary control of stochastic linear distributed parameter systems described by analytic semigroups

A stochastic adaptive control problem is formulated and solved for some unknown linear, stochastic distributed parameter systems that are described by analytic semigroups. The control occurs on the boundary. The “highest-order” operator is assumed to be known but the “lower-order” operators contain unknown parameters. Furthermore, the linear operators of the state and the control on the boundary contain unknown parameters. The noise in the system is a cylindrical white Gaussian noise. The performance measure is an ergodic, quadratic cost functional. For the identification of the unknown parameters a diminishing excitation is used that has no effect on the ergodic cost functional but ensures sufficient excitation for strong consistency. The adaptive control is the certainty equivalence control for the ergodic, quadratic cost functional with switchings to the zero control.

On Ljung’s Approach to System Parameter Identification

The well-known paper by Ljung (1977) is revisited. By using a stochastic approximation theorem, the main result of Ljung (1977) is reproved under conditions weaker than those required in Ljung (1977). The proof is straightforward and consists in verifying conditions which should be satisfied by noise in the stochastic approximation theorem given in Section 2. An application of the main result of the paper to a stochastic adaptive con trol problem is described in Section 4. This application also demonstrates that weakening conditions achieved in the paper is significant.

Robust adaptive systems and self stabilization

A methodology for the global stability analysis and, consequently, for the design of robust deterministic and stochastic adaptive control, filtering, and prediction is introduced. The methodology represents a mathematical formalization of the self-stabilization mechanism which is a natural characteristic of every properly designed adaptive system. The underlying idea is the construction of a suitable Lyapunov function for different periods of adaptation. The effectiveness of the proposed approach is demonstrated by solving the robust deterministic and stochastic adaptive control problems. It is shown that very small algorithm gains may produce very large signals in the adaptive loop, which are unacceptable for practical applications. The intensity of the admissible unmodeled dynamics does not depend on the algorithm gain, and it is specified in terms of the corresponding H/sup infinity / norm. In order to establish goal stability, persistency exciting conditions are not required in the present results.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Stochastic adaptive control and Martingale limit theory

Recently, S.P. Meyn and P.E. Caines (ibid., vol.AC-32, p.220-6, 1987) have used ergodic theory for Markov processes to give the first asymptotic stability analysis of a nontrivial stochastic adaptive control problem. By nontrivial is meant a stochastic adaptive control problem whose parameter variation has finite nonzero power. They correctly observed that the stochastic Lyapunov function methods fail here, because there is no almost sure parameter convergence. It is shown here how Martingale asymptotics can be used to produce many results close to those of Meyn and Caines, as well as to supply some new observations. Strengths and weaknesses of both approaches are discussed. >

Decentralized stochastic adaptive control of interconnected systems

AbstractThis paper considers the stochastic adaptive control problem for a class of large‐scale systems formed by arbitrary interconnection of subsystems with unknown parameters and non‐linearities. For the estimation of the unknown parameters of the local controllers, stochastic approximation algorithms are used. Conditions sufficient for global stability of the overall system are established. It is shown that the overall tracking error is bounded by a quantity depending on the size of interconnections.

Certainty equivalence control with forcing: revisited

Certainty equivalence control with forcing has been shown to be optimal for several stochastic adaptive control problems with the average cost per unit time criterion. Recently researchers have started looking at stochastic adaptive control problems with a view to minimizing the rate of increase of the learning loss. This criterion is stronger than the average cost per unit time criterion. Certainty equivalence control with forcing does not usually suffice for the learning loss criterion and one has to develop fairly complicated schemes in order to achieve optimality. The objective of this paper is to see how well one might be able to do with a certainty-equivalence-control-with-forcing type of scheme. In particular we construct a class of such schemes whose learning loss is O(( log n) 1+δ) for δ > 0, whereas optimal schemes typically have a O( log n) learning loss.

Comments on "An algorithm for a solution of a stochastic adaptive linear quadratic optimal control problem

It is shown that there is an inconsistency in the derivation of the adaptive estimation equations by R. Rishel and L. Harris (see ibid., vol.AC-31, p.1165-70, Dec. 1986). An extra condition that is required is noted. >

The Structural Robustness of Stochastic Systems

In stochastic system theory the analysis of a controlled system is typically carried out by assuming some idealized system model, and then establishing the desired conclusions for the ideal system. An underlying assumption is that the results obtained for the model will apply to the true system if the model is good enough. In this paper we apply the theory of Markov processes to attempt to justify this assumptionIt is assumed that a Markov chain Φ evolving on Euclidean space exists, and that the input and output processes appear as functions of Φ. Stochastic systems of this form are commonly found in stochastic adaptive control problems. For example, the controlled systems of [Meyn and Caines, 1987] and [Goodwin, Ramadge, and Caines, 1981] are of this form when the disturbance processes are independent and identically distributed (i.i.d.).It is demonstrated that invariant probabilities on the state process determine the asymptotic behavior of the overall system. The robustness questions of interest are then explored by introducing a notion of convergence for stochastic systems and investigating the behavior of the invariant probabilities corresponding to a convergent sequence of stochastic systemsThese general results are applied to the analysis of linear state space systems under nonlinear feedbackI am indebted to Peter Caines of the Department of Electrical Engineering at McGill University for suggesting that I apply the “Markov state space” theory of this paper and [Meyn and Caines, 1988] to the analysis of linear state space systems under nonlinear feedback. This application forms the second half of this paper

Theory-based inductive learning: An integration of symbolic and quantitative methods : Spencer Star Department of Computer Science, Carnegie-Mellon University, Pittsburgh, Pennsylvania 15213

The objective of this paper is to propose a method that will generate a causal explanation of observed events in an uncertain world and then make decisions based on that explanation. Feedback can cause the explanation and decisions to be modified. I call the method Theory-Based Inductive Learning (T-BIL). T-BIL integrates deductive learning, based on a technique called Explanation-Based Generalization (EBG) from the field of machine learning, with inductive learning methods from Bayesian decision theory. T-BIL takes as inputs (1) a decision problem involving a sequence of related decisions over time, (2) a training example of a solution to the decision problem in one period, and (3) the domain theory relevant to the decision problem. T-BIL uses these inputs to construct a probabilistic explanation of why the training example is an instance of a solution to one stage of the sequential decision problem. This explanation is then generalized to cover a more general class of instances and is used as the basis for making the next-stage decisions. As the outcomes of each decision are observed, the explanation is revised, which in turn affects the subsequent decisions. A detailed example is presented that uses T-BIL to solve a very general stochastic adaptive control problem for an autonomous mobile robot.

A Stochastic Adaptive Control Problem and Related Non-adaptive Problems

Abstract For a continuous time non-linear stochastic adaptive control problem in which the unknown parameters enter the state equation in an affine way three related nonadaptive stochastic control problems are obtained. Under appropriate conditions optimal controls for any of the related problems give optimal controls for the original adaptive control problem.

An algorithm for a solution of a stochastic adaptive linear quadratic optimal control problem

A variational approach is taken to derive optimality conditions for a discrete-time linear quadratic adaptive stochastic optimal control problem. These conditions lead to an algorithm for computing optimal control laws which differs from the dynamic programming algorithm.

A Comparative Study of Self-Tuning Regulators

The stochastic adaptive control problem for a single-input single output system is considered. The use of a general control scheme allows us to study the servo and regulator problems at the same time.In the case where the model parameters are assumed known we present several control algorithms in a unifying framework .These algorithms are then studied using the self-tuning approach. The theoretical results are illustrated via simulated examples, and an evaluation of the different methods is given by means of performance indices, i.e.response-time, output variance and control variance.

Sensitivity adaptive feedback with estimation redistribution

A stochastic adaptive control problem that incorporates estimation cost considerations is formulated. A sensitivity, index is introduced to represent the total estimation cost. The effort the control must expend on estimation is redistributed according to the accuracy required to achieve a given control objective. The effect of the control on the sensitivity functions and the weighting in the sensitivity index are used to influence the future uncertainties of the system. A dynamic feedback-control algorithm with duality properties is proposed. The control and the sensitivity-weighting matrix are designed such that the control objective function is minimized, subject to an upper bound in the total estimation cost. This control is called Sensitivity Adaptive Feedback with Estimation Redistribution (SAFER) control.