Certainty Equivalence Control Research Articles

An energy amplification condition for decentralized adaptive stabilization

The author is interested here in the problem of global decentralized adaptive regulation (of the plant output to zero) of square multivariable linear time-invariant systems without any restrictions on relative degrees or matching assumptions. The first solution to this problem was recently reported by the author using Morse's new dynamic certainty equivalent adaptive controller to prove that global stabilization is possible if the unmodeled interconnections do not induce amplification of the energy of the signals in all channels. In this paper the author shows that to preserve global convergence, it is actually enough to have only one nonamplifying channel. Instrumental for the establishment of the authors' result is the fundamental S-procedure losslessness theorem of Megretsky and Treil, together with some basic loop transformations and D-scalings.

Dual Control of Linearly Parameterised Models via Prediction of Posterior Densities

A suboptimal dual control policy is presented for linearly parameterised systems with unknown parameters, additive Gaussian noise and quadratic cost. The dual effect of the control action is taken into account through the prediction of the future posterior densities of the model parameters θ. If θ has a normal prior density, the model response is linear in θ and contains no autoregressive part, then the posterior densities of θ are normal and their covariance matrices are known functions of the control actions. Replacing the unknown future posterior means by the current parameter estimates, one can easily approximate the costto- go. Two examples of FIR models illustrate the superiority ofthis dual control policy over two classical passive policies, namely heuristic certainty equivalence control and open-loop-feedback-optimal control.

The Continuum-Armed Bandit Problem

In this paper we consider the multiarmed bandit problem where the arms are chosen from a subset of the real line and the mean rewards are assumed to be a continuous function of the arms. The problem with an infinite number of arms is much more difficult than the usual one with a finite number of arms because the built-in learning task is now infinite dimensional. We devise a kernel estimator-based learning scheme for the mean reward as a function of the arms. Using this learning scheme, we construct a class of certainty equivalence control with forcing schemes and derive asymptotic upper bounds on their learning loss. To the best of our knowledge, these bounds are the strongest rates yet available. Moreover, they are stronger than the $o(n)$ required for optimality with respect to the average-cost-per-unit-time criterion.

Cyclic Switching and Supervisory Control

Recently a newly introduced concept called “cyclic switching” was was shown provide a viable solution to the long standing certainty equivalence stabilizability problem which arises in the design of estimator-based parameter adaptive controls because of the existence of points in parameter space where the estimated model upon which certainty equivalence control is based, looses stabilizability (Pait and Morse, 1994). Still more recently a simple, ‘high-level’ controller called a ‘supervisor’ was proposed for the purpose of coordinating the switching into feedback with a siso process, of a sequence of linear positioning or set-point controllers from a family of candidate controllers, so as to cause the output of the process to approach and track a constant reference input (Morse, n.d.). The purpose of this paper is to demonstrate that cyclic switching and supervisory control are compatible concepts.

Motorway network control via nonlinear optimization

The on-line application of nonlinear optimization methods for control in motorway networks is presented. The considered control measure is route diversion by Variable Message Signs (VMS). The control task is formulated as a dynamic, nonlinear, discrete-time optimal control problem with constrained control variables and is solved by a gradient-based search. A modified algorithm for the heuristic search of optimal discrete (binary value) control sequences is presented. Feedback control is realized by solving the optimization problem for each control interval over a sufficiently long future time horizon (certainty equivalent open-loop feedback control). As a prerequisite, a state-space description of the network traffic dynamics is derived from a macroscopic model as used also in the simulation program METANET. Simulation results for route diversion on a hypothetical example network are given. The required computational effort is quantified and the control benefits are given in terms of network performance and the switching frequency of VMS.

Route diversion control in motorway networks via nonlinear optimization

The online application of nonlinear optimization methods to feedback control of motorway networks is presented. The considered control measure is route diversion via variable message signs (VMS). The control task is formulated as a dynamic, nonlinear, discrete-time optimal control problem with constrained control variables and is solved by a gradient based search. An algorithm for the heuristic search of optimal discrete (binary) control sequences is also presented. Feedback control is realized by solving the optimization problem for each control interval over a sufficiently long future time horizon (certainty equivalent open-loop feedback control). As a prerequisite, a state-space description of the network traffic dynamics is derived from a macroscopic model that is also used in the simulation program METANET. Simulation results for route diversion on a hypothetical example network are provided. The required computational effort is assessed and the control benefits are evaluated in terms of global network performance, switching frequency of VMS and user optimality.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Transient Bounds of Dynamic Certainty Equivalent Adaptive Controllers

In this paper we show that the transient bounds of Morse’s dynamic certainty equivalent adaptive controller [1] established in [2] can be considerably strenghtened. Specifically, we derive a computable bound on the L∞ norm of the tracking error which, in contrast with the local bound obtained in [2], holds for all systems initial conditions. Also, we add an “adaptation gain” in the high order estimator to prove that, by increasing this gain, the L2 norm of the tracking error can be made arbitrarily small without increasing its L∞ norm. This is an improvement over the results obtained with backstep-ping designs [6] where these performance measures must be traded-off in the selection of the adaptation speed.

A cyclic switching strategy for parameter-adaptive control

This paper introduces a new strategy, called "cyclic switching," to deal with the well-known certainty equivalence control synthesis problem which arises in the design of identifier-based adaptive controllers because of the existence of points in parameter space where the design model /spl Sigma//sub D/, upon which certainty equivalence synthesis is based, loses stabilizability. Unlike most previously suggested methods for handling this problem, the technique proposed here can be employed with or without process excitation. For the technique to work it is not necessary for there to be a mechanism for moving tuned parameters away from values at which /spl Sigma//sub D/ loses stabilizability, and no such mechanism is used.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Adaptive Boundary and Point Control of Linear Stochastic Distributed Parameter Systems

An adaptive control problem for the boundary or the point control of a linear stochastic distributed parameter system is formulated and solved in this paper. The distributed parameter system is modeled by an evolution equation with an infinitesimal generator for an analytic semigroup. Since there is boundary or point control, the linear transformation for the control in the state equation is also an unbounded operator. The unknown parameters in the model appear affinely in both the infinitesimal generator of the semigroup and the linear transformation of the control. Strong consistency is verified for a family of least squares estimates of the unknown parameters. An Itô formula is established for smooth functions of the solution of this linear stochastic distributed parameter system with boundary or point control. The certainty equivalence adaptive control is shown to be self-tuning by using the continuity of th solution of a stationary Riccati equation as a function of parameters in a uniform operator topology. For a quadratic cost functional of the state and the control, the certainty equivalence control is shown to be self-optimizing; that is, the family of average costs converges to the optimal ergodic cost. Some examples of stochastic parabolic problems with boundary control and a structurally damped plate with random loading and point control are described that satisfy the assumptions for the adaptive control problem solved in this paper.

On the certainty equivalence principle and the optimal control of partially observed dynamic games

The purpose of this paper is to explain that while in general the certainty equivalence controller does not achieve the optimal value of a dynamic min-max game with partial information, the optimal controller may be determined using the recently developed information state approach. Examples are provided illustrating when the certainty equivalence controller is or is not optimal. >

MIMO design models and internal regulators for cyclicly switched parameter-adaptive control systems

In Pait and Morse (1994) the concept of "cyclic switching" was proposed as a means of dealing with the well-known stabilizability problem, which arises in adaptive controller synthesis because of the existence of points in parameter space where the design model upon which certainty equivalence control is based loses stabilizability. While the procedure outlined in the above paper is specific enough to enable one to construct cyclicly switched adaptive controllers for SISO process models, for the MIMO case two key issues are left unresolved. The objective of this paper is to address both of these issues, thereby extending excitation free, identifier-based adaptive control methods to minimum and nonminimum phase MIMO linear systems.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

A solution to the continuous-time adaptive decoupling problem

In this paper we present a solution to the problem of designing a globally convergent adaptive decoupling controller for multiinput/multioutput linear time invariant continuous-time systems with unknown (possibly nondiagonal) interactor matrix. The only assumptions about the plant are that it is minimum phase and that an upper bound on its McMillan degree and the relative degrees of each of the entries of its transfer matrix are known. The decoupling is minimal in the sense that asymptotic tracking is achieved with the minimal infinity structure, i.e., the smallest number of integrators. Instrumental for our analysis are the utilization of Morse's new dynamic certainty equivalent adaptive controller (1992) and the output reordering procedure proposed by Mutoh and Ortega (1993) to estimate the interactor structure. >

Transient bounds of dynamic certainty equivalent adaptive controllers

AbstractIn this paper we show that the transient bounds of Morse's dynamic certainty equivalent adaptive controller established previously by us can be considerably strengthened. Specifically, we derive a computable bound on the ℒ︁∞‐norm of the tracking error which, in contrast with the local bound obtained previously by us, holds for all system initial conditions. Also, we add an ‘adaptation gain’ in the high‐order estimator to prove that, by increasing this gain, the ℒ︁∞‐norm of the tracking error can be made arbitrarily small without increasing its ℒ︁∞ norm. This is an improvement over the results obtained with backstepping designs, where these performance measures must be traded off in the selection of the adaptation speed.

A solution to the decentralized adaptive stabilization problem

In this paper we present for the first time a solution to the problem of designing a globally convergent truly decentralized adaptive controller for systems of arbitrary relative degree without any matching assumptions. Specifically, we prove that using Morse's new dynamic certainty equivalent adaptive controller we can determine a class of unmodelled interconnections in the face of which regulation of the plant utput to zero with internal stability is still possible. Furthermore, we prove that the output regulation happens with some guaranteed transient performance bounds. Namely, we show that the L 2 norm of the output is uniformly bounded by the initial parameter estimation error and that, if the latter and the plant initial conditions are sufficiently small, then we can ensure a bound for the L ∞ norm of the output.

Discrete-time entropy formulation of optimal and adaptive control problems

The discrete-time version of the entropy formulation of optimal control of problems developed by G.N. Saridis (1988) is discussed. Given a dynamical system, the uncertainty in the selection of the control is characterized by the probability distribution (density) function which maximizes the total entropy. The equivalence between the optimal control problem and the optimal entropy problem is established, and the total entropy is decomposed into a term associated with the certainty equivalent control law, the entropy of estimation, and the so-called equivocation of the active transmission of information from the controller to the estimator. This provides a useful framework for studying the certainty equivalent and adaptive control laws. >

Stable Indirect Adaptive Control of Continuous-Time Systems with No A Priori Knowledge on the Parameters

Abstract A crucial drawback of indirect adaptive control is the stabilizability or controllability of the estimated plant model which is required in order to calculate the controller parameters. Loss of stabilizability at a particular instant of time may lead to unbounded signais. In this paper, we propose an indirect adaptive control scheme that overcomes the stabilizability problem, and meets the control objective exactly with no additional a priori knowledge on the unknown plant other than its order. The control scheme switches from the usual certainty equivalence control law to a rich open loop control input, according to an adjustable lower bound for the degree of stabilizability of the estimated plant model. The switching terminates in finite time after which no loss of stabilizability can occur, and the certainty equivalence control law is on for the rest of the: me. In contrast to previous schemes, the proposed scheme, referred to as the switched-excitation approach, requires no knowledge of bounds for the stabilizability degree of the unknown plant and guarantees closed-loop stability and zero residual tracking error.

Towards a unified theory of parameter adaptive control. II. Certainty equivalence and implicit tuning

For pt.I see ibid., vol.35, p.1002-12 (1990). It is shown that for a large class of linear multivariable process models a properly designed certainty equivalence controller results in a tunable closed-loop parameterized system. The importance of this result to both the analysis and synthesis of parameter adaptive control systems is discussed. It is demonstrated by means of examples how the connection between certainty equivalence and tunability, together with the concept of tunability itself, can be used to markedly simplify the stability analysis of adaptive control systems. A new family of indirect parameterized controllers is introduced which has capabilities comparable to those of the well-known direct parameterized controllers which have served as basic building blocks for adaptive control systems of all types. The concept of implicit tuning is formalized and its potential importance to adaptive control is briefly discussed. It is explained how the idea of tunability can be used to analyze an implicitly tuned system.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Global tunability of one-dimensional SISO systems

It is shown by example that with a suitably defined certainty equivalence controller Sigma /sub R/, it is possible to make a closed-loop parameterized system Sigma tunable on its entire parameter space P, even though P may contain points at which the parameterized Sigma /sub D/, upon which Sigma /sub R/'s definition is based, is not stabilizable. The implications of this discovery are briefly discussed. >

Minimizing the learning loss in adaptive control of Markov chains under the weak accessibility condition

We consider the adaptive control of Markov chains under the weak accessibility condition with a view to minimizing the learning loss. A certainty equivalence control with a forcing scheme is constructed. We use a stationary randomized control scheme for forcing and compute a maximum likelihood estimate of the unknown parameter from the resulting observations. We obtain an exponential upper bound on the rate of decay of the probability of error. This allows us to choose the rate of forcing appropriately, whereby we achieve a o(f(n) log n) learning loss for any function as .

Minimizing the learning loss in adaptive control of Markov chains under the weak accessibility condition

We consider the adaptive control of Markov chains under the weak accessibility condition with a view to minimizing the learning loss. A certainty equivalence control with a forcing scheme is constructed. We use a stationary randomized control scheme for forcing and compute a maximum likelihood estimate of the unknown parameter from the resulting observations. We obtain an exponential upper bound on the rate of decay of the probability of error. This allows us to choose the rate of forcing appropriately, whereby we achieve a o(f(n) log n) learning loss for any function as .