LQG Optimal Control Problem Research Articles

Irregular LQG optimal control problem involving multiplicative noise

Irregular LQG optimal control problem involving multiplicative noise

Convex analysis for LQG systems with applications to major–minor LQG mean–field game systems

We develop a convex analysis approach for solving LQG optimal control problems and apply it to major–minor (MM) LQG mean–field game (MFG) systems. The approach retrieves the best response strategies for the major agent and all minor agents that attain an ϵ-Nash equilibrium. An important and distinctive advantage to this approach is that unlike the classical approach in the literature, we are able to avoid imposing assumptions on the evolution of the mean–field. In particular, this provides a tool for dealing with complex and non–standard systems.

Design of coherent quantum observers for linear quantum systems

Quantum versions of control problems are often more difficult than their classical counterparts because of the additional constraints imposed by quantum dynamics. For example, the quantum LQG and quantum optimal control problems remain open. To make further progress, new, systematic and tractable methods need to be developed. This paper gives three algorithms for designing coherent quantum observers, i.e., quantum systems that are connected to a quantum plant and their outputs provide information about the internal state of the plant. Importantly, coherent quantum observers avoid measurements of the plant outputs. We compare our coherent quantum observers with a classical (measurement-based) observer by way of an example involving an optical cavity with thermal and vacuum noises as inputs.

Coherent quantum LQG control

Based on a recently developed notion of physical realizability for quantum linear stochastic systems, we formulate a quantum LQG optimal control problem for quantum linear stochastic systems where the controller itself may also be a quantum system and the plant output signal can be fully quantum. Such a control scheme is often referred to in the quantum control literature as “coherent feedback control”. It distinguishes the present work from previous works on the quantum LQG problem where measurement is performed on the plant and the measurement signals are used as the input to a fully classical controller with no quantum degrees of freedom. The difference in our formulation is the presence of additional non-linear and linear constraints on the coefficients of the sought after controller, rendering the problem as a type of constrained controller design problem. Due to the presence of these constraints, our problem is inherently computationally hard and this also distinguishes it in an important way from the standard LQG problem. We propose a numerical procedure for solving this problem based on an alternating projections algorithm and, as an initial demonstration of the feasibility of this approach, we provide fully quantum controller design examples in which numerical solutions to the problem were successfully obtained. For comparison, we also consider the case of classical linear controllers that use direct or indirect measurements, and show that there exists a fully quantum linear controller which offers an improvement in performance over the classical ones.

Quantum LQG Control with Quantum Mechanical Controllers

Based on a recently developed notion of physical realizability for quantum linear stochastic systems, we formulate a quantum LQG optimal control problem for quantum linear stochastic systems where the controller itself may also be a quantum system and the plant output signal can be fully quantum. This is distinct from previous works on the quantum LQG problem where measurement is performed on the plant and the measurement signals are used as input to a fully classical controller with no quantum degrees of freedom. The difference in our formulation is the presence of additional non-linear and linear constraints on the coefficients of the sought after controller, rendering the problem as a type of constrained controller problem. Due to the presence of these constraints our problem is inherently computationally hard and this distinguishes it in an important way from the standard LQG problem. We propose a numerical procedure for solving this problem based on an alternating projections algorithm and, as initial demonstration of the feasibility of this approach, we provide a fully quantum controller design example in which a numerical solution to the problem was successfully obtained.

The problem of LQG optimal control via a limited capacity communication channel

The paper addresses a LQG optimal control problem involving bit-rate communication capacity constraints. A discrete-time partially observed system perturbed by white noises is studied. Unlike the classic LQG control theory, the control signal must be first encoded, then transmitted to the actuators over a digital communication channel with a given bandwidth, and finally decoded. Both the control law and the algorithms of encoding and decoding should be designed to archive the best performance. The optimal control strategy is obtained. It is shown that where the estimator-coder separation principle holds, the controller-coder one fails to be true.

Adaptive Control using Controllers of Restricted Structure

A novel approach to adaptive control is presented, based on a multiple-model LQG optimal control problem. The control law calculation is for a restricted structure controller, in this case a modified-PID form. The restricted structure algorithm given enables a simplified cost-function to be optimised numerically. The n+1 models are obtained from a non-linear system description linearised at n appropriate operating points, plus one on-line linear model identified with recursive least squares, thus providing the adaptation. The main advantage of the approach is the cautious nature of the solution. Each new update of the controller involves averaging the cost across fixed and present models, providing cautious robust adaptive control action. The method is applied to a non-linear dynamic ship-positioning simulation, and results are given.

Restricted-structure LQG optimal control for continuous-time systems

A novel LQG optimal-control problem is developed for continuous-time systems, where the structure of the controller is assumed to be fixed a priori. The controller may be chosen to be of reduced order, lead/lag or PID forms, and the optimal causal controller is required to minimise the usual LQG cost index. The theoretical problem considered is to obtain the causal, stabilising, controller, of a prespecified form, that minimises an LQG criterion. The underlying practical problem of importance is to obtain a simple method of tuning low-order controllers given only an approximate model of the process. The robustness of the solution can be improved by tuning the controller using QFT methods.

An exact solution to continuous-time mixed H/sub 2//H/sub /spl infin/#x221E;/ control problems

Multiobjective control problems have been the object of much attention in the past few years, since they allow for handling multiple, perhaps conflicting, performance specifications and model uncertainty. One of the earliest multiobjective problems is the mixed H/sub 2//H/sub /spl infin// control problem, which can be motivated as a nominal LQG optimal control problem subject to robust stability constraints. This problem has proven to be surprisingly difficult to solve, and at this time no closed-form solutions are available. Moreover, it has been shown that except in some trivial cases, the optimal controller is infinite-dimensional. In this paper, we propose a solution to general continuous-time mixed H/sub 2//H/sub /spl infin// problems, based upon constructing a family of approximating problems, obtained by solving an equivalent discrete-time problem. Each of these approximations can be solved efficiently, and the resulting controllers converge strongly in the H/sub 2/ topology to the optimal solution.

LQG optimal control of constrained discrete time systems

In this paper, we study the discrete time LQG optimal control problem, with convex quadratic con straints. We show that the Separation Theorem does not hold. However, a generalization of this result which we call the Quasi-Separation Theorem applies. The optimal control is obtained by solving a convex, finite-dimensional optimization problem that is equivalent to solving a sequence of unconstrained LQG control problems.

An exact solution to general four-block discrete-time mixed ℋ/sub 2//ℋ/sub ∞/ problems via convex optimization

The mixed /spl Hscr//sub 2//spl Hscr//sub /spl infin// control problem can be motivated as a nominal LQG optimal control problem subject to robust stability constraints, expressed in the form of an /spl Hscr//sub /spl infin// norm bound. This paper contains a solution to a general four-block mixed /spl Hscr//sub 2///spl Hscr//sub /spl infin// problem, based upon constructing a family of approximating problems. Each one of these problems consists of a finite-dimensional convex optimization and an unconstrained standard /spl Hscr//sub /spl infin// problem. The set of solutions is such that in the limit the performance of the optimal controller is recovered, allowing one to establish the existence of an optimal solution. Although the optimal controller is not necessarily finite-dimensional, it is shown that a performance arbitrarily close to the optimal can be achieved with rational controllers. Moreover, the computation of a controller yielding a performance /spl epsiv/-away from optimal requires the solution of a single optimization problem, a task that can be accomplished in polynomial time.

A quasi-separation theorem for LQG optimal control with IQ constraints

We consider the deterministic, the full observation and the partial observation LQG optimal control problems with finitely many IQ (integral quadratic!) constraints, and show that Wohnam's famous Separation Theorem for stochastic control has a generalization to this case. Although the problems of filtering and control are not independent, we show that the interdependence of these two problems is so superficial that in effect, they are problems which can be treated separately. It is in this context that the label Quasi-Separation Theorem is to be understood. We conclude with a discussion of computation issues and show how gradient-type optimization algorithms can be used to solve these problems. In this way, a systematic computation algorithm is derived.

On the relative entropy of discrete-time Markov processes with given end-point densities

Given a Markov process x(k) defined over a finite interval I=[0,N], I/spl sub/Z we construct a process x*(k) with the same initial density as x, but a different end-point density, which minimizes the relative entropy of x and x*. It is shown that x* is a Markov process in the same reciprocal class as x. In the Gaussian case, the minimum relative entropy problem is related to a minimum energy LQG optimal control problem.

Relating H2 and H∞ bounds for finite-dimensional systems

For a linear time invariant system, the infinity-norm of the transfer function can be used as a measure of the gain of the system. This notion of system gain is ideally suited to the frequency domain design techniques such as H ∞ optimal control. Another measure of the gain of a system is the H 2 norm, which is often associated with the LQG optimal control problem. The only known connection between these two norms is that, for discrete time transfer functions, the H 2 norm is bounded by the H ∞ norm. It is shown in this paper that, given precise or certain partial knowledge of the poles of the transfer function, it is possible to obtain an upper bound of the H ∞ norm as a function of the H 2 norm, both in the continuous and discrete time cases. It is also shown that, in continuous time, the H 2 norm can be bounded by a function of the H ∞ norm and the bandwidth of the system.

State-Space Approach to LQG Multivariable Predictive and Feedforward Optimal Control

Two extensions of the LQG optimal control problem are presented which enable prediction and feedforward action to be introduced in a state equation framework. Predictive control is first considered using a discrete-time state-space system model. Future reference values are assumed to be known p-steps ahead and the controller has a two-degrees of freedom structure. The mechanism for introducing feedforward control and hence providing a third degree of freedom is also considered. Both prediction and feedforward action are simple to design and implement using the proposed state-space approach.

An exact solution to general SISO mixed ℋ/sub 2//ℋ/sub ∞/ problems via convex optimization

The mixed /spl Hscr//sub 2///spl Hscr//sub /spl infin// control problem can be motivated as a nominal LQG optimal control problem, subject to robust stability constraints, expressed in the form of an /spl Hscr//sub /spl infin// norm bound. A related modified problem consisting on minimizing an upper bound of the /spl Hscr//sub 2/ cost subject to /spl Hscr//sub /spl infin// constraints was introduced by Bernstein-Haddad (1989). Although there presently exist efficient methods to solve this modified problem, the original problem remains, to a large extent, still open. In this paper we propose a method for solving general discrete-time SISO /spl Hscr//sub 2///spl Hscr//sub /spl infin// problems. This method involves solving a sequence of problems, each one consisting of a finite-dimensional convex optimization and an unconstrained Nehari approximation problem.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Program CC's implementation of the wiener‐hopf lqg optimal control problem

AbstractThe linear quadratic Gaussian (LQG) optimal control problem of single‐input‐single‐output (SISO) analogue and digital systems can be solved using Wiener‐Hopf techniques of spectral and partial factorization. A new form of the Wiener‐Hopf result is derived, followed by a description of how the solution is implemented in Program CC, a user‐friendly, command driven, computer‐aided‐control‐system‐design (CACSD) package.

LQG multivariable controllers : minimum variance interpretation for use in self-tuning systems

It is first demonstrated that every LQG optimal control problem can be rewritten as a minimum output variance control problem for a related system model. This system may be defined in either minimum or non-minimum phase forms. The optimal controller derived from the equivalent minimum variance problem is identical to the LQG optimal controller. During the solution of the equivalent problem an equation is derived which is similar to the implicit relationships used in direct self-tuning control schemes. This enables the problems in constructing a multivariable implicit LQG self-tuning controller, using polynomial matrix methods, to be explored. The inherent difficulties become clear but a possible strategy is evident. Finally the results are specialized to the scalar case and a very simple implicit LQG self-tuning algorithm is described.

The adaptive LQG problem--Part I

This paper is concerned with a rigorous study of the dual problem of Fel'dbaum, i.e., the LQG optimal control problem in the presence of Bayesian parameter uncertainty. The solution of this problem involves two parts, one relating to filtering and one to control. Although we establish our filtering result in complete generality in the last section, most of the paper concentrates on the finite parameter case to ease the exposition. The control result that we establish is incomplete in the sense that the smoothness of the optimal cost function is assumed rather than proved. Nevertheless, our results and methods are such that we arrive at a new proof of the classical separation theorem showing that the well-known LQG feedback law is optimal within the widest possible class of admissible controls. As this new proof avoids all talk of dependence of the sigma algebra on the control, weak solutions, measure transformation techniques, etc., we feel that this result will help to clarify what is involved in the classical separation theorem.

Design of closed-loop optimal controllers for systems with shape deterministic inputs

The design of closed-loop optimal controllers for systems with output-feedback is considered. The disturbance, initial condition and reference signals in the system are assumed to be either deterministic or shape-deterministic or a sum of both, The shape-deterministic signals have a specified time response, but the magnitude of the response is a random variable with known mean and covariance. The performance criterion includes the expectation averaging operator, and the cost is measured over the positive-time interval only. The performance criterion also includes dynamical operators which may be chosen to achieve specified response characteristics. The solution for the closed-loop controller is obtained, for both tracking and regulating problems, in the s-domain. The controller is not, in general, the same as those obtained from either the LQP or LQG optimal-control problems. The advantages of the technique are that output feedback, rather than state-feedback, is used, and both random and deterministic signals are optimised. In other approaches to such problems, there are severe difficulties in defining a performance criterion which is suitable for both deterministic and stochastic signals.