Control Problem For Continuous-time Systems Research Articles

Reinforcement Learning and Adaptive Optimal Control for Continuous-Time Nonlinear Systems: A Value Iteration Approach.

This article studies the adaptive optimal control problem for continuous-time nonlinear systems described by differential equations. A key strategy is to exploit the value iteration (VI) method proposed initially by Bellman in 1957 as a fundamental tool to solve dynamic programming problems. However, previous VI methods are all exclusively devoted to the Markov decision processes and discrete-time dynamical systems. In this article, we aim to fill up the gap by developing a new continuous-time VI method that will be applied to address the adaptive or nonadaptive optimal control problems for continuous-time systems described by differential equations. Like the traditional VI, the continuous-time VI algorithm retains the nice feature that there is no need to assume the knowledge of an initial admissible control policy. As a direct application of the proposed VI method, a new class of adaptive optimal controllers is obtained for nonlinear systems with totally unknown dynamics. A learning-based control algorithm is proposed to show how to learn robust optimal controllers directly from real-time data. Finally, two examples are given to illustrate the efficacy of the proposed methodology.

Optimal Control for Nonlinear Systems Driven by a Known Exogenous Signal

We consider optimal control problems for continuous-time systems with time-dependent dynamics, in which the time-dependence arises from the presence of a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">known</i> exogenous signal. The problem has been elegantly solved in the case of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">linear</i> input-affine systems, for which it has been shown that the solution has a remarkable structure: It is given by the sum of two contributions; a state feedback, which coincides with the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">unperturbed</i> optimal control law, and a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">purely feedforward</i> term in charge of compensating the effect of the exogenous signal. The objective of this article is to extend the above result to <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">nonlinear</i> input-affine systems. It is shown that, while some of the relevant features of the linear case indeed rely heavily on <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">linearity</i> and are not preserved in the nonlinear setting, several structural claims can be proved also in the nonlinear case.

Optimal LQ tracking control for continuous-time systems with pointwise time-varying input delay

This paper investigates the linear quadratic (LQ) tracking problem for linear continuous-time systems with pointwise time-varying input delays. It is assumed that the time-varying input delay takes only one value from a finite set at a given time instant. By defining an indicator, the investigated LQ tracking problem is firstly transformed into a special optimal control problem for continuous-time systems with multiple delays in a single input channel. Then the duality between the LQ tracking for the system with multiple input delay and the smoothing for an associated backward delay free system is established. Finally, an explicit analytical solution to the proposed LQ tracking problem is presented by solving the smoothing problem.

Application of the Preview Control Method to the Optimal Tracking Control Problem for Continuous-Time Systems with Time-Delay

This paper considers the application of the preview control method to the optimal tracking control problem for a class of continuous-time systems with state and input delays. First, through a transformation, the system is transformed into a nondelayed one. Then, the tracking problem of the time-delay system is transformed into one of a nondelayed system via processing of the reference signal. We then apply preview control theory to derive an augmented system for the nondelayed system and design a controller with preview function assuming that the reference signal is previewable. Finally, we obtain the optimal control law of the augmented error system and thus obtain that of the original system by letting the preview length of the reference signal go to zero. Numerical simulations are provided to illustrate the effectiveness and validity of our conclusions.

Decomposition-Based Distributed Control for Continuous-Time Multi-Agent Systems

In this technical note, we consider the problem of designing distributed controller for a continuous-time system composed of a number of identical dynamically coupled subsystems. The underlying mathematical derivation is based on Kronecker product and a special similarity transformation constructed from interconnection pattern matrix that decomposes the system into modal subsystems. This along with the result of extended linear matrix inequality (LMI) formulation for continuous-time systems makes it possible to derive explicit expressions for computing the parameters of distributed controllers for both static state feedback and dynamic output feedback cases. The main contribution of the technical note is the solution of distributed control problem for continuous-time systems under <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">H</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</sub> / α-stability and <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">H</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> / α-stability performances by solving a set of LMIs with noncommon Lyapunov variables. The effectiveness of this method is demonstrated by means of a satellite formation example.

Optimal Control Systems

1R6. Optimal Control Systems. - DS Naidu (Idaho State Univ, Pocatello ID). CRC Press LLC, Boca Raton FL. 2003. 433 pp. ISBN 0-8493-0892-5. $99.95.Reviewed by I Kolmanovsky (Sci Res Lab, MD-2036, Ford Motor Co, 2101 Village Rd, Dearborn MI 48124).The book, Optimal Control Systems, by DS Naidu provides an introduction to key concepts and methods of the optimal control theory for deterministic continuous-time and discrete-time finite-dimensional dynamic systems. The material of the book is based on the author’s experience teaching graduate level optimal control courses and has grown out of the lecture notes used by the author in these courses. After an Introduction (the first chapter of the book), the second chapter describes in detail the main ideas in the calculus of variations (culminating in the study of Euler-Lagrange equations and second variation-based Legendre conditions) and seamlessly transitions to the treatment of the optimal control problems. The Hamiltonian function is introduced early on which subsequently features prominently in the treatment of the Pontryagin’s maximum principle. The end result of Chapter 2 is a solution to several types of optimal control problems (such as fixed end-time, with no control constraints) using the principles of the calculus of variations but stated in the formalism that will later be useful for more general optimal control problems. These developments are already sufficient for the in-depth treatment of Linear Quadratic Optimal Control problem for continuous-time systems, which is one of the central tools in the control theory. This treatment is a subject of Chapters 3 and 4 where the finite horizon and infinite horizon cases, set point and trajectory tracking, fixed-end point regulator system, Linear Quadratic Regulator with a specified degree of stability, and other topics are considered in detail. Frequency domain properties of the linear quadratic regulator are analyzed and the celebrated results on one half to infinity gain margin and 60 degrees phase margin are derived. Chapter 5 treats similar issues as in Chapters 2-4 but for discrete-time systems. In Chapter 6, the treatment of constrained control problems begins with the introduction of the Pontryagin maximum principle for these systems (using the notion of admissible variations) and dynamic programming techniques. The latter are also used to illustrate an alternative approach to deriving continuous-time and discrete-time linear quadratic regulator solutions. Time-optimal control problems are treated in Chapter 7 including the analysis of the number of switchings in the optimal “bang-bang” control laws for linear systems. Other special optimal control problems (energy and fuel-optimal control problems with integral costs) including the case of singular controls are touched upon as well. The developments are completed by considering the point wise-in-time state constraints and describing how they can be treated using either the penalty function method or the slack variable method. The appendices help make the book self-contained by reviewing some of the key results in linear algebra and state space analysis of linear systems, and by providing the listings of the relevant Matlab files (available electronically from the author). The book is very readable and careful attention has certainly been given to provide the reader with a comprehensive set of tools without overburdening with the complex mathematical details and notations (although the key mathematical ideas are, indeed, well exposed and the logic underlying the developments is transparent and thorough). It should be accessible to a wide audience including graduate students and practitioners from engineering and other fields. Optimal Control Systems can be utilized both as a textbook in the introductory courses in the optimal control theory, or as a quick reference by practicing engineers. The use of Matlab/Simulink to treat the examples (including the snippets of the actual code), historical remarks about the scientists behind the major discoveries in the optimal control theory, recipe-style summaries of the key methodologies, exercise problems at the end of each of the chapters are undoubtedly the strong points of this book which significantly contribute to its pedagogical value. On a somewhat more critical side, most of the examples in the book are, in fact, of relatively low order linear systems. This can actually be viewed as an advantage in that the in-depth treatment of these examples is possible so that the main ideas can be illustrated rather well. At the same time, the treatment of more of higher order nonlinear system examples in combination with a more extended introduction into the numerical methods of the optimal control could have been beneficial for the readers to gain an additional insight into how the underlying techniques can be applied.

Min-max model predictive control of nonlinear systems using discontinuous feedbacks

This note proposes a model predictive control (MPC) algorithm for the solution of a robust control problem for continuous-time systems. Discontinuous feedback strategies are allowed in the solution of the min-max problems to be solved. The use of such strategies allows MPC to address a large class of nonlinear systems, including among others nonholonomic systems. Robust stability conditions to ensure steering to a certain set under bounded disturbances are established. The use of bang-bang feedbacks described by a small number of parameters is proposed, reducing considerably the computational burden associated with solving a differential game. The applicability of the proposed algorithm is tested to control a unicycle mobile robot.

A hybrid adaptive control scheme using sampled data and intersampling compensation

An adaptive control problem for continuous-time systems using discrete adaptation technique is examined. We propose a hybrid adaptive scheme whose controller parameters are updated only at sampling instants using sampled data. In order to establish stability for the closed-loop adaptive system, an additional feedback loop is designed to compensate possible poor intersampling behavior. A stability analysis is given for the resulting adaptive control system. Furthermore, it is shown that the intersampling compensation can be designed such that for any given sampling rate, the asymptotic tracking performance of the closed-loop hybrid adaptive control system can be made arbitrarily close to that of the continuous adaptive control system.

Sampled-Data H2 Control for Symmetric Systems

This paper considers digital control problems for continuous-time systems having symmetric transfer functions. It is supposed that digital controllers are implemented with zero-order hold, discrete-time controller, and ideal or integral sampler. The following results are obtained : (1) stabilization : transfer functions of discretized systems with both types of samplers keep symmetries of original continuous-time transfer functions. A condition for discrete-time stabilizing controllers to be symmetric is obtained. (2) discrete-time H2 control synthesis : discrete-time H2 optimal controllers for symmetric discrete-time generalized plants are symmetric. (3) sampled-data H2 control synthesis : for symmetric continuous-time generalized plants with integral samplers, sampled-data H2 optimal controllers are symmetric, while symmetries can not be utilized with ideal samplers.

The equivalent discrete-time optimal control problem for continuous-time systems with stochastic parameters

This paper considers the transformation of the digital optimal control problem to an equivalent discrete-time optimal control problem for linear continuous-time systems with continuous-time white stochastic parameters and additive continuous-time white noise. The observations available at the sampling instants are in general non-linear and corrupted by discrete-time noise. The equivalent discrete-time system has white stochastic parameters. Expressions are derived for the first and second moments of these parameters and for the cost criterion parameters which are explicit in the parameters and the statistics of the continuous-time system.

Optimal control for linear continuous-time systems with general noises based upon sampled data

This paper is concerned with the optimal control problem for continuous-time systems with general noises, based upon sampled data, under the quadratic cost functional. The system is described by a linear stochastic integral equation, the observations are made at discrete times, and the noise processes are not assumed zero-moan Gaussian and/or white. Derived is the optimal controller algorithm, where the optimal input consists of the following two: (1) the optimal input for usual linear-quadratic-Gaussian control systems and (2) its correction input due to the fact that the noise processes arc non-white and have non-zero means.