Linear Quadratic Optimal Control Problem Research Articles

Optimal Control and Stabilization for Networked Systems With Input Delay and Markovian Packet Losses

The linear quadratic regulation (LQR) problem for discrete-time networked control systems (NCSs) is investigated in this article. The difference from most previous works is that input delay and packet losses occur simultaneously in the communication channel connecting the controller to the actuator. Moreover, the data packet dropout is modeled as a time-homogeneous Markov process which poses challenges due to the temporal correlation. The contributions of this article are twofold. First, by applying the maximum principle involving Markov jumps and delay, the linear quadratic optimal control problem in finite horizon is solved and the solution is given in terms of a forward and a backward stochastic difference equations (FBSDEs-M). Second, under a basic assumption, the infinite horizon optimal control problem is solved and the necessary and sufficient condition for mean square stabilization is given in terms of the solutions to coupled algebraic Riccati-type equations with Markov jumps. The presented results are new to the best of our knowledge since there is no existing work that tackles delay and Markovian packet dropouts simultaneously. It is a generalization of the previous work in which the packet dropouts is modeled as an independent identically distributed Bernoulli process.

Two Inverse Problems Solution by Feedback Tracking Control

Two inverse ill-posed problems are considered. The first problem is an input restoration of a linear system. The second one is a restoration of time-dependent coefficients of a linear ordinary differential equation. Both problems are reformulated as auxiliary optimal control problems with regularizing cost functional. For the coefficients restoration problem, two control models are proposed. In the first model, the control coefficients are approximated by the output and the estimates of its derivatives. This model yields an approximating linear-quadratic optimal control problem having a known explicit solution. The derivatives are also obtained as auxiliary linear-quadratic tracking controls. The second control model is accurate and leads to a bilinear-quadratic optimal control problem. The latter is tackled in two ways: by an iterative procedure and by a feedback linearization. Simulation results show that a bilinear model provides more accurate coefficients estimates.

Second Chebyshev wavelets (SCWs) method for solving finite-time fractional linear quadratic optimal control problems

In this paper, we present an indirect computational procedure based on the truncated second kind Chebyshev wavelets for finding the solutions of Caputo fractional time-invariant linear optimal control systems which the functional cost consists of the finite-time quadratic cost function. Utilizing the operational matrices of second Chebyshev wavelets (SCWs) of the Riemann–Liouville fractional integral, the corresponding linear two-point boundary value problem (TPBVP), obtained from the fractional Euler–Lagrange equations, is reduced to a coupled Sylvester-type matrix equation. An equivalent linear matrix form using the Kronecker product is constructed. The upper bound of the error of the SCWs approximation and the convergence of the proposed method are investigated. Low computational complexity and flexible accuracy are two important superiorities of this approach. Numerical experiments provide satisfactory results compared to the exiting techniques.

Linear Quadratic Optimal Control Problems of Delayed Backward Stochastic Differential Equations

This paper is concerned with a linear quadratic optimal control problem of delayed backward stochastic differential equations. An explicit representation is derived for the optimal control, which is a linear feedback of the entire past history and the expected value of the future state trajectory in a short period of time. To obtain the optimal feedback, a new class of delayed Riccati equations and delayed-advanced forward-backward stochastic differential equations are introduced. Furthermore, the unique solvability of their solutions are discussed in detail.

Open-loop solvability for mean-field stochastic linear quadratic optimal control problems of Markov regime-switching system

<p style="text-indent:20px;">This paper investigates the mean-field stochastic linear quadratic optimal control problem of Markov regime switching system (M-MF-SLQ, for short). The representation of the cost functional for the M-MF-SLQ is derived using the technique of operators. It is shown that the convexity of the cost functional is necessary for the finiteness of the M-MF-SLQ problem, whereas uniform convexity of the cost functional is sufficient for the open-loop solvability of the problem. By considering a family of uniformly convex cost functionals, a characterization of the finiteness of the problem is derived and a minimizing sequence, whose convergence is equivalent to the open-loop solvability of the problem, is constructed. We demonstrate with a few examples that our results can be employed for tackling some financial problems such as mean-variance portfolio selection problem.</p>

Interpreting the dual Riccati equation through the LQ reproducing kernel

In this study, we provide an interpretation of the dual differential Riccati equation of Linear-Quadratic (LQ) optimal control problems. Adopting a novel viewpoint, we show that LQ optimal control can be seen as a regression problem over the space of controlled trajectories, and that the latter has a very natural structure as a reproducing kernel Hilbert space (RKHS). The dual Riccati equation then describes the evolution of the values of the LQ reproducing kernel when the initial time changes. This unveils new connections between control theory and kernel methods, a field widely used in machine learning.

The linear quadratic optimal control problem for discrete-time Markov jump linear singular systems

In this paper, we address the linear quadratic optimal control problem for discrete-time Markov jump linear singular systems. The results are obtained under conditions that bring some additional structure to the quite general and complex class of systems being considered. The approach involves base transformations and restricts the control actions into an appropriate subspace to ensure regularity of the closed-loop system. Both finite and infinite time horizon problems are considered. The results are evaluated by means of an example.

A BSDE approach to stochastic linear quadratic control problem

AbstractIn this article, we study a kind of linear quadratic optimal control problem driven by forward–backward stochastic differential equations (FBSDEs in short) with deterministic coefficients. The cost functional is defined by the solution of FBSDEs. By means of the Girsanov transformation, the original issue is turned equivalently into the classical LQ problem. By functional analysis approach, some necessary and sufficient conditions for the existence of optimal controls have been obtained. Moreover, we investigate the relationship between two groups of first‐order and second‐order adjoint equations. A new stochastic Riccati equation is derived, which leads to the state feedback form of optimal control. By introducing a new Hamiltonian function with an exponential factor, we establish the stochastic maximum principle to deal with the stochastic linear quadratic problem for forward–backward stochastic system with nonconvex control domain using first‐order adjoint equation. An illustrative example is given as well.

Indefinite stochastic linear-quadratic optimal control problems with random coefficients: Closed-loop representation of open-loop optimal controls

This paper is concerned with a stochastic linear-quadratic optimal control problem in a finite time horizon, where the coefficients of the control system are allowed to be random, and the weighting matrices in the cost functional are allowed to be random and indefinite. It is shown, with a Hilbert space approach, that for the existence of an open-loop optimal control, the convexity of the cost functional (with respect to the control) is necessary; and the uniform convexity, which is slightly stronger, turns out to be sufficient, which also leads to the unique solvability of the associated stochastic Riccati equation. Further, it is shown that the open-loop optimal control admits a closed-loop representation. In addition, some sufficient conditions are obtained for the uniform convexity of the cost functional, which are strictly more general than the classical conditions that the weighting matrix-valued processes are positive (semi-) definite.

Open-loop and closed-loop solvabilities for stochastic linear quadratic optimal control problems of Markovian regime switching system

This paper investigates the stochastic linear quadratic (LQ, for short) optimal control problem of Markovian regime switching system. The representation of the cost functional for the stochastic LQ optimal control problem of Markovian regime switching system is derived by the technique of Itô’s formula with jumps. For the stochastic LQ optimal control problem of Markovian regime switching system, we establish the equivalence between the open-loop (closed-loop, resp.) solvability and the existence of an adapted solution to the corresponding forward-backward stochastic differential equation with constraint. (i.e., the existence of a regular solution to Riccati equations). Also, we analyze the interrelationship between the strongly regular solvability of Riccati equations and the uniform convexity of the cost functional. Finally, we present an example which is open-loop solvable but not closed-loop solvable.

A concise introduction to control theory for stochastic partial differential equations

<p style='text-indent:20px;'>The aim of this notes is to give a concise introduction to control theory for systems governed by stochastic partial differential equations. We shall mainly focus on controllability and optimal control problems for these systems. For the first one, we present results for the exact controllability of stochastic transport equations, null and approximate controllability of stochastic parabolic equations and lack of exact controllability of stochastic hyperbolic equations. For the second one, we first introduce the stochastic linear quadratic optimal control problems and then the Pontryagin type maximum principle for general optimal control problems. It deserves mentioning that, in order to solve some difficult problems in this field, one has to develop new tools, say, the stochastic transposition method introduced in our previous works.

Решение линейно-квадратичной задачи оптимального управления на основе конечномерных моделей

We consider a linear-quadratic optimal control problem with indefinite matrices and the interval control constraint. The problem also has a regularizationparameter in the functional. The approximate solution of the problem is carried out on subsets of admissible controls, which are formed using linear combinations of special functions with an orientation to the optimal control structure due to the maximum principle. As a result of this procedure, a finite-dimensional quadratic optimization problem with the interval constraint on variables is obtained. The following relations between the variational problem and its finite-dimensional model are established: the convexity property of the optimal control problem is preserved for finite-dimensional model; a nonconvex optimal control problem under a certain condition on the regularization parameter (estimate from below) is approximated by a convex quadratic problem, which is solved in a finite number of operations;a special non-convex optimal control problem with an upper bound on the regularization parameter passes into the problem of minimizing a concave function on a finite set of points. A special case of a non-convex optimal control problem for the maximum of the norm of the final state is distinguished. Two procedures for improving the extreme points of finite-dimensional model are constructed, which reduce the computational costs for the global solution of the problem within the framework of the linearization method.

Direct method to solve linear-quadratic optimal control problems

&lt;p style='text-indent:20px;'&gt;In this work, we have proposed a new approach for solving the linear-quadratic optimal control problem, where the quality criterion is a quadratic function, which can be convex or non-convex. In this approach, we transform the continuous optimal control problem into a quadratic optimization problem using the Cauchy discretization technique, then we solve it with the active-set method. In order to study the efficiency and the accuracy of the proposed approach, we developed an implementation with MATLAB, and we performed numerical experiments on several convex and non-convex linear-quadratic optimal control problems. The obtained simulation results show that our method is more accurate and more efficient than the method using the classical Euler discretization technique. Furthermore, it was shown that our method fastly converges to the optimal control of the continuous problem found analytically using the Pontryagin's maximum principle.&lt;/p&gt;

Rate Constrained Discrete-time Maximum Principle

Abstract Actuators in real-world systems operate under limited bandwidth and saturation constraints. Mathematically, these limitations manifest as certain types of constraints being imposed on the sequence of control action. While saturation constraints have been studied extensively, bandwidth limitations in the actuators have received very little attention in the control community. While an attempt has been made in the direction of incorporating frequency constraints at the synthesis stage, rate constraints on the control at the design stage has not been studied extensively in the regime of discrete-time. This article tries to fill in that gap by incorporating rate constraints on the control trajectories. We propose a new discrete-time Pontryagin maximum principle (PMP) with rate constraints being imposed on the control and derive a set of first order necessary conditions for optimality. As an illustration, the linear quadratic (LQ) optimal control problem under rate constraints has been considered.

Linear-quadratic optimal control for backward stochastic differential equations with random coefficients

This paper is concerned with a linear-quadratic (LQ, for short) optimal control problem for backward stochastic differential equations (BSDEs, for short), where the coefficients of the backward control system and the weighting matrices in the cost functional are allowed to be random. By a variational method, the optimality system, which is a coupled linear forward-backward stochastic differential equation (FBSDE, for short), is derived, and by a Hilbert space method, the unique solvability of the optimality system is obtained. In order to construct the optimal control, a new stochastic Riccati-type equation is introduced. It is proved that an adapted solution (possibly non-unique) to the Riccati equation exists and decouples the optimality system. With this solution, the optimal control is obtained in an explicit way.

Error Estimates for the Optimal Control of a Parabolic Fractional PDE

We consider the integral definition of the fractional Laplacian and analyze a linear-quadratic optimal control problem for the so-called fractional heat equation; control constraints are also consi...

A New Run Algorithm for Solving the Continuous Linear-Quadratic Optimal Control Problem with Unseparated Boundary Conditions

A New Run Algorithm for Solving the Continuous Linear-Quadratic Optimal Control Problem with Unseparated Boundary Conditions

New regularity results and finite element error estimates for a class of parabolic optimal control problems with pointwise state constraints

We study first-order necessary optimality conditions and finite element error estimates for a class of distributed parabolic optimal control problems with pointwise state constraints. It is demonstrated that, if the bound in the state constraint and the differential operator in the governing PDE fulfil a certain compatibility assumption, then locally optimal controls satisfy a stationarity system that allows to significantly improve known regularity results for adjoint states and Lagrange multipliers in the parabolic setting. In contrast to classical approaches to first-order necessary optimality conditions for state-constrained problems, the main arguments of our analysis require neither a Slater point, nor uniform control constraints, nor differentiability of the objective function, nor a restriction of the spatial dimension. As an application of the established improved regularity properties, we derive new finite element error estimates for the dG(0) − cG(1)-discretization of a purely state-constrained linear-quadratic optimal control problem governed by the heat equation. The paper concludes with numerical experiments that confirm our theoretical findings.

Multistage Uncertain Random Linear Quadratic Optimal Control

In this paper, linear quadratic (LQ) optimal control problems are investigated for two types of uncertain random systems which consider the coefficient of the perturbed term as a constant vector or a vector-valued function of state vector and control vector. First, the uncertain random optimal control model is established under expected value criterion. Second, based on Bellman’s principle, recurrence equations are presented for settling such problem. Then by applying the recurrence equations and chance theory, the analytical expressions of the optimal results for the LQ problems are derived. Furthermore, some examples and an application are given to show the effectiveness of our results.

Autonomous Vehicle Convoy Formation Control with Size/Shape Switching for Automated Highways

Today’s semi-autonomous vehicles are gradually moving towards full autonomy. This transition requires developing effective control algorithms for handing complex autonomous tasks. Driving as a group of vehicles, referred to as a convoy, on automated highways is a highly important and challenging task that autonomous driving systems must deal with. This paper considers the control problem of a vehicle convoy modeled with linear dynamics. The convoy formation requirement is presented in terms of a quadratic performance index to minimize. The convoy formation control is formulated as a receding horizon linear-quadratic (LQ) optimal control problem. The receding horizon control law is innovatively defined via the solution to the algebraic Riccati equation. The solution matrix and therefore the receding horizon control law are obtained in the closed-form. A control architecture consisting of four algorithms is proposed to handle formation size/shape switching. The closed-form control law is at the core of these algorithms. Simulation results are provided to justify the models, solutions, and proposed algorithms.