Linear Quadratic Optimal Control Problem Research Articles

The linear quadratic optimal control problem for discrete-time Markov jump linear singular systems

In this paper, we address the linear quadratic optimal control problem for discrete-time Markov jump linear singular systems. The results are obtained under conditions that bring some additional structure to the quite general and complex class of systems being considered. The approach involves base transformations and restricts the control actions into an appropriate subspace to ensure regularity of the closed-loop system. Both finite and infinite time horizon problems are considered. The results are evaluated by means of an example.

A BSDE approach to stochastic linear quadratic control problem

AbstractIn this article, we study a kind of linear quadratic optimal control problem driven by forward–backward stochastic differential equations (FBSDEs in short) with deterministic coefficients. The cost functional is defined by the solution of FBSDEs. By means of the Girsanov transformation, the original issue is turned equivalently into the classical LQ problem. By functional analysis approach, some necessary and sufficient conditions for the existence of optimal controls have been obtained. Moreover, we investigate the relationship between two groups of first‐order and second‐order adjoint equations. A new stochastic Riccati equation is derived, which leads to the state feedback form of optimal control. By introducing a new Hamiltonian function with an exponential factor, we establish the stochastic maximum principle to deal with the stochastic linear quadratic problem for forward–backward stochastic system with nonconvex control domain using first‐order adjoint equation. An illustrative example is given as well.

Indefinite stochastic linear-quadratic optimal control problems with random coefficients: Closed-loop representation of open-loop optimal controls

This paper is concerned with a stochastic linear-quadratic optimal control problem in a finite time horizon, where the coefficients of the control system are allowed to be random, and the weighting matrices in the cost functional are allowed to be random and indefinite. It is shown, with a Hilbert space approach, that for the existence of an open-loop optimal control, the convexity of the cost functional (with respect to the control) is necessary; and the uniform convexity, which is slightly stronger, turns out to be sufficient, which also leads to the unique solvability of the associated stochastic Riccati equation. Further, it is shown that the open-loop optimal control admits a closed-loop representation. In addition, some sufficient conditions are obtained for the uniform convexity of the cost functional, which are strictly more general than the classical conditions that the weighting matrix-valued processes are positive (semi-) definite.

Open-loop and closed-loop solvabilities for stochastic linear quadratic optimal control problems of Markovian regime switching system

This paper investigates the stochastic linear quadratic (LQ, for short) optimal control problem of Markovian regime switching system. The representation of the cost functional for the stochastic LQ optimal control problem of Markovian regime switching system is derived by the technique of Itô’s formula with jumps. For the stochastic LQ optimal control problem of Markovian regime switching system, we establish the equivalence between the open-loop (closed-loop, resp.) solvability and the existence of an adapted solution to the corresponding forward-backward stochastic differential equation with constraint. (i.e., the existence of a regular solution to Riccati equations). Also, we analyze the interrelationship between the strongly regular solvability of Riccati equations and the uniform convexity of the cost functional. Finally, we present an example which is open-loop solvable but not closed-loop solvable.

A concise introduction to control theory for stochastic partial differential equations

<p style='text-indent:20px;'>The aim of this notes is to give a concise introduction to control theory for systems governed by stochastic partial differential equations. We shall mainly focus on controllability and optimal control problems for these systems. For the first one, we present results for the exact controllability of stochastic transport equations, null and approximate controllability of stochastic parabolic equations and lack of exact controllability of stochastic hyperbolic equations. For the second one, we first introduce the stochastic linear quadratic optimal control problems and then the Pontryagin type maximum principle for general optimal control problems. It deserves mentioning that, in order to solve some difficult problems in this field, one has to develop new tools, say, the stochastic transposition method introduced in our previous works.

Решение линейно-квадратичной задачи оптимального управления на основе конечномерных моделей

We consider a linear-quadratic optimal control problem with indefinite matrices and the interval control constraint. The problem also has a regularizationparameter in the functional. The approximate solution of the problem is carried out on subsets of admissible controls, which are formed using linear combinations of special functions with an orientation to the optimal control structure due to the maximum principle. As a result of this procedure, a finite-dimensional quadratic optimization problem with the interval constraint on variables is obtained. The following relations between the variational problem and its finite-dimensional model are established: the convexity property of the optimal control problem is preserved for finite-dimensional model; a nonconvex optimal control problem under a certain condition on the regularization parameter (estimate from below) is approximated by a convex quadratic problem, which is solved in a finite number of operations;a special non-convex optimal control problem with an upper bound on the regularization parameter passes into the problem of minimizing a concave function on a finite set of points. A special case of a non-convex optimal control problem for the maximum of the norm of the final state is distinguished. Two procedures for improving the extreme points of finite-dimensional model are constructed, which reduce the computational costs for the global solution of the problem within the framework of the linearization method.

Direct method to solve linear-quadratic optimal control problems

&lt;p style='text-indent:20px;'&gt;In this work, we have proposed a new approach for solving the linear-quadratic optimal control problem, where the quality criterion is a quadratic function, which can be convex or non-convex. In this approach, we transform the continuous optimal control problem into a quadratic optimization problem using the Cauchy discretization technique, then we solve it with the active-set method. In order to study the efficiency and the accuracy of the proposed approach, we developed an implementation with MATLAB, and we performed numerical experiments on several convex and non-convex linear-quadratic optimal control problems. The obtained simulation results show that our method is more accurate and more efficient than the method using the classical Euler discretization technique. Furthermore, it was shown that our method fastly converges to the optimal control of the continuous problem found analytically using the Pontryagin's maximum principle.&lt;/p&gt;

Rate Constrained Discrete-time Maximum Principle

Abstract Actuators in real-world systems operate under limited bandwidth and saturation constraints. Mathematically, these limitations manifest as certain types of constraints being imposed on the sequence of control action. While saturation constraints have been studied extensively, bandwidth limitations in the actuators have received very little attention in the control community. While an attempt has been made in the direction of incorporating frequency constraints at the synthesis stage, rate constraints on the control at the design stage has not been studied extensively in the regime of discrete-time. This article tries to fill in that gap by incorporating rate constraints on the control trajectories. We propose a new discrete-time Pontryagin maximum principle (PMP) with rate constraints being imposed on the control and derive a set of first order necessary conditions for optimality. As an illustration, the linear quadratic (LQ) optimal control problem under rate constraints has been considered.

Linear-quadratic optimal control for backward stochastic differential equations with random coefficients

This paper is concerned with a linear-quadratic (LQ, for short) optimal control problem for backward stochastic differential equations (BSDEs, for short), where the coefficients of the backward control system and the weighting matrices in the cost functional are allowed to be random. By a variational method, the optimality system, which is a coupled linear forward-backward stochastic differential equation (FBSDE, for short), is derived, and by a Hilbert space method, the unique solvability of the optimality system is obtained. In order to construct the optimal control, a new stochastic Riccati-type equation is introduced. It is proved that an adapted solution (possibly non-unique) to the Riccati equation exists and decouples the optimality system. With this solution, the optimal control is obtained in an explicit way.

Error Estimates for the Optimal Control of a Parabolic Fractional PDE

We consider the integral definition of the fractional Laplacian and analyze a linear-quadratic optimal control problem for the so-called fractional heat equation; control constraints are also consi...

A New Run Algorithm for Solving the Continuous Linear-Quadratic Optimal Control Problem with Unseparated Boundary Conditions

A New Run Algorithm for Solving the Continuous Linear-Quadratic Optimal Control Problem with Unseparated Boundary Conditions

New regularity results and finite element error estimates for a class of parabolic optimal control problems with pointwise state constraints

We study first-order necessary optimality conditions and finite element error estimates for a class of distributed parabolic optimal control problems with pointwise state constraints. It is demonstrated that, if the bound in the state constraint and the differential operator in the governing PDE fulfil a certain compatibility assumption, then locally optimal controls satisfy a stationarity system that allows to significantly improve known regularity results for adjoint states and Lagrange multipliers in the parabolic setting. In contrast to classical approaches to first-order necessary optimality conditions for state-constrained problems, the main arguments of our analysis require neither a Slater point, nor uniform control constraints, nor differentiability of the objective function, nor a restriction of the spatial dimension. As an application of the established improved regularity properties, we derive new finite element error estimates for the dG(0) − cG(1)-discretization of a purely state-constrained linear-quadratic optimal control problem governed by the heat equation. The paper concludes with numerical experiments that confirm our theoretical findings.

Multistage Uncertain Random Linear Quadratic Optimal Control

In this paper, linear quadratic (LQ) optimal control problems are investigated for two types of uncertain random systems which consider the coefficient of the perturbed term as a constant vector or a vector-valued function of state vector and control vector. First, the uncertain random optimal control model is established under expected value criterion. Second, based on Bellman’s principle, recurrence equations are presented for settling such problem. Then by applying the recurrence equations and chance theory, the analytical expressions of the optimal results for the LQ problems are derived. Furthermore, some examples and an application are given to show the effectiveness of our results.

Autonomous Vehicle Convoy Formation Control with Size/Shape Switching for Automated Highways

Today’s semi-autonomous vehicles are gradually moving towards full autonomy. This transition requires developing effective control algorithms for handing complex autonomous tasks. Driving as a group of vehicles, referred to as a convoy, on automated highways is a highly important and challenging task that autonomous driving systems must deal with. This paper considers the control problem of a vehicle convoy modeled with linear dynamics. The convoy formation requirement is presented in terms of a quadratic performance index to minimize. The convoy formation control is formulated as a receding horizon linear-quadratic (LQ) optimal control problem. The receding horizon control law is innovatively defined via the solution to the algebraic Riccati equation. The solution matrix and therefore the receding horizon control law are obtained in the closed-form. A control architecture consisting of four algorithms is proposed to handle formation size/shape switching. The closed-form control law is at the core of these algorithms. Simulation results are provided to justify the models, solutions, and proposed algorithms.

Asymptotic approximation to a solution of a singularly perturbed linear-quadratic optimal control problem with second-order linear ordinary differential equation of state variable

&lt;p style='text-indent:20px;'&gt;The direct scheme method is applied to construct an asymptotic approximation of any order to a solution of a singularly perturbed optimal problem with scalar state, controlled via a second-order linear ODE and two fixed end points. The error estimates for state and control variables and for the functional are obtained. An illustrative example is given.&lt;/p&gt;

Collision-geometry-based generalized optimal impact angle guidance for various missile and target motions

A unified optimal impact angle guidance law is designed for different types of interceptors and targets. By solving a linear quadratic optimal control problem with the energy consumption weighted by an arbitrary positive function, the generalized guidance command is determined in a closed-loop feedback form. The main feature of the proposed algorithm is that it can be applied to varying-speed interceptors and targets. The weighting function also brings the command shaping capability of the proposed guidance law. To show the applicability of the proposed guidance law, several illustrative examples covering different types of interceptors and targets are provided. Numerical simulations are conducted to demonstrate the effectiveness of the proposed guidance algorithm.

Solvability for indefinite mean‐field stochastic linear quadratic optimal control with random jumps and its applications

SummaryIn this article, we study the open‐loop and closed‐loop solvability for indefinite mean‐field stochastic linear quadratic (MF‐SLQ) optimal control problem and its application in finance, where the controlled stochastic system is driven by a Brownian motion and a Poisson random martingale measure and also disturbed by some stochastic processes. The intrinsic property of stochastic systems results in the inequivalence of those two solvabilities, which is different from deterministic case. Based on a well‐posedness result of the problem, it is shown that the uniform convexity of cost functional is sufficient for the open‐loop solvability of the problem. By a matrix minimum principle, the necessity condition of regular solvability for a decoupled Riccati equation is established, meanwhile, the closed‐loop solvability is turned out to be equivalent to the regular solvability of Riccati equations with some constraints on the solutions of disturbances equations, moreover, the optimal closed‐loop strategy is characterized by regular solutions of Riccati equations and adapted solutions of disturbances equations. And then a mean‐variance model is considered to solve optimal portfolio selection strategy problem of the insurance company with liability. Our study fills a gap in the field of solvability for mean‐field stochastic optimal control with random jumps and provides a different way for solving mean‐variance portfolio selection model.

Output Feedback Q-Learning for Linear-Quadratic Discrete-Time Finite-Horizon Control Problems.

An algorithm is proposed to determine output feedback policies that solve finite-horizon linear-quadratic (LQ) optimal control problems without requiring knowledge of the system dynamical matrices. To reach this goal, the Q -factors arising from finite-horizon LQ problems are first characterized in the state feedback case. It is then shown how they can be parameterized as functions of the input-output vectors. A procedure is then proposed for estimating these functions from input/output data and using these estimates for computing the optimal control via the measured inputs and outputs.

A New Method for Determining the Degree of Controllability of State Variables for the LQR Problem Using the Duality Theorem

The control performance of a dynamic system can be checked by the degree of controllability. In this work, we present a new method for determining the degree of observability of state variables for the linear quadratic optimal estimation (LQE) problem. We carried out the calculation of the degree of controllability for the linear quadratic optimal control (LQR) problem using a duality theorem. Compared with the traditional measures of controllability such as determinant, trace, and maximal eigenvalue of the inverse controllability Gramian, the proposed degree of controllability was developed for each state variable and takes into account both the controllability Gramian and the cost function. The new method is convenient to apply to LQR problem. In the numerical simulation, we determined the influence of the model parameters on the degree of controllability. Besides that, we analyzed the degree of controllability, which gives an insight into the relationship between the system model design and the control performance.

Linear quadratic optimal control for a class of continuous-time nonhomogeneous Markovian jump linear systems in infinite time horizon

In this paper, the infinite time horizon linear quadratic optimal control problem is investigated for the continuous-time Itô stochastic Markovian jump linear systems (MJLSs) with time-varying transition rates. It is assumed that the time-varying transition rates of the MJLSs possess piecewise homogeneous time-varying property, which implies that the transition rates are time-varying in the whole time domain but they are time-invariant in some small time intervals. The variations of the transition rates in these small time intervals are considered to be in two cases: arbitrary variation and stochastic variation. With this, the considered system becomes a piecewise homogeneous Itô stochastic MJLS. The main contribution of this paper is that two linear quadratic optimal controllers in infinite time horizon are proposed for the above modeled continuous-time piecewise homogeneous Itô stochastic MJLS in the sense of arbitrary variation and stochastic variation, respectively. Moreover, the sufficient and necessary conditions for the existence of the designed controllers are established based on the existence of the unique positive definite solution of two coupled algebraic Riccati matrix equations. Finally, a simulation example is provided to illustrate the effectiveness of the proposed linear quadratic optimal controllers.