LQR Problems Research Articles

A Numerical Comparison of Different Solvers for Large-Scale, Continuous-Time Algebraic Riccati Equations and LQR Problems

In this paper, we discuss numerical methods for solving large-scale continuous-time algebraic Riccati equations. These methods have been the focus of intensive research in recent years, and significant progress has been made in both the theoretical understanding and efficient implementation of various competing algorithms. There are several goals of this manuscript: first, to gather in one place an overview of different approaches for solving large-scale Riccati equations, and to point to the recent advances in each of them. Second, to analyze and compare the main computational ingredients of these algorithms, to detect their strong points and their potential bottlenecks. And finally, to compare the effective implementations of all methods on a set of relevant benchmark examples, giving an indication of their relative performance.

Solving Stochastic LQR Problems by Polynomial Chaos

We consider the infinite dimensional stochastic linear quadratic optimal control problem for the infinite horizon case. We provide a numerical framework for solving this problem using a polynomial chaos expansion approach. By applying the method of chaos expansions to the state equation, we obtain a system of deterministic partial differential equations in terms of the coefficients of the state and the control variables. We set up a control problem for each equation, which results in a set of infinite horizon deterministic linear quadratic regulator problems. We prove the optimality of the solution expressed in terms of the expansion of these coefficients compared to the direct approach. We perform numerical experiments which validate our approach and compare the finite and infinite horizon case.

Fourier-splitting method for solving hyperbolic LQR problems

We consider the numerical approximation to linear quadratic regulator problems for hyperbolic partial differential equations where the dynamics is driven by a strongly continuous semigroup. The optimal control is given in feedback form in terms of Riccati operator equations. The computational cost relies on solving the associated Riccati equation and computing the optimal state. In this paper we propose a novel approach based on operator splitting idea combined with Fourier's method to efficiently compute the optimal state. The Fourier's method allows to accurately approximate the exact flow making our approach computational efficient. Numerical experiments in one and two dimensions show the performance of the proposed method.

An efficient cost reduction procedure for bounded-control LQR problems

A novel approach has been developed for approximating the solution to the constrained LQR problem, based on updating the final state and costate of a related regular problem, and on slightly shifting the switching times (the instants when the control meets the constraints). The main result is the expression of a suboptimal control in feedback form using the solution of some compatible Riccati equation. The gradient method is applied to reduce the cost via explicit algebraic formula for its partial derivatives with respect to the hidden final state/costate of the related regular problem and to the switching times. The numerical method is termed efficient because it does not involve integrations of states or cost trajectories, and reduces to its minimum the dimension of the unknown parameters at the final condition. All the relevant objects are calculated from a few auxiliary matrices, which are computed only once. The scheme is here applied to two case studies whose optimal solutions are known. The first example is a two-dimensional model of the ‘cheapest stop of a train’ problem. The second one refers to the temperature control of a metallic strip leaving a multi-stand rolling mill, a problem with a high-dimensional state.

A new extension of the infinite-dimensional KYP lemma in the coercive case

The Kalman-Yakubovich-Popov (KYP) Lemma, giving the frequency-domain criteria for solvability of LQR problems and feasibility of related LMIs and Lur'e-Riccati equations, is undoubtedly one of the cornerstone results in modern control theory. Numerous applications of the KYP lemma in nonlinear and robust control motivated its extension to systems with distributed parameters. Unlike the finite-dimensional case, where the KYP lemma admits elegant proofs by means of algebraic techniques and methods from convex analysis, a prerequisite for its infinite-dimensional extensions is feasibility of the corresponding LQR problem, resulting on assumptions of exponential or L2-stabilizability. This property is restrictive and not easily verifiable for general infinite-dimensional systems. Being quite natural in the problems of absolute stability, it seems unnecessary in other applications of the KYP lemma which require only solvability of the operator inequality, but not positivity of its solution, such as e.g. instability or dichotomy criteria. In the present paper we show, that in the “strict” or coercive case the KYP lemma retains its validity, replacing stabilizability assumption with some technical assumption which holds, for instance, whenever the linear system is exponentially dichotomic.

Constrained LQR for low-precision data representation

Performing computations with a low-bit number representation results in a faster implementation that uses less silicon, and hence allows an algorithm to be implemented in smaller and cheaper processors without loss of performance. We propose a novel formulation to efficiently exploit the low (or non-standard) precision number representation of some computer architectures when computing the solution to constrained LQR problems, such as those that arise in predictive control. The main idea is to include suitably-defined decision variables in the quadratic program, in addition to the states and the inputs, to allow for smaller roundoff errors in the solver. This enables one to trade off the number of bits used for data representation against speed and/or hardware resources, so that smaller numerical errors can be achieved for the same number of bits (same silicon area). Because of data dependencies, the algorithm complexity, in terms of computation time and hardware resources, does not necessarily increase despite the larger number of decision variables. Examples show that a 10-fold reduction in hardware resources is possible compared to using double precision floating point, without loss of closed-loop performance.

Optimal saturated feedback laws for LQR problems with bounded controls

Optimal and suboptimal strategies are substantiated and illustrated for linear-quadratic problems with penalized endpoints, when bounds in control values are imposed. The optimal solution for a given process with restricted controls, starting at a known initial state, is shown to coincide with the saturated solution to some unrestricted problem that has the same coefficients, except for the final penalization matrix \(S\), and starts at a generally different initial state. This result reduces the searching span for the solution: from the infinite-dimensional set of admissible control trajectories to the finite-dimensional space of symmetric positive semi-definite symmetric matrices \(\hat{S}\) and initial states \(\hat{x}_{0}\). An efficient scheme is also proposed to approximate (and eventually to find) the optimal feedback strategy on-line, based on the updating of \(\hat{S}\) at successive sampling times \(t_{k},\) and on the possibility to generate the corresponding Riccati matrix \(P(t,T,\hat{S})\) for \(t_{k}<t\le t_{k+1}\) from auxiliary matrices stored in memory. Numerical simulations are provided, compared, and checked against the analytical solutions of two classical case-studies.

Infinite-Horizon Switched LQR Problems in Discrete Time: A Suboptimal Algorithm With Performance Analysis

This paper studies the quadratic regulation problem for discrete-time switched linear systems (DSLQR problem) on an infinite time horizon. A general relaxation framework is developed to simplify the computation of the value iterations. Based on this framework, an efficient algorithm is developed to solve the infinite-horizon DSLQR problem with guaranteed closed-loop stability and suboptimal performance. Due to these guarantees, the proposed algorithm can be used as a general controller synthesis tool for switched linear systems.

On infinite horizon switched LQR problems with state and control constraints

This paper studies the Discrete-Time Switched LQR problem over an infinite time horizon, subject to polyhedral constraints on state and control inputs. Specifically, we aim to find an infinite-horizon hybrid-control sequence, i.e., a sequence of continuous and discrete (switching) control inputs, that minimizes an infinite-horizon quadratic cost function, subject to polyhedral constraints on state and (continuous) control input. The overall constrained, infinite-horizon problem is split into two subproblems: (i) an unconstrained, infinite-horizon problem and (ii) a constrained, finite-horizon one. We derive a stationary suboptimal policy for problem (i) with analytical bounds on its optimality, and develop a novel formulation of problem (ii) as a Mixed-Integer Quadratic Program. By introducing the concept of a safe set, the solutions of the two subproblems are combined to achieve the overall control objective. Through the connection between (i) and (ii) it is shown that, by proper choice of the design parameters, the error of the overall suboptimal solution can be made arbitrarily small. The approach is tested on a numerical example.

Non-recursive Haar Connection Coefficients Based Approach for Linear Optimal Control

In the present paper, two-fold contributions are made. First, non-recursive formulations of various Haar operational matrices, such as Haar connection coefficients matrix, backward integral matrix, and product matrix are developed. These non-recursive formulations result in computationally efficient algorithms, with respect to execution time and stack-and-memory overflows in computer implementations, as compared to corresponding recursive formulations. This is demonstrated with the help of MATLAB PROFILER. Later, a unified method is proposed, based on these non-recursive connection coefficients, for solving linear optimal control problems of all types, irrespective of order and nature of the system. This means that the single method is capable of optimizing both time-invariant and time-varying linear systems of any order efficiently; it has not been reported in the literature so far. The proposed method is applied to solve finite horizon LQR problems with final state control. Computational efficiency of the proposed method is established with the help of comparison on computation-time at different resolutions by taking several illustrative examples.

On stochastic Riccati equations for the stochastic LQR problem

This paper studies an approximation of stochastic Riccati equations for stochastic LQR problems some of which may be even with indefinite control weight costs.

Constrained LQR Problems in Elliptic Distributed Control Systems with Point Observations—-Convergence Results

Constrained LQR Problems in Elliptic Distributed Control Systems with Point Observations—-Convergence Results

Constrained LQR problems in elliptic distributed control systems with point observations—convergence results

In this paper constrained LQR problems in distributed control systems governed by the elliptic equation with point observations are studied. A variational inequality approach coupled with potential theory in a Banach space setting is adopted. First the admissible control set is extended to be bounded by two functions, and feedback characterization of the optimal control in terms of the optimal state is derived; then two numerical algorithms proposed in [5] are modified, and the strong convergence and uniform convergence in Banach space are proved. This verifies that the numerical algorithm is insensitive to the partition number of the boundary. Since our control variables are truncated below and above by two functions inL p and in our numerical computation only the layer density not the control variable is assumed to be piecewise smooth, uniform convergence guarantees a better convergence. Finally numerical computation for an example is carried out and confirms the analysis.

Constrained LQR Problems in Elliptic Distributed Control Systems with Point Observations--- on Convergence Rates

In this paper, we continue to study the (bound constrained) linear-quadratic regulator problem in distributed boundary control systems governed by the elliptic equation with point observations. Due to the appearance of singularities in the problem, the traditional Galerkin variational method that leads to an adjoint system is not desirable, and the classical Lagrangian multiplier method is not reliable for providing numerical results. A characterization formula of the optimal control and its singularity decomposition formula are derived in [Z. Ding, L. Ji, and J. Zhou, SIAM J. Control Optim., 34 (1996), pp. 264--294; Z. Ding and J. Zhou, Appl. Math. Optim.}, to appear] by using the boundary integral equation and potential theory coupled with a variational inequality in a Banach space setting. Based on the characterization formula, a conditioned gradient projection method (CGPM) has been proposed in [Z. Ding and J. Zhou, Appl. Math. Optim., to appear]. Numerical experiment has shown that CGPM is efficient and also insensitive to the partition number of the boundary. In this paper, we estimate the rate of convergence for CGPM. First it is proved that for ${\cal N}=2$, CGPM converges exponentially in the L2 norm, and for ${\cal N}=3$, CGPM converges subexponentially in the L p norm. Then, under a reasonable condition, it is proved that for ${\cal N}=2$, CGPM converges uniformly exponentially, and for ${\cal N}=3$, CGPM converges uniformly subexponentially.

Constrained LQR Problems in Elliptic Distributed Control Systems with Point Observations

In this paper, we study (bound constrained) LQR problems in distributed control systems governed by the elliptic equation with point observations, which are motivated by problems in corrosion engineering and contemporary “smart materials.” Several regularity and characterization theorems have been established. In particular, three decomposition formulas are obtained to characterize the optimal control and optimal layer density, and are used to direct the numerical computations. These results cannot be obtained by the traditional Galerkin variational method. In the process, several useful lemmas are established, which are of independent interest. We point out that the classical Lagrangian multiplier method (LMM) may fail to provide a reliable numerical algorithm. Based on our characterization results and the boundary element method, two algorithms are proposed to carry out numerical computations. It has been shown by our numerical experiments that both algorithms are efficient and insensitive to the partition number of the boundary. An adaptive local refinement scheme has also been designed to handle the rough behavior of the optimal solution around sensor locations.

The point spectra of some LQR problems

We present a sufficient and in some cases necessary condition for determining the spectra of a wide class of LQR problems in an abstract Hilbert space.