Infinite Horizon Linear Quadratic Regulator Research Articles

Design of infinite horizon LQR controller for discrete delay systems in satellite orbit control: A predictive controller and reduction method approach

In the realm of satellite orbit control, powerful controller design plays a pivotal role in minimizing fuel consumption and ensuring orbit stability. This article introduces an advanced approach to the design of a Linear Quadratic Regulator (LQR) controller with an infinite horizon, tailored for discrete delay systems. The proposed methodology integrates predictive control with a reduction method, aiming for optimality while addressing performance and system constraints. Formulating the control problem as a quadratic program, the predictive control method generates a sequence of control inputs using a reducing horizon strategy. Stability analysis, employing Lyapunov-Krasovsky functions and linear matrix inequalities, yields delay-independent conditions for exponential convergence. A numerical example showcases the controller's effectiveness in maintaining orbit and reducing fuel consumption, underlining its capacity to achieve control objectives despite uncertainties and time delays. This research contributes to robust control strategies in satellite orbit systems, enhancing control performance and operational efficiency.

Suboptimal consensus protocol design for a class of multiagent systems

This article presents a new technique for suboptimal consensus protocol design for a class of multiagent systems. The proposed technique is based upon the extension of newly developed sufficient conditions for suboptimal linear–quadratic optimal control design, which are derived in this paper by an explication of a noniterative solution technique of the infinite-horizon linear quadratic regulation problem in Krotov framework. For suboptimal consensus protocol design, the structural requirements on the overall feedback gain matrix, which are inherently imposed by agents dynamics and their interaction topology, are recast on a specific matrix introduced in a suitably formulated convex optimization problem. As a result, preassigning the identical feedback gain matrices to a network of homogeneous agents, which act on the relative state variables with respect to their neighbors is not required. The suboptimality of the computed control laws is quantified by implicitly deriving an upper bound on the cost in terms of the solution of a convex optimization problem and initial conditions of the agents instead of specifying it a priori. Numerical examples are provided to demonstrate the implementation of proposed approaches and their comparison with existing methods in the literature.

A novel PID type iterative learning controller optimized by two‐dimensional infinite horizon linear quadratic iterative learning control for batch processes

AbstractAlthough a proportional integral derivative type iterative learning control (PIDILC) scheme can achieve good performance, its application is limited by open‐loop structure, parameter tuning, and initial state. In this paper, a novel PIDILC optimized by two‐dimensional infinite horizon linear quadratic regulator (PIDILC‐2D‐IHLQR) is proposed. First, by synthesizing the advantage of the conventional PIDILC method and proportional integral derivative (PID) strategy, a novel closed‐loop PIDILC scheme is obtained. Then, a novel two‐dimensional infinite horizon linear quadratic regulator (2D‐IHLQR) is developed to optimize the parameters of the PIDILC strategy. The limitations of parameter tuning and initial state are solved by this PIDILC‐2D‐IHLQR method. Therefore, the proposed method not only solves the aforementioned limitations but also inherits the advantages of PIDILC algorithm, PID method, and the novel 2D‐IHLQR scheme. Moreover, a stability condition is given based on Lyapunov theory and it can help judge whether the selection of control parameters meets the stability condition. The effectiveness of the proposed method is demonstrated by the case study on an injection modelling process.

Convergence and Sample Complexity of Gradient Methods for the Model-Free Linear–Quadratic Regulator Problem

Model-free reinforcement learning attempts to find an optimal control action for an unknown dynamical system by directly searching over the parameter space of controllers. The convergence behavior and statistical properties of these approaches are often poorly understood because of the nonconvex nature of the underlying optimization problems and the lack of exact gradient computation. In this article, we take a step toward demystifying the performance and efficiency of such methods by focusing on the standard infinite-horizon linear–quadratic regulator problem for continuous-time systems with unknown state-space parameters. We establish exponential stability for the ordinary differential equation (ODE) that governs the gradient-flow dynamics over the set of stabilizing feedback gains and show that a similar result holds for the gradient descent method that arises from the forward Euler discretization of the corresponding ODE. We also provide theoretical bounds on the convergence rate and sample complexity of the random search method with two-point gradient estimates. We prove that the required simulation time for achieving <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\epsilon$</tex-math></inline-formula> -accuracy in the model-free setup and the total number of function evaluations both scale as <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\log \, (1/\epsilon)$</tex-math></inline-formula> .

Robust Policy Iteration for Continuous-Time Linear Quadratic Regulation

This article studies the robustness of policy iteration in the context of continuous-time infinite-horizon linear quadratic regulator (LQR) problem. It is shown that Kleinman's policy iteration algorithm is small-disturbance input-to-state stable, a property that is stronger than Sontag's local input-to-state stability but weaker than global input-to-state stability. More precisely, whenever the error in each iteration is bounded and small, the solutions of the policy iteration algorithm are also bounded and enter a small neighborhood of the optimal solution of the LQR problem. Based on this result, an off-policy data-driven policy iteration algorithm for the LQR problem is shown to be robust when the system dynamics are subject to small additive unknown bounded disturbances. The theoretical results are validated by a numerical example.

Preview-based leader-following consensus control of distributed multi-agent systems

The preview-based leader-following consensus control of distributed multi-agent systems (MAS) with a directed acyclic graph is investigated in this paper. The external disturbance considered in this paper is the step function that can be previewed, and the information of the leader can be obtained in advance. The leader-following consensus control of MAS is solved by using the state augmentation technique and the preview control method that transforms the consensus control problem into the finite horizon linear quadratic regulator (LQR) problem and the infinite horizon LQR problem. In the finite horizon, the minimum principle is utilized to design the optimal controller. In the infinite horizon, the standard infinite horizon LQR is applied to devise the corresponding controller. Assuming the original systems are stabilisable and detectable, the distributed consensus controller guarantees the asymptotic consensus of the preview MAS. The simulation illustrates the effectiveness of the distributed leader-following consensus controller designed in this paper.

Move blocked model predictive control with guaranteed stability and improved optimality using linear interpolation of base sequences

To mitigate the online computational load of model predictive control, move blocking, which parameterises either the input sequence or offset from the base sequence by fixing the decision variables over arbitrary time intervals, is commonly used. However, existing move blocking schemes use a fixed base sequence only and do not fully exploit the valuable properties from various base sequences. Thus, we propose the interpolated solution-based move blocking strategy which parameterises the offset from the convex combination of two complementary base sequences – infinite-horizon linear quadratic regulator solution and shifted previous solution – and optimises the interpolation parameter as an additional decision variable in the optimal control problem. This allows the controller to exploit the valuable properties from both solutions by choosing the optimal interpolation parameter and blocked offset according to the current state online. The proposed approach efficiently improves the optimality performance whileguaranteeing the recursive feasibility and closed-loop stability.

Solution for the Continuous-Time Infinite-Horizon Linear Quadratic Regulator Subject to Scalar State Constraints

This letter provides a solution for the continuous-time linear quadratic regulator (LQR) subject to a scalar state constraint. Using a dichotomy transformation, novel properties for the finite-horizon LQR are derived; the unknown boundary conditions are explicitly expressed as a function of the horizon length, the initial state, and the final state or cost of the final state. Practical relevance of these novel properties are demonstrated with an algorithm to compute the continuous-time LQR subject to a scalar state constraint. The proposed algorithm uses the analytical conditions for optimality, without a priori discretization, to find only those sampling time instances that mark the start and end of a constrained interval. Each subinterval consists of a finite-horizon LQR, hence, a solution can be efficiently computed and the computational complexity does not grow with the horizon length. In fact, an infinite horizon can be handled. The algorithm is demonstrated with a simulation example.

Almost Every Single-Input LQR Optimal Control Problem Admits a PD Feedback Solution

In this letter, we prove that almost every infinite-horizon linear quadratic regulator (LQR) control problem with single-input admits an optimal solution in the form of a feedback that is a suitable constant linear combination of the state and its first derivative—a proportional plus derivative state feedback. The only assumption that we make is that the associated Hamiltonian matrix pencil has no eigenvalues on the imaginary axis. This condition is known to be generically true. Since a regular infinite-horizon LQR problem is known to admit an optimal control in the form of a static state feedback (a proportional feedback), we concentrate in this letter only on singular LQR problems.

Constrained Generalized Continuous Algebraic Riccati Equations (CGCAREs) Are Generically Unsolvable

In this letter, we show that a constrained generalized continuous algebraic Riccati equation (CGCARE), arising in the context of a singular infinite-horizon linear quadratic regulator (LQR) problem, is generically 1 unsolvable. We achieve this result in three steps: first, we provide a set of necessary and sufficient conditions for the solvability of a CGCARE. Then, using these conditions, we show that the determinant of a certain matrix pencil being identically zero is necessary for a CGCARE to be solvable. Finally, we show that this necessary condition is generically false. It is well-known that CGCARE solvability is a necessary and sufficient condition for a singular infinite-horizon LQR problem to admit a solution that is implementable as a static state-feedback control law. Thus, our main result reveals that a singular infinite-horizon LQR problem generically disallows solution by a static state-feedback law.

Linear quadratic regulation problem for discrete-time systems with multi-channel multiplicative noise

This paper investigates the linear quadratic regulation (LQR) problem for discrete-time systems with multiplicative noise. Multiplicative noise is usually assumed to be a scalar in existing literature works. Motivated by recent applications of networked control systems and MIMO communication technology, we consider multi-channel multiplicative noise represented by a diagonal matrix. We first show that the finite horizon LQR problem can be solved using a generalized Riccati equation. We then prove the convergence of the generalized Riccati equation under the conditions of stabilization and exact observability, and obtain the solution to the infinite horizon LQR problem. Finally, we provide a numerical example to demonstrate the proposed approach.

Convex optimization of nonlinear feedback controllers via occupation measures

The construction of feedback control laws for underactuated nonlinear robotic systems with input saturation limits is crucial for dynamic robotic tasks such as walking, running, or flying. Existing techniques for feedback control design are either restricted to linear systems, rely on discretizations of the state space, or require solving a nonconvex optimization problem that requires feasible initialization. This paper presents a method for designing feedback controllers for polynomial systems that maximize the size of the time-limited backwards reachable set (BRS). In contrast to traditional approaches based on Lyapunov’s criteria for stability, we rely on the notion of occupation measures to pose this problem as an infinite-dimensional linear program which can then be approximated in finite dimension via semidefinite programs (SDPs). The solution to each SDP yields a polynomial control policy and an outer approximation of the largest achievable BRS which is well suited for use in a trajectory library or feedback motion planning algorithm. We demonstrate the efficacy and scalability of our approach on six nonlinear systems. Comparisons to an infinite-horizon linear quadratic regulator approach and an approach relying on Lyapunov’s criteria for stability are also included in order to illustrate the improved performance of the presented technique.

Infinite horizon LQR for systems with multiple delays in a single input channel

This paper is concerned with the linear quadratic regulation (LQR) problem for both linear discrete-time systems and linear continuous-time systems with multiple delays in a single input channel. Our solution is given in terms of the solution to a two-dimensional Riccati difference equation for the discrete-time case and a Riccati partial differential equation for the continuous-time case. The conditions for convergence and stability are provided.

Linear Quadratic Regulation for Discrete-Time Systems with Input Delay: Spectral Factorization Approach

The infinite-horizon linear quadratic regulation (LQR) problem is settled for discrete-time systems with input delay. With the help of an autoregressive moving average (ARMA) innovation model, solutions to the underlying problem are obtained. The design of the optimal control law involves in resolving one polynomial equation and one spectral factorization. The latter is the major obstacle of the present problem, and the reorganized innovation approach is used to clear it up. The calculation of spectral factorization finally comes down to solving two Riccati equations with the same dimension as the original systems.

Constrained Infinite-Horizon Linear Quadratic Regulation of Discrete-Time Systems

We are concerned with the output norm-constrained infinite-horizon linear quadratic regulation problem, where the underlying state-control constraints are specified by curved, rather than polyhedral, surfaces. Each suboptimal problem admits an exact convex synthesis condition expressed by an increasing union of linear matrix inequalities and results in an eventually time-invariant controller satisfying the desired performance level subject to given constraints and closed-loop stability. The output norm-constrained minimum-time control problem is also dealt with as a corollary.

Continuous Time Linear Quadratic Regulator With Control Constraints via Convex Duality

A continuous time infinite horizon linear quadratic regulator with input constraints is studied. Optimality conditions, both in the open loop and feedback form, and continuity and differentiability properties of the optimal value function and of the optimal feedback are shown. Arguments rely on basic ideas of convex conjugacy, and in particular, use a dual optimal control problem

Infinite horizon model predictive control with no terminal constraint

This paper investigates the constrained infinite horizon linear quadratic regulator (CIHLQR). In particular, it is shown that the terminal set inclusion condition that is associated with this control law introduces limitations into the control system. By relaxing this condition, it is demonstrated that the resulting controller will in many cases remain equivalent to CIHLQR. It is further shown that CIHLQR with a relaxed terminal set inclusion condition offers significant benefits, in terms of computational expense and performance when compared with standard finite horizon model predictive control (MPC) approaches.

Minimising conservatism in infinite-horizon LQR control

This paper studies the formulation of the constrained infinite horizon linear quadratic regulator control law (CIHLQR). Results from recent studies in this area are extended to show that conditions used in the standard formulation of the CIHLQR law are not necessary, but merely sufficient. Through the use of a novel proof it is shown that for a general SISO system with input constraints and certain conditions imposed, saturated LQR provides the same control sequence as CIHLQR. It is further shown that saturated LQR is equivalent to the CIHLQR in the case of first-order systems, subject to both state and control constraints. Finally, the region of constrained stabilisability is characterised for the case of open-loop unstable first-order systems.

Constrained linear quadratic regulation

The paper is a contribution to the theory of the infinite-horizon linear quadratic regulator (LQR) problem subject to inequality constraints on the inputs and states, extending an approach first proposed by Sznaier and Damborg (1987). A solution algorithm is presented, which requires solving a finite number of finite-dimensional positive definite quadratic programs. The constrained LQR outlined does not feature the undesirable mismatch between open-loop and closed-loop nominal system trajectories, which is present in the other popular forms of model predictive control (MPC) that can be implemented with a finite quadratic programming algorithm. The constrained LQR is shown to be both optimal and stabilizing. The solution algorithm is guaranteed to terminate in finite time with a computational cost that has a reasonable upper bound compared to the minimal cost for computing the optimal solution. Inherent to the approach is the removal of a tuning parameter, the control horizon, which is present in other MPC approaches and for which no reliable tuning guidelines are available. Two examples are presented that compare constrained LQR and two other popular forms of MPC. The examples demonstrate that constrained LQR achieves significantly better performance than the other forms of MPC on some plants, and the computational cost is not prohibitive for online implementation.

Infinite Horizon Linear Quadratic Control with Constraints

A model predictive control law is considered that optimizes the infinite horizon linear quadratic regulator (LQR) objective over the infinite dimensional sequence of future controls that satisfy user-defined input and state constraints. The resulting scheme is called infinite horizon LQR with constraints. The problem of transient infeasibility of hard state constraints is also discussed and we consider the use of soft constraints as a solution. We also introduce the concept of exact soft constraints, which allows constraints to be enforced as though they were hard, when that is feasible, and to be relaxed only when necessary.