Standard Linear Quadratic Regulator Research Articles

Distributed Minmax Strategy for Multiplayer Games: Stability, Robustness, and Algorithms.

This article studies a distributed minmax strategy for multiplayer games and develops reinforcement learning (RL) algorithms to solve it. The proposed minmax strategy is distributed, in the sense that it finds each player's optimal control policy without knowing all the other players' policies. Each player obtains its distributed control policy by solving a distributed algebraic Riccati equation in a multiplayer noncooperative game. This policy is found against the worst policies of all the other players. We guarantee the existence of distributed minmax solutions and study their L2 and asymptotic stabilities. Under mild conditions, the resulting minmax control policies are shown to improve robust gain and phase margins of multiplayer systems compared to the standard linear-quadratic regulator controller. Distributed minmax solutions are found using both model-based policy iteration and data-driven off-policy RL algorithms. Simulation examples verify the proposed formulation and its computational efficiency over the nondistributed Nash solutions.

Lnertia Weight-free Particle Swarm Optimization in Optimal Control Design for Vehicle Active Suspension Systems

Vehicle active suspension systems play an important role in ride comfort and driving safety. This study considers the problem of an efficient control scheme design for vehicle active suspension systems. The active suspension systems aim to get more comfortable riding and good handling for random road disturbances. The purpose of this work is to reduce the driver’s entire body acceleration and thereby improve ride comfort. The inertial weight-free particle swarm optimization (PSO) method is utilized to obtain weighting matrices of the optimal control namely linear quadratic regulator (LQR) for the active suspension systems. The designed state-feedback controller is applied to the quarter-car suspension system under different road profiles. Simulation results of the inertia weight-free PSO-tuned LQR are compared with the results of the classical-tuned controller and standard PSO-tuned LQR controller to show the effectiveness.

Efficient Learning of a Linear Dynamical System With Stability Guarantees

We propose a principled method for projecting an arbitrary square matrix to the non-convex set of asymptotically stable matrices. Leveraging ideas from large deviations theory, we show that this projection is optimal in an information-theoretic sense and that it simply amounts to shifting the initial matrix by an optimal linear quadratic feedback gain, which can be computed exactly and highly efficiently by solving a standard linear quadratic regulator problem. The proposed approach allows us to learn the system matrix of a stable linear dynamical system from a single trajectory of correlated state observations. The resulting estimator is guaranteed to be stable and offers statistical bounds on the estimation error.

Parametric Dynamic Distributed Containment Control of Continuous-Time Linear Multi-Agent Systems with Specified Convergence Speed

This paper focuses on the distributed containment control of continuous-time linear multi-agent systems (MASs) with multiple leaders over fixed topology. A parametric dynamic compensated distributed control protocol is proposed in which both the information from the observer in the virtual layer and actual adjacent agents are employed. The necessary and sufficient conditions of the distributed containment control are derived based on the standard linear quadratic regulator (LQR). On this basis, the dominant poles are configured by using the modified linear quadratic regulator (MLQR) optimal control and Geršgorin's circle criterion, hence the containment control with specified convergence speed of the MAS is achieved. Another main advantage of the proposed design is, in the case of virtual layer failure, by adjusting parameters the dynamic control protocol reduces to static, and the convergence speed can still be specified through the dominant pole assignment method combined with inverse optimal control. Finally, typical numerical examples are presented to demonstrate the effectiveness of theoretical results.

Risk-Aware Linear Quadratic Control Using Conditional Value-at-Risk

Stochastic linear quadratic control problems are considered from the viewpoint of risks. In particular, a worst-case conditional value-at-risk (CVaR) of quadratic objective function is minimized subject to additive disturbances whose first two moments of the distribution are known. The study focuses on three problems of finding the optimal feedback gain that minimizes the quadratic cost of: stationary distribution, one-step, and infinite time horizon. For the stationary distribution problem, it is proved that the optimal control gain that minimizes the worst-case CVaR of the quadratic cost is equivalent to that of the standard (stochastic) linear quadratic regulator. For the one-step problem, an approach to an optimal solution as well as analytical suboptimal solutions are presented. For the infinite time horizon problem, two suboptimal solutions that bound the optimal solution and an approach to an optimal solution for a special case are discussed. The presented theorems are illustrated with numerical examples.

On the Relationship of Optimal State Feedback and Disturbance Response Controllers

This paper studies the relationship between state feedback policies and disturbance response policies for the standard Linear Quadratic Regulator (LQR). For open-loop stable plants, we establish a simple relationship between the optimal state feedback controller ut = K*xt and the optimal disturbance response controller ut = L(H)*;1wt-1 +...+ L(H)*;1wt-H with H-order. Here xt, wt, ut stands for the state, disturbance, control action of the system, respectively. Our result shows that L(H)*,1 is a good approximation of K* and the approximation error ∥K* — L(H)*;1∥ decays exponentially with H. We further extend this result to LQR for open-loop unstable systems, when a pre-stabilizing controller K0 is available.

High fidelity modelling and GA optimized control of yaw dynamics of a custom built remotely operated unmanned underwater vehicle

In this paper, a custom built, underactuated, inspection class micro Remotely Operated Unmanned Underwater Vehicle (ROV) is employed as a testbed to investigate control and modelling problems related to underwater vehicles in shallow waters and pools. Dynamic model for yaw is obtained via mathematical modelling and system identification techniques. To instil confidence in the identified model, residuals and cross-validation tests are carried out to obtain high fidelity vehicle model for subsequent stabilizing closed-loop control design. Following the modelling exercise, design, real-time implementation, and analysis of a GA optimized PI controller for yaw is carried out. The performance of the GA optimized controller is benchmarked against the experimental results of a multi-parameter root-locus tuned PI controller and simulated responses of a standard linear quadratic regulator (LQR) controller. In addition, the efficacy of the GA-PI controller is gauged employing recently developed marine predator algorithm (MPA). The need for a controller with optimized performance motivated the utilization of GA and MPA optimization techniques. The results from real-time pool experiments indicate substantially enhanced performance of GA optimized controller, outperforming other controllers by as much as 22% in performance indicators such as settling time and maximum overshoot. Furthermore, the GA optimized controller demonstrated far better robustness and disturbance rejection capabilities.

Preview-based leader-following consensus control of distributed multi-agent systems

The preview-based leader-following consensus control of distributed multi-agent systems (MAS) with a directed acyclic graph is investigated in this paper. The external disturbance considered in this paper is the step function that can be previewed, and the information of the leader can be obtained in advance. The leader-following consensus control of MAS is solved by using the state augmentation technique and the preview control method that transforms the consensus control problem into the finite horizon linear quadratic regulator (LQR) problem and the infinite horizon LQR problem. In the finite horizon, the minimum principle is utilized to design the optimal controller. In the infinite horizon, the standard infinite horizon LQR is applied to devise the corresponding controller. Assuming the original systems are stabilisable and detectable, the distributed consensus controller guarantees the asymptotic consensus of the preview MAS. The simulation illustrates the effectiveness of the distributed leader-following consensus controller designed in this paper.

Ters Sarkaç Sistemi İçin LQR Kontrolcü Tasarımında Genetik Algoritma Optimizasyonu

Bu çalışmada kontrol tekniklerinin performanslarını incelemek için sıklıkla tercih edilen ters sarkaç sistemi ele alınmıştır. Ters sarkaç sisteminin doğrusal olmayan yapısı nedeniyle de kontrolü zor bir mühendislik problemidir. Ters sarkaç problemine yönelik sistemin hareket denklemleri çıkartılmış, durum-uzay formunda ifade edilmiş ve tasarım kriterleri belirlenmiştir. Ters sarkaç sisteminde tasarım kriteri olarak arabanın ve sarkacın pozisyonlarını kontrol etmek hedeflenmiştir. Bu hedeflere uygun olarak kontrol tekniği belirlenmiştir. Kontrolcü olarak Lineer Kuadratik Regülatör (LQR) tekniği kullanılmıştır. LQR kontrolcüsü ile birden fazla durum değişkenleri kontrol edilebildiği için ters sarkaç sisteminde tercih edilmiştir. Gerçekleştirilen çalışmada LQR kontrolcüsünün performansını doğrudan etkileyen Q ve R matrisleri Genetik Algoritma ile optimize edilmiştir. Optimize edilmiş LQR kontrolcüsü ve standart LQR kontrolcüsü olarak iki farklı yöntem uygulanmıştır. Genetik Algoritma geniş arama algoritmalarının aksine en iyiye ulaşmak için bir yaklaşımı olmadığından en iyiye ulaşamayabilir fakat zaman kısıtlamalarını dikkate almada en iyi algoritmalardan birisi olduğu için tercih edilmiştir. Ters sarkaç sisteminde yapılan optimizasyonlarda amaç fonksiyonları genellikle referans değere yükselme süresi, oturma süresi ve kalıcı durum hatalarının toplanması olarak kullanılmaktadır. Gerçekleştirilen çalışmada farklı olarak Genetik Algoritmanın uygunluk fonksiyonu için bir öneri sunulmuştur. Bu öneri, arabanın referans pozisyon değeri ile arabanın pozisyon değeri arasındaki farkın minimize edilmesi şeklinde tasarlanmıştır. Genetik Algoritma (GA) uygunluk fonksiyonunun çalışmada önerilen formül ile kullanıldığında kabul edilebilir sonuçlar ürettiği gösterilmiştir. Gerçekleştirilen deneyler sonucunda Genetik Algoritma ile optimizasyonu yapılan LQR kontrolcüsü, deneme yanılma yöntemiyle bulunan değerler ile çalışan LQR kontrolcüsüne göre daha başarılı olduğu gözlemlenmiştir. Aynı zamanda Q ve R matrisleri Genetik Algoritma ile belirlendiği için bu katsayıların belirlenmesinde kaybedilen zamanın önüne geçilmiştir.

Feedback control of chaotic systems using multiple shooting shadowing and application to Kuramoto-Sivashinsky equation.

We propose an iterative method to evaluate the feedback control kernel of a chaotic system directly from the system's attractor. Such kernels are currently computed using standard linear optimal control theory, known as linear quadratic regulator theory. This is however applicable only to linear systems, which are obtained by linearizing the system governing equations around a target state. In the present paper, we employ the preconditioned multiple shooting shadowing (PMSS) algorithm to compute the kernel directly from the nonlinear dynamics, thereby bypassing the linear approximation. Using the adjoint version of the PMSS algorithm, we show that we can compute the kernel at any point of the domain in a single computation. The algorithm replaces the standard adjoint equation (that is ill-conditioned for chaotic systems) with a well-conditioned adjoint, producing reliable sensitivities which are used to evaluate the feedback matrix elements. We apply the idea to the Kuramoto-Sivashinsky equation. We compare the computed kernel with that produced by the standard linear quadratic regulator algorithm and note similarities and differences. Both kernels are stabilizing, have compact support and similar shape. We explain the shape using two-point spatial correlations that capture the streaky structure of the solution of the uncontrolled system.

Vehicular Lateral Tracking Control with Winding Road Disturbance Compensation

Abstract In this paper, an innovative vehicular lateral tracking control scheme is proposed to compensate for a winding road disturbance (WRD) on a curved road. Using a dynamic lateral motion model with look-ahead distance, we propose a WRD compensator (WRDC) to sufficiently achieve satisfactory vehicular lateral tracking control performance on the curved road in the presence of WRD. The proposed tracking control scheme is designed by the WRDC gain from the dominant state for the lateral offset in the lane keeping control. By developing the WRDC, state-space reference is redesigned to compensate for tracking error. We showed that the proposed WRDC ensures that the projected tracking error converges to zero. To verify the usefulness of the tracking control, the WRDC applied to a linear quadratic regulator (LQR) controller was compared to the standard LQR, LQR with an integrator, and LQR with an anti-windup. We observed that the proposed WRDC is not only robust against road curvature variation but also outperforming the tracking performance of LQR with anti-windup.

Indirect Adaptive State-Feedback Control of Rotary Inverted Pendulum Using Self-Mutating Hyperbolic-Functions for Online Cost Variation

This paper presents the development of an indirect adaptive state-feedback controller to improve the disturbance-rejection capability of under-actuated multivariable systems. The ubiquitous Linear-Quadratic-Regulator (LQR) is employed as the baseline state-feedback controller. Despite its optimality, the LQR lacks robustness against parametric uncertainties. Hence, the main contribution of this paper is to devise and retrofit the LQR with a stable online gain-adjustment mechanism that dynamically adjusts the state weighting-coefficients of LQR's quadratic cost-function via state-error dependent nonlinear-scaling functions. An original self-mutating phase-based adaptive modulation scheme is systematically formulated in this paper to self-adjust the state weighting-coefficients. The scheme employs pre-calibrated secant-hyperbolic-functions whose waveforms are dynamically reconfigured online based on the variations in magnitude and polarity of state-error variables. This augmentation dynamically alters the solution of the Riccati-Equation which modifies the state-feedback gains online. The proposed adaptation flexibly manipulates the system's control effort as the response converges to or diverges from the reference. The efficacy of proposed adaptive controller is validated by conducting hardware-in-the-loop experiments to vertically stabilize the QNET 2.0 Rotary Pendulum system. As compared to the standard LQR, the proposed adaptive controller renders rapid transits in system's response with improved damping against oscillations, while maintaining its asymptotic-stability, under bounded exogenous disturbances.

Optimal guaranteed cost control of discrete-time linear systems subject to structured uncertainties

ABSTRACT In this paper, we propose state- and static output-feedback generalised guaranteed cost control (GCC) approaches for discrete-time linear systems subject to norm-bounded structured parametric uncertainties. This method enables the convex synthesis for a more general class of systems, where uncertainties are uncorrelated block diagonal, and no feed-through uncertainty is multiplicative with control input ones. It also provides necessary and sufficient conditions for state-feedback and sufficient conditions for static output-feedback. We also present a comparative study among the proposed controllers, standard Linear Quadratic Regulator, and Guaranteed Cost Controller found in the literature.

Controlling an Underactuated Two-Wheeled Mobile Robot: A Constraint-Following Approach

Controlling underactuated systems is a challenging problem in control engineering. This paper presents a novel constraint-following approach for control design of an underactuated two-wheeled mobile robot (2 WMR), which has two degrees-of-freedom (DOF) to be controlled but only one actuator. The control goal is to drive the 2 WMR to follow a set of constraints, which may be holonomic or nonholonomic constraints. The constraint is considered in a more general form than the previous studies on constraint-following control (hence including a wider range of constraints). No auxiliary variables or pseudo variables are required for the control design. The proposed control only uses physical variables. We show that the proposed control is able to deal with both holonomic and nonholonomic constraints by forcing the constraint-following error to converge to zero, even if the system is not initially on the constraint manifold. Using this control design, we investigate two cases regarding different constraints on the 2 WMR motion, one for a holonomic constraint and the other for a nonholonomic constraint. Simulation results show that the proposed control is able to drive the 2 WMR to follow the constraints in both cases. Furthermore, the standard linear quadratic regulator (LQR) control is applied as a comparison in the simulations, which reflects the advantage of the proposed control.

Control Performance Aware Cooperative Transmission in Multiloop Wireless Control Systems for Industrial IoT Applications

The wide application of Internet of Things (IoT) in industrial automation encourages the emergence of a new paradigm of industrial IoT systems, wireless control system (WCS), where the system and/or control information is delivered over wireless channels. In practical systems, WCSs would consist of multiple control-loops in general, the resource competition among which would seriously increase mutual interferences and transmission collisions, making it is difficult to provide the required transmission reliability for the control strategy. To address this issue, we design the control strategy together with the hybrid cooperative transmission scheme for multiloop WCSs in a proactive way. We first define the overall system cost function to explore the impacts of standard linear quadratic regulator control cost and wireless transmission reliability on the control performance. In order to further minimize the overall system cost while guaranteeing the control stability, we then propose a control performance aware cooperative transmission scheme, which is formulated as a constrained optimization problem. Decomposition method and heuristic algorithms are designed based on the feature of network structure to solve the formulated mixed integer nonlinear programming problem efficiently. Finally, simulation results demonstrate that by using the proposed strategy, the overall system cost is significantly reduced, decreasing by 78% and 82% compared to the cases without considerations of system dynamics and without cooperative transmission, respectively.

A new method for proving the separation principle for the infinite-dimensional LQG regulator problem

Abstract This paper gives an alternative perspective on the standard linear quadratic Gaussian regulator problem for infinite-dimensional state-space systems. We will show that when considered on an extended Hilbert state-space, the originally stochastic control problem can be phrased as a deterministic one. In this setting we obtain a new method of proof for the well-known separation principle which does not involve probabilistic considerations.

Integrated Mechatronic Design in the Flexure-Linked Dual-Drive Gantry by Constrained Linear–Quadratic Optimization

A dual-drive H-gantry is commonly used in many industrial processes to meet the requirement of high-precision Cartesian motion. Unlike the rigid-linked gantry stage, the flexure-linked counterpart allows a small degree of rotation of the crossarm to prevent possible damages. However, by this design, the chattering of control signals and inappropriate stiffness of the flexure may induce the resonant modes of the gantry. Hence, to maintain the precision tracking of the midpoint position and the orientation of the gantry, as well as to minimize the vibration on the end effector, we seek the most suitable flexure stiffness and controller parameters by formulating a constrained linear–quadratic optimization problem. Since such a mechatronic design problem is not solvable via standard linear–quadratic regulator formulas, we convert it to a constrained projection gradient-based optimization problem, which can be efficiently solved by direct computation of projection gradient and line search of optimal step length. A fast convergence of parameters is achieved after first several iterations. Through a series of comparative experiments, the effectiveness of the proposed method is validated.

Recursive Robust Regulator for Discrete-Time Markovian Jump Linear Systems

In this paper, we propose a robust regulator for discrete-time Markovian jump linear systems (DMJLS) subject to structured parameter uncertainties. States of Markov chain are considered known at each instant and the transition probabilities matrix is time-varying. The framework proposed to deal with this problem is developed through a regularization approach based on robust least-squares and penalty functions. Recursive solutions in terms of coupled Riccati equations, which do not depend on any auxiliary parameter to be tuned, are provided. These equations resemble the recursive solutions of DMJLS standard regulators when the system is not subject to uncertainties. Conditions for convergence and stability of this robust regulator are established. Numerical examples present comparative studies considering the proposed robust regulator, the standard linear quadratic regulator, and an $H_{\infty }$ -controller for DMJLS.

Predictive control for networked systems affected by correlated packet loss

SummaryIn this paper, we consider the predictive control problem of designing receding horizon controllers for networked linear systems subject to random packet loss in the controller to actuator link. The packet dropouts are temporarily correlated in the sense that they obey a Markovian transition model. Our design task is to solve the optimal controller that minimizes a given receding horizon cost function, using the available packet loss history. Due to the correlated nature of the packet loss, standard linear quadratic regulator methods do not apply. We first present the optimal control law by considering the correlations. This controller turns out to depend on the packet loss history and would typically require a large lookup table for implementation when the Markovian order is high. To address this issue, we present and compare several suboptimal design approaches to reduce the number of control laws.

Co‐design of stabilisation and transmission scheduling for wireless control systems

Wireless control system (WCS) refers to the feedback control system with state and (or) control information transmitted over wireless channels. The control system stability is severely affected by the wireless transmission reliability, which in turn relies heavily on the scheduling of limited wireless communication resources, such as channel, time and power. Therefore, co-design of control stabilisation and transmission scheduling is of vital importance. In this study, the authors firstly explore the effect of wireless transmission reliability on control performance for a WCS with multiple subsystems, local sensors and a remote combination of controller-scheduler. Then a feedback control method is designed to mitigate the impact of unreliable transmission in the loop of sensor-to-controller and controller-to-actuator. Stabilisation and transmission scheduling are co-designed to stabilise the system and minimise the control-communication cost by solving a mixed-integer non-linear programming problem. A decomposition scheme is finally proposed to optimise the control performance by minimising the gap with standard linear quadratic regulator controller as well as the communication cost by allocating the transmission channels, time-slots and power. The effectiveness and advantage of the proposed scheme are demonstrated by the simulation results for a slab temperature control system in hot rolling process.