Linear Quadratic Regulator Research Articles

Reinforcement Learning Control of Hypersonic Vehicles and Performance Evaluations

This work presents a new framework for model-based continuous-time reinforcement learning (CT-RL) control of hypersonic vehicles (HSVs). The predominant classes of CT-RL methods for general nonlinear systems in adaptive dynamic programming (ADP) and deep RL tend to either present substantial theoretical results but lack practical synthesis capability (ADP) or show empirical promise without offering theoretical guarantees (deep RL). Meanwhile, RL control frameworks developed directly for HSVs tend to require a simplified model and complicated control structure, and they lack the substantial numerical evaluations essential for real-world flight implementation. To directly address these challenges, we propose a new decentralized excitable integral reinforcement learning framework within which the reference input-based exploration improves persistence of excitation. Together with new insights on prescaling and established decentralized control structure for HSVs, we demonstrate the resulting controller for significant performance improvement over classical Linear Quadratic Regulator (LQR) and feedback linearization methods. Additionally, we provide convergence, optimality, and closed-loop stability guarantees of the proposed method. We demonstrate these performance guarantees over a set of substantial and systematic numerical evaluations on an unstable, nonminimum phase HSV model subject to varying modeling errors and initial conditions.

Active Vibration Control of Landing Gear System

The vibration of the aircraft when taxing, landing, and taking off is a great problem. Vibration induced due to the dynamic interaction of aircraft with the ground causes serious problems. Therefore, the vibration should be reduced or completely absorbed if possible by some external means. So, in this study active vibration control of an aircraft LG is investigated. The equation of motion of the aircraft LG system is obtained by Newton’s second law and simulated in MATLAB/Simulink environment. Two different controllers are fitted to the landing gear system. Proportional, integral derivative (PID) and Linear Quadratic Regulator (LQR) controllers are used in controlling active suspension structure. In the study, vertical displacement and acceleration of the fuselage are set as objective since it has harmful effects on the performance of the pilot’s capability. Simulation is run under B grade road and bump road for passive and active systems simultaneously. The performance of controllers is compared to one another and also with that of a passive system. From the results, it was obtained that under bump road excitation, LQR and PID controllers make 57.32% and 53.34% improvement in fuselage displacement, respectively. Similarly, under B Grade road, compared to passive systems LQR and PID controllers improve the vertical acceleration of the fuselage by 92.86% and 53.55% respectively.

Experimental backward integration for state-dependent differential Riccati equation (SDDRE): A case study on flapping-wing flying robot

Backward integration (BI) is a classical approach for solving optimal control arising from linear quadratic regulator (LQR) design in differential form. It proposes a two-round solution, the first one starting from a final boundary condition to the initial one to generate the optimal gain, and the next one, solving the control system in a forward loop. Implementation of the BI on the nonlinear optimal control, the state-dependent differential Riccati equation (SDDRE), is a challenge since in the backward loop, the state information is missing and is not straightforward like the LQR. Hence, a control for backward motion is required to regulate the system from the terminal to the initial desired states. While there have been some valuable works on the theoretical implementation of the BI in simulations for the SDDRE, this approach has not been reported in experiments for the best knowledge of the authors. Here in this work, the SDDRE is solved using the BI and two backward and forward solution steps, and it is experimentally applied for a flapping-wing flying robot (FWFR). The execution of the control system is done onboard using Raspberry Pi 4B as the processor which has limited computational capacity. The trajectory tracking of a line in a closed limited space is proposed for the FWFR flight. The objective is to position the robot bird at the corner of the rectangular limited space with minimum error in translation. The results have been presented for 21 flights to show the repeatability and presented the best-case minimum error of 10 (cm) at the end of the trajectory, in the YZ-plane. Considering approximately 12 (m) flight path, the error is found less than 1 percent of the travel distance. The results were compared with forward integration to confirm the correctness of the computation. The experimental flight dataset, MATLAB simulation codes, and experimental Python codes are available as supplementary material for this work in the online version of the paper.

The Analytical Design and Implementation of Inverted Pendulum Control System using Linear Quadratic Regulator

The Analytical Design and Implementation of Inverted Pendulum Control System using Linear Quadratic Regulator

Optimal and robust control methodologies for 4DOF active suspension systems: A comparative study with uncertainty considerations

AbstractDrawing from recent developments in the field, this article explores advanced control methodologies for active suspension systems with the aim of enhancing ride comfort and vehicle handling. The study systematically and comprehensively implements, simulates, and compares five control methods: Proportional‐integral‐derivative (PID), linear quadratic regulator (LQR), , , and synthesis in the context of half‐vehicle active suspension systems. By using a detailed system model that includes parameter uncertainties and performance weights, analysis, and simulations are conducted to evaluate the performance of each control approach. The results provide valuable insights into the strengths and limitations of these methods, offering a comprehensive comparative analysis. Notably, the study reveals that control may not ensure stability for all possible combinations within a broad range of uncertainties, indicating the need for careful consideration in its application. The results and simulations thoroughly evaluate and compare the performance of each control strategy across various output responses, contributing to the advancement of more effective and reliable active suspension systems.

A novel development of h-CSLQFL controller for optimal sizing and economic pricing of battery in electric vehicle system

This paper proposes a novel design of h-CSLQFL controller for reducing the battery size (kWh) and battery cost (US$/kWh) of electric vehicle (EV) system. The h-CSLQFL controller is a hybrid controller which is formed by the combination of Cuckoo search optimization (CSO), linear quadratic regulator (LQR), and fuzzy logic controller (FLC). The higher battery cost is one of the biggest challenges with EV due to which a significant number of consumers cannot afford to purchase it. The replacement of conventional vehicles with EV creates ecofriendly and sustainable solutions. For attaining the optimal reduced cost of battery, a strategy has been demonstrated by framing a multi-objective function which is controlled with h-CSLQFL controller. It commences with the design of mathematical model of EV in which supply voltage and current are the input parameters whereas the speed and torque are the output parameters. Thereafter, LQR controller was designed and implemented for analyzing the performance parameter of EV. Afterwards, CSO controller in support with LQR controller has been formulated in closed loop manner. At the end, the multi-objective function is controlled with LQR based CSO and LQR controller. It is observed that the least optimal economic cost and size of EV battery have been attained with h-CSLQFL controller in comparison to the LQR based CSO and LQR controller. In addition to this, the least risk factor and reduced sensitivity have been also attained with the proposed approach. The validation of the proposed h-CSLQFL controller has been done on a hardware set up.

Vibration control of structure using active tuned mass damper: A new control algorithm

This paper investigates the vibration behavior of a 10-story benchmark building controlled with an active tuned mass damper (ATMD) under earthquake loads. An ATMD, typically installed on the top floor, is a modified form of a tuned mass damper (TMD) that includes a mass, spring, damper, and an actuator enhancing system performance and structural damping. The force and movement direction of the ATMD are controlled by a processor and actuator. In the ATMD system, control algorithms determine the magnitude and direction of the applied force. This research introduces a new control algorithm based on the structure’s dynamic response and ATMD stiffness. To evaluate the performance of this algorithm, linear–quadratic regulator (LQR) and fuzzy logic controller (FLC) algorithms were designed and presented for comparison. The results indicate that the proposed algorithm outperformed the other algorithms, reducing the structure’s displacement and acceleration responses by an average of 40% and 28.16%, respectively, compared to the uncontrolled state.

Decentralized Retrofit Model Predictive Control of Inverter-Interfaced Small-Scale Microgrids

In recent years, small-scale microgrids have become popular in the power system industry because they provide an efficient electrical power generation platform to guarantee autonomy and independence from the power grid, which is a critical feature in cases of catastrophic events or remote areas. On the other hand, due to the short distances among multiple distribution generation systems in small-scale microgrids, the interconnection couplings among them increase significantly, which jeopardizes the stability of the entire system. Therefore, this work proposes a novel method to design decentralized robust controllers based on a retrofit model predictive control scheme to tackle the issue of instability due to the short distances among generation systems. In this approach, the retrofit model predictive controller receives the measured feedback signal from the interconnection current and generates a control command signal to limit the interconnection current to prevent instability. To design a retrofit controller, only the model of a robust closed-loop system, as well as an interconnection line, is required. The model predictive control signal is added in parallel to the control signal from the existing robust voltage source inverter controller. Simulation results demonstrate the superior performance of the proposed technique as compared with the virtual impedance and retrofit linear quadratic regulator techniques (benchmarks) with respect to peak-load demand, plug-and-play capability, nonlinear load, and inverter efficiency.

Ship Autonomous Berthing Strategy Based on Improved Linear-Quadratic Regulator

There has been significant interest in the research field of ship automatic navigation, particularly in the area of autonomous berthing. To address the key challenges of path planning and control during ship berthing, we propose an enhanced Linear−Quadratic Regulator (LQR) control approach, reinforced by the Covariance Matrix Adaptation Evolution Strategy (CMA−ES), along with an adaptive berthing strategy decision model. This integrated framework encompasses ship motion control, path planning, and berthing strategy selection to facilitate adaptive and autonomous ship berthing. Initially, a dynamic mathematical model of ship motion is established, taking into account wind and current interference effects. Subsequently, an adaptive environment−aware berthing strategy model is introduced to enable automatic selection of berthing strategies based on spatial relationships between environmental factors and the berth. By utilizing the refined LQR method, autonomous motion control for ship berthing is achieved. To validate the effectiveness of our controller, comprehensive simulation analyses are conducted under varying operating conditions to encompass crucial factors such as large drift angle characteristics of ships, shallow water effects, and bank effects across seven diverse working conditions. The simulation results underscore the robustness of our proposed method in responding to environmental interference while demonstrating its capability to select appropriate berthing strategies based on varying operational scenarios.

Hybrid coati–grey wolf optimization with application to tuning linear quadratic regulator controller of active suspension systems

Vehicle suspension systems have become increasingly crucial for both driving safety and comfort. Active suspension systems can dynamically adjust suspension characteristics in real-time by introducing force into the system. Designing a controller for the real-time adjustment of the control force in active suspension systems is essential to meet challenging control objectives, including body acceleration, suspension deflection, and tire deflection. This article proposes a hybrid optimization approach named Coati–Grey Wolf Optimization (COAGWO), which combines the strengths of the coati optimization algorithm and grey wolf optimization to tune the gains of linear quadratic control applied to vehicle suspension systems. The COAGWO algorithm incorporates a unique strategy inspired by the Coati Optimization Algorithm, allowing wolves to climb trees. This enhancement significantly improves the wolves’ ability to explore the global search space and reduces the likelihood of being trapped in local optima. Initially, we conduct extensive experiments using a suite of challenging optimization problems from the CEC2019 benchmark to evaluate the effectiveness of the COAGWO algorithm. The effectiveness of COAGWO is compared against several state-of-the-art algorithms, including grey wolf, coati, aquila-grey wolf, whale, reptile search, tunicate swarm, and seagull optimization algorithms. The experimental results demonstrate that COAGWO consistently outperforms these algorithms in terms of solution quality and convergence speed. For the optimal weight selection problem of linear quadratic control applied to the control of vehicle suspension systems, the excellent performance of the proposed method is illustrated through comparative simulation studies under various road disturbance conditions. The results indicate that the COAGWO algorithm achieves a more efficient active suspension system compared to competitor algorithms by reducing the overall acceleration of the driver’s body, thereby enhancing ride comfort.

Optimal linear quadratic regulator-based reduced order model for hybrid AC/DC microgrid

The stability of hybrid microgrids represents an elementary requirement to ensure their security and reliability. This paper aims to propose a supplementary controller that can enhance the hybrid microgrid dynamic performance while preserving its stability. At first, the paper derives the small signal state-space model of a hybrid microgrid that includes inverter-based and conventional generation. This model is then reduced using the balanced truncation method. The reduced order model is utilized to design a supplementary linear quadratic output regulator known for its efficiency and high performance. A Kalman observer is employed to measure the states of the reduced order model to complete the control process. Finally, time domain simulations are used to validate the reduced order model and verify the performance of the proposed controller. The results revealed the accuracy of the model and the effectiveness of the controller.

Active vibration control of rotating carbon nanotube reinforced composite cylindrical shells via piezoelectric patches

This paper addresses the active vibration control of rotating carbon nanotube reinforced composite (CNTRC) cylindrical shells via piezoelectric actuator and sensor pairs. Considering circumferential initial stresses and Coriolis forces induced by rotation, an electromechanical coupling model of a simply supported CNTRC cylindrical shell, covered with surface-bonded piezoelectric sensors/actuators is established using the Lagrange equations and the model validation is carried out through a comparative analysis with existing literature. To suppress vibrations of rotating CNTRC cylindrical shells over a range of speeds, an LQR (Linear Quadratic Regulator) closed-loop controller is designed and its effectiveness is analyzed and evaluated through dynamic response analysis. Furthermore, the optimization of piezoelectric patch layout is performed by analyzing the performance of the controller for rotating CNTRC shells with typical piezoelectric sensors/actuators distributions. This paper presents and validates a strategy for vibration control of rotating CNTRC cylindrical shells using piezoelectric patches. The findings derived can offer guidance for vibration suppression of rotating thin-walled structures in practical engineering applications.

Vibration control and robustness analysis of tensegrity structures via fuzzy dynamic sliding mode control method

This study addresses the problem of mitigating vibration responses in tensegrity structures subjected to complex dynamic loading conditions. The overall purpose is to develop an effective control method to enhance the stability and performance of these structures. To achieve this, a fuzzy dynamic sliding mode control (FDSMC) method is proposed. The methodology involves deriving the state space expression of the controlled system based on the tensegrity structure's dynamic model, followed by designing a dynamic sliding mode controller using the reaching law. Fuzzy rules are incorporated to adaptively adjust the dynamic sliding mode parameters, accounting for uncertainties in controller parameters, thereby ensuring high efficiency and smooth controller output. A comparative analysis scheme, focusing on identical energy input on actuators, is utilized to evaluate the performance of various active vibration control methods. Two illustrative examples, a spatial double-layer tensegrity beam and a complex tensegrity spiral tower, are investigated in detail. The findings demonstrate that the FDSMC method significantly reduces structural dynamic responses and improves algorithm robustness compared to the conventional linear quadratic regulator (LQR) method, indicating its promising potential for relevant engineering applications.

Inferring Human Control Intent Using Inverse Linear Quadratic Regulator With Output Penalty Versus Gain Penalty: Better Fit but Similar Intent

Abstract To identify the underlying mechanisms of human motor control, parametric models are utilized. One approach of employing these models is the inferring the control intent (estimating motor control strategy). A well-accepted assumption is that human motor control is optimal; thus, the intent is inferred by solving an inverse optimal control (IOC) problem. Linear quadratic regulator (LQR) is a well-established optimal controller, and its inverse LQR (ILQR) problem has been used in the literature to infer the control intent of one subject. This implementation used a cost function with gain penalty, minimizing the error between LQR gain and a preliminary estimated gain. We hypothesize that relying on an estimated gain may limit ILQR optimization capability. In this study, we derive an ILQR optimization with output penalty, minimizing the error between the model output and the measured output. We conducted the test on 30 healthy subjects who sat on a robotic seat capable of rotation. The task utilized a physical human–robot interaction with a perturbation torque as input and lower and upper body angles as output. Our method significantly improved the goodness of fit compared to the gain-penalty ILQR. Moreover, the dominant inferred intent was not statistically different between the two methods. To our knowledge, this work is the first that infers motor control intent for a sample of healthy subjects. This is a step closer to investigating control intent differences between healthy subjects and subjects with altered motor control, e.g., low back pain.

LQR controller performance via particle swarm optimization and neural networks

AbstractThe inverted pendulum‐cart (IPC) control problem has long been a benchmark in the field of control systems due to its inherent instability and nonlinear dynamics. The linear quadratic regulator (LQR) control technique has been proven to be effective in stabilizing the inverted pendulum; however, the challenge lies in finding the optimal control gains that provide the best performance. This article presents a method to address the LQR control design problem for the IPC system which is compared against a particle swarm optimization based LQR and a neural network optimized LQR. The method provides a deterministic approach to finding the weighing matrices Q and R in accordance with the time domain characteristics chosen by the designer, such as settling time and maximum peak overshoot. Results from MATLAB simulations indicate that the suggested strategy has good performance.

Compound Control Design of Near-Space Hypersonic Vehicle Based on a Time-Varying Linear Quadratic Regulator and Sliding Mode Method

The design of a hypersonic vehicle controller has been an active research field in the last decade, especially when the vehicle is studied as a time-varying system. A time-varying compound control method is proposed for a hypersonic vehicle controlled by the direct lateral force and the aerodynamic force. The compound control method consists of a time-varying linear quadratic regulator (LQR) control law for the aerodynamic rudder and a sliding mode control law for the lateral thrusters. When the air rudder cannot continuously produce control force and torque, the direct lateral force is added to the system. To solve the problem that LQR cannot directly obtain the analytical solution of the time-varying system, a novel approach to approximate analytical solutions using Jacobi polynomials is proposed in this paper. Finally, the stability of the time-varying compound control system is proven by the Lyapunov–Krasovskii functional (LKF). The simulation results show that the proposed compound control method is effective and can improve the fast response ability of the system.

Linear Quadratic Gaussian Controller for Single-Ended Primary Inductor Converter via Integral Linear Quadratic Regulator Merged with an Offline Kalman Filter

This paper introduces a Linear Quadratic Gaussian (LQG) controller for a Single-Ended Primary Inductor Converter (SEPIC). The LQG design is based on merging an integral Linear Quadratic Regulator (LQR) with an offline Kalman Filter (commonly referred to as a Linear Quadratic Estimator (LQE)). The robustness of the LQG controller is guaranteed based on the separation principle. This manuscript addresses the need to use observer-based systems for the fourth-order SEPIC, which needs a sensor reduction as an essential requirement. This paper provides a comprehensive, yet systematic, approach to designing the LQG system. The work validates the convergences of the states in an LQG system to an actual value. Furthermore, it compares the performance of an LQG system with a benchmark Type-II industrial controller by means of a simulation of the switched converter model in the Simulink/MATLAB 2023a environment.

Experimental Evaluation of the Robust Controllers Applied on a Single Inductor Multiple Output DC-DC Buck Converter to Minimize Cross Regulation Considering Parametric Uncertainties and CPL Power Variations

This paper presents two control design strategies for voltage regulation in a single inductor dual-output DC-DC buck converter system. Based on a nominal multiple input multiple output plant model and performance requirements, both a Linear Quadratic Regulator and a Decoupled PI control law are designed to control the power converter system under parametric uncertainties such as voltage source variation, constant power load variation, and load resistance variation. Therefore, the converter can cascade operation, which may cause the undesired effects of voltage oscillation and reduce the system’s stability margin. The control performance was assessed under both the uncertainties of resistive loading and the power variation in a constant power load. A single inductor dual-output DC-DC buck converter board was developed for experimental tests. The experimental results show that the Linear Quadratic Regulator strategy, compared to the PI strategy, presented a robust performance in the presence of parametric uncertainties in the resistive load-bearing; however, it was sensitive to uncertainties due to the presence of constant power loads.

On the Analysis of Model-Free Methods for the Linear Quadratic Regulator

On the Analysis of Model-Free Methods for the Linear Quadratic Regulator

Secondary voltage regulation based on full-state feedback — performance assessment with Dyna[formula omitted]o

This paper discusses the performance assessment of new secondary voltage regulators currently under study in France to address challenges posed by the energy transition. With renewable integration and increasing cross-border interconnections, power flow variations surge across the network. Coupled with the historical SVRs’ relatively slow response, operational since the late 1970s, recent observations show higher voltage volatility. Prior research has already highlighted the potential of two novel regulators: the Average Q-SVR and the LQ-SVR, in mitigating these issues. Our focus here centres on the latter and begins by discussing the implications of control design hypotheses on closed-loop system performance. These include linearisation, neglecting communication delays, and assuming uniform dynamic behaviour across all resources. We then propose a two-step control design, leveraging Linear Quadratic Regulator (LQR) and eigenstructure placement methods and demonstrate that this approach surpasses the LQR method alone in terms of transient reactive power tracking.