Classical Linear Quadratic Regulator Research Articles

Neural algorithm for optimization of multidimensional object controller parameters

Optimal control of multivariable systems is a complex dynamic process that minimizes the cost function to obtain the optimal control strategy. Unfortunately, for nonlinear systems, it is not possible to use the traditional linear quadratic regulator (LQR), which would be optimal over the entire range of parameter variation. The problem of nonlinear multivariable systems and their optimal control is very momentous. The solution presented in this paper is based on the application of Reinforcement Learning (RL) networks in controlling a five-degree-of-freedom overhead crane system. Additionally, unlike the classical approach, the algorithm is adapted to directly analyze tabular data of inputs and outputs of the controlled model instead of analyzing its state as feedback (model-free). Implementing the new control structure for the multivariable system improved control quality compared to the classical LQR controller with linearization at the operating point. In addition to quality, the resource indicators, which in the LQR controller are represented by the matrix R, have been significantly improved. The architecture of the neural control system is presented, ensuring that over the entire range of nonlinearity, the quality of control is preserved while reducing the cost of its resource intensity. Obtaining optimal control with reduced resources for its implementation induces a wide range of applications of such neural control in engineering systems. The effectiveness of the proposed control system has been demonstrated in simulation studies. The simulation results present the system’s excellent control performance and adaptability over the entire range of object nonlinearity. The neural algorithm resulted in significantly shorter adjustment time and better control quality with significantly less system resource consumption and increased system dynamics.

Reinforcement Learning Control of Hypersonic Vehicles and Performance Evaluations

This work presents a new framework for model-based continuous-time reinforcement learning (CT-RL) control of hypersonic vehicles (HSVs). The predominant classes of CT-RL methods for general nonlinear systems in adaptive dynamic programming (ADP) and deep RL tend to either present substantial theoretical results but lack practical synthesis capability (ADP) or show empirical promise without offering theoretical guarantees (deep RL). Meanwhile, RL control frameworks developed directly for HSVs tend to require a simplified model and complicated control structure, and they lack the substantial numerical evaluations essential for real-world flight implementation. To directly address these challenges, we propose a new decentralized excitable integral reinforcement learning framework within which the reference input-based exploration improves persistence of excitation. Together with new insights on prescaling and established decentralized control structure for HSVs, we demonstrate the resulting controller for significant performance improvement over classical Linear Quadratic Regulator (LQR) and feedback linearization methods. Additionally, we provide convergence, optimality, and closed-loop stability guarantees of the proposed method. We demonstrate these performance guarantees over a set of substantial and systematic numerical evaluations on an unstable, nonminimum phase HSV model subject to varying modeling errors and initial conditions.

Wind and Wave-Induced Vibration Reduction Control for Floating Offshore Wind Turbine Using Delayed Signals

Active vibration control is a critical issue of the wind turbine in the field of marine energy. First, based on a three-degree-of-freedom wind turbine, a state space model subject to wind and wave loads is obtained. Then, a delayed state feedback control scheme is illustrated to reduce the vibration of platform pitch angle and tower top foreaft displacement, where the control channel includes time-delay state signals. The designed controller’s existence conditions are investigated. The simulation results show that the delayed feedback H∞ controller can significantly suppress wind- and wave-induced vibration of the wind turbine. Furthermore, it presents potential advantages over the delay-free feedback H∞ controller and the classic linear quadratic regulator in two aspects: vibration control performance and control cost.

Wavelet LQR based gain scheduling for power regulation and vibration control of floating horizontal axis wind turbine with double-pitched rotor

This paper introduces a wavelet-based linear quadratic regulator (LQR) control strategy for double-pitched rotor systems of an offshore horizontal axis wind turbines. A comprehensive aero-servo-elastic model incorporating bending-torsion coupling is developed to assess the performance of the proposed controller under combined turbulent wind and wave loading. The weight matrices of the classical LQR algorithm are optimized using a nonlinear optimization scheme. Additionally, frequency-dependent gain scheduling is proposed using wavelet transform where the analytic Morse wavelet serves as the basis function. The resulting LQR algorithm operates in the time-frequency domain and solves scale-dependent Algebraic Riccati equations to determine optimal gains. Numerical simulations demonstrate the superior performance of the proposed gain scheduling compared to the classical LQR approach. The effectiveness of the algorithm is evaluated across various flow conditions and wind-wave misalignment, with benchmark results used for performance comparison.

Double-loop LQR depth tracking control of underactuated AUV: Methodology and comparative experiments

In this paper, a double-loop depth tracking control scheme based on linear quadratic regulator (LQR) is proposed for underactuated autonomous underwater vehicles (AUV). The motivation is to address the problem that the classic single-loop LQR depth controller which is easy to use in practice yet cannot effectively overcome the underactuated AUV features and complex disturbances. First, a double-loop depth control framework is proposed which divides complex disturbances into outer-loop kinematic disturbances and inner-loop dynamic disturbances. Second, the adaptive line-of-sight (ALOS) guidance law is designed in the outer loop to account for the underactuated characteristics. The depth tracking error is converted to the desired pitch and the kinematic disturbance caused by the unknown angle of attack is compensated. Furthermore, based on the linear model in the inner-loop, a concise LQR method is designed to track the desired pitch. Meanwhile, an extended state observer (ESO) is introduced to estimate dynamic disturbances such as model nonlinearity, uncertainty and external environmental disturbances. Finally, the proposed scheme is validated by tank test and compared with the single-loop LQR method. The experiment results show that the proposed double-loop depth tracking method can effectively overcome complex disturbances and the depth tracking accuracy is increased by 83%.

A deep reinforcement learning framework to modify LQR for an active vibration control applied to 2D building models

Abstract Deep reinforcement learning (DRL) has emerged as a promising approach for optimizing control policies in various fields. In this article, we explore the use of DRL for controlling vibrations in building structures. Specifically, we focus on the problem of reducing vibrations induced by external sources such as wind or earthquakes. We propose a DRL-based control framework that learns to adjust the control signal of a classical adaptive linear quadratic regulator (LQR)-based model to mitigate the vibration of building structures in real-time. The framework combines the proximal policy optimization method and a deep neural network that is trained using a simulation environment. The network takes input sensor readings from the building and outputs signals that work as a corrector to the signals from the LQR model. It demonstrates the approach’s effectiveness by simulating a 3-story building structure. The results show that our DRL-based control approach outperforms the classical LQR model in reducing building vibrations. Moreover, we show that the approach is robust for learning the system’s dynamics. Overall, the work highlights the potential of DRL for improving the performance of building structures in the face of external disturbances. The framework can be easily integrated into existing building control systems and extended to other control problems in structural engineering.

Active control for vehicle suspension using a self-powered dual-function active electromagnetic damper

Concerning vehicle suspension techniques, active suspension tends to offer the most precise control performance compared to passive and semi-active alternatives. However, due to its high-power consumption, its practical use is often limited. This study aims to develop an innovative self-powered active suspension by assessing the performance and feasibility of a dual-function active electromagnetic damper (DF-AEMD), which is capable of providing simultaneous force tracking and energy harvesting functions. In addition to outlining its working mechanism, the DF-AEMD is utilized to track the full-loop target control force calculated from the classic linear quadratic regulator in a self-powered manner. From the force tracking standpoint, the DF-AEMD can not only provide damping forces, but also generate actuation forces, and thus achieve better tracking accuracies of target control force and control performances for vehicle suspension than the traditional semi-active damper; while from the energy harvesting aspect, sufficient energy can be harvested simultaneously to realize the self-powered active control. Systematic simulations are conducted to discuss the relationship between control performance and energy harvesting efficiency of the DF-AEMD.

Optimal LQR Controller Methods for Double Inverted Pendulum System on a Cart

Most of the systems in our lives are inherently nonlinear and unstable. In control problems in the field of engineering, the aim is to define the control laws that maximize the operating efficiency of these systems under diverse security coefficients, and constraints and minimize error rates. This study aimed to model and optimally control a Double-Inverted Pendulum System on a Cart (DIPSC). A DIPSC was modeled using the Lagrange-Euler method, and classical and optimal Linear Quadratic Regulator (LQR) control methods were designed for the control of the system. The purpose of the designed controllers is to keep the arms of the double inverted pendulum on the moving cart vertically in balance and to bring the cart to the determined balance position. The critically important Q and R parameters of the LQR control technique that is one of the optimal control techniques were obtained using the Genetic Algorithm (GA), Particle Swarm Optimization (PSO), and Grey Wolf Optimization (GWO) algorithms. The DIPSC system was checked using classical LQR and optimal LQR methods. All obtained results are given graphically. The proposed methods are presented and analyzed in tabular form using Settling time and Mean-Square-Error (MSE) performance criteria.

Contribution to strengthening Bus voltage stability and power exchange balance of a decentralized DC-multi-microgrids: Performance assessment of classical, optimal, and nonlinear controllers for hybridized energy storage systems control

Stability is the primary goal in standalone power systems due to the high penetration and uncertainties of renewable energy resources, so the system reliability is affected, and the hybrid energy storage system (HESS) and diesel generator (DG) compensate for renewables' failures. This study contributes to managing and controlling DC-coupled multi-Microgrids fed by Photovoltaic, HESS, and DG resources and subjected to pulsed loading and uncertain PV outputs, besides battery SOCs and DG fuel constraints. The Dual-loop control structure compares classical Proportional Integral (CPI), Super-Twisting-Sliding-Mode-Control (ST-SMC), and Linear Quadratic Regulator with Integral Action (LQR-I) to evaluate the robustness and control performance of the HESS. For the supercapacitor (SC), a hysteresis current control (HCC) tracks its setpoint currents using a frequency decoupling-based power-split control. Matlab simulations confirmed system reliability by combining controllers with different Key Performance Indicators, where adopting the LQR-I for bus voltage regulation and the ST-SMC for HESS current control outperformed the other combinations and revealed superior system performance, improving Bus voltage regulation, DG current tracking, overall system efficiency, and loading convergence by 67.63(%), 83.63(%), 77.92(%), and 2.72(%), respectively. Instead, the basic CPI enhanced battery and SC current control with 51.55 and 63.94 (%) compared to the winner scenario.

Novel Model-free Optimal Active Vibration Control Strategy Based on Deep Reinforcement Learning

Neural networks (NNs) can provide a simple solution to complex structural vibration control problems. However, most past NN-based control strategies cannot guarantee an optimal policy in structural vibration control. In this study, a novel active vibration control strategy based on deep reinforcement learning is proposed, which utilizes the learning ability of NN controllers and simultaneously provides control performance comparable to traditional model-based optimal controllers. The proposed learning algorithm can determine the control policy through interaction with the environment without knowing dynamic system models. This study shows that the proposed model-free strategy can provide optimal control performance to various systems and excitations. The proposed control strategy is first verified on a single-degree-of-freedom model and subsequently extended to a multi-degree-of-freedom shear-building model. Its control performance with full-state feedback is nearly the same as that of a classical linear quadratic regulator. Moreover, the learned policy can outperform a traditional output feedback controller in a partially observed system. The robustness of the proposed control strategy against measurement noise is also tested.

Competitive Equilibriums of Multi-Agent Systems over an Infinite Horizon

In this paper, we investigate the properties of competitive equilibriums for dynamic multi-agent systems (MAS) over an infinite horizon. When there is no external resource supply, a group of dynamic agents with distributed resource allocations form a transactive market to share their resources at a specific price. Each agent makes decisions on the locally consumed resource and the traded resource. The system reaches a competitive equilibrium when each agent's payoff is maximized and the tradings are balanced. Firstly, we consider general utility functions and show that under feasibility assumptions, any competitive equilibrium maximizes the social welfare. Secondly, we prove that for sufficiently small initial conditions, the social welfare maximization solution constitutes a competitive equilibrium with zero price. We also prove for general feasible initial conditions, there exists a time instant after which the optimal price, corresponding to a competitive equilibrium, becomes zero. Finally, we specifically focus on quadratic MAS for which the system-level social welfare optimization becomes a classical constrained linear quadratic regulator (CLQR) problem. We construct explicitly a feasible set of initial conditions under which the price under a competitive equilibrium is zero for all time.

A Real-Time Gaussian Process-Based Stochastic Controller for Periodic Disturbances

Using a non-parametric Gaussian process regression (GPR) framework, we present in this paper a practical learning technique to model, filter, and predict periodic disturbances. Augmenting a known system model with the GP model of the disturbance results in a structure referred to as latent force model (LFM), which is also a GP with an appropriate covariance function. By representing the LFM as a stochastic state-space model using spectral factorization, we apply a full Bayesian inference through Kalman filter and smoother, and design a linear quadratic regulator (LQR) to achieve an optimal closed-loop performance. On a real-time application, LJ MS15 DC motor under induced periodic disturbances, the proposed approach is implemented successfully. Real-time experiments demonstrate the efficiency and practicality of the proposed LFM-based LQR compared to the classical LQR approach.

Optimal State Feedback-Integral Control of Fuel-Cell Integrated Boost Converter

Clean energy and reduction in carbon footprint have motivated the adoption of Fuel Cells as an alternative source of energy, merely having any detrimental byproducts. In this work, the control of the Proton-exchange membrane fuel cell (PEMFC) integrated boost converter has been carried out. State-feedback with integral-based controllers has been designed using the classical pole placement method and optimal linear quadratic regulator approach. The first method aims at achieving the desired performance. The second method also minimizes the error while optimizing the control effort. In this approach, the control energy required for attaining the time and frequency domain objective at steady state operation is also optimized. The same avoids the limitation of existing schemes by avoiding sub-optimal solutions. Both control techniques have been verified in terms of transient performance during the load disturbance using numerical simulations and experimentation. In addition, comparative state trajectories and overall efficiency analysis of the system are presented.

Control Analysis with Modified LQR Method of Anti-Tank Missile with Vectorization of the Rocket Engine Thrust

This article approaches the issue of the optimal control of a hypothetical anti-tank guided missile (ATGM) with an innovative rocket engine thrust vectorization system. This is a highly non-linear dynamic system; therefore, the linearization of such a mathematical model requires numerous simplifications. For this reason, the application of a classic linear-quadratic regulator (LQR) for controlling such a flying object introduces significant errors, and such a model would diverge significantly from the actual object. This research paper proposes a modified linear-quadratic regulator, which analyzes state and control matrices in flight. The state matrix is replaced by a Jacobian determinant. The ATGM autopilot, through the LQR method, determines the signals that control the control surface deflection angles and the thrust vector via calculated Jacobians. This article supplements and develops the topics addressed in the authors’ previous work. Its added value includes the introduction of control in the flight direction channel and the decimation of the integration step, aimed at speeding up the computational processes of the second control loop, which is the LQR based on a linearized model.

Evaluation of a NMPC Flowrate Controller for a Hot Rolling Finishing Mill for Steel Bars

In this work, the material flowrate control of a hot rolling finishing mill for steel bars is evaluated in simulation. First, a reduced model for control purpose is developed and described. This model uses nonlinear equations from the rolling theory and leads to the nonlinear control structure. The simulations are tested on a hot rolling mill simulator that implements a more detailed process behavior and is outside of the scope of this work. For controlling this plant, a Nonlinear Model Predictive Control (NMPC) is tested in various conditions. It is compared as a benchmark with a classical Linear Quadratic Regulator with integral action (LQI). The controller is implemented with help of CasADi and the control problem is solved by a SQP solver. The NMPC uses an underlying EKF as disturbance observer to handle the reference tracking problem.

Intelligent Dynamic Inversion Controller Design for Ball and Beam System

The Ball and beam system (BBS) is an attractive laboratory experimental tool because of its inherent nonlinear and open-loop unstable properties. Designing an effective ball and beam system controller is a real challenge for researchers and engineers. In this paper, the control design technique is investigated by using Intelligent Dynamic Inversion (IDI) method for this nonlinear and unstable system. The proposed control law is an enhanced version of conventional Dynamic Inversion control incorporating an intelligent control element in it. The Moore-Penrose Generalized Inverse (MPGI) is used to invert the prescribed constraint dynamics to realize the baseline control law. A sliding mode-based intelligent control element is further augmented with the baseline control to enhance the robustness against uncertainties, nonlinearities, and external disturbances. The semi-global asymptotic stability of IDI control is guaranteed in the sense of Lyapunov. Numerical simulations and laboratory experiments are carried out on this ball and beam physical system to analyze the effectiveness of the controller. In addition to that, comparative analysis of RGDI control with classical Linear Quadratic Regulator and Fractional Order Controller are also presented on the experimental test bench.

Active Vibration Control of Truss Structures for Large Space Telescopes Based on Cable Actuators

The membrane diffraction is a new imaging method for space telescopes. It makes a hot research topic in space telescope technology with lots of advantages, such as light weight, easy foldability and high optical imaging accuracy. The active vibration control of the truss structure of a kind of membrane diffraction space telescope was investigated, and an active vibration control strategy was proposed based on cable actuators. Firstly, the dynamic model for the telescope truss structure was established. Then the particle swarm optimization algorithm was used to study the arrangement optimization of cable actuators. The active control law for the structure vibration was designed with the classical linear quadratic regulator method. Finally, the numerical simulation results verify the effectiveness of the proposed method. In the numerical simulations, the relationship between the number of cable actuators and the required time for the structure to regain stability was studied in detail.

Discrete‐time decentralized linear quadratic control for linear time‐varying systems

AbstractThis article addresses the problem of designing a decentralized control solution for a network of agents modeled by linear time‐varying (LTV) dynamics, in a discrete‐time framework. A general scheme is proposed, in which the problem is formulated as a classical linear quadratic regulator problem, for the global system, subject to a given sparsity constraint on the gain, which reflects the decentralized nature of the network. A method able to compute a sequence of well‐performing stabilizing regulator gains is presented and validated resorting to simulations of two randomly generated LTV systems, one stable and the other unstable. Moreover, a tracking solution is developed, building on the solution to the regulator problem. Both methods rely on a closed‐form solution, thus they can be computed very rapidly. Similarly to the centralized solution, both the presented methods require that a window of the future system dynamics is known. Both methods are validated resorting to simulations of: (i) a nonlinear network of four interconnected tanks; and (ii) a large‐scale nonlinear network of interconnected tanks. When implemented to a nonlinear network, approximated by an LTV system, the proposed methods are able to compute well‐performing gains that track the desired output. Finally, both algorithms are scalable, being adequate for implementation in large‐scale networks.

Model-based and model-free designs for an extended continuous-time LQR with exogenous inputs

We present an extended linear quadratic regulator (LQR) design for continuous-time linear time invariant (LTI) systems in the presence of exogenous inputs with a novel feedback control structure. We first propose a model-based solution with cost minimization guarantees for states and inputs using dynamic programming (DP) that out-performs classical LQR with exogenous inputs. The control law consists of a combination of the optimal state feedback and an additional optimal term which is dependent on the exogenous inputs. The control gains for the two components are obtained by solving a set of matrix differential equations. We provide these solutions for both finite horizons and steady state cases. In the second part of the paper, we formulate a reinforcement learning (RL) based algorithm which does not need any model information except the input matrix, and can compute approximate steady-state extended LQR gains using measurements of the states, the control inputs, and the exogenous inputs. Both model-based and data-driven optimal control algorithms are tested with a numerical example under different exogenous inputs showcasing the effectiveness of the designs.

Intelligent optimal controller design applied to quarter car model based on non-asymptotic observer for improved vehicle dynamics

A novel active suspension control strategy is introduced to improve dynamic response of vehicle suspension systems. The proposed algorithm is a fusion of classical controller design methods together with an online observer and is based on the cancelation of system disturbances. The operational calculus method and the differential algebraic theory are applied to build the observer/compensator that is appended to the classical linear quadratic regulator. An ultra-local model based on linear algebraic rules is presented avoiding the use of a precise mathematical model while guaranteeing the stability of the overall system. Simplicity of implementation, low power demand and significant enhancement of active suspension performance are the observed features of the proposed controller. The numerical simulations illustrate the effectiveness and the robustness against sprung mass variation of the proposed control method compared to proportional–integral–derivative controller, intelligent proportional–derivative controller, linear quadratic regulator and active disturbance rejection—linear quadratic regulator.