Linear Quadratic Regulator Problem Research Articles

Linear Quadratic Regulator: II. Robust Formulations

The classical linear quadratic regulation problem is considered in the robust formulations where the matrices of the system and/or initial conditions are not know precisely. Several approaches are proposed where the quadratic cost is minimized against the worst-case uncertainties. Finding such controllers is performed via reducing the matrix Riccati equation with uncertainty to a single linear matrix inequality. The properties of the solutions are discussed and the comparison with previously known approaches is performed.

Robust Optimization in Possibility Theory

Abstract In this contribution, the optimization of systems under uncertainty is considered. The possibilistic evaluation of the fuzzy-valued constraints and the adoption of a multicriteria decision making technique for the fuzzy-valued objective function enable a meaningful solution to general fuzzy-valued optimization problems. The presented approach is universally applicable, which is demonstrated by reformulating and solving the linear quadratic regulator problem for fuzzy-valued system matrices and initial conditions.

Primal-Dual Q-Learning Framework for LQR Design

Recently, reinforcement learning (RL) is receiving more and more attentions due to its successful demonstrations outperforming human performance in certain challenging tasks. The goal of this paper is to study a new optimization formulation of the linear quadratic regulator (LQR) problem via the Lagrangian duality theories in order to lay theoretical foundations of potentially effective RL algorithms. The new optimization problem includes the Q-function parameters so that it can be directly used to develop Q-learning algorithms, known to be one of the most popular RL algorithms. We prove relations between saddle-points of the Lagrangian function and the optimal solutions of the Bellman equation. As an example of its applications, we propose a model-free primal-dual Q-learning algorithm to solve the LQR problem and demonstrate its validity through examples.

Linear quadratic regulator problem governed by granular neutrosophic fractional differential equations

Quadratic cost functions estimation in the linear optimal control systems governed by differential equations (DEs) or partial differential equations (PDEs) has a well-established discipline in mathematics with many interfaces to science and engineering. During its development, the impact of uncertain phenomena to objective function and the complexity of the systems to be controlled have also increased significantly. Many engineering problems like magnetohydromechanical, electromagnetical and signal analysis for the transmission and propagation of electrical signals under uncertain environment can be dealt with. In this paper, we study the optimal control problem with operating a fractional DEs and PDEs at minimum quadratic objective function in the framework of neutrosophic environment and granular computing. However, there has been no studies appeared on the neutrosophic calculus of fractional order. Hence, we will introduce some derivatives of fractional order, including the neutrosophic Riemann–Liouville fractional derivatives and neutrosophic Caputo fractional derivatives. Next, we propose a new setting of two important problems in engineering. In the first problem, we investigate the numerical and exact solutions of some neutrosophic fractional DEs and neutrosophic telegraph PDEs. In the second problem, we study the optimality conditions together with the simulation of states of a linear quadratic optimal control problem governed by neutrosophic fractional DEs and PDEs. Some key applications to DC motor model and one-link robot manipulator model are investigated to prove the effectiveness and correctness of the proposed method.

Development of bi-directional DC/DC converter for fuel cell hybrid vehicle

Dual active full-bridge (DAB) DC/DC converters are widely used in energy storage systems with bidirectional energy flow, such as fuel cell hybrid vehicles. This paper presents a robustness design method for DAB converter power system stabilizers. Based on the single-phase-shift control model of the DAB converter, the small signal model and the multicell structure of the converter are established and the state equation of the system is obtained. Then, the controller design problem is transformed into a linear quadratic regulator problem, which is solved by linear matrix inequality. The controller designed by this method can make the DC/DC converter run stably even when the parameters are uncertain, which is verified by simulation and experiments.

Dynamic Multiobjective Control for Continuous-Time Systems Using Reinforcement Learning

This paper presents an extension of the reinforcement learning algorithms to design suboptimal control sequences for multiple performance functions in continuous-time systems. The first part of the paper provides the theoretical development and studies the required conditions to obtain a state-feedback control policy that achieves Pareto optimal results for the multiobjective performance vector. Then, a policy iteration algorithm is proposed that takes into account practical considerations to allow its implementation in real-time applications for systems with partially unknown models. Finally, the multiobjective linear quadratic regulator problem is solved using the proposed control scheme and employing a multiobjective optimization software to solve the static optimization problem at each iteration.

On the structure of the set of active sets in constrained linear quadratic regulation

The constrained linear quadratic regulation problem is solved by a continuous piecewise affine function on a set of state space polytopes. It is an obvious question whether this solution can be built up iteratively by increasing the horizon, i.e., by extending the classical backward dynamic programming solution for the unconstrained case to the constrained case. Unfortunately, however, the piecewise affine solution for horizon N is in general not contained in the piecewise affine law for horizon N+1. We show that backward dynamic programming does, in contrast, result in a useful structure for the set of the active sets that defines the solution. Essentially, every active set for the problem with horizon N+1 results from extending an active set for horizon N, if the constraints are ordered stage by stage. Consequently, the set for horizon N+1 can be found by only considering the constraints of the additional stage. Furthermore, it is easy to detect which polytopes and affine pieces are invariant to increasing the horizon, and therefore persist in the limit N→∞. Several other aspects of the structure of the set of active sets become evident if the active sets are represented by bit tuples. There exists, for example, a subset of special active sets that generates a positive invariant and persistent (i.e., horizon-invariant) set around the origin. It is very simple to identify these special active sets, and the positive invariant and persistent region can be found without solving optimal control or auxiliary optimization problems. The paper briefly discusses the use of these results in model predictive control. Some opportunities for uses in computational methods are also briefly summarized.

Linear State-Feedback Primary Control for Enhanced Dynamic Response of AC Microgrids

This paper proposes a state feedback primary control strategy for microgrids with multiple distributed energy resource units, improving their transient behavior in both islanded and grid-connected modes of operation. To that end, the interaction of each distributed energy resource unit within the microgrid is modeled as a lumped dynamic system, which results to be nonlinear and multivariable. For the closed-loop control of such multivariable systems, the full state feedback formulation is preferred which requires a suitable state observer. For the design of the observer and feedback gains, the solution of the linear-quadratic estimation and regulation problems is considered. For simplicity, an approximate linear model at a representative operating point is derived. The linear quadratic Gaussian/loop-transfer recovery is adopted as the design procedure to optimize the trajectory of the state variables subject to a desirable actuation effort, in this case of the voltage amplitude and frequency, yielding a solution that is robust to model uncertainties. The effectiveness of the strategy is assessed through time-domain simulation on the CIGRE benchmark medium-voltage distribution network with three distributed energy resource units. These results are compared to those obtained with conventional static droop gains and with a state-of-the-art technique from the literature.

Almost Every Single-Input LQR Optimal Control Problem Admits a PD Feedback Solution

In this letter, we prove that almost every infinite-horizon linear quadratic regulator (LQR) control problem with single-input admits an optimal solution in the form of a feedback that is a suitable constant linear combination of the state and its first derivative—a proportional plus derivative state feedback. The only assumption that we make is that the associated Hamiltonian matrix pencil has no eigenvalues on the imaginary axis. This condition is known to be generically true. Since a regular infinite-horizon LQR problem is known to admit an optimal control in the form of a static state feedback (a proportional feedback), we concentrate in this letter only on singular LQR problems.

Dynamics and control of a flexible rotating clamped-free beam by SDRE strategy

PurposeThe purpose of this paper is the dynamic analysis of the coupled rotation and vibration motion of a system containing a central rigid body to which is attached a flexible beam.Design/methodology/approachThe methodology includes the Lagrange’s formulation by using the extended Hamilton’s Principle in conjunction with the assumed modes method to describe the system of equations by ordinary differential equations. The first unconstrained mode of vibration was considered as the solution for the transversal displacement. Such mode emerges as the eigenvalue problem solution associated to the dynamics of the system. The control strategy adopted is a nonlinear analogy of the linear quadratic regulator problem as the Riccati equation is solved at every integration step during the numerical solutions. This strategy is known as state-dependent Riccati equation.FindingsBy means of computational simulations, it was found the relation between controlled motion and inertia ratio.Research limitations/implicationsThis work is limited to planar case and fixed hub.Practical implicationsPractical implications of this work realize the design of lighter yet dexterous structures.Originality/valueThe contribution of this paper is the position and vibration control of a flexible beam accounting for nonlinearity effects and the fact that the structure to where it is clamped has a comparable inertia.

Disturbance-observer-based LQR control of singularly perturbed systems via recursive decoupling methods

ABSTRACTThis paper addresses the disturbance-observer-based linear quadratic regulation (LQR) problem of singularly perturbed systems subject to harmonic disturbances. To reduce the computational burden and facilitate the on-board implementation, a recursive matrix decoupling method is proposed to decompose the stiff singularly perturbed model into the block-diagonal form. Then, a robust LQR controller is designed based on the equivalent decoupled system, which consists of a state feedback controller for performance optimisation and a robust controller for disturbance compensation. Moreover, the designed controller is integrated with a finite frequency disturbance observer to attenuate the effect of harmonic disturbances. Finally, a simulation example on a DC motor system is provided to verify the effectiveness of the proposed design method.

Reinforcement Learning-Based Linear Quadratic Regulation of Continuous-Time Systems Using Dynamic Output Feedback.

In this paper, we propose a model-free solution to the linear quadratic regulation (LQR) problem of continuous-time systems based on reinforcement learning using dynamic output feedback. The design objective is to learn the optimal control parameters by using only the measurable input-output data, without requiring model information. A state parametrization scheme is presented which reconstructs the system state based on the filtered input and output signals. Based on this parametrization, two new output feedback adaptive dynamic programming Bellman equations are derived for the LQR problem based on policy iteration and value iteration (VI). Unlike the existing output feedback methods for continuous-time systems, the need to apply discrete approximation is obviated. In contrast with the static output feedback controllers, the proposed method can also handle systems that are state feedback stabilizable but not static output feedback stabilizable. An advantage of this scheme is that it stands immune to the exploration bias issue. Moreover, it does not require a discounted cost function and, thus, ensures the closed-loop stability and the optimality of the solution. Compared with earlier output feedback results, the proposed VI method does not require an initially stabilizing policy. We show that the estimates of the control parameters converge to those obtained by solving the LQR algebraic Riccati equation. A comprehensive simulation study is carried out to verify the proposed algorithms.

LQR Robust Control for Active and Reactive Power Tracking of a DFIG based WECS

This research work sets forward a new formulation of Linear Quadratic Regulator problem (LQR) applied to a Wind Energy Conversion System (WECS). A new necessary and sufficient condition of Lyapunov asymptotic stability is also established. The problem is mathematically described in form of Linear Matrix Inequalities (LMIs). The considered WECS is based on a Doubly Fed Induction Generator (DFIG). An appropriate Linear Parameter Varying (LPV) model is designed. This model stands for a realistic representation of the randomly time varying wind velocity. Stability and robustness of the controller over the admissible values of time varying parameter are investigated. The newly lifted Lyapunov condition gives less conservative conditions for LMI approach in case of parameter-dependent Lyapunov functions PDLF. The considered PDLF has the same variation dynamics as the system matrix. The intrinsic objective for our research is to offer more freedom degrees to the control problem and to improve the efficiency of the controller in case of uncertainties or parametric variations. The performances of the proposed theorems are validated to achieve active and reactive powers tracking of the WECS over the admissible range of wind speeds. The interesting features of the proposed solution are the simpler implementation and the larger robustness margin. It also has the advantage of providing a linear control to the considered nonlinear system without resorting to linearization. The LMIs implementation is performed on Yalmip Matlab toolbox. The proposed controller is verified on a Matlab Simulink emulator. This work presents an extension of the LQR control problem to LPV systems.

Constrained Generalized Continuous Algebraic Riccati Equations (CGCAREs) Are Generically Unsolvable

In this letter, we show that a constrained generalized continuous algebraic Riccati equation (CGCARE), arising in the context of a singular infinite-horizon linear quadratic regulator (LQR) problem, is generically 1 unsolvable. We achieve this result in three steps: first, we provide a set of necessary and sufficient conditions for the solvability of a CGCARE. Then, using these conditions, we show that the determinant of a certain matrix pencil being identically zero is necessary for a CGCARE to be solvable. Finally, we show that this necessary condition is generically false. It is well-known that CGCARE solvability is a necessary and sufficient condition for a singular infinite-horizon LQR problem to admit a solution that is implementable as a static state-feedback control law. Thus, our main result reveals that a singular infinite-horizon LQR problem generically disallows solution by a static state-feedback law.

Dynamic Collective Choice: Social Optima

We consider a dynamic collective choice problem where a large number of agents are cooperatively choosing between multiple destinations while being influenced by the behavior of the group. For example, in a robotic swarm exploring a new environment, a robot might have to choose between multiple sites to visit, but at the same time it should remain close to some group to achieve coordinated tasks. Finding a social optimum for our problem reduces to solving a set of linear quadratic regulator problems, whose number, however, increases exponentially with the size of the population. Alternatively, we develop via the mean field games methodology a set of decentralized strategies characterized via the fixed points of a suitable operator. In the homogeneous parameter agents case, the procedure reduces to solving a vector fixed point equation of size $l$ equal to the number of destinations, followed by each agent comparing only $l$ regulator costs, independently of the number of agents. When the latter is sufficiently large, the strategies qualify as approximately socially optimal. To compute the approximate social optimum, each agent only needs to know its own state and the statistical distributions of the agents’ initial states and problem parameters. A numerical example illustrates the benefits of cooperative strategies compared to noncooperative ones in achieving an adequate splitting of agents among multiple destinations that each require collective attention.

Solving Stochastic LQR Problems by Polynomial Chaos

We consider the infinite dimensional stochastic linear quadratic optimal control problem for the infinite horizon case. We provide a numerical framework for solving this problem using a polynomial chaos expansion approach. By applying the method of chaos expansions to the state equation, we obtain a system of deterministic partial differential equations in terms of the coefficients of the state and the control variables. We set up a control problem for each equation, which results in a set of infinite horizon deterministic linear quadratic regulator problems. We prove the optimality of the solution expressed in terms of the expansion of these coefficients compared to the direct approach. We perform numerical experiments which validate our approach and compare the finite and infinite horizon case.

A novel Mittag–Leffler stable estimator for nonlinear fractional-order systems: a linear quadratic regulator approach

In this article, the estimation problem in a class of nonlinear fractional-order systems is solved by a stable estimator that generalizes the classical exponential observers, establishing a new class called Mittag–Leffler observers. The solution to the linear quadratic regulator problem is proposed to design optimal control laws for fractional-order linear systems and its applications. All the results are based on the Caputo derivative for commensurate fractional-order systems. Numerical simulations validate the proposed theory.

Optimal and Systematic Design of Current Controller for Grid-Connected Inverters

A design approach for grid-connected inverter controllers for distributed and renewable energy system applications is proposed, where a high number of controller gains can be designed optimally in a fairly systematic way. The linear quadratic regulator problem is first modified to accommodate sinusoidal signal tracking through a meaningful state-space transformation. The cost function is modified to explicitly include the tracking error, so that its weights are designed in a transparent and systematic way. The proposed technique is applied to the well-known control structure comprising the fundamental and harmonic resonant controllers. Second, the control structure is rearranged to reject the distortions from both the grid voltage and the reference signal, and the proposed design technique is applied to this new structure. It is shown that the desired features of active damping for $LCL$ filters and robust performance against system uncertainties, harmonics, and disturbances are achieved. Third, the controller and the systematic design procedure are extended for inverters with $LLCL$ filters. Proposed control structures are designed and simulated and then implemented on a digital signal processor. Results confirm features of the method and its ability to address control system challenges in high power density and efficient inverter applications.

Uncertain DC-DC Zeta Converter Control in Convex Polytope Model Based on LMI Approach

<span>A dc-dc zeta converter is a switch mode dc-dc converter that can either step-up or step-down dc input voltage. In order to regulate the dc output voltage, a control subsystem needs to be deployed for the dc-dc zeta converter. This paper presents the dc-dc zeta converter control. Unlike conventional dc-dc zeta converter control which produces a controller based on the nominal value model, we propose a convex polytope model of the dc-dc zeta converter which takes into account parameter uncertainty. A linear matrix inequality (LMI) is formulated based on the linear quadratic regulator (LQR) problem to find the state-feedback controller for the convex polytope model. Simulation results are presented to compare the control performance between the conventional LQR and the proposed LMI based controller on the dc-dc zeta converter. Furthermore, the reduction technique of the convex polytope is proposed and its effect is investigated.</span>

One Layer Nonlinear Economic Closed-Loop Generalized Predictive Control for a Wastewater Treatment Plant

The main scope of this paper is the proposal of a new single layer Nonlinear Economic Closed-Loop Generalized Predictive Control (NECLGPC) as an efficient advanced control technique for improving economics in the operation of nonlinear plants. Instead of the classic dual-mode MPC (model predictive controller) schemes, where the terminal control law defined in the terminal region is obtained offline solving a linear quadratic regulator problem, here the terminal control law in the NECLGPC is determined online by an unconstrained Nonlinear Generalized Predictive Control (NGPC). In order to make the optimization problem more tractable two considerations have been made in the present work. Firstly, the prediction model consisting of a nonlinear phenomenological model of the plant is expressed with linear structure and state dependent matrices. Secondly, instead of including the nonlinear economic cost in the objective function, an approximation of the reduced gradient of the economic function is used. These assumptions allow us to design an economic unconstrained nonlinear GPC analytically and to state the NECLGPC allow for the design of an economic problem as a QP (Quadratic Programing) problem each sampling time. Four controllers based on GPC that differ in designs and structures are compared with the proposed control technique in terms of process performance and energy costs. Particularly, the methodology is implemented in the N-Removal process of a Wastewater Treatment Plant (WWTP) and the results prove the efficiency of the method and that it can be used profitably in practical cases.