Optimal State Feedback Control Research Articles

Optimal quadratic full state feedback control of nonlinear affine systems

The solution of the optimal quadratic full-state feedback closed-loop control of a nonlinear affine system problem is presented and solved. This solution uses the State-Dependent Coefficient (SDC) form of a nonlinear system and the Calculus of Variations. Further, it is shown that this solution enables the computation of the value function associated with the respective Hamilton-Jacobi-Bellman equation thus satisfying the necessary and sufficient conditions for optimality. The use of the SDC form representation in the optimal solution for a nonlinear affine system results in an explicit and exact full-state feedback closed-loop control implementation as opposed to the existing open-loop solutions of the optimal control. This solution, like for the linear case, is non-causal and thus cannot be solved in real-time. The closed-loop application involves precomputing a time-varying full-state feedback matrix. The optimal feedback is a linear combination of the current state. The time-varying gain matrix is the backward solution of a non-symmetric matrix Riccati equation. Conditions for the stability of the full-state feedback are presented. This new representation of the optimal solution gives a well-understandable algorithm that enables implementation in the closed loop of the optimal feedback control of a non-linear system which is not possible with the existing optimal open loop representations.

Density estimation based soft actor-critic: deep reinforcement learning for static output feedback control with measurement noise

ABSTRACT The state-of-the-art deep reinforcement learning (DRL) methods, including Deep Deterministic Policy Gradient (DDPG), Twin Delayed DDPG (TD3), Proximal Policy Optimization (PPO), Soft Actor-Critic (SAC), among others, demonstrate significant capability in solving the optimal static state feedback control (SSFC) problem. This problem can be modeled as a fully observed Markov decision process (MDP). However, the optimal static output feedback control (SOFC) problem with measurement noise is a typical partially observable MDP (POMDP), which is difficult to solve, especially for the continuous state-action-observation space with high dimensions. This paper proposes a two-stage framework to address this challenge. In the laboratory stage, both the states and the noisy outputs are observable; the SOFC policy is converted to a constrained stochastic SSFC policy, of which the probability density function is generally not analytical. To this end, a density estimation based SAC algorithm is proposed to explore the optimal SOFC policy by learning the optimal constrained stochastic SSFC. Consequently, in the real-world stage, only the noisy outputs and the learned SOFC policy are required to solve the optimal SOFC problem. Numerical simulations and the corresponding experiments with robotic arms are provided to illustrate the effectiveness of our method. The code is available at https://github.com/RanKyoto/DE-SAC.

Dynamic optimal control of flow front position in injection molding process: A control parameterization-based method

High dynamic and precise control of process variables within permissible limits is the key to high product quality in the injection molding industry. Among these variables, the injection flow rate of molten polymer is particularly significant. In this paper, an effective optimal open-loop and state feedback controller design method is designed to realize the optimal tracking control of the melt polymer flow front position flow of melt polymer fluid during injection molding. To this end, the dynamic model of the filling process is firstly established, taking into account the nonlinearity of hydraulic control system and the effect of molten polymer flow on the dynamics of the injection ram, and then the dynamic optimal tracking control problem of flow front position of molten polymer in injection molding machine is proposed. Then, an open-loop controller and a multi-level feedback control law are well designed, wherein the control signal and the control feedback kernels are parameterized by control parameterization. The controller design problems are transformed into sequential optimal parameter decision problems. The explicit expressions of the gradient information of the objective function and constraints on the parameters of the decision variables are solved by using the state sensitivity equation analysis method, and the gradient information is combined with the nonlinear programming algorithm to solve the optimization problems efficiently. Finally, the viability and effectiveness of the proposed design method are verified by numerical simulations.

Optimized FSF Controller for Bulk Power Control of Large-Core PHWRs

This paper considers an optimized full state feedback (FSF) optimal controller for bulk power control of a 700-MW(electric) pressurized heavy water reactor (PHWR) that minimizes the controller norm to reduce the effect of disturbances. Lyapunov’s linear matrix inequalities (LMIs) have been considered for stability of the model. For the closed loop, these inequalities, which become nonlinear in the unknowns, are converted to LMIs by a suitable variable substitution. The controller’s optimization is achieved by minimizing the upper bound of the state feedback vector’s norm. As a result of this optimization, the controller gain is reduced, which reduces the effect of the disturbance input to the system. We study the stability of the closed loop system and the nonlinear transient performance using the state feedback. We demonstrate that the proposed controller’s transient performance is superior to that of a nonoptimized controller when compared to a conventional proportional-derivative controller. The designed controller has a norm that is about five orders lower than that obtained without optimization while still providing acceptable transient performance.

Singular Optimal Linear Quadratic Regulator of Switched Systems

In this paper, we will investigate the singular linear quadratic regulator (LQR) problem of switched linear system in finite time horizon. The proposed method is transforming into a switched LQR problem by adopting linear transformation. Next, we adopt an embedding transformation method to convert the switched LQR problem to a traditional optimal control problem, so the bang-bang-type solution of the embedded optimal control problem in finite time is the optimal solution to the switched LQR problem. The switching sequence of modes and the switching instants can be calculated by solving a closed-form optimal switching condition. The optimal state feedback control law is determined simultaneously. Then, by solving a sequence of Riccati equation, we find some conditions that ensure switched LQR problem can be convert to the singular LQR problem. Finally, a numerical example is presented to demonstrate the effectiveness of the proposed method.

Finite-Time Disturbance Decoupling of Boolean Control Networks

In this article, the finite-time disturbance decoupling problem (FTDDP) of Boolean networks (BNs) and Boolean control networks (BCNs) is studied. A novel necessary and sufficient condition is first provided to determine whether a given BN is finite-time disturbance decoupled. Then, an algorithm is devised to find the shortest time that guarantees that a given BN is finite-time disturbance decoupled. In addition, using a constructed truth matrix, a necessary and sufficient condition for solving the disturbance decoupling problem of BCNs is proposed, and all possible time optimal state feedback controllers are designed. Finally, two examples are introduced to illustrate the feasibility of the obtained results.

A Robust State Feedback Optimal Control Law with Backstepping Approach for Steering Control of an Autonomous Underwater Vehicle Using Semi-definite Programming

A Robust State Feedback Optimal Control Law with Backstepping Approach for Steering Control of an Autonomous Underwater Vehicle Using Semi-definite Programming

Robust Control Based on Observed States Designed by Means of Linear Matrix Inequalities for Grid-Connected Converters

This paper provides a procedure to design robust controllers based on observed states applied to three-phase inverters with LCL filters connected to a grid with uncertain and possibly time-varying impedances, which can arise in renewable energy systems and microgrid applications. Linear matrix inequalities are used to rapidly compute, off-line, based only on the choice of two scalar parameters for pole location, sets of gains for the controller and the observer, and also to provide a theoretical certificate of the closed-loop stability, including a limit for the rate of variations of the grid impedances. The proposed design procedure allows the easy implementation of robust state feedback controllers with a reduced number of sensors, ensuring good performance for different sets of grid impedances. Additionally, larger regions of guaranteed stability are provided by the proposed procedure, when compared with a similar condition from the literature. The control law using the observed states can ensure grid currents with low harmonic content, complying with the IEEE 1547 Standard requirements, with negligible loss of performance concerning the feedback of the measured state variables. Three optimal state feedback controllers from the literature are reproduced here and successfully implemented using the observed state variables based on the proposed procedure. In all cases, the viability of the proposal was confirmed by simulations and experimental results.

3D Path Following Control of an Autonomous Underwater Robotic Vehicle Using Backstepping Approach Based Robust State Feedback Optimal Control Law

This work renders the design of a robust state feedback optimal control strategy for an Autonomous Underwater Robotic Vehicle (AURV). The control strategy is developed using a polytopic approach based on hydrodynamic parameter variation. Besides, a backstepping approach is designed to control the kinematics of the system. However, the dynamics of the AURV system are controlled by a robust optimal control technique. In this work, the decoupled systems for both horizontal and vertical dynamics of AURV are used for the development of the control algorithms. Furthermore, the 3-D path following is achieved by integrating the control algorithms of both horizontal and vertical dynamics of AURV. The proposed controller is formulated using semi-definite programming (SDP). To track the 3-D path, it is intended to track both the desired depth and desired yaw in diving and steering planes. The simulation studies are conducted through MATLAB/Simulink environment using the YALMIP tool. Furthermore, the robust behavior of the proposed control algorithm is verified by considering the uncertain hydrodynamic parameters.

On the Relationship of Optimal State Feedback and Disturbance Response Controllers

This paper studies the relationship between state feedback policies and disturbance response policies for the standard Linear Quadratic Regulator (LQR). For open-loop stable plants, we establish a simple relationship between the optimal state feedback controller ut = K*xt and the optimal disturbance response controller ut = L(H)*;1wt-1 +...+ L(H)*;1wt-H with H-order. Here xt, wt, ut stands for the state, disturbance, control action of the system, respectively. Our result shows that L(H)*,1 is a good approximation of K* and the approximation error ∥K* — L(H)*;1∥ decays exponentially with H. We further extend this result to LQR for open-loop unstable systems, when a pre-stabilizing controller K0 is available.

Robust Tube-Based TS-MPC for Safe Coordination of Autonomous Vehicle

In this work, a robust vehicle control scheme is proposed, which is capable of coordinating with nearby vehicles in order to optimally compute control actions that achieve collision-free overtaking maneuvers. The control actions are computed online by a global model predictive control (MPC) controller, which assumes a nominal disturbance-free vehicle model. To reduce the computational burden of the MPCs optimization problem, the vehicle model is reformulated into a pseudo-linear Takagi-Sugeno (TS) representation. Furthermore, the mismatch error between the real and the nominal model is corrected by a local TS H∞-optimal state-feedback controller. Moreover, the robust feasibility of the MPCs optimization problem is guaranteed by implementing a tube-based architecture. Finally, the proposed control scheme is tested and validated in a high-fidelity simulation, in which the controlled vehicle was capable of overtaking multiple vehicles while rejecting disturbances.

Stability analysis and design of cooperative control for linear delta operator system

<abstract><p>This paper investigates the cooperative state feedback control problem for delta operator-based large-scale systems with independent subsystems. First, the state feedback controller is introduced to interconnect the adjacent subsystems into a closed-loop system. Second, the Lyapunov function in delta domain is constructed, and the linear matrix inequality method is used to design the cooperative state feedback stability controller for the whole large-scale interconnected system. Third, a performance index is introduced for the design of the optimal cooperative state feedback controller. Finally, stability of the closed-loop system is proved on the basis of stability theory, and simulation examples are given for showing the effectiveness of the design method.</p></abstract>

Design of a backstepping technique-based robust state feedback optimal control law for autonomous underwater vehicle in depth plane using semi-definite programming

Design of a backstepping technique-based robust state feedback optimal control law for autonomous underwater vehicle in depth plane using semi-definite programming

Sampled-data Control of Probabilistic Boolean Control Networks: A Deep Reinforcement Learning Approach

The rise of reinforcement learning (RL) has guided a new paradigm: unraveling the intervention strategies to control systems with unknown dynamics. Model-free RL provides an exhaustive framework to devise therapeutic methods to alter the regulatory dynamics of gene regulatory networks (GRNs). This paper presents an RL-based technique to control GRNs modeled as probabilistic Boolean control networks (PBCNs). In particular, a double deep-Q network (DDQN) approach is proposed to address the sampled-data control (SDC) problem of PBCNs, and optimal state feedback controllers are obtained, rendering the PBCNs stabilized at a given equilibrium point. Our approach is based on options, i.e., the temporal abstractions of control actions in the Markov decision processes (MDPs) framework. First, we define options and hierarchical options and give their properties. Then, we introduce multi-time models to compute the optimal policies leveraging the options framework. Furthermore, we present a DDQN algorithm: i) to concurrently design the feedback controller and the sampling period; ii) wherein the controller intelligently decides the sampled period to update the control actions under the SDC scheme. The presented method is model-free and offers scalability, thereby providing an efficient way to control large-scale PBCNs. Finally, we compare our control policy with state-of-the-art control techniques and validate the presented results.

Model‐free distributed optimal control for continuous‐time linear systems

This paper studies the distributed optimal consensus problem for a group of continuous-time linear systems in the scenario that the dynamics of linear systems are completely unknown. Both an optimal state feedback control law and an optimal output feedback control law are constructed by solving the Algebraic Riccati Equation (ARE) that contains no global information, which guarantees the consensus of linear systems and the global optimality. Adaptive Dynamic Programming (ADP) method is presented to solve the ARE, thus the control algorithms are not relying on the dynamics of linear systems any more. Finally, numerical simulation results are presented to demonstrate the efficiency of proposed approach.

A multi‐strategy fuzzy control method based on the Takagi‐Sugeno model

AbstractIn this work, a new multi‐strategy fuzzy control (MSFC) based on fuzzy fusion techniques for the control of multivariable nonlinear systems is proposed. This new nonlinear control method is a mixture of different control techniques by fuzzy interpolation using the fuzzy Takagi‐Sugeno model. It combines various control strategies to achieve smooth transient responses and zero steady state error. The following techniques are merged within the proposed MSFC structure: an optimal state feedback control that yields an optimal transient response, optimal control through an incremental state model that returns a zero steady state error, and a constant input control to maintain the system behavior within predefined boundaries whenever the MSFC method is applied.

Diversified-Intensified Current Search Algorithm and Its Application to Optimal State-Feedback Controller Design for Tractor Active Suspension System

Wydawnictwo SIGMA-NOT wydaje czasopisma fachowe informujące swoich czytelników o najnowszych osiągnięciach naukowych i nowoczesnych rozwiązaniach technicznych w Polsce i na świecie, popularyzuje problemy techniczne oraz poszerza wiedzę i kulturę techniczną.

Indefinite linear quadratic optimal control for discrete time‐varying linear rectangular descriptor systems

AbstractIn this article, the indefinite linear quadratic (LQ) optimal control problem for discrete time‐varying rectangular descriptor systems is investigated, in which the indefinite weight matrices are considered in the quadratic cost function. With a suite of equivalent transformations, the indefinite LQ problem for discrete time‐varying rectangular descriptor systems can be substituted by an equivalent indefinite LQ problem for standard state space systems. Then, the transformed equivalent indefinite LQ problem is solved by establishing a dual Krein space model, and the optimal control and optimal cost value are obtained by invoking the Kalman filter of Krein space. The form of state feedback optimal control corresponding to the original system is given. Next, to guarantee the dynamic part for the resulting optimal closed‐loop system has a unique solution, some necessary and sufficient conditions are proposed. Finally, a numerical example is carried out to demonstrate the availability of established results.

Constrained Decoupling Adaptive Dynamic Programming for A Partially Uncontrollable Time-Delayed Model of Energy Systems

The lack of feedback-loop design and uncontrollable state reconstruction, such as load demand and renewable generation, are key factors to hinder the development of energy control. Adaptive dynamic programming is an efficient method to break through the bottleneck. However, most current works assume that the evolution of uncontrollable state satisfies Markovian property. This paper proposes a time-delayed adaptive dynamic programming framework for optimal energy control, utilizing more comprehensive historical data for state reconstruction. The uncontrollable state is reconstructed by an attention mechanism-based Transformer network based on online measured data. Then, a self-learning iterative algorithm is developed to obtain the optimal state feedback control policy under a discounted performance index function. The problem of solving iterative control policy, which is a mixed-integer nonlinear programming problem with an enormous computational burden, is decomposed into three sub-problems, i.e. nonlinear programming with no discrete variable, which is solved by the Projected Newton method, mixed-integer quadratic programming, and quadratic programming. Numerical results illustrate that the developed control algorithm minimizes the electricity cost in the long term and avoids peak load.

Optimal adaptive state feedback control for a three degree-of-freedom series-type double inverted pendulum system by using particle swarm optimisation

ABSTRACT This research tries to stabilise a three degree-of-freedom series-type double inverted pendulum system by employing an adaptive state feedback control approach optimised by a particle swarm optimisation (PSO) algorithm. To this end, the dynamical equations are derived via Lagrangian method and linearised by utilising approximation approaches. Then, the state feedback scheme is implemented to control the system states, that is, the angle and angular velocity of the first and second poles as well as the position and velocity of the cart. In order to timely regulate the control gains, the adaptation rules based on the gradient descent method and sliding surface relations are applied. The particle swarm optimisation algorithm is implemented to optimally tune the design parameters of the controller based on a proper objective function defined as summation of integrals of absolute values of the system errors. The time responses of the angles of the poles and the position of the cart clearly depict the feasibility and efficiency of the introduced control strategy to stabilise the system from different initial conditions to the final desired values.