Static State Feedback Controller Research Articles

Density estimation based soft actor-critic: deep reinforcement learning for static output feedback control with measurement noise

ABSTRACT The state-of-the-art deep reinforcement learning (DRL) methods, including Deep Deterministic Policy Gradient (DDPG), Twin Delayed DDPG (TD3), Proximal Policy Optimization (PPO), Soft Actor-Critic (SAC), among others, demonstrate significant capability in solving the optimal static state feedback control (SSFC) problem. This problem can be modeled as a fully observed Markov decision process (MDP). However, the optimal static output feedback control (SOFC) problem with measurement noise is a typical partially observable MDP (POMDP), which is difficult to solve, especially for the continuous state-action-observation space with high dimensions. This paper proposes a two-stage framework to address this challenge. In the laboratory stage, both the states and the noisy outputs are observable; the SOFC policy is converted to a constrained stochastic SSFC policy, of which the probability density function is generally not analytical. To this end, a density estimation based SAC algorithm is proposed to explore the optimal SOFC policy by learning the optimal constrained stochastic SSFC. Consequently, in the real-world stage, only the noisy outputs and the learned SOFC policy are required to solve the optimal SOFC problem. Numerical simulations and the corresponding experiments with robotic arms are provided to illustrate the effectiveness of our method. The code is available at https://github.com/RanKyoto/DE-SAC.

Adaptive robust bearing-based formation control for multi-agent systems

The problem of bearing-based formation control with disturbance rejection has been studied by quite a few papers under various assumptions on the disturbances such as constant disturbances, trigonometric polynomial disturbances or bounded uniformly continuous disturbances with known bounds. In this paper, we further study the same problem with the disturbances being bounded piecewise continuous time functions. We first consider the case where the bounds of the disturbances are known. We show that the problem can be solved by a distributed static state feedback control law. Then we further consider the case where the bounds of the disturbances are unknown. We manage to solve this problem by introducing the adaptive control technique to deal with the unknown bounds of the disturbances. Our results include all the existing results on the bearing-based formation control with disturbance rejection as special cases. Moreover, we present a sufficient condition on the initial values of the closed-loop system for each case that prevents the collision.

Parametric uncertain LTI systems: an LMI optimisation formulation for robust feedback control design

This article considers the following problems for a parametric uncertain (affine) linear time-invariant system. The first problem is: Let a feedback gain matrix be designed such that the states of the closed loop system satisfy the specified transient performance bounds. Then, compute a ball in the uncertain parameter space such that for all parameter perturbations within it, the state response continues to satisfy the same transient performance bounds. The second problem is on the synthesis of a robust static state feedback control, which i) minimises the norm of the gain matrix, ii) maximises the radius of the ball in the uncertain parameter space, and iii) ensures achieving specified transient performance in the closed loop. For this, a sub-optimal linear matrix inequality optimisation is formulated. The developed results are demonstrated with numerical examples. It also highlights how the proposed methodology can be applied to design robust static state feedback control for a polytopic uncertain system.

Data-Driven $\mathcal {L}_{2}$-Stability Analysis for Dynamic Event-Triggered Networked Control Systems: A Hybrid System Approach

This paper addresses the problem of data-driven control and stability analysis for networked control systems (NCSs). The systems considered in this paper are assumed to be linear with unknown system matrices and bounded external disturbance. Firstly, a static state-feedback controller is derived directly from noisy data in the absence of network. Secondly, the NCSs including a dynamic event-triggering mechanism (DETM) and time-varying transmission delays are modeled as a hybrid system. Then the <inline-formula><tex-math notation="LaTeX">$\mathcal {L}_{2}$</tex-math></inline-formula>-stability conditions for the hybrid system are established by using the Lyapunov stability approach. Thirdly, by combining the model-based conditions and the parametric representation of the unknown system matrices, data-based stability conditions are derived. Based on the methods proposed in this paper, many well-established model-based theorems can be readily applied to the unknown linear systems. Finally, the proposed theories are verified on a turbofan engine hardware-in-the-loop (HIL) simulation platform.

Leader‐following consensus of a class of heterogeneous uncertain multi‐agent systems with a distributed model reference adaptive control law

SummaryThis paper considers the leader‐following consensus problem of a multi‐agent system whose agents have heterogeneous uncertain dynamics in homogeneous state dimensions. Each agent utilizes a distributed model reference adaptive control (MRAC) law to deal with uncertainties. The nominal part of the MRAC is a distributed static state feedback control law. The reference model is the leader‐following consensus problem of reference agents with homogeneous linear time‐invariant dynamics. This problem becomes an N‐player graphical differential game under a given cost function. The reference agents utilize distributed static state feedback controllers that constitute a Nash equilibrium solution to the graphical differential game. This paper provides the conditions for the distributed MRAC to guarantee that each agent asymptotically tracks the corresponding reference agent; consequently, the multi‐agent system solves the leader‐following consensus problem as the reference model does. These conditions yield a straightforward design method for the MRAC. A numerical example demonstrates an application of the proposed approach to a multi‐robot system with nonlinear dynamics.

Policy Optimization for Markovian Jump Linear Quadratic Control: Gradient Method and Global Convergence

Recently, policy optimization has received renewed attention from the control community due to various applications in reinforcement learning tasks. In this article, we investigate the global convergence of the gradient method for quadratic optimal control of discrete-time Markovian jump linear systems (MJLS). First, we study the optimization landscape of direct policy optimization for MJLS, with static-state feedback controllers and quadratic performance costs. Despite the nonconvexity of the resultant problem, we are still able to identify several useful properties such as coercivity, gradient dominance, and smoothness. Based on these properties, we prove that the gradient method converges to the optimal-state feedback controller for MJLS at a linear rate if initialized at a controller, which is mean-square stabilizing. This article brings new insights for understanding the performance of the policy gradient method on the Markovian jump linear quadratic control problem.

Data-driven [formula omitted] control of constrained systems: An application to bilateral teleoperation system

A novel, identification-free, data-driven (DD) H∞ control method is presented for discrete-time (DT) linear time-invariant (LTI) systems under physical limitations and norm-bounded disturbances. The presented approach does not demand information on system matrices or any measurements of disturbance affecting the system. The only information needed to develop a static state-feedback (SF) controller is the bounds on disturbances, states and control signals. It is assumed that only the disturbance input matrix and the performance matrices the user generally defines are known, and all others are entirely unknown. The proposed method relies on the closed-loop (CL) parametrization of the LTI system with control input and state measurements. The disturbances affecting the system states are handled as affine uncertainties, later represented as Linear Fractional Transformation (LFT). For obtaining a less conservative controller, a full block S-procedure method (FBSPM) is used, which takes advantage of relaxations such as convex hull relaxation or Pólya relaxation for the inner approximation of the disturbance set with arbitrary precision. Numerical illustrations and extensive case studies on a bilateral teleoperation system indicate that the proposed design method allows us to obtain very effective controllers which never exceed the bounds of the state and input variables and are capable of reference and force tracking.

Sensitivity-Based Iterative State-Feedback Tuning for MIMO Systems

In this paper, a new approach to tuning and optimisation of controlled systems regarding the tracking behaviour is presented. This approach can be understood as an extension to the iterative feedback tuning (IFT) approach known from the literature. Motivated by the sensitivity concept, the IFT algorithm is extended to both linear and nonlinear systems in state-space description with static state-feedback control, resulting in the proposed iterative state-feedback tuning (ISFT) method. The main contribution consists of the derivation of first and second-order sensitivity functions for quadratic cost functions, which is typical for control optimisation problems. The proposed approach results in a gradient-based iterative algorithm for control parameter adaptation. Finally, exemplary simulation results will be presented for a nonlinear system, demonstrating the usability of this new approach.

A Data-Driven Approach to the L₂ Stabilization of Linear Systems Subject to Input Saturations

This paper revisits the L2 stabilization problem for linear systems subject to saturating input and energy-bounded additive disturbance from a data-driven point of view. The backbone of the results resides in writing the stability conditions in a compact dedicated structure allowing to exhibit LMI conditions. Based on the assumption that the data collection is informative, the finite L2-gain stabilization of the closed-loop system is addressed by considering a static state-feedback control law. Then, the L2-gain of the closed loop and an inner-approximation of the basin of attraction of the origin for the disturbance-free closed-loop system are characterized. The potential of the method is discussed along the treatment of an academic example.

LMI-based Decentralized Load Frequency Control of a Hybrid Power System with a Virtual Synchronous Generator and Battery Storage

This paper proposes the participation of wind generation in the decentralized control of load frequency of a hybrid power system consisting of, in addition to wind generation, conventional generation and battery storage. The wind generation is modelled as a ‘Virtual Synchronous Generator (VSG)’ in a separate control area with its own virtual frequency. It has also been proposed to operate the wind generation system in a ‘de-loaded’ mode, thereby allowing it to take part in frequency regulation services. For the purposes of a decentralized control design, the overall system model is decomposed into three subsystems. Static state-feedback control gains are computed by posing the decentralized control problem as a set of linear matrix inequalities (LMIs) subject to structural and stabilizing constraints.

Orbital stabilization of point-to-point maneuvers in underactuated mechanical systems

The task of inducing, via continuous static state-feedback control, an asymptotically stable heteroclinic orbit in a nonlinear control system is considered in this paper. The main motivation comes from the problem of ensuring convergence to a so-called point-to-point maneuver in an underactuated mechanical system. Namely, to a smooth curve in its state–control space which is consistent with the system dynamics and connects two (linearly) stabilizable equilibrium points. The proposed method uses a particular parameterization, together with a state projection onto the maneuver as to combine two linearization techniques for this purpose: the Jacobian linearization at the equilibria on the boundaries and a transverse linearization along the orbit. This allows for the computation of stabilizing control gains offline by solving a semidefinite programming problem. The resulting nonlinear controller, which simultaneously asymptotically stabilizes both the orbit and the final equilibrium, is time-invariant, locally Lipschitz continuous, requires no switching, and has a familiar feedforward plus feedback–like structure. The method is also complemented by synchronization function–based arguments for planning such maneuvers for mechanical systems with one degree of underactuation. Numerical simulations of the non-prehensile manipulation task of a ball rolling between two points upon the “butterfly” robot demonstrates the efficacy of the synthesis.

Feedback control for structured descriptor systems with minimum free-entry pattern gain vectors

In this article, a structured descriptor system (SDS) is considered, where the entries of the system matrices are either fixed zeros or free parameters. When the free-entries are assigned with some real numbers, the resulting system becomes a descriptor system in the usual sense. The objective of this work is to devise a methodology to verify whether there exists a static state feedback control for a SDS such that almost all the DSs (in a generic sense), characterized by the given SDS, can be made impulse free and place their finite poles at any desired locations in the complex plane. In addition, if such a feedback control exists, then compute a structured gain vector, which has minimum free-entries. To address the underlying problems, graph theoretic approach is used, where it is shown that the computation of spanning cycle families of a directed graph, corresponding to a suitably defined structured matrix, plays an important role. The efficacy of the developed results are demonstrated with a numerical example.

Global asymptotic stability of input-saturated one degree-of-freedom Euler–Lagrange systems with Rayleigh dissipation under nonlinear control

ABSTRACT In this manuscript, the regulation of one degree-of-freedom Euler–Lagrange systems subject to input saturation is addressed. In particular, the design and analysis of a nonlinear static state feedback controller is presented. As a result, it is proven via Lyapunov's direct method that, in the presence of Rayleigh dissipation, the closed-loop equilibrium point is globally asymptotically stable with a strict Lyapunov function. Since saturation occurs in the system which contains the actuator model, the proposed control law is unconstrained and can be simplified to a proportional-derivative with desired gravity compensation algorithm. As a by-product global asymptotic stability is also proven for the case where Rayleigh dissipation is null. Numerical simulations on a crank-slider mechanism are presented. Moreover, experimental results on a DC-DC buck power converter are also shown and confirm the viability of our approach.

Reducing conservatism in robust stability analysis and stabilisation of arbitrary switched linear systems

Parametric modelling of uncertainties has been considered for switched linear systems in the past few years to improve the efficiency of robust stability analysis. The main goal of this paper is simultaneously alleviating conservatism and computational effort for robust stability analysis and stabilisation of arbitrary switched linear systems with uncertain parameters. For this purpose, stability conditions are revised and some constraints are relaxed. First, an improved weighted stability bound is proposed which is larger than previous results. Since the obtained stability bound depends on the chosen Lyapunov function, an optimisation problem is also given to determine its maximum value. Then, using the stability results, robust stabilisation of parametric uncertain switched linear systems under arbitrary switching is investigated. In this regard, a design method is presented to offer a static state feedback control law. The conservatism of the proposed control law is reduced by relaxing the constraint on upper bound of a matrix variable. Finally, illustrative examples are included to demonstrate the effectiveness of the proposed method.

A scheduling algorithm for networked control systems

This paper deals with the design of scheduling logics for Networked Control Systems (NCSs) whose communication networks have limited capacity. We assume that only a subset of the plants can communicate with their controllers at any time instant. Our contributions are twofold. First, we present a probabilistic algorithm to design scheduling logics that, under certain conditions on the plant and controller dynamics and the capacity of the network, ensure stochastic stability of each plant in an NCS. Second, given the plant dynamics and the capacity of the network, we design static state-feedback controllers such that the conditions for stability under our scheduling logics are satisfied. The main apparatus for our analysis is a Markovian jump linear system representation of the individual plants in an NCS. Our stability conditions involve sets of matrix inequalities. We present numerical experiments to demonstrate our results.

Controlling the variable length pendulum: Analysis and Lyapunov based design methods

The analysis and design of methods to damp the swing of the variable length pendulum by adjusting its length are presented here. To analyze the theoretical limits of such Coriolis force based damping, a comprehensive open-loop numerical analysis is performed for a two-dimensional model having the string length as the controlled input. Further, for a four dimensional model, having the force applied to the string as the controlled input, a smooth static state feedback controller is designed using backstepping. Results are verified both in simulations and through extensive laboratory experiments, and compared with previously published results achievable using an identical experimental setting.

On Periodic Scheduling and Control for Networked Systems Under Random Data Loss

This article deals with networked control systems (NCSs) whose shared networks have limited communication capacity and are prone to data losses. Our contributions are two-fold. First, we present necessary and sufficient conditions on the plant and controller dynamics and the network parameters such that there exist purely time-dependent periodic scheduling sequences under which the stability of each plant is preserved for all admissible data loss signals. Second, given a period for the scheduling sequence, we identify sufficient conditions on the plant dynamics and the network parameters such that the plants admit static state-feedback controllers that are favorable for the stability conditions presented. The main apparatus for our analysis is a switched system representation of the individual plants in an NCS whose switching signals are time-inhomogeneous Markov chains. Our stability conditions involve the existence of sets of symmetric and positive definite matrices that satisfy certain (in)equalities. We identify the existence of stabilizing periodic scheduling sequences and their corresponding favorable state-feedback controllers under ideal communication between the plants and their controllers as special cases of our results. A numerical experiment is presented to demonstrate the proposed techniques.

Guaranteed cost controller for discrete time-delayed systems with actuator saturation

The problem of guaranteed cost (GC) control using static-state feedback controllers for uncertain linear discrete time-delayed systems subjected to actuator saturation is studied in this paper. The stability analysis of closed-loop systems is carried out using a Lyapunov-Krasovskii functional. Conditions for the existence of state-feedback GC controllers are developed using a linear matrix inequality (LMI)-based criterion. The approach ensures a sufficient performance bound over all the acceptable parameter uncertainties. The scheme of the optimal GC controller problem is framed as a convex optimization problem with LMI constraints. The design of GC controllers for discrete-time systems subjected to actuator saturation without considering the effect of state-delay is also discussed. The effectiveness of the proposed approach is illustrated using suitable examples.

Finite-time boundedness and control of positive coupled differential-difference equations with bounded time-varying delay

This paper is concerned with the problems of finite-time boundedness and finite-time control for positive coupled differential-difference equations (CDDEs) with bounded time-varying delay. The finite-time stability of such systems is analyzed by constructing an estimate of the solutions over a finite time interval. And, sufficient conditions based on linear programming (LP) are provided for finite-time stability of positive CDDEs with bounded time-varying delay. Then, by coordinate transformation, the obtained results are extended to the finite-time bounedness of positive CDDEs with bounded time-varying delay. By the obtained result of finite-time boundedness, static output-feedback controllers and static state-feedback controllers are designed and a sufficient condition is derived to ensure the positivity and finite-time boundedness of closed-loop system. Three illustrative examples are given to show the validity of our results.

A novel approach for robust control design of hidden synthetic inertia for variable speed wind turbines

This work presents a novel design methodology for hidden synthetic inertia control of variable speed wind turbines that considers the compromise between grid frequency response and operating limits of the wind turbine states. Two state space design strategies are proposed: one based on H∞ robust control with dynamic output feedback; and other based on H∞ static state feedback control; both strategies operate as supplementary controllers. The study was carried out on simulation with dynamic mathematical modeling of a variable speed wind turbine and an isolated system with conventional power plants, aiming to evaluate the grid frequency response to load-generation disturbances and wind speed variations. The proposed control strategies are compared with two topologies: without wind turbine frequency control and another with synthetic inertia contribution by Proportional-Derivative control. The proposed strategies allow to optimize performance of synthetic inertia control in accordance with wind turbine operating limits. H∞-based topologies presented greater attenuation of the impacts of load-generation disturbances on the frequency response, and also minimizes a cost function based on H2 norm of the frequency signal in comparison with other controllers, thus providing better performance against reference variations and changes of operation point.