Optimal Output Feedback Research Articles

Output feedback fault-tolerant Q-learning for discrete-time linear systems with actuator faults

This paper presents an innovative output feedback fault-tolerant Q-learning algorithm that can be implemented online without relying on explicit system models or fault details. In the face of actuator faults, finding optimal Fault-Tolerant Control (FTC) solutions that can stabilize the faulty system poses significant challenges. The proposed approach is implemented online without the need for system dynamics and actuator fault information. Furthermore, it operates without relying on full state measurements, utilizing only the input-output data of the faulty system. An innovative representation of the output feedback Fault-Tolerant Q-function (FTQF) is established using input-output data. Subsequently, a model-free optimal output feedback FTC policy is obtained from the developed FTQF. Then, a fault-tolerant Q-learning algorithm is formulated to iteratively acquire the optimal FTC policy in real-time, eliminating the necessity for system dynamics and actuator fault information. The proposed algorithm exhibits immunity to excitation noise bias, even without considering a discounting factor. Furthermore, the proposed Q-learning approach is proven to be effective in stabilizing faulty closed-loop system. Finally, the proposed algorithm's efficiency is validated through numerical simulations of F-16 autopilot aircraft dynamics.

Optimal output feedback control for networked control systems with dual multi‐memoryless channels

AbstractThis paper delves into the output feedback control of systems with dual multi‐memoryless channels, where the data packets transmitted from the sensor to the controller (S/C) and from the controller to the actuator (C/A) are suffered from two different multi‐parallel Markovian fading channels. Two new multi‐state Markovian chains are introduced, by which the multiple fading channels system is transformed into a new jumping parameter system. Then a stochastic maximum principle is given along with an optimal state feedback controller for the case that the state is known. When the state is unknown, a Markov jump linear estimator is designed by using the completing square method. In this case, an output feedback controller is obtained by substituting the real system state with its estimation. Meanwhile, the separation principle is shown. Finally, the validity of these methods is verified by simulation examples.

Data‐driven based optimal output feedback control with low computation cost

SummaryA partial model‐free, data‐driven adaptive optimal output feedback (OPFB) control scheme with low computational cost continuous‐time is proposed in this paper. The design objective is to obtain the optimal control law by using measurable input and output data, without some knowledge of system model information. Firstly, the system states are decoupled into measurable and unmeasurable parts, and a new state‐space equation is built to estimate the unmeasurable states by using a reduced‐order observer. Based on this, a parametrization method is utilized to reconstruct the system states. Subsequently, by using the reconstructed states, the adaptive dynamic programming (ADP) Bellman equations based on policy‐iteration (PI) and value‐iteration (VI) are presented to solve the control problems with initially stable and unstable conditions, respectively. Then, the convergence of the system is proved. Compared with the early proposed OPFB algorithms, only the unknown internal state needs to be reconstructed. Therefore, the computation cost and design complexity are reduced for the proposed scheme. The effectiveness of the proposed scheme is verified through two numerical simulations. In addition, a practical inverted pendulum experiment is carried out to demonstrate the performance of the proposed scheme.

Inverse Optimal Adaptive Fuzzy Output Feedback Control for Nonlinear Systems With Output Quantization

Inverse Optimal Adaptive Fuzzy Output Feedback Control for Nonlinear Systems With Output Quantization

Optimal Output-Feedback Control Over Markov Wireless Communication Channels

Optimal Output-Feedback Control Over Markov Wireless Communication Channels

A linear matrix inequality approach to optimal reset control design for a class of nonlinear systems

AbstractIn this article, the problem of the optimal reset control design for Lipschitz nonlinear systems is addressed. The reset controller includes a base linear controller and a reset law that enforces resets to the controller states. The reset law design is strongly dependent on the appropriate design of the base controller. For this reason, in this article, the base controller and reset law are simultaneously designed. More precisely, an optimal dynamic output feedback is considered as the base controller which minimizes the upper bound of a quadratic performance index, and a reset law is used to improve the transient response of the closed‐loop system. This design is done in a full offline procedure. The problem is transformed into a set of linear matrix inequalities (LMIs), and the reset controller is obtained by solving an offline LMI optimization problem. Finally, two examples are presented to illustrate the effectiveness and validity of the proposed method.

Optimal fuzzy output feedback tracking control for unmanned surface vehicles systems

This study investigates an optimal fuzzy output feedback tracking control problem with a Q-learning algorithm on unmanned surface vehicles (USVs) with immeasurable velocities information. Firstly, the USVs are modeled by a Takagi–Sugeno (T–S) fuzzy system. Based on the fuzzy system, an ideal optimal fuzzy output feedback tracking controller is proposed by utilizing reconstruction states, measured output and input data. Theoretically, the analytical solution of the designed optimal controller is reduced to solving the algebraic Riccati equations (AREs). Due to the difficulty of solving AREs directly, a Q-learning value iteration algorithm is proposed to obtain its approximation solution. Furthermore, it is rigorously mathematically proved that the proposed Q-learning algorithm can approximate the ideal solution and guarantee the USVs output to track the desired reference signal. Ultimately, the simulation and comparison results with the existing control method verify the efficacy and advantage of the presented fuzzy optimal output feedback tracking control approach with a Q-learning algorithm.

Density estimation based soft actor-critic: deep reinforcement learning for static output feedback control with measurement noise

ABSTRACT The state-of-the-art deep reinforcement learning (DRL) methods, including Deep Deterministic Policy Gradient (DDPG), Twin Delayed DDPG (TD3), Proximal Policy Optimization (PPO), Soft Actor-Critic (SAC), among others, demonstrate significant capability in solving the optimal static state feedback control (SSFC) problem. This problem can be modeled as a fully observed Markov decision process (MDP). However, the optimal static output feedback control (SOFC) problem with measurement noise is a typical partially observable MDP (POMDP), which is difficult to solve, especially for the continuous state-action-observation space with high dimensions. This paper proposes a two-stage framework to address this challenge. In the laboratory stage, both the states and the noisy outputs are observable; the SOFC policy is converted to a constrained stochastic SSFC policy, of which the probability density function is generally not analytical. To this end, a density estimation based SAC algorithm is proposed to explore the optimal SOFC policy by learning the optimal constrained stochastic SSFC. Consequently, in the real-world stage, only the noisy outputs and the learned SOFC policy are required to solve the optimal SOFC problem. Numerical simulations and the corresponding experiments with robotic arms are provided to illustrate the effectiveness of our method. The code is available at https://github.com/RanKyoto/DE-SAC.

Inverse optimally adaptive neural output‐feedback control of stochastic nonlinear systems

SummaryIn this article, for a class of stochastic nonlinear systems with non‐strict feedback, a neural adaptive inverse optimal output feedback control design scheme is presented. First, according to the existing inverse optimal criterion, an auxiliary system is established. On this basis, a novel observer is built to evaluate the unpredictable states. Second, in the control process, neural networks (NNs) are applied to estimate the unknown functions. Based on NNs and the backstepping technology, an adaptive neural inverse optimal output feedback controller is established. It is indicated that the proposed scheme could ensure the semi‐globally uniformly ultimately bounded of the closed‐loop system and also achieve the objective of inverse optimality. Eventually, an example is applied to testify the feasibility of this scheme.

Direct integral pseudospectral and integral spectral methods for solving a class of infinite horizon optimal output feedback control problems using rational and exponential Gegenbauer polynomials

This study is concerned with the numerical solution of a class of infinite-horizon linear regulation problems with state equality constraints and output feedback control. We propose two numerical methods to convert the optimal control problem into nonlinear programming problems (NLPs) using collocations in a semi-infinite domain based on rational Gegenbauer (RG) and exponential Gegenbauer (EG) basis functions. We introduce new properties of these basis functions and derive their quadratures and associated truncation errors. A rigorous stability analysis of the RG and EG interpolations is also presented. The effects of various parameters on the accuracy and efficiency of the proposed methods are investigated. The performance of the developed integral spectral method is demonstrated using two benchmark test problems related to a simple model of a divert control system and the lateral dynamics of an F-16 aircraft. Comparisons of the results of the current study with available numerical solutions show that the developed numerical scheme is efficient and exhibits faster convergence rates and higher accuracy.

Novel extended Kalman filter + H∞$$ {\mathbf{H}}_{\infty } $$ optimal output feedback control configuration for small satellites with high pointing stability requirements

SummaryOne of the most challenging subsystems for the CubeSats is the attitude determination and control system (ADCS) because it involves hardware and software integration, advanced strategies and algorithms to determine and control the attitude that impacts on space maneuvering and well working of a large set of payload (e.g., the optical payload). Although ADCS is largely studied, it still requires further investigations and analysis in order to achieve high‐performance and a reduced efforts. This paper presents an effective and reliable solution for the ADCS of CubeSats involved in Earth observation missions. The solution is based on an innovative framework that leverages the strengths of two distinct methodologies—H‐infinity optimal output feedback control and the extended Kalman filter (EKF)‐to significantly enhance the pointing stability of satellite systems. While H‐infinity optimal output feedback control is traditionally associated with partial state knowledge, we intentionally extend its application to scenarios with complete state information. The study includes a comparative performance analysis involving three algorithm configurations: the classic combination of EKF and model predictive control (MPC), the utilization of H‐infinity optimal output feedback control without EKF, and our proposed integrated approach. Performance evaluations are based on extrinsic indicators, demonstrating the advantages of our method in terms of pointing stability and overall control system performance. This research underscore the significance of combining innovative thinking with well‐established methodologies, unlocking new possibilities for stability enhancement in a range of engineering applications.

Robust H[formula omitted]-PID control Stability of fractional-order linear systems with Polytopic and two-norm bounded uncertainties subject to input saturation

This paper deals a type of H∞ proportional–integral–derivative (PID) control mechanism for a type of structural uncertain fractional order linear systems by convex Polytopic and two-norm bounded uncertainties subject to input saturation which mainly focuses on the case of a fractional order α such that 0<α<1. The Gronwall–Bellman lemma and the sector condition of the saturation function are investigated for system stability analysis and stabilization. The main strategy of the presented strategy is to restore fractional order PID controller design under input saturation problem from static output feedback controller design. Unlike existing strategies, non-iterative strategy is used to get optimal output feedback based on the LMI. On the premise of a linear matrix inequality algorithm, the SOF control laws can be obtained. After that, the fractional-order PID controller is recovered from the SOF controller. A numerical example is provided in order to show the validity and superiority of the proposed method.

Output feedback Q-learning for discrete-time finite-horizon zero-sum games with application to the [formula omitted] control

In this paper, we present a Q-learning framework for solving finite-horizon zero-sum game problems involving the H∞ control of linear system without knowing the dynamics. Research in the past mainly focused on solving problems in infinite horizon with completely measurable state. However, in the practical engineering, the system state is not always directly accessible, and it is difficult to solve the time-varying Riccati equation associated with the finite-horizon setting directly either. The main contribution of the proposed model-free algorithm is to determine the optimal output feedback policies without measurement state in finite-horizon setting. To achieve this goal, we first describe the Q-function caused by finite-horizon problems in the context of state feedback, then we parameterize the Q-functions as input–output vectors functions. Finally, the numerical examples on aircraft dynamics demonstrate the algorithm’s efficiency.

Data-driven Optimal Output Feedback Control of Linear Systems from Input-Output Data

We design optimal output feedback controllers for linear systems where only outputs are available for measurement and control actuation. The goal is to render the closed-loop system stable while minimizing a quadratic cost function balancing performance and control effort. We first provide a model-based solution to this problem, where an optimal dynamic output feedback controller is computed by solving a semidefinite program. Then, we shift our attention to data-based solutions, bypassing the system identification step. We derive semidefinite programs that are explicitly stated in terms of input-output data. The effectiveness of the method is illustrated via a power system case study.

Optimal Static Output Feedback Stabilization of Fractional-Order Systems With Caputo Derivative Order 1 ≤ α &lt; 2

Optimal Static Output Feedback Stabilization of Fractional-Order Systems With Caputo Derivative Order 1 ≤ α &lt; 2

Adaptive inverse optimal backstepping control strategy for longitudinal vibration of high-speed elevator system based on fuzzy observer

To effectively suppress the longitudinal vibration of the car under the conditions of high-speed operation and emergency braking, and improve the ride comfort of the car system, this paper proposes an adaptive fuzzy inverse optimal output feedback control strategy. Firstly, the dynamic model of the high-speed elevator system is established and the nonlinear dynamic model is approximated by the fuzzy logic system, and the auxiliary system model is established. Fuzzy state observer is designed to estimate the unmeasurable state. Furthermore, an adaptive inverse optimal output feedback controller based on fuzzy observer is designed by using adaptive backstepping technology and inverse optimal control principle. The stability analysis shows that the proposed adaptive fuzzy inverse optimal output feedback control strategy not only ensures the stability of the car attitude of high-speed elevator but also realizes the inverse optimization of the target cost function. Finally, the acceleration time–frequency response analysis of the two typical stages of high-speed elevator uniform running and emergency braking is carried out, and the numerical results are compared with the linear quadratic controller optimized by stepping quantum genetic algorithm (GA-LQR) and controller based on the state-dependent Ricatti equation (SDRE). The analysis verifies the effectiveness of the controller.

Robust output feedback-based neuro-fuzzy controller for seismically excited tall buildings with ATMD accounting for variations in the type of supporting soil

Recently, developing the control algorithms for tuning the voltage/force command of smart control devices for application in seismic-excited buildings and bridges has been well addressed considering soil-structure interaction (SSI) effects. A controller is often designed for a soil pre-set condition. Moreover, considering an exact model of soil in the controller design process is a tedious task due to high computational costs. Consequently, the seismic performance and robustness of the adopted controller are questionable against the variations in soil conditions. The motivation of the present paper is to propose a robust controller for seismic-excited tall buildings against the variations of soil conditions than the pre-set condition. Taking advantage of the optimal output feedback controller (OOFC), a simple and practical controller based on OOFC is proposed for application in tall buildings equipped with an active tuned mass damper (ATMD) accounting for SSI effects. Considering different soil types, a dataset is generated from the proposed OOFC and an adaptive neuro-fuzzy inference system (ANFIS) is trained to fit them in a robust dataset. Then, a robust output feedback-based ANFIS controller, namely ROFBANFISC, is proposed. The robustness of the OOFC and ROFBANFISC are validated for different soil types. Considering six performance criteria, the numerical studies are conducted on a 40-story structure subjected to five real ground motions. The simulation results carried out for different soil conditions show that the proposed ROFBANFISC is more robust than the OOFC. Also, it can maintain its appropriate seismic performance in different soil types.

Control of drill string torsional vibrations using optimal static output feedback

Failures during oil extraction can have impacts on exploration costs and time, especially those caused by vibrations during the drilling phase. A proper way to reduce these detrimental failures is to design an active controller that efficiently mitigates vibrations with the constraint of using the limited amount of real time data available in field operations. In this context, a technique that might be suitable for the controller design is the optimal static output feedback (OSOF), that considers only the measured signals for feedback. Recently, the authors proposed some modifications to this technique, involving the addition of sensors locations and a new way of dealing with the dependence of the controller parameters on the system initial condition. The aim of this contribution is to apply this novel control technique to suppress drill string torsional vibrations and address all the particular aspects regarding the application of the proposed controller for the drill string problem, i.e., reformulation of the equations of motion as a stabilization problem, determination of the output matrix and linearization. Analyses and comparison of the proposed controller with proportional–integral (PI) and linear quadratic regulator (LQR) controllers are performed considering numerical simulations of a finite element model with a non-regularized dry friction function. Results indicate that the proposed controller has a performance similar to the LQR with a significantly reduced number of sensors, and also exhibits superior performance and robustness with an equivalent control effort over the PI controller.

Observer-Based Finite-Time Inverse Optimal Output Regulation for Uncertain Nonlinear Systems

This paper deals with the finite-time inverse optimal output regulation problem for a class of uncertain nonlinear systems that are subjected to an exosystem. The nonlinear systems not only contain external disturbances in the exosystem but also take the unknown nonlinear functions and unmeasured states into account. The output regulation problem is first converted into a stabilization problem through the internal model. Then, fuzzy logic systems (FLSs) are employed to approximate the unknown nonlinear functions. An auxiliary system is constructed and, based on the auxiliary system, a fuzzy state observer is designed to estimate the unmeasured states. Furthermore, a novel adaptive fuzzy finite-time inverse optimal output feedback controller and adaptive law are designed by combining backstepping technology, adaptive control technology, finite-time stability theory, and the inverse optimal control method. The control algorithm ensures that all the signals of the closed-loop system are semiglobally practical finite-time stable (SGPFS), so the newly well-defined cost function that is connected with the auxiliary system can be minimized. Finally, the validity of the approach is confirmed by virtue of the simulation results.

Model‐free distributed optimal control for continuous‐time linear systems

This paper studies the distributed optimal consensus problem for a group of continuous-time linear systems in the scenario that the dynamics of linear systems are completely unknown. Both an optimal state feedback control law and an optimal output feedback control law are constructed by solving the Algebraic Riccati Equation (ARE) that contains no global information, which guarantees the consensus of linear systems and the global optimality. Adaptive Dynamic Programming (ADP) method is presented to solve the ARE, thus the control algorithms are not relying on the dynamics of linear systems any more. Finally, numerical simulation results are presented to demonstrate the efficiency of proposed approach.