Discrete-time Linear Time-invariant Systems Research Articles

Data-driven distributed output consensus control for multi-agent systems with unknown internal state

This paper discusses the topic of optimal output consensus control of linear time-invariant discrete-time multi-agent systems (MASs). The attainment of optimal output consensus control for MASs hinges upon the resolution of the interconnected Hamilton–Jacobi-Bellman equation, a task typically precluded by the inherent intractability of analytical solutions. Furthermore, most real-world systems are too complex to obtain the internal states of systems. To address these issues, a modified deep Q-learning network is constructed using current and historical system data rather than a precise model of the system. First, reconstructing the internal state of each agent using an adaptive distributed observer based on output feedback prevents the system instability brought on by the augmented systems. Then the local error system of the agent can be redefined. Based on the redefined error system, a data-driven adaptive dynamic programming (ADP) method is introduced, realized by using the actor–critic neural network structure. In addition, an experience replay strategy is proposed to reduce the propagation of estimation bias and improve the learning speed. Finally, the comparative numerical simulations substantiate the efficacy of the proposed algorithm in a quantifiable manner.

Output null controllability for linear time-invariant structured discrete-time systems: A graph theoretic condition

In this paper, we consider a linear time-invariant discrete-time system and study the output null controllability problem, i.e., the problem of steering the output to zero in a finite number of steps. We assume that we only know the structure of the system, i.e., the zero/nonzero location in the system matrices. Hence, we consider a structural version of the output null controllability problem. We represent the structure of the system by means of a directed graph and present a graph theoretic sufficient condition for the problem to be generically solvable. Here generically solvable means that the problem is solvable for almost all systems with the same structure. We illustrate the conditions using an example.

Data-driven adaptive optimal control for discrete-time linear time-invariant systems

In this paper, the problem of data-driven quadratic optimal control is investigated for discrete-time linear systems without the exact knowledge of system dynamics. A value iteration-based algorithm is proposed to obtain the optimal feedback gain. The main idea of the proposed approach is to obtain two sequences for the approximation of the unique positive definite solution of the corresponding algebraic Riccati matrix equation and the optimal feedback gain by using the collected data of the system states and control inputs. The proposed algorithm could be activated by an arbitrary bounded control input and a simple initial value of the Lyapunov matrix. An important feature of this algorithm is that an original stabilising feedback gain is not required. Finally, the effectiveness of the proposed approach is verified by two examples.

Optimization Landscape of Policy Gradient Methods for Discrete-Time Static Output Feedback.

In recent times, significant advancements have been made in delving into the optimization landscape of policy gradient methods for achieving optimal control in linear time-invariant (LTI) systems. Compared with state-feedback control, output-feedback control is more prevalent since the underlying state of the system may not be fully observed in many practical settings. This article analyzes the optimization landscape inherent to policy gradient methods when applied to static output feedback (SOF) control in discrete-time LTI systems subject to quadratic cost. We begin by establishing crucial properties of the SOF cost, encompassing coercivity, L -smoothness, and M -Lipschitz continuous Hessian. Despite the absence of convexity, we leverage these properties to derive novel findings regarding convergence (and nearly dimension-free rate) to stationary points for three policy gradient methods, including the vanilla policy gradient method, the natural policy gradient method, and the Gauss-Newton method. Moreover, we provide proof that the vanilla policy gradient method exhibits linear convergence toward local minima when initialized near such minima. This article concludes by presenting numerical examples that validate our theoretical findings. These results not only characterize the performance of gradient descent for optimizing the SOF problem but also provide insights into the effectiveness of general policy gradient methods within the realm of reinforcement learning.

Singularity-free adaptive control of discrete-time linear systems without prior knowledge of the high-frequency gain

This paper proposes a new output feedback model reference adaptive control (MRAC) method for discrete-time linear time-invariant systems with arbitrary relative degrees. The proposed method does not require any prior knowledge of the high-frequency gain represented by kp, thereby completely eliminating the common design condition in the traditional discrete-time MRAC framework: the sign of kp is known, as well as an upper bound on |kp|. Specifically, an output feedback adaptive control law is developed, which incorporates a time-varying gain function to effectively address the singularity issue. The developed adaptive control law leads to the derivation of a linear estimation error equation for the closed-loop system. Consequently, a gradient algorithm based parameter update law is directly formulated by utilizing the estimation error and some other available signals without requiring any prior knowledge of kp. In comparison to the traditional MRAC and Nussbaum function-based methods, it does not necessitate any additional design conditions or involve any transient performance issues, while still ensuring closed-loop stability and asymptotic output tracking for any given bounded reference signal. The simulation study showcases the design procedure and evaluates the efficacy of the proposed control method.

Guaranteed Robust Performance of $\mathcal {H}_{\infty }$ Filters With Sparse and Low Precision Sensing

The performance of estimation algorithms depends on the available sensors and their precisions. While higher precisions of sensors provide better estimation accuracy, it may lead to unnecessarily expensive designs and higher operational costs due to higher costs of high-precision sensing modalities. Also, higher precision can cause interference for other sensors in the environment, such as RADARS, resulting in degradation in the overall system's performance. This paper presents a framework for co-designing a sparse sensing network with the least precise sensors and the filter that guarantees the prescribed <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathcal {H}_{\infty }$</tex-math></inline-formula> estimation accuracy. Convex optimization problems for minimizing sensor precisions are formulated for continuous and discrete-time linear time-invariant systems, with and without model uncertainties. Different heuristics for determining a sparse sensor set with the least precise sensors are discussed, and their performances are compared using numerical simulations. The application of the proposed framework is demonstrated using numerical examples.

Distributed State Estimation Under Jointly Connected Switching Networks: Continuous-Time Linear Systems and Discrete-Time Linear Systems

In this paper, we address the distributed state estimation problem for both continuous-time linear time-invariant (LTI) systems and discrete-time LTI systems under switching networks. The observed system is jointly observable, i.e., each agent can only access a part of the measurement output of the observed system and cannot recover the full state by itself. The full state estimation has to be achieved by network communication of neighboring agents. In contrast to existing works, the salient feature of this work is that the developed approach can deal with the jointly connected switching network and thus is more resilient to unreliable communication. First, we propose a new observability decomposition method for linear systems in modal canonical form. Then, we design distributed observers for both the continuous-time and the discrete-time system. Based on the common Lyapunov function approach, we show that the switched estimation error system is asymptotically stable and thus, the full state estimation can be achieved under jointly connected switching networks.

On the frequency and damping of resonances in electrical networks: a discrete state-space approach using bounce Diagrams

The paper presents a novel method to compute resonances in a power system network using a discrete state-space approach based on bounce diagrams. The approach models the distributed parameter line as a lattice diagram, considering the refection/transmission of traveling waves. This model is then represented as a discrete-time linear time-invariant system, enabling the computation of resonances in the network. Two applications are demonstrated: fault location and load placement to improve damping in black start scenario. The method is particularly useful to understand and control resonances in electric grids. Theoretical analysis and numerical examples illustrate the behavior of resonances as resistive loads are connected along the transmission line, providing insights for fault location and load allocation strategies.

Parameter Identification of Discrete-time Linear Time-invariant Systems Using State and Input Data

Parameter Identification of Discrete-time Linear Time-invariant Systems Using State and Input Data

PROPERTIES OF SCHWARZ MATRICES IN DISCRETE-TIME LINEAR SYSTEMS

In this paper, we investigate the properties of the Schwarz matrix, a specific type of matrix that appears in the stability analysis of discrete-time linear time-invariant systems. We derive a formula for the determinant of the Schwarz matrix and a formula for its permanent. We also provide conditions on the entries of the Schwarz matrix that ensure the system described by the state update equation $x_{k+1} = Bx_k$ is stable, as well as conditions that guarantee the eigenvalues of the Schwarz matrix are real. These findings provide insights into the stability properties of systems characterized by Schwarz matrices and offer new tools for the analysis of interconnected subsystems in a cascaded structure.

Efficient Off-Policy Q-Learning for Data-Based Discrete-Time LQR Problems

This paper introduces and analyzes an improved Q-learning algorithm for discrete-time linear time-invariant systems. The proposed method does not require any knowledge of the system dynamics, and it enjoys significant efficiency advantages over other data-based optimal control methods in the literature. This algorithm can be fully executed off-line, as it does not require to apply the current estimate of the optimal input to the system as in on-policy algorithms. It is shown that a persistently exciting input, defined from an easily tested matrix rank condition, guarantees the convergence of the algorithm. A data-based method is proposed to design the initial stabilizing feedback gain that the algorithm requires. Robustness of the algorithm in the presence of noisy measurements is analyzed. We compare the proposed algorithm in simulation to different direct and indirect data-based control design methods.

Data-driven [formula omitted] control of constrained systems: An application to bilateral teleoperation system

A novel, identification-free, data-driven (DD) H∞ control method is presented for discrete-time (DT) linear time-invariant (LTI) systems under physical limitations and norm-bounded disturbances. The presented approach does not demand information on system matrices or any measurements of disturbance affecting the system. The only information needed to develop a static state-feedback (SF) controller is the bounds on disturbances, states and control signals. It is assumed that only the disturbance input matrix and the performance matrices the user generally defines are known, and all others are entirely unknown. The proposed method relies on the closed-loop (CL) parametrization of the LTI system with control input and state measurements. The disturbances affecting the system states are handled as affine uncertainties, later represented as Linear Fractional Transformation (LFT). For obtaining a less conservative controller, a full block S-procedure method (FBSPM) is used, which takes advantage of relaxations such as convex hull relaxation or Pólya relaxation for the inner approximation of the disturbance set with arbitrary precision. Numerical illustrations and extensive case studies on a bilateral teleoperation system indicate that the proposed design method allows us to obtain very effective controllers which never exceed the bounds of the state and input variables and are capable of reference and force tracking.

Theoretical and Algorithmic Study of Inverses of Arbitrary High-Dimensional Multi-Input Multi-Output Linear-Time-Invariant Systems

Nowadays, systems need to be built and/or characterized to handle exceptional circumstances or adapt to a world itself more complex. A typical phenomenon can often be found that systems are required to accommodate high-dimensional inputs and outputs. Although the theories and methods for inverting a single-input single-output (SISO) linear-time-invariant (LTI) system have been well established, the generalized framework (consisting of theories and algorithms) for extending to arbitrary high-dimensional multi-input multi-output (MIMO) scenarios is still unsubstantial in the existing literature as this extension is far from trivial. In this work, we would like to develop such a new framework for governing the inversion of arbitrary high-dimensional discrete-time MIMO LTI systems, where any individual transfer function from a certain input to a certain output may have the infinite-impulse-response (IIR) characteristics. We propose two new inversion algorithms to invert the transfer-function tensors (TFTs) of arbitrary MIMO LTI systems. The pertinent computational complexities are also investigated for our proposed two TFT-inversion algorithms. The approximation of the inverse of an arbitrary TFT by a finite-impulse-response (FIR) TFT is studied and the corresponding approximation-error analysis is derived as well. Finally, numerical evaluations are presented to study the computational complexities with respect to different TFT dimensions and ranks.

Event-Triggered Neural Network Control for LTI Systems

This paper addresses the problem of event-triggered control (ETC) of discrete-time linear time-invariant (LTI) systems stabilized by neural network controllers. The event-triggering mechanisms (ETMs) is proposed to update only a portion of the layers required to maintain stability and satisfactory performance of the feedback system, thus reducing the computational cost associated with the evaluation of the neural network. Sufficient convex conditions in the form of linear matrix inequalities (LMIs) are provided to compute the triggering parameters and characterize an estimate of the domain of attraction for the feedback system. The formulation is based on the use of Lyapunov theory and a set of generalized sector constraints that deal with the nonlinear activation functions, represented in this case by the saturation function. Optimization procedures are also formulated to effectively reduce the amount of computation in the neural network. An example borrowed from the literature is used to illustrate the effectiveness of the proposal.

Adaptive Observers for MIMO Discrete-Time LTI Systems

In this paper, an adaptive observer is proposed for multi-input multi-output (MIMO) discrete-time linear time-invariant (LTI) systems. Unlike existing MIMO adaptive observer designs, the proposed approach is applicable to LTI systems in their general form. Further, the proposed method uses recursive least square (RLS) with covariance resetting for adaptation that is shown to guarantee that the estimates are bounded, irrespective of any excitation condition, even in the presence of a vanishing perturbation term in the error used for updation in RLS. Detailed analysis for convergence and boundedness has been provided along with simulation results for illustrating the performance of the developed theory.

On the impulse response peak of discrete-time uncertain LTI systems*

This paper addresses the problem of determining the peak of the admissible impulse responses of a discrete-time uncertain linear time-invariant (LTI) system. The uncertainty is represented by a vector of parameters constrained into a semialgebraic set that affect polynomially the matrices of the system. It is shown that an upper bound of the sought peak can be obtained by solving a semidefinite program (SDP) constructed by introducing a novel approach, which consists of splitting the responses into two temporal parts: head and tail. The head of the responses is analyzed via inspection of the response formula, while the tail of the responses is analyzed via quadratic Lyapunov functions polynomially parameterized by the uncertainty. As shown by some numerical examples, the conservatism of the upper bound can be reduced by increasing the length of the head or the degree of the Lyapunov functions in the uncertainty.

Discrete-time Linear Time-invariant Distributed Minimum Energy Estimator

Proper monitoring of large complex spatially critical infrastructures often requires a sensor network capable of inferring the state of the system. Such networks enable the design of distributed estimators considering only local (partial) measurements, local communication capabilities with nearby sensors, as well as the system model. Solutions often assume perfect knowledge of the system model, and white process and measurement noise, which are limiting in engineering settings. In this paper, we consider the minimum energy setting where the model uncertainty and process and measurement noises are bounded but unknown. We provide the first distributed minimum energy estimator for discrete-time linear time-invariant systems, and we show that the error dynamics is input-to-state stable. Lastly, we illustrate the performance in some pedagogical examples, and compare the performance with respect to the centralized implementation of the minimum energy estimator.

Adaptive Output Feedback Model Predictive Control

Model predictive control (MPC) for uncertain systems in the presence of hard constraints on state and input is a non-trivial problem, and the challenge is increased manyfold in the absence of state measurements. In this paper, we propose an adaptive output feedback MPC technique, based on a novel combination of an adaptive observer and robust MPC, for single-input single-output discrete-time linear time-invariant systems. At each time instant, the adaptive observer provides estimates of the states and the system parameters that are then leveraged in the MPC optimization routine while robustly accounting for the estimation errors. The solution to the optimization problem results in a homothetic tube where the state estimate trajectory lies. The true state evolves inside a larger outer tube obtained by augmenting a set, invariant to the state estimation error, around the homothetic tube sections. The proof for recursive feasibility for the proposed `homothetic and invariant' two-tube approach is provided, along with simulation results on an academic system.

Learning Safety Filters for Unknown Discrete-Time Linear Systems

A learning-based safety filter is developed for discrete-time linear time-invariant systems with unknown models subject to Gaussian noises with unknown covariance. Safety is characterized using polytopic constraints on the states and control inputs. The empirically learned model and process noise covariance with their confidence bounds are used to construct a robust optimization problem for minimally modifying nominal control actions to ensure safety with high probability. The optimization problem relies on tightening the original safety constraints. The magnitude of the tightening is larger at the beginning since there is little information to construct reliable models, but shrinks with time as more data becomes available.

Data-Driven Control Design With LMIs and Dynamic Programming

The goal of this paper is to study model-free data-driven control evaluation and design strategies for discrete-time linear time-invariant systems, where the system model is unknown. In particular, our main contribution is twofold: 1) new state-input exploration and data collection schemes from experiences; 2) new data-driven linear matrix inequalities and dynamic programming methods for stabilization and optimal control problems. The proposed exploration and data collection schemes theoretically guarantee to acquire sufficient information from the system’s state-input trajectories that can solve the underlying control design problems. We prove that under mild assumptions, as more and more data is accumulated, the collected data can solve the problems with higher probability along with the proposed algorithms.