Discrete-time Nonlinear Dynamical Systems Research Articles

PAC-Bayes Generalisation Bounds for Dynamical Systems including Stable RNNs

In this paper, we derive a PAC-Bayes bound on the generalisation gap, in a supervised time-series setting for a special class of discrete-time non-linear dynamical systems. This class includes stable recurrent neural networks (RNN), and the motivation for this work was its application to RNNs. In order to achieve the results, we impose some stability constraints, on the allowed models. Here, stability is understood in the sense of dynamical systems. For RNNs, these stability conditions can be expressed in terms of conditions on the weights. We assume the processes involved are essentially bounded and the loss functions are Lipschitz. The proposed bound on the generalisation gap depends on the mixing coefficient of the data distribution, and the essential supremum of the data. Furthermore, the bound converges to zero as the dataset size increases. In this paper, we 1) formalize the learning problem, 2) derive a PAC-Bayesian error bound for such systems, 3) discuss various consequences of this error bound, and 4) show an illustrative example, with discussions on computing the proposed bound. Unlike other available bounds the derived bound holds for non i.i.d. data (time-series) and it does not grow with the number of steps of the RNN.

Set-based state estimation for discrete-time constrained nonlinear systems: An approach based on constrained zonotopes and DC programming

This paper introduces a novel state estimator designed for discrete-time nonlinear dynamical systems that encompass unknown-but-bounded uncertainties, along with state linear inequality and nonlinear equality constraints. Our algorithm is based on constrained zonotopes (CZs) and employs a DC programming approach (where DC stands for difference of convex functions). Recently, techniques such as mean value extension and first-order Taylor extension have been adapted from zonotopes to facilitate the propagation of CZs across nonlinear mappings. While the resulting algorithms, known as CZMV and CZFO, achieve higher precision compared to the original zonotopic versions, they still exhibit sensitivity to the wrapping and dependency effects inherent in interval arithmetic. These interval-related challenges can be mitigated through the use of DC programming, as it allows for the determination of approximation error bounds by solving optimization problems. One direct advantage of this technique is the elimination of the dependency effect. Our set-membership filter, referred to as CZDC, provides an alternative solution to CZMV and CZFO. To showcase the effectiveness of our proposed approach, we conducted experiments using CZDC on two numerical examples.

Discrete-time nonlinear feedback linearization via physics-informed machine learning

We present a physics-informed machine learning (PIML) scheme for the feedback linearization of nonlinear discrete-time dynamical systems. The PIML finds the nonlinear transformation law, thus ensuring stability via pole placement, in one step. In order to facilitate convergence in the presence of steep gradients in the nonlinear transformation law, we address a greedy training procedure. We assess the performance of the proposed PIML approach via a benchmark nonlinear discrete map for which the feedback linearization transformation law can be derived analytically; the example is characterized by steep gradients, due to the presence of singularities, in the domain of interest. We show that the proposed PIML outperforms, in terms of numerical approximation accuracy, the traditional numerical implementation, which involves the construction –and the solution in terms of the coefficients of a power-series expansion–of a system of homological equations as well as the implementation of the PIML in the entire domain, thus highlighting the importance of continuation techniques in the training procedure of PIML schemes.

Generalized Semistability and Stochastic Semistability for Switched Nonlinear Systems Using Fixed Point Theory

Abstract This paper generalizes our previous results on semistability and stochastic semistability for switched nonlinear systems published in the Proceedings of 2021 Modeling, Estimation and Control Conference. The paper also provides the results on semistability in mean square for switched nonlinear discrete-time systems. The theoretical result involves generalized sufficient conditions for (stochastic) semistability and semistability in mean square of discrete-time nonlinear dynamical systems under time-varying or random (arbitrary) switching by means of Fixed Point Theory. An advantage of these results is to overcome fundamental challenges arising from using existing methods such as Lyapunov and LaSalle methods. As an application of the theoretical results presented, a constrained distributed consensus problem over random multi-agent networks is considered for which a generalized asynchronous and totally asynchronous iterative algorithm is derived. The algorithm is able to converge even if the weighted matrix of the graph is periodic and irreducible under synchronous protocol. Finally, a numerical example is given in which there is a distribution dependency among communication graphs to demonstrate the results.

Learning Control Policies for Stochastic Systems with Reach-Avoid Guarantees

We study the problem of learning controllers for discrete-time non-linear stochastic dynamical systems with formal reach-avoid guarantees. This work presents the first method for providing formal reach-avoid guarantees, which combine and generalize stability and safety guarantees, with a tolerable probability threshold p in [0,1] over the infinite time horizon. Our method leverages advances in machine learning literature and it represents formal certificates as neural networks. In particular, we learn a certificate in the form of a reach-avoid supermartingale (RASM), a novel notion that we introduce in this work. Our RASMs provide reachability and avoidance guarantees by imposing constraints on what can be viewed as a stochastic extension of level sets of Lyapunov functions for deterministic systems. Our approach solves several important problems -- it can be used to learn a control policy from scratch, to verify a reach-avoid specification for a fixed control policy, or to fine-tune a pre-trained policy if it does not satisfy the reach-avoid specification. We validate our approach on 3 stochastic non-linear reinforcement learning tasks.

Modeling the Combined Effect of Travelers’ Contrarian Behavior, Learning and Inertia on the Day-to-Day Dynamics of Route Choice

Understanding the many facets of repeated route choice behavior in traffic networks is essential for obtaining accurate flow forecasts and enhancing the effectiveness of traffic management measures. This paper presents a model of the day-to-day evolution of route choices incorporating travelers’ contrarian behavior, learning and inertia. The model is formulated as a discrete-time nonlinear dynamical system, and its properties are investigated analytically and numerically with a focus on the effect of the fraction of individuals adopting a contrarian route choice behavior. The findings of the study indicate that the extent of contrarian behavior may have significant impacts on the attractiveness and stability of network equilibria as well as on global system performance. We show that a properly balanced combination of direct and contrarian subjects can protect the system from instabilities triggered by other behavioral and network features. Our results also suggest that the fixed point stability range may depend to a considerable extent on travelers’ inertia and memory of previous experiences, as well as on the form of the travel cost functions used in the model. The occurrence of contrarian behavior should be explicitly taken into account in the design of traffic management schemes involving the deployment of Advanced Traveler Information Systems (ATISs), as it may act as a mitigating factor against the concentration of choices on the recommended routes. The analytical framework proposed in this paper represents a novel contribution, since contrarian behavior in repeated route choice has been investigated mainly by means of empirical or simulation approaches thus far.

System identification using autoregressive Bayesian neural networks with nonparametric noise models

System identification is of special interest in science and engineering. This article is concerned with a system identification problem arising in stochastic dynamic systems, where the aim is to estimate the parameters of a system along with its unknown noise processes. In particular, we propose a Bayesian nonparametric approach for system identification in discrete time nonlinear random dynamical systems assuming only the order of the Markov process is known. The proposed method replaces the assumption of Gaussian distributed error components with a flexible family of probability density functions based on Bayesian nonparametric priors. Additionally, the functional form of the system is estimated by leveraging Bayesian neural networks, which leads to flexible uncertainty quantification. Hamiltonian Monte Carlo sampler within a Gibbs sampler for posterior inference is proposed and its effectiveness is illustrated in real time series.

Bayesian computational methods for state-space models with application to SIR model

The state-space model is a powerful statistical tool to estimate linear or non-linear discrete-time dynamic systems. This model naturally leads to the estimation problem of the time-varying parameters of the discovery-time demographic version of the susceptible-infected-recovered (SIR) model that we consider. In this paper, we consider computational methods to perform Bayesian inference on state-space models for analysing time-series data. We compare the three popular Bayesian computational methods for state-space models: the adaptive Metropolis-within-Gibbs algorithm, Liu and West's algorithm and variational approximation method based on Gaussian distributions. The performances of the three methods are compared based on synthetic datasets. Furthermore, we analyse the trend of the spread of COVID-19 in South Korea to point out the limitations of existing methods and derive meaningful results.

Consensus in networks of dynamical systems with limited communication capacity

This paper provides a solution for the preservation of consensus in a network of several agents, described by discrete-time nonlinear dynamical systems. Consensus preservation is the problem of maintaining a certain distance between the states of several systems, given that the systems’ initial states are close to each other. The agents are equipped with both a smart sensor, capable of measuring the state of the system and performing some computations, and a controller. The sensors and controllers are placed at locations that are remote from one another. A network of communication channels with limited transmission capacity connects the agents by allowing the sensors to send messages to the controllers of their system as well as to the controllers of other systems. The controllers use the messages that they receive to steer the agents such that the consensus is preserved. Sensors and controllers that achieve this feature are called consensus-preserving protocols. In this paper, three distinct consensus-preserving protocols are presented, each with an increasing degree of interaction between the systems. For each of these protocols, a theorem providing conditions on the minimum communication capacity sufficient to implement them is presented. The protocols are tested by simulations for a network of logistic maps and a network of Hénon maps. For both of these networks, analytical bounds on the sufficient transmission capacities in the communication channels to implement the protocols are provided. These bounds are compared to rates observed in simulations of the consensus-preserving protocols.

An efficient implementation of graph-based invariant set algorithm for constrained nonlinear dynamical systems

The graph-based invariant set (GIS) algorithm is a promising set-based technique for computing the largest (with respect to inclusion) control invariant set of general discrete-time nonlinear dynamical systems. However, like other invariant set algorithms for nonlinear systems, the GIS algorithm may require a lot of resources when computing the control invariant set. This limits its applicability to higher dimensional systems. In this work, we present an improved and efficient implementation of the GIS algorithm for general discrete-time controlled nonlinear systems. We first identify the bottlenecks through extensive analysis, and then provide remedial procedures to improve the implementation of the GIS algorithm. Specifically, we developed an adaptive subdivision scheme using a supervised machine learning-based algorithm to reduce the cell growth rate and parallelize the graph construction step. We extensively demonstrate the performance of the improved GIS algorithm using a numerical example and compare the result to that of the standard GIS algorithm. The results show that the adaptive subdivision and the parallelization improved the speed of the algorithm by about 8x and 3x respectively, that of the standard GIS algorithm.

Lyapunov theorems for stability and semistability of discrete-time stochastic systems

This paper develops Lyapunov and converse Lyapunov theorems for discrete-time stochastic semistable nonlinear dynamical systems expressed by Itô-type difference equations possessing a continuum of equilibria. Specifically, we provide necessary and sufficient Lyapunov conditions for stochastic semistability and show that stochastic semistability implies the existence of a continuous Lyapunov function whose difference operator involves a discrete-time analog of the infinitesimal generator for continuous-time Itô dynamical systems and decreases along the dynamical system sample solution sequences satisfying an inequality involving the average distance to the set of the system equilibria.

Robust tube-based model predictive control with Koopman operators

Koopman operators are of infinite dimension and capture the characteristics of nonlinear dynamics in a lifted global linear manner. The finite data-driven approximation of Koopman operators results in a class of linear predictors, useful for formulating linear model predictive control (MPC) of nonlinear dynamical systems with reduced computational complexity. However, the robustness of the closed-loop Koopman MPC under modeling approximation errors and possible exogenous disturbances is still a crucial issue to be resolved. Aiming at the above problem, this paper presents a robust tube-based MPC solution with Koopman operators, i.e., r-KMPC, for nonlinear discrete-time dynamical systems with additive disturbances. The proposed controller is composed of a nominal MPC using a lifted Koopman model and an off-line nonlinear feedback policy. The proposed approach does not assume the convergence of the approximated Koopman operator, which allows using a Koopman model with a limited order for controller design. Fundamental properties, e.g., stabilizability, observability, of the Koopman model are derived under standard assumptions with which, the closed-loop robustness and nominal point-wise convergence are proven. Simulated examples are illustrated to verify the effectiveness of the proposed approach.

Finite Time Stability and Optimal Finite Time Stabilization for Discrete-Time Stochastic Dynamical Systems

In this paper, we address finite time stability in probability of discrete-time stochastic dynamical systems. Specifically, a stochastic comparison lemma is constructed along with a scalar system involving a generalized deadzone function to establish almost sure convergence and finite time stability in probability. This result is used to provide Lyapunov theorems for finite time stability in probability for Itô-type stationary nonlinear stochastic difference equations involving Lyapunov difference conditions on the minimum of the Lyapunov function itself along with a fractional power of the Lyapunov function. In addition, we establish sufficient conditions for almost sure lower semicontinuity of the stochastic settling-time capturing the average settling time behavior of the discrete-time nonlinear stochastic dynamical system. Furthermore, a stochastic finite-time optimal control framework is developed by exploiting connections between Lyapunov theory for finite time stability in probability and stochastic Bellman theory. In particular, we show that finite time stability in probability of the closed-loop nonlinear system is guaranteed by means of a Lyapunov function that can clearly be seen to be the solution to the steady state form of the stochastic Bellman equation, and hence, guaranteeing both stochastic finite time stability and optimality.

Data-Driven Abstraction and Model Invalidation for Unknown Systems With Bounded Jacobians

In this paper, we consider data-driven abstraction and model invalidation problems for unknown nonlinear discrete-time dynamical systems with bounded Jacobians, where only prior noisy sampled data of the systems, instead of mathematical models, are available. First, we introduce a novel non-parametric learning approach to over-approximate the unknown model/dynamics with upper and lower functions, i.e., to find model abstractions, under the assumption of known bounded Jacobians. Notably, the resulting data-driven models can be mathematically proven to be equal to or more accurate than componentwise Lipschitz continuity-based methods. Further, we show that the resulting data-driven model can be used to determine its (in)compatibility with a newly observed length-T output trajectory, i.e., to (in)validate models, using a tractable feasible check. Finally, we propose a method to estimate the Jacobian bounds if they are not known or given.

A non-linear discrete-time dynamical system related to epidemic SISI model

Abstract We consider SISI epidemic model with discrete-time. The crucial point of this model is that an individual can be infected twice. This non-linear evolution operator depends on seven parameters and we assume that the population size under consideration is constant, so death rate is the same with birth rate per unit time. Reducing to quadratic stochastic operator (QSO) we study the dynamical system of the SISI model.

Converging outer approximations to global attractors using semidefinite programming

This paper develops a method for obtaining guaranteed outer approximations for global attractors of continuous and discrete time nonlinear dynamical systems. The method is based on a hierarchy of semidefinite programming problems of increasing size with guaranteed convergence to the global attractor. The approach taken follows an established line of reasoning, where we first characterize the global attractor via an infinite dimensional linear programming problem (LP) in the space of Borel measures. The dual to this LP is in the space of continuous functions and its feasible solutions provide guaranteed outer approximations to the global attractor. For systems with polynomial dynamics, a hierarchy of finite-dimensional sum-of-squares tightenings of the dual LP provides a sequence of outer approximations to the global attractor with guaranteed convergence in the sense of volume discrepancy tending to zero. The method is very simple to use and based purely on convex optimization. Numerical examples with the code available online demonstrate the method.

Dissipativity, inverse optimal control, and stability margins for nonlinear discrete-time stochastic feedback regulators

ABSTRACT In this paper, we derive stability margins for optimal and inverse optimal stochastic feedback regulators. Specifically, gain, sector, and disk margin guarantees are obtained for discrete-time nonlinear stochastic dynamical systems controlled by nonlinear optimal and inverse optimal controllers that minimise a nonlinear-nonquadratic performance criterion. Furthermore, using the newly developed notion of stochastic dissipativity we derive a return difference inequality to provide connections between stochastic dissipativity and optimality of nonlinear controllers for discrete-time stochastic dynamical systems. In particular, using extended Kalman-Yakubovich-Popov conditions characterising stochastic dissipativity we show that our optimal feedback control law satisfies a return difference inequality predicated on a difference operator of a controlled Markov dispersion process and is stochastically dissipative with respect to a specific quadratic supply rate.

Set-Membership Filter for Discrete-Time Nonlinear Systems Using State-Dependent Coefficient Parameterization

In this article, a recursive set-membership filtering algorithm for discrete-time nonlinear dynamical systems subject to unknown but bounded process and measurement noises is proposed. The nonlinear dynamics are represented in a pseudolinear form using the state-dependent coefficient (SDC) parameterization. Matrix Taylor expansions are utilized to expand the state-dependent matrices about the state estimates. Upper bounds on the norms of remainders in the matrix Taylor expansions are calculated online using a nonadaptive random search algorithm at each time step. Utilizing these upper bounds and the ellipsoidal set description of the uncertainties, a two-step filter is derived that utilizes the “correction–prediction” structure of the standard Kalman filter variants. At each time step, correction and prediction ellipsoids are constructed that contain the true state of the system by solving the corresponding semidefinite programs. Finally, a simulation example is included to illustrate the effectiveness of the proposed approach.

Virtual unmodeled dynamic compensation and data-driven nonlinear generalized predictive control

For a class of discrete-time nonlinear dynamic systems, a nonlinear generalized predictive control method is proposed, which combines the conventional generalized predictive control with data-driven control, the feedback control with feedforward compensation, and the signal compensation with the regulator principle. The input-output dynamic equations of the closed-loop system are derived, and the performance of the proposed control algorithm is analyzed. The advantages and effectiveness of the proposed control algorithm are demonstrated through comparative experiments.

Applications of the Poincaré–Hopf Theorem: Epidemic Models and Lotka–Volterra Systems

This article focuses on properties of equilibria and their associated regions of attraction for continuous-time nonlinear dynamical systems. The classical Poincaré–Hopf theorem is used to derive a general result providing a sufficient condition for the system to have a unique equilibrium. The condition involves the Jacobian of the system at possible equilibria and ensures that the system is in fact locally exponentially stable. We apply this result to the susceptible–infected–susceptible (SIS) networked epidemic model, and a generalized Lotka–Volterra system. We use the result further to extend the SIS model via the introduction of decentralized feedback controllers, which significantly change the system dynamics, rendering existing Lyapunov-based approaches invalid. Using the Poincaré–Hopf approach, we identify a necessary and sufficient condition, under which the controlled SIS system has a unique nonzero equilibrium (a diseased steady state), and monotone systems theory is used to show that this nonzero equilibrium is attractive for all nonzero initial conditions. A counterpart condition for the existence of a unique equilibrium for a nonlinear discrete-time dynamical system is also presented.