Discrete-time Systems Research Articles

An approach to mismatched disturbance rejection control for uncontrollable systems

This study focuses on the problem of optimal mismatched disturbance rejection control for uncontrollable linear discrete-time systems. In contrast to previous studies, by introducing a quadratic performance index such that the regulated state can track a reference trajectory and minimize the effects of disturbances, mismatched disturbance rejection control is transformed into a linear quadratic tracking problem. The necessary and sufficient conditions for the solvability of this problem over a finite horizon and a disturbance rejection controller are derived by solving a forward–backward difference equation. In the case of an infinite horizon, a sufficient condition for the stabilization of the system is obtained under the detectable condition. Additionally, in combination with the generalized extended state observer, a controller design method is proposed, and the stability analysis of the system under this controller is presented. This paper details our novel approach to disturbance rejection. Finally, four examples are provided to demonstrate the effectiveness of the proposed method.

Model-free aperiodic tracking for discrete-time systems using hierarchical reinforcement learning

In the operation of practical systems, it is often a challenging task to deal with limited knowledge of the system dynamics, therefore, it is needed to obtain a framework for the simultaneous learning and control of dynamic systems, able to cope with uncertain dynamics. This paper proposes a model-free aperiodic tracking control method based on MAXQ hierarchical reinforcement learning for unknown dynamics in discrete-time systems. Firstly, a MAXQ hierarchical framework is developed for aperiodic tracking control that decomposes the task into pre-control and accurate control subtasks. The former achieves sub-optimal tracking which prompts the implementation of accurate control. The latter implements aperiodic tracking based on the pre-control result to reduce resource occupation. The structure of the MAXQ framework simplifies the complexity of the tracking task, and the incorporation of pre-control accelerates the learning process in the formal control stage. Secondly, considering the disparities between the control inputs and the control update instants, pre-control is decomposed to learn the triggering strategy and control policy, respectively. Meanwhile, the control policy learns optimal control inputs by minimizing the cost function. Similarly, the triggering strategy and control policy are also separately learned in accurate control. In contrast to traditional aperiodic triggering mechanisms, in accurate control stage, the triggering strategy is developed based on the cumulative error of the pre-control result, thus avoiding the influence of accidental factors and the cumulative effects of errors, enhancing system robustness. Thirdly, the proposed tracking control is derived by using only the input, output, and reference signal data from the system, without relying on system dynamics. It is applicable to both linear and nonlinear systems, demonstrating strong generalizability. Finally, simulation examples are provided to validate the effectiveness and superiority of the proposed method.

A linearized BDF2 virtual element method for the unsteady Brinkman–Forchheimer equations with variable time step

This paper is devoted to a lowest order virtual element method for the unsteady incompressible Brinkman–Forchheimer equations on polygonal mesh, while a linearized variable-time-step second order backward differentiation formula is adopted in time. We utilize discrete orthogonal convolution and discrete complementary convolution kernels to obtain error estimates for the solutions of time-discrete system. By virtue of the temporal–spatial error splitting approach, and under the current mildest adjacent time-step ratio condition: 0<rk≔τk/τk−1≤rmax≈4.8645, we have established the L∞ boundedness of the fully discrete velocity solution and its L2-norm optimal error estimate. Numerical experiments are conducted on various polygonal meshes, validating the accuracy of the theoretical analysis.

Snapback Repellers, Computational Chaos and Extreme Multistability in Discrete-Time Memristor Murali–Lakshmanan–Chua Circuit

Discretization schemes such as Euler method and Runge–Kutta techniques are extensively used to find approximate solutions of Continuous-Time (CT) dynamical system. While the approximation is good for small discretization step sizes, as pointed out by Lorenz, when the step size increases, computational chaos and computational instability are frequently observed, the former phenomenon being a precursor to the latter. By computational instability, it is meant that there is a blow up of trajectories for the Discrete-Time (DT) system. Computational chaos instead means that for certain step sizes, the DT system displays chaos while the CT counterpart is not chaotic. This paper studies the dynamics of a class of second-order maps obtained via the discretization of a Memristor Murali–Lakshmanan–Chua Circuit (MMLCC). The discretization, which is based on the DT Flux–Charge Analysis Method (FCAM), guarantees that the first integrals of a CT-MMLCC are preserved exactly for the DT system. Hence the dynamics of DT-MMLCC evolves on invariant manifolds and it is characterized by the coexistence of infinitely many different attractors (extreme multistability). The paper uses analytic techniques introduced by Marotto, based on the concept of snapback repellers and transverse homoclinic orbits, to study the chaotic behaviors of the maps. Regions of the parameter space are singled out where there exist snapback repellers for DT-MMLCC, thus implying that the maps display chaos in the Marotto and in the Li–Yorke sense. Since the corresponding CT-MMLCC does not display chaos, the observed chaos of DT-MMLCC is genuinely a consequence of the discretization scheme used in the paper, i.e. it can be actually considered as computational chaos. It is also verified that computational chaos is a precursor to computational instability of the DT-MMLCC maps. Finally, the paper analyzes the effect on chaos obtained by changing the invariant manifold where the dynamics evolves.

A compact meta-learned neuro-fuzzy technique for noise-robust nonlinear control

Neuro-fuzzy systems show promise for adaptive control but can become complex due to the need to learn many parameters. This paper presents a resilient nonlinear controller that combines a simplified neuro-fuzzy system (Simp_NFS) and simplified neural network (Simp_NN) with only two meta-learnable parameters. This architecture enables fast and stable adaptation in uncertain nonlinear discrete-time systems. Simp_NFS utilizes interpretable hyperplane-based rules without antecedent parameters, simplifying the learning process to consequent weights. Simp_NN reduces complexity by replacing hidden-output weights with their mean. The hybrid auto-adaptive controller (HAC) combines the advantages of Simp_NFS and Simp_NN, significantly reducing the number of adaptive parameters compared to standard neuro-fuzzy methods for real-time control with limited resources. Simp_NFS provides structural adaptivity to handle uncertainties, while Simp_NN ensures reliable disturbance attenuation. The stability of HAC is proven using Lyapunov analysis. Extensive testing on challenging single-input single-output (SISO) and multi-input multi-output systems (MIMO) demonstrates that HAC improves performance by up to 82.55% compared to existing techniques. Key innovations include an ultra-compact meta-learned architecture, transparent online evolution of hyperplane clusters, and enhanced modeling capability for nonlinear uncertain systems. This interpretable neuro-fuzzy approach could enhance autonomy and safety by maintaining model transparency. The implementation of HAC is publicly available on GitHub at https://github.com/m-ferdaus/HAC.

Formation control of high-order linear discrete-time multiple unmanned aerial vehicles systems in noisy environments

Formation control of high-order linear discrete-time multiple unmanned aerial vehicles (multi-UAVs) systems with multiplicative noises is studied in this article. Because of the existence of multiplicative noises, classical methods for discrete-time system control problems should be improved. Here, we develop a new discrete-time formation protocol for multi-UAVs systems. The stochastic Lyapunov stability theory and the positive definiteness theory of a generalized algebraic Riccati equation are then used to derive sufficient conditions for formation feasibility.

Novel generalized policy iteration for efficient evolving control of nonlinear systems

In this article, we construct a novel generalized policy iteration framework to address optimal regulation problems for discrete-time nonlinear systems in a more efficient way. Relevant properties are investigated for the framework, including monotonicity and convergence of the iterative value function sequence as well as the admissibility of the iterative control policy. Additionally, an innovative approach is developed to seek an initial admissible control policy for the framework with an adjustable searching speed. Based on these, an evolving control algorithm is presented with stability guarantee. This algorithm employs iterative control policies for system control during the computation of the optimal control policy, as opposed to waiting for the generation of the optimal control policy before implementing control. Eventually, two simulation experiments are conducted with real-world physical backgrounds, in order to illustrate the performance of the proposed strategy.

Positive semi-definite Lyapunov function-based adaptive control for nonlinear discrete-time systems with application to chaotic Duffing oscillator

Positive semi-definite Lyapunov function-based adaptive control for nonlinear discrete-time systems with application to chaotic Duffing oscillator

Learning-based robust output tracking control for unknown discrete-time nonlinear systems with dynamic uncertainty

Although numerous approximate dynamic programming (ADP) methods have been proposed and applied to robust control problems, research on the use of ADP methods for robust tracking control in nonlinear systems remains limited, especially for discrete time (DT) nonlinear systems. In this paper, we propose an efficient robust learning-based output tracking control method, referred to as r-LTC, for a class of unknown DT nonlinear systems with dynamic uncertainty. The objectives are achieved by: 1) using a system identification technique based on network-type coefficients Auto-Regressive with eXogenous variables (ARX) model to identify completely unknown systems; 2) reformulating the identified model into a state space model in terms of deviations from the reference trajectory, such that the robust tracking problem can be converted into a robust regulation problem; 3) transforming the robust regulation problem into an equivalent optimal regulation problem for a nominal system with a modified utility function; 4) alternately implementing the performance evaluation and control policy update through two neural networks (NNs) to numerically solve DT Hamilton-Jacobi-Bellman (HJB) equation, such that the approximated optimal feedback control inputs for nominal system can be obtained, allowing for realization of all unknown bounded uncertainties. The proposed method does not require a state observer and any steady state knowledge, only the measurable system input and output data are utilized. The convergence of NNs’ approximate weights and stability of the closed-loop uncertain systems are rigorously proven using the Lyapunov method. Lastly, a case study is performed using real data collected from the Shenzhen Metro Line 12 tunneling project to examine the performance of the proposed r-LTC algorithm in terms of the set value tracking accuracy, control system stability, and robustness against different types of disturbances.

GA based order abatement technique for linear dynamic systems for continuous time system and discrete time system

GA based order abatement technique for linear dynamic systems for continuous time system and discrete time system

A novel control approach for discrete-time 2D Roesser systems under input saturation and intrinsic nonlinearity: A region of stability investigation

This paper presents the control system design for the nonlinear discrete-time Roesser systems under saturated control input. A deficiency in the existing literature has been addressed in this study for the control of two-dimensional (2D) systems. A generalized boundary condition analysis is presented in the form of tractable invariant sets for the derivation of region of stability to design a suitable state feedback control. A mechanism for the evaluation of controller parameters is presented by considering the generalized structure of 2D Lyapunov function. The work has been extended when external disturbances are present in the 2D systems, and a robust design condition by accounting the perturbations as well as the input saturation has been established. In comparison to the existing works, a control system design for 2D Roesser systems under input saturation and intrinsic nonlinearity by application of a generalized Lyapunov function has been revealed. Furthermore, a tractable 2D region of stability for the local control of the 2D systems is achieved, rather than the conventional global controllers. Furthermore, a method has also been studied for obtainment of the controller gain matrix through convex optimization regarding the complete structure of the 2D Lyapunov function. Simulation results are provided for the stabilization of an unstable nonlinear 2D system under input saturation.

Robust Invariance Conditions of Uncertain Linear Discrete Time Systems Based on Semidefinite Programming Duality

This article proposes a novel robust invariance condition for uncertain linear discrete-time systems with state and control constraints, utilizing a method of semidefinite programming duality. The approach involves approximating the robust invariant set for these systems by tackling the dual problem associated with semidefinite programming. Central to this method is the formulation of a dual programming through the application of adjoint mapping. From the standpoint of semidefinite programming dual optimization, the paper presents a novel linear matrix inequality (LMI) conditions pertinent to robust positive invariance. Illustrative examples are incorporated to elucidate the findings.

Structure-preserving model reduction based on multiorder Arnoldi for discrete-time second-order systems

This paper investigates the new structure-preserving model reduction methods based on multiorder Arnoldi for discrete-time second-order systems by using discrete Laguerre polynomials. The expansion coefficients of discrete-time second-order system and its equivalent first-order system satisfy the implicit recursive expressions in space spanned by discrete Laguerre polynomials, and then multiorder Arnoldi algorithm is applied to the implicit recursive expressions to produce the column-orthogonal matrices, where the orthogonal projection matrix for the equivalent first-order system is constructed by partitioning the resulting column-orthogonal matrix appropriately. The reduced-order systems can preserve the special structures of the original systems as well as match a desired number of expansion coefficients of the original outputs. The error estimate of output variables is also given for discrete-time second-order system. Two benchmark examples are simulated to illustrate the effectiveness of the proposed methods.

Explainable data-driven Q-learning control for a class of discrete-time linear autonomous systems

Explaining what a reinforcement learning (RL) control agent learns play a crucial role in the safety critical control domain. Most of the approaches in the state-of-the-art focused on imitation learning methods that uncover the hidden reward function of a given control policy. However, these approaches do not uncover what the RL agent learns effectively from the agent-environment interaction. The policy learned by the RL agent depends in how good the state transition mapping is inferred from the data. When the state transition mapping is wrongly inferred implies that the RL agent is not learning properly. This can compromise the safety of the surrounding environment and the agent itself. In this paper, we aim to uncover the elements learned by data-driven RL control agents in a special class of discrete-time linear autonomous systems. Here, the approach aims to add a new explainable dimension to data-driven control approaches to increase their trust and safe deployment. We focus on the classical data-driven Q-learning algorithm and propose an explainable Q-learning (XQL) algorithm that can be further expanded to other data-driven RL control agents. Simulation experiments are conducted to observe the effectiveness of the proposed approach under different scenarios using several discrete-time models of autonomous platforms.

Proportional relations between the wave number and amplitude of spiral waves near Neimark-Sacker bifurcations

The spatiotemporal distribution patterns of interacting populations are broadly accepted as a pivotal factor in sustaining species diversity. Spiral waves represent common spatiotemporal patterns observed in ecosystems and biological systems, encompassing both continuous-time and discrete-time systems. The study of the dynamics and regulation of spiral waves in continuous-time systems, often observed in the vicinity of Hopf bifurcations, has been comprehensively examined. However, the dynamical characteristics and rules governing spiral waves near Hopf bifurcations in discrete-time systems, also named Neimark-Sacker bifurcations, are still not fully understood. Here, we investigate spiral waves in a discrete-time predator-pest model caused by a Neimark-Sacker bifurcation. Our results suggest a linear relationship between the amplitude and wave number of spiral waves near the Neimark-Sacker bifurcation. At last, we propose a model that can describe the behaviors of spiral waves in discrete-time systems near Neimark-Sacker bifurcations. Our findings illuminate the process of pattern formation in discrete-time systems, offering potential insights for forecasting and managing pest distribution.

Self-Triggered Impulsive Control for Lyapunov Stability of Nonlinear Systems in Discrete Time.

This article studies Lyapunov stability for nonlinear systems based on discrete-time self-triggered impulsive control (STIC). With the help of comparison approach, some Lyapunov-based sufficient conditions guaranteeing nonZeno behavior and global asymptotic stability of nonlinear systems under STIC are derived. Different from the existing self-triggered mechanisms using implicit expressions to compute the next impulse instant, which may result in computational burden, we propose a novel discrete-time self-triggered mechanism (STM) by which the next impulse instant can be predicted directly by the information of the comparison system. Moreover, the resulting algorithm is easy to be implemented. It is shown that the designed STIC strategy can not only achieve a tradeoff between computational complexity, communication resource usage and control performance, but also has strong robustness and flexibility. Finally, an illustrative simulation is provided showing the effectiveness of the results.

Asynchronous iterative Q-learning based tracking control for nonlinear discrete-time multi-agent systems

This paper addresses the tracking control problem of nonlinear discrete-time multi-agent systems (MASs). First, a local neighborhood error system (LNES) is constructed. Then, a novel tracking algorithm based on asynchronous iterative Q-learning (AIQL) is developed, which can transform the tracking problem into the optimal regulation of LNES. The AIQL-based algorithm has two Q values QiA and QiB for each agent i, where QiA is used for improving the control policy and QiB is used for evaluating the value of the control policy. Moreover, the convergence of LNES is given. It is shown that the LNES converges to 0 and the tracking problem is solved. A neural network-based actor-critic framework is used to implement AIQL. The critic network of AIQL is composed of two neural networks, which are used for approximating QiA and QiB respectively. Finally, simulation results are given to verify the performance of the developed algorithm. It is shown that the AIQL-based tracking algorithm has a lower cost value and faster convergence speed than the IQL-based tracking algorithm.

Convergence and Stability of Optimal Regulation via Generalized N-Step Value Gradient Learning.

In this article, the generalized N -step value gradient learning (GNSVGL) algorithm, which takes a long-term prediction parameter λ into account, is developed for infinite horizon discounted near-optimal control of discrete-time nonlinear systems. The proposed GNSVGL algorithm can accelerate the learning process of adaptive dynamic programming (ADP) and has a better performance by learning from more than one future reward. Compared with the traditional N -step value gradient learning (NSVGL) algorithm with zero initial functions, the proposed GNSVGL algorithm is initialized with positive definite functions. Considering different initial cost functions, the convergence analysis of the value-iteration-based algorithm is provided. The stability condition for the iterative control policy is established to determine the value of the iteration index, under which the control law can make the system asymptotically stable. Under such a condition, if the system is asymptotically stable at the current iteration, then the iterative control laws after this step are guaranteed to be stabilizing. Two critic neural networks and one action network are constructed to approximate the one-return costate function, the λ -return costate function, and the control law, respectively. It is emphasized that one-return and λ -return critic networks are combined to train the action neural network. Finally, via conducting simulation studies and comparisons, the superiority of the developed algorithm is confirmed.

Distributed Dynamic Event-Triggered Leader-Following Consensus for Nonlinear Multiagent Systems Over Fading Channel.

This article investigates the leader-following consensus problem of discrete-time nonlinear multiagent systems (MASs) over fading channels. With the consideration of the transmission among followers maybe affected by the fading networks, the nonidentical fading channels model is constructed. To reduce the transmission network burden, the dynamical event-triggered mechanism (DETM) is developed. Different from most of existing event-triggered strategies, the threshold parameter in the developed dynamical event-triggering condition is dynamically adjusted according to a dynamic rule. Based on the DETM, a distributed consensus control protocol is designed under fading channels. Then, sufficient criteria are provided to ensure that MASs can achieve the leaser-following consensus, and satisfy the H∞ performance index in the presence of fading channels. The desired controller parameters can be derived in terms of solutions of matrix inequalities that are lightly solvable. In the end, simulation results show that the designed dynamical event transmission policy is capable of diminishing communication burden more promptly and effectively than some existing ones.

Encryption–decryption-based distributed fault-tolerant consensus tracking control for multi-agent systems

This study investigates fault-tolerant consensus tracking for discrete-time multi-agent systems (MASs) subject to external eavesdropping threats and additive actuator faults. First, actuator faults are modeled by difference equations, and decentralized observers are constructed to estimate actuator faults as well as system states. To offset fault-induced effects, ensure secure communication, and alleviate communication congestion, neighboring encrypted state information based on the encryption–decryption strategy (EDS) and estimated fault are integrated into a distributed active fault-tolerant consensus tracking control (FCTC) protocol. Through the properties of compatible norms, criteria for the controller, observer, and dynamic encryption key in EDS are derived to achieve leader-following consensus (LFC) of MASs with bias and drift actuator faults. Simulation results confirm the validity of the encryption–decryption-based distributed FCTC strategy.