Discrete-time Integrator Research Articles

A Memory-Efficient Neural Ordinary Differential Equation Framework Based on High-Level Adjoint Differentiation

Neural ordinary differential equations (neural ODEs) have emerged as a novel network architecture that bridges dynamical systems and deep learning. However, the gradient obtained with the continuous adjoint method in the vanilla neural ODE is not reverse-accurate. Other approaches suffer either from an excessive memory requirement due to deep computational graphs or from limited choices for the time integration scheme, hampering their application to large-scale complex dynamical systems. To achieve accurate gradients without compromising memory efficiency and flexibility, we present a new neural ODE framework, PNODE, based on high-level discrete adjoint algorithmic differentiation. By leveraging discrete adjoint time integrators and advanced checkpointing strategies tailored for these integrators, PNODE can provide a balance between memory and computational costs, while computing the gradients consistently and accurately. We provide an open-source implementation based on PyTorch and PETSc, one of the most commonly used portable, scalable scientific computing libraries. We demonstrate the performance through extensive numerical experiments on image classification and continuous normalizing flow problems. We show that PNODE achieves the highest memory efficiency when compared with other reverse-accurate methods. On the image classification problems, PNODE is up to two times faster than the vanilla neural ODE and up to 2.3 times faster than the best existing reverse-accurate method. We also show that PNODE enables the use of the implicit time integration methods that are needed for stiff dynamical systems.

Uncertainty in discrete-time integration — The case of static gas meters

We consider the evaluation of uncertainty in a particular case of discrete-time integration, i.e., that of static gas meters. We show that the current approach, which does not take correlations into account, can lead to underrating of the uncertainty associated with the estimate of the delivered volume. Now, a correct evaluation of uncertainty in the case e.g. of online measurements along large offshore pipelines is important in order to remove technical barriers to international trade.We focus our discussion on the practical example of gas meters, but the framework of uncertainty evaluation we provide is valid in general for all those measurements involving discrete-time integration, and has therefore a wide applicability.

Fully-discrete spatial eigenanalysis of discontinuous spectral element methods: Insights into well-resolved and under-resolved vortical flows

This study presents a comprehensive spatial eigenanalysis of fully-discrete discontinuous spectral element methods, now generalising previous spatial eigenanalysis that did not include time integration errors. The influence of discrete time integration is discussed in detail for different explicit Runge–Kutta (1st to 4th order accurate) schemes combined with either Discontinuous Galerkin (DG) or Spectral Difference (SD) methods, both here recovered from the Flux Reconstruction (FR) scheme. Selected numerical experiments using the improved SD method by Liang et al. (2009) [53,54] and Jameson (2010) [55] are performed to quantify the influence of time integration errors on actual simulations. These involve test cases of varied complexity, from one-dimensional linear advection equation studies to well-resolved and under-resolved inviscid vortical flows. When simulations are well-resolved, the overall order of accuracy of the (fully-discrete) method of choice is limited to that of the time integration scheme. Moreover, it is shown that, while both well-resolved and under-resolved simulations of linear problems correlate well with the eigenanalysis prediction of time integration errors, the correlation can be much worse for under-resolved nonlinear problems as observed via numerical experiments. In fact, in the numerical simulation of under-resolved vortical flows, the predominance of spatial errors made it practically impossible for time integration errors to be distinctly identified. As a result, the eigenanalysis predictions are expected to hold (even if partially) in direct numerical simulations of turbulence. This highlights that the interaction between space and time discretisation errors is more complex than otherwise anticipated, contributing to the current understanding about when eigenanalysis can effectively predict the behaviour of numerical errors in practical under-resolved nonlinear problems, including under-resolved turbulence computations.

The Feature Extraction of Impact Response and Load Reconstruction Based on Impulse Response Theory

Impact load is a kind of aperiodic excitation with a short action time and large amplitude, it had more significant effect on the structure than static load. The reconstruction (or identification namely) of impact load is of great importance for validating the structural strength. The aim of this article was to reconstruct the impact load accurately. An impact load identification method based on impulse response theory (IRT) and BP (Back Propagation) neural network is proposed. The excitation and response signals were transformed to the same length by extracting the peak value (amplitude of sine wave) in the rising oscillation period of the response. First, we deduced that there was an approximate linear relationship between the discrete-time integral of impact load and the amplitude of the oscillation period of the response. Secondly, a BP neural network was used to establish a linear relationship between the discrete-time integral of the impact load and the peak value in the rising oscillation period of the response. Thirdly, the network was trained and verified. The error between the actual maximum amplitude of impact load and the identification value was 2.22%. The error between the actual equivalent impulse and the identification value was 0.67%. The results showed that this method had high accuracy and application potential.

A numerical method for suspensions of articulated bodies in viscous flows

An articulated body is defined as a finite number of rigid bodies connected by a set of arbitrary constraints that limit the relative motion between pairs of bodies. Such a general definition encompasses a wide variety of situations in the microscopic world, from bacteria to synthetic micro-swimmers, but it is also encountered when discretizing inextensible bodies, such as filaments or membranes. In this work we consider hybrid articulated bodies, i.e. constituted of both linear chains, such as filaments, and closed-loop chains, such as membranes. Simulating suspensions of such articulated bodies requires to solve the hydrodynamic interactions between large collections of objects of arbitrary shape while satisfying the multiple constraints that connect them. Two main challenges arise in this task: limiting the cost of the hydrodynamic solves, and enforcing the constraints within machine precision at each time-step. To address these challenges we propose a formalism that combines the body mobility problem in Stokes flow with a velocity formulation of the constraints, resulting in a mixed mobility-resistance problem. While resistance problems are known to scale poorly with the particle number, our preconditioned iterative solver is not sensitive to the system size, therefore allowing to study large suspensions with quasilinear computational cost. Additionally, constraint violations, e.g. due to discrete time-integration errors, are prevented by correcting the particles' positions and orientations at the end of each time-step. Our correction procedure, based on a nonlinear minimisation algorithm, has negligible computational cost and preserves the accuracy of the time-integration scheme. The versatility of our method allows to study a plethora of articulated systems within a unified framework. We showcase its robustness and scalability by exploring the locomotion modes of a model microswimmer inspired by the diatom colony Bacillaria Paxillifer, and by simulating large suspensions of bacteria interacting near a no-slip boundary. Finally, we provide a Python implementation of our framework in a collaborative publicly available code, where the user can prescribe a set of constraints through a single input file to study a wide spectrum of applications involving suspensions of articulated bodies.

Verifying Provable Stability Domains for Discrete-Time Systems Using Ellipsoidal State Enclosures

Stability contractors, based on interval analysis, were introduced in recent work as a tool to verify stability domains for nonlinear dynamic systems. These contractors rely on the property that - in case of provable asymptotic stability - a certain domain in a multi-dimensional state space is mapped into its interior after a certain integration time for continuous-time processes or after a certain number of discretization steps in a discrete-time setting. However, a disadvantage of the use of axis-aligned interval boxes in such computations is the omnipresent wrapping effect. As shown in this contribution, the replacement of classical interval representations by ellipsoidal domain enclosures reduces this undesirable effect. It also helps to find suitable ratios for the edge lengths if interval-based domain representations are investigated. Moreover, ellipsoidal domains naturally represent the possible regions of attraction of asymptotically stable equilibrium points that can be analyzed with the help of quadratic Lyapunov functions, for which stability criteria can be cast into linear matrix inequality (LMI) constraints. For that reason, this paper further presents possible interfaces of ellipsoidal enclosure techniques with LMI approaches. This combination aims at the maximization of those domains that can be proven to be stable for a discrete-time range-only localization algorithm in robotics. There, an Extended Kalman Filter (EKF) is applied to a system for which the dynamics are characterized by a discrete-time integrator disturbance model with additive Gaussian noise. In this scenario, the measurement equations correspond to the distances between the object to be localized and beacons with known positions.

Berry-Esseen bounds for functionals of independent random variables

We derive Berry-Esseen approximation bounds for general functionals of independent random variables, based on a continuous-time integration by parts setting and discrete chaos expansions methods. Our approach improves on related results obtained in discrete-time integration by parts settings and applies to U-statistics satisfying the weak assumption of decomposability in the Hoeffding sense, and yield Kolmogorov distance bounds instead of the Wasserstein bounds previously derived in the special case of degenerate U-statistics. Linear and quadratic functionals of arbitrary sequences of independent random variables are included as particular cases, with new fourth moment bounds, and applications are given to Hoeffding decompositions, weighted U-statistics, quadratic forms, and random subgraph weighing. In the case of quadratic forms, our results recover and improve the bounds available in the literature, and apply to matrices with non-empty diagonals.

Fully-Discrete Spatial Eigenanalysis of Discontinuous Spectral Element Methods: Insights into Well-Resolved and Under-Resolved Vortical Flows

This study presents a comprehensive spatial eigenanalysis of fully-discrete discontinuous spectral element methods, now generalizing previous spatial eigenanalysis that did not include time integration errors. The influence of discrete time integration is discussed in detail for different explicit Runge-Kutta (1st to 4th order accurate) schemes combined with either Discontinuous Galerkin (DG) or Spectral Difference (SD) methods, both here recovered from the Flux Reconstruction (FR) scheme. Selected numerical experiments using the improved SD method by Liang and Jameson [1] are performed to quantify the influence of time integration errors on actual simulations. These involve test cases of varied complexity, from one-dimensional linear advection equation studies to well-resolved and under-resolved inviscid vortical flows. It is shown that, while both well-resolved and under-resolved simulations of linear problems correlate well with the eigenanalysis prediction of time integration errors, the correlation can be much worse for under-resolved nonlinear problems. The effect of mesh regularity is also considered, where time integration errors are found to be, in the case of irregular grids, less pronounced than those of the spatial discretisation. In fact, for the under-resolved vortical flows considered, the predominance of spatial errors made it practically impossible for time integration errors to be distinctly identified. Nevertheless, for well-resolved nonlinear simulations, the effect of time integration errors could still be recognized. This highlights that the interaction between space and time discretisation errors is more complex than otherwise anticipated, contributing to the current understanding about when eigenanalysis can effectively predict the behaviour of numerical errors in practical under-resolved nonlinear problems, including under-resolved turbulence computations.

Dynamic Event-triggered Control in Discrete-time Multi-agent Consensus over Directed Topologies

In this paper, we study the problem of multi-agent consensus in networks of discrete-time integrator agents with directed topologies. With the aim to reduce the frequency of interaction among agents for saving energy expenses, we develop three triggering-based protocols for strongly connected agent networks. Our common strategy is to use triggering protocols that take account of the state values received from neighbors in determining the event times. The first two protocols are event-triggering type, where the thresholds are state dependent so that when the agents are far from consensus, the events take place less frequently. The two protocols are called static and dynamic depending on the use of auxiliary variables in determining the thresholds. The third protocol is self-triggering type, where at each triggering time, an agent computes the time for its next transmission, which is communicated to its neighbors together with its state data. For these protocols, we present Lyapunov-based design methods, and demonstrate their efficacy through numerical examples.

Dynamic output-feedback control of a chain of discrete-time integrators with arbitrary zeros and asymmetric input saturation

For a chain of discrete-time integrators with arbitrary zeros and asymmetric input saturation, this paper develops globally asymptotically stabilizing full-state-feedback and dynamic-output-feedback controllers. These controllers extend prior results on nested-saturation controllers for discrete-time chains of integrators, and they are applied to setpoint command following and rejection of constant disturbances.

67 dB SNDR 20 kHz BW SC third‐order modulator with single Op‐Amp and 20 µW power consumption for bio‐medical applications

This paper presents a single loop 3rd order discrete-time Sigma–Delta Modulator ( Σ Δ M ) using a single operational amplifier (Op-Amp). The proposed Σ Δ M utilises delay-based discrete-time integrators and a single Op-Amp is used to implement all the integrators of the proposed design. The delay-based design helps in relaxing the settling time requirement of an integrator, also the error due to slewing in the output of the integrator is avoided. A switched capacitor Op-Amp based proposed design is simulated in standard 180~nm CMOS technology with a resolution of 10.75 bits. The proposed Σ Δ M achieves a signal-to-noise-distortion ratio (SNDR) of 66.48 dB and dynamic range (DR) of 70 dB. An input signal of −4 dBFS magnitude at a frequency of 2.125 kHz is sampled with a sampling frequency of 1.024 MHz. The proposed circuit is designed for a bandwidth (BW) of 20 kHz with an over-sampling-ratio (OSR) of 25.6. The proposed modulator achieves a figure-of-merit (FOMS/FOMW) of 156.48 dB/0.290 (pJ/conversion-step) with the total power consumption of 20 µW at 1.8 V supply voltage (VDD). A Monte-Carlo simulation for the SNDR of the proposed design is done and the achieved mean and standard deviation (SD) is 66.09 dB and 1.025 respectively for 1000 samples.

36 μW fourth order sigma-delta modulator using single operational amplifier

ABSTRACT This paper presents a single loop fourth-order Discrete-Time Sigma-Delta Modulator (ΣΔM) using single Operational Amplifier (Op-Amp). Proposed ΣΔM utilises delay-based discrete-time integrators and an Op-Amp to implement all the integrators. Op-Amp reuse helps in saving power. A switched-capacitor based proposed design is simulated in standard 180 nm CMOS technology with an effective resolution of 12.04 bits. The proposed ΣΔM achieves a Signal to Noise Distortion Ratio (SNDR) of 74.24 dB with 78 dB of Dynamic Range (DR). An input signal of −4 dBFS magnitudes and 1.125 kHz of frequency is sampled at a sampling frequency of 320 kHz. The proposed circuit is designed for 10 kHz Bandwidth (BW) with Oversampling Ratio (OSR) of 16. The proposed modulator achieves Figure-of-Merit (FOM) of 158.68 dB/0.427 (pJ/conversion-step) with a total power consumption of 36 µW at 1.8 V supply voltage. The layout area of the proposed circuit is 134 µm ×134 µm.

A Passivity-Based Approach for Simulating Satellite Dynamics With Robots: Discrete-Time Integration and Time-Delay Compensation

This article proposes a passivity-based approach for simulating satellite dynamics on a position-controlled robot equipped with a force–torque sensor. Time delays intrinsic in the computational loop and discrete-time integration degrade the behavior of the satellite dynamics reproduced by the robot. These factors can generate an energy-inconsistent simulation and can even render the system unstable. In this article, time delay and discrete-time integration effects are analyzed from an energetic perspective and compensated through a passivity-based control strategy to ensure a faithful and stable dynamic simulation with position-controlled robots. The benefit of the proposed strategy is validated by simulations and experiments on the On-Orbit Servicing Simulator (OOS-SIM), a robotic facility used for simulating free-floating dynamics.

Robust quantized consensus of discrete multi-agent systems under input saturation

In this paper, we consider the quantized consensus problem of multiple discrete-time integrator agents which suffer from input saturation. As agents transmit state information through communication networks with limited bandwidth, the states of agents have to be quantized into a finite number of bits before transmission. To handle this quantized consensus problem, we introduce an internal time-varying saturation function into the controllers of all agents and ensure that the range of the state of each agent can be known in advance by its neighboring agents. Based on such shared state range information, we construct a quantized consensus protocol which implements a finite-bit quantization strategy to all states of agents and can guarantee the achievement of the asymptotic consensus under any given input saturation threshold. Such desired consensus can be guaranteed at as low bit rate as 1 bit per time step for each agent. Moreover, we can place an upper bound on the convergence rate of the consensus error of agents. We further improve that quantized consensus protocol to a robust version whose parameters are determined with only an upper bound on the number of agents and does not require any more global information of the inter-agent network. Simulations are done to confirm the effectiveness of our quantized consensus protocols.

Robust Quantized Consensus of Discrete Multi-Agent Systems in the Input-to-State Sense

In this paper, we consider the quantized consensus problem of multiple discrete-time integrator agents which suffer from additive noise. Due to the limited communication resources, each agent can only exchange quantized information of finite length with its neighbors through unreliable channels, where the transmitted signals may be delayed or even lost. Quantization errors, network delay, and packet loss result in quantization mismatch. To handle the additive noise and quantization mismatch, a quantized consensus protocol is proposed by implementing dynamic encoding and decoding policies at a finite bit rate. Particularly, the proposed consensus protocol introduces an internal saturation function into all the controllers of agents so that the control inputs of neighbor agents can be predicted locally by each agent even under distributed control law and quantization mismatch. Based on such shared control input information, the proposed consensus protocol can guarantee the achievement of the approximate consensus of all agents in the input-to-state sense, i.e., the consensus errors of agents are bounded from above by the sum of a class $\mathcal {K}_\infty $ function of the upper bound of the additive noise and a class $\mathcal {KL}$ function of the upper bound on the initial states of agents. It is shown that such approximate consensus in the input-to-state sense can be guaranteed at an as low bit rate as 1 bit per time step for each agent without requiring any global information, except an upper bound of the number of agents. Two specific situations, namely packet loss, and network delay, are further analyzed with the explicit expressions of quantization mismatch. The simulations are done to confirm the effectiveness of the proposed quantized consensus protocol.

A robust quantized consensus protocol for discrete-time multi-agent systems with additive noise

This paper investigates the consensus problem of multiple discrete-time integrator agents with communication constraints and additive process noise. It proposes a protocol to achieve the approximate consensus of agents over inter-agent communication networks with finite bit rates. Under that protocol, dynamic encoding and decoding algorithms are implemented for each pair of neighboring agents to transmit quantized states at a finite bit rate. With received quantized states of neighboring agents, the control input of each agent is locally computed. Particularly input saturation is introduced into the local controllers of agents and places both lower and upper bounds on the local control inputs of agents. These control input bounds can be known in advance and greatly enhance the robustness of the consensus protocol. Under the proposed protocol, the approximate consensus can be guaranteed at any finite bit rate to encode the states of agents. It is shown that even a single bit per time step is enough for the desired approximate consensus. The additive process noise does not increase the bit rate required for that approximate consensus. Moreover, the proposed consensus protocol can be designed with only an upper bound on the number of agents and is more robust than some previous consensus protocols which may require the global information of the inter-agent network topology, such as the second largest eigenvalue of the Laplacian matrix. Even when some communication links are broken due to communication failure or some nodes leave, the same set of consensus parameters can still robustly guarantee the expected approximate consensus. Simulations are conducted to illustrate the effectiveness of the proposed quantized consensus protocol.

A Systematic Controller Design for a Grid-Connected Inverter with LCL Filter Using a Discrete-Time Integral State Feedback Control and State Observer

Inductive-capacitive-inductive (LCL)-type filters are currently preferred as a replacement for L-type filters in distributed generation (DG) power systems, due to their superior harmonic attenuation capability. However, the third-order dynamics introduced by LCL filters pose a challenge to design a satisfactory controller for such a system. Conventionally, an LCL-filtered grid-connected inverter can be effectively controlled by using a full-state feedback control. However, this control approach requires the measurement of all system state variables, which brings about more complexity for the inverter system. To address this issue, this paper presents a systematic procedure to design an observer-based integral state feedback control for a LCL-filtered grid-connected inverter in the discrete-time domain. The proposed control scheme consists of an integral state feedback controller and a full-state observer which uses the control input, grid-side currents, and grid voltages to predict all the system state variables. Therefore, only the grid-side current sensors and grid voltage sensors are required to implement the proposed control scheme. Due to the discrete-time integrator incorporated in the state feedback controller, the proposed control scheme ensures both the reference tracking and disturbance rejection performance of the inverter system in a practical and simple way. As a result, superior control performance can be achieved by using the reduced number of sensors, which significantly reduces the cost and complexity of the LCL-filtered grid-connected inverter system in DG applications. To verify the practical usefulness of the proposed control scheme, a 2 kW three-phase prototype grid-connected inverter has been constructed, and the proposed control system has been implemented based on 32-bit floating-point digital signal processor (DSP) TMS320F28335. The effectiveness of the proposed scheme is demonstrated through the comprehensive simulation and experimental results.

Time Integration of Rank-Constrained Tucker Tensors

Dynamical low-rank approximation in the Tucker tensor format of given large time-dependent tensors and of tensor differential equations is the subject of this paper. In particular, a discrete time integration method for rank-constrained Tucker tensors is presented and analyzed. It extends the known projector-splitting integrator for dynamical low-rank approximation of matrices to Tucker tensors and is shown to inherit the same favorable properties. The integrator is based on iteratively applying the matrix projector-splitting integrator to tensor unfoldings but with inexact solution in a substep. It has the property that it reconstructs time-dependent Tucker tensors of the given rank exactly. The integrator is also shown to be robust to the presence of small singular values in the tensor unfoldings. Numerical examples with time-dependent problems from quantum physics and tensor optimization methods illustrate our theoretical results.

Event-Triggered Sliding-Mode Control for Multi-Area Power Systems

This paper solves the load frequency control problem and proposes a new event-triggered sliding-mode control (SMC) algorithm in multi-area interconnected power systems. First, we consider a three-area interconnected power system, and construct its corresponding mathematical model. The prescribed H <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</sub> , or energy-to-energy, performance is utilized to test the effectiveness of the corresponding disturbance attenuation. Then, an appropriate discrete-time integral switching surface is designed for each subsystem, and the discontinuous controller is designed to guarantee the resulting subsystem asymptotically stable and robust. Using time-delay system analysis technique, the event-triggered scheme, sliding mode dynamics, and the network-induced delays are converted into a network dynamic system with time-varying delays. Thus, corresponding sufficient conditions are derived and an event-triggered SMC law for reaching motion is designed to ensure the reachability of the sliding mode surface in a finite time. Finally, examples and simulations are provided to illustrate the feasibility and the effectiveness of the developed event-triggered SMC scheme.

Discrete Time Approximation of Continuous Time Nonlinear State Space Models

In this paper we discuss the problem of approximating a continuous time nonlinear state space model with an explicit discrete time state space model. Two steps are made to do so. The first one is to replace the classical zero-order-hold (ZOH) excitation signal assumption by a more general concept of low pass (LP) signals. It will be shown that opposed to the ZOH-property, the LP-property is maintained for a wide class of (non)linear systems, including cascaded and closed loop systems. The second step is based on a result from the signal processing theory that states that a BL-signal can be exactly predicted from its past samples by a fixed causal linear filter. This result is generalized to the discrete time integration of LP-signals by introducing an error bound that can be made arbitrarily small by increasing the sample frequency. Both ideas are then combined to bound the approximation errors of an explicit discrete time nonlinear state space approximation for a continuous time nonlinear state space model. The order of the decay of the approximation error as a function of the sample frequency is given. It is shown that eventually the aliasing error is the dominating error, at a cost of a (slightly) increased model complexity. These results are directly applicable to nonlinear system identification, and will be experimentally verified on the identification of a closed loop nonlinear system (the silverbox) using a discrete time nonlinear state space model.