Second-order Dynamical Systems Research Articles

Fast convergence of the primal-dual dynamical system and corresponding algorithms for a nonsmooth bilinearly coupled saddle point problem

AbstractThis paper studies the convergence rate of a second-order dynamical system associated with a nonsmooth bilinearly coupled convex-concave saddle point problem, as well as the convergence rate of its corresponding discretizations. We derive the convergence rate of the primal-dual gap for the second-order dynamical system with asymptotically vanishing damping term. Based on an implicit discretization scheme, we propose a primal-dual algorithm and provide a non-ergodic convergence rate under a general setting for the inertial parameters when one objective function is continuously differentiable and convex and the other is a proper, convex and lower semicontinuous function. For this algorithm we derive a $$O\left( 1/k^2 \right) $$ O 1 / k 2 convergence rate under three classical rules proposed by Nesterov, Chambolle-Dossal and Attouch-Cabot without assuming strong convexity, which is compatible with the results of the continuous-time dynamic system. For the case when both objective functions are continuously differentiable and convex, we further present a primal-dual algorithm based on an explicit discretization. We provide a corresponding non-ergodic convergence rate for this algorithm and show that the sequence of iterates generated weakly converges to a primal-dual optimal solution. Finally, we present numerical experiments that indicate the superior numerical performance of both algorithms.

Exponential series approximation of the SIR epidemiological model

IntroductionThe SIR (Susceptible-Infected-Recovered) model is one of the simplest and most widely used frameworks for understanding epidemic outbreaks.MethodsA second-order dynamical system for the R variable is formulated using an infinite exponential series expansion, and a recursion relation is established between the series coefficients. A numerical approximation scheme for the R variable is also developed.ResultsThe proposed numerical method is compared to a double exponential (DE) nonlinear approximate analytic solution, which reveals two coupled timescales: a relaxation timescale, determined by the ratio of the model’s time constants, and an excitation timescale, dictated by the population size. The DE solution is applied to estimate model parameters for a well-known epidemiological dataset—the boarding school flu outbreak.DiscussionFrom a theoretical standpoint, the primary contribution of this work is the derivation of an infinite exponential, Dirichlet, series for the model variables. Truncating the series yields a finite approximation, known as a Prony series, which can be interpreted as a sequence of coupled exponential relaxation processes, each with a distinct timescale. This apparent complexity can be approximated well by the DE solution, which appears to be of main practical interest.

Tikhonov regularized second-order plus first-order primal-dual dynamical systems with asymptotically vanishing damping for linear equality constrained convex optimization problems

In this paper, in the setting of Hilbert spaces, we consider a Tikhonov regularized second-order plus first-order primal-dual dynamical system with asymptotically vanishing damping for a linear equality constrained convex optimization problem. It is shown that convergence properties of the proposed inertial dynamical system depend upon the choice of the Tikhonov regularization parameter. When the Tikhonov regularization parameter decays rapidly to zero, we derive the fast convergence rates of the primal-dual gap, the objective residual, the feasibility violation and the gradient norm of the objective function along the generated trajectory. When the Tikhonov regularization parameter decreases slowly to zero, we prove the strong convergence of the primal trajectory of the Tikhonov regularized dynamical system to the minimal norm solution of the linear equality constrained convex optimization problem. Numerical experiments are performed to illustrate the efficiency of our approach.

A Second-Order Continuous-Time Dynamical System for Solving Sparse Image Restoration Problems

A Second-Order Continuous-Time Dynamical System for Solving Sparse Image Restoration Problems

Nonautonomous spectral submanifolds for model reduction of nonlinear mechanical systems under parametric resonance.

We use the recent theory of spectral submanifolds (SSMs) for model reduction of nonlinear mechanical systems subject to parametric excitations. Specifically, we develop expressions for higher-order nonautonomous terms in the parameterization of SSMs and their reduced dynamics. We provide these results for both general first-order and second-order mechanical systems under periodic and quasiperiodic excitation using a multi-index based approach, thereby optimizing memory requirements and the computational procedure. We further provide theoretical results that simplify the SSM parametrization for general second-order dynamical systems. More practically, we show how the reduced dynamics on the SSM can be used to extract the resonance tongues and the forced response around the principal resonances in parametrically excited systems. In the case of two-dimensional SSMs, we formulate explicit expressions for computing the steady-state response as the zero-level set of a two-dimensional function for systems that are subject to external as well as parametric excitation. This allows us to parallelize the computation of the forced response over the range of excitation frequencies. We demonstrate our results on several examples of varying complexity, including finite-element-type examples of mechanical systems. Furthermore, we provide an open-source implementation of all these results in the software package SSMTool.

Time-Varying Optimal Formation Control for Second-Order Multiagent Systems Based on Neural Network Observer and Reinforcement Learning.

This article addresses a distributed time-varying optimal formation protocol for a class of second-order uncertain nonlinear dynamic multiagent systems (MASs) based on an adaptive neural network (NN) state observer through the backstepping method and simplified reinforcement learning (RL). Each follower agent is subjected to only local information and measurable partial states due to actual sensor limitations. In view of the distributed optimized formation strategic needs, the uncertain nonlinear dynamics and undetectable states may jointly affect the stability of the time-varying cooperative formation control. Furthermore, focusing on Hamilton-Jacobi-Bellman optimization, it is almost incapable of directly dealing with unknown equations. Above uncertainty and immeasurability processed by adaptive state observer and NN simplified RL are further designed to achieve desired second-order formation configuration at the least cost. The optimization protocol can not only solve the undetectable states and realize the prescribed time-varying formation performance on the premise that all the errors are SGUUB, but also prove the stability and update the critics and actors easily. Through the above-mentioned approaches offer an optimal control scheme to address time-varying formation control. Finally, the validity of the theoretical method is proven by the Lyapunov stability theory and digital simulation.

Output Feedback Dynamical Compensation via EigenStructure Assignment for Active Magnetic Bearings-Rigid Rotor Systems

A rotor system supported by active magnetic bearings (AMBs) is actually a second-order dynamical system, whose dynamical characteristics should be considered in the design process of the controller. The relationships between the dynamical parameters and the control parameters can be established by the eigenstructure assignment. Output feedback dynamical compensators (OFDCs) are proposed based on the eigenstructure assignment for an AMBs-rigid rotor system in this paper. First, the model of the four-degree-of-freedom (4-DOF) AMBs-rigid rotor system was established, and it was decoupled into two single-degree-of-freedom (S-DOF) translational sub-systems and two S-DOF rotational sub- systems through the modal decoupling method. Aiming to the control of the four S-DOF systems, the relationships among the parameters of the OFDCs, the eigenvalues, and the eigenvectors were derived based on the eigenstructure assignment. Then, a multi-objective optimization problem was constructed in which the system stability, the robustness to the bias current perturbation, and the maximum amplitudes of the unbalance displacement and control current were considered. Several Pareto solutions were obtained through NSGA-II, and different solutions were selected to compare the performance of the corresponding systems. The simulation and experimental results verify the effectiveness of the proposed OFDCs and the rationality of the proposed design process.

Brain-Inspired Reservoir Computing Using Memristors with Tunable Dynamics and Short-Term Plasticity.

Recent advancements in reservoir computing (RC) research have created a demand for analogue devices with dynamics that can facilitate the physical implementation of reservoirs, promising faster information processing while consuming less energy and occupying a smaller area footprint. Studies have demonstrated that dynamic memristors, with nonlinear and short-term memory dynamics, are excellent candidates as information-processing devices or reservoirs for temporal classification and prediction tasks. Previous implementations relied on nominally identical memristors that applied the same nonlinear transformation to the input data, which is not enough to achieve a rich state space. To address this limitation, researchers either diversified the data encoding across multiple memristors or harnessed the stochastic device-to-device variability among the memristors. However, this approach requires additional preprocessing steps and leads to synchronization issues. Instead, it is preferable to encode the data once and pass them through a reservoir layer consisting of memristors with distinct dynamics. Here, we demonstrate that ion-channel-based memristors with voltage-dependent dynamics can be controllably and predictively tuned through the voltage or adjustment of the ion channel concentration to exhibit diverse dynamic properties. We show, through experiments and simulations, that reservoir layers constructed with a small number of distinct memristors exhibit significantly higher predictive and classification accuracies with a single data encoding. We found that for a second-order nonlinear dynamical system prediction task, the varied memristor reservoir experimentally achieved an impressive normalized mean square error of 1.5 × 10-3, using only five distinct memristors. Moreover, in a neural activity classification task, a reservoir of just three distinct memristors experimentally attained an accuracy of 96.5%. This work lays the foundation for next-generation physical RC systems that can exploit the complex dynamics of their diverse building blocks to achieve increased signal processing capabilities.

Experimental Realization of Stable Exceptional Chains Protected by Non-Hermitian Latent Symmetries Unique to Mechanical Systems.

Lines of exceptional points are robust in the three-dimensional non-Hermitian parameter space without requiring any symmetry. However, when more elaborate exceptional structures are considered, the role of symmetry becomes critical. One such case is the exceptional chain (EC), which is formed by the intersection or osculation of multiple exceptional lines (ELs). In this Letter, we investigate a non-Hermitian classical mechanical system and reveal that a symmetry intrinsic to second-order dynamical equations, in combination with the source-free principle of ELs, guarantees the emergence of ECs. This symmetry can be understood as a non-Hermitian generalized latent symmetry, which is absent in prevailing formalisms rooted in first-order Schrödinger-like equations and has largely been overlooked so far. We experimentally confirm and characterize the ECs using an active mechanical oscillator system. Moreover, by measuring eigenvalue braiding around the ELs meeting at a chain point, we demonstrate the source-free principle of directed ELs that underlies the mechanism for EC formation. Our Letter not only enriches the diversity of non-Hermitian exceptional point configurations, but also highlights the new potential for non-Hermitian physics in second-order dynamical systems.

Fast Optimistic Gradient Descent Ascent (OGDA) Method in Continuous and Discrete Time

AbstractIn the framework of real Hilbert spaces, we study continuous in time dynamics as well as numerical algorithms for the problem of approaching the set of zeros of a single-valued monotone and continuous operator V. The starting point of our investigations is a second-order dynamical system that combines a vanishing damping term with the time derivative of V along the trajectory, which can be seen as an analogous of the Hessian-driven damping in case the operator is originating from a potential. Our method exhibits fast convergence rates of order $$o \left( \frac{1}{t\beta (t)} \right) $$ o 1 t β ( t ) for $$\Vert V(z(t))\Vert $$ ‖ V ( z ( t ) ) ‖ , where $$z(\cdot )$$ z ( · ) denotes the generated trajectory and $$\beta (\cdot )$$ β ( · ) is a positive nondecreasing function satisfying a growth condition, and also for the restricted gap function, which is a measure of optimality for variational inequalities. We also prove the weak convergence of the trajectory to a zero of V. Temporal discretizations of the dynamical system generate implicit and explicit numerical algorithms, which can be both seen as accelerated versions of the Optimistic Gradient Descent Ascent (OGDA) method for monotone operators, for which we prove that the generated sequence of iterates $$(z_k)_{k \ge 0}$$ ( z k ) k ≥ 0 shares the asymptotic features of the continuous dynamics. In particular we show for the implicit numerical algorithm convergence rates of order $$o \left( \frac{1}{k\beta _k} \right) $$ o 1 k β k for $$\Vert V(z^k)\Vert $$ ‖ V ( z k ) ‖ and the restricted gap function, where $$(\beta _k)_{k \ge 0}$$ ( β k ) k ≥ 0 is a positive nondecreasing sequence satisfying a growth condition. For the explicit numerical algorithm, we show by additionally assuming that the operator V is Lipschitz continuous convergence rates of order $$o \left( \frac{1}{k} \right) $$ o 1 k for $$\Vert V(z^k)\Vert $$ ‖ V ( z k ) ‖ and the restricted gap function. All convergence rate statements are last iterate convergence results; in addition to these, we prove for both algorithms the convergence of the iterates to a zero of V. To our knowledge, our study exhibits the best-known convergence rate results for monotone equations. Numerical experiments indicate the overwhelming superiority of our explicit numerical algorithm over other methods designed to solve monotone equations governed by monotone and Lipschitz continuous operators.

A projected Newton-like inertial dynamics for modeling day-to-day traffic evolution with elastic demand

This paper proposes a projected Newton-like inertial dynamics for modeling second-order day-to-day (DTD) traffic evolution with elastic travel demand. The proposed DTD model describes double dynamics of traffic flow and travel cost based on a class of second-order gradient-like dissipative dynamic systems. We use the projection operator to prevent the existence of negative flow, which is regarded as a major pitfall of the existing second-order DTD traffic models. To our knowledge, this would be the first attempt to address the problem of negative flow in the second-order DTD traffic models. Meanwhile, we show that the proposed model inherits the properties of Newton-like inertial dynamics and behaves similarly to the existing second-order DTD models. The proposed model admits a Hessian-driven component, which is closely related to the congestion externality associated with the marginal link travel cost. The proposed model also extends the existing second-order DTD models from the fixed demand case to the elastic demand case. We characterize several theoretical properties of the proposed projected second-order DTD model, such as the equivalence between its fixed points and the user equilibrium with elastic demand, the convergence of the DTD traffic evolution process, and the stability analysis with different stability concepts. We show that the proposed model can be reduced to the well-known network tatonnement model. Finally, we demonstrate the properties of the projected second-order DTD model via numerical examples.

An Adaptive Frequency Sampling Algorithm for Dynamic Condensation-Based Frequency Response Analysis

This paper proposed an efficient and adaptive frequency sampling algorithm for frequency response analysis using dynamic condensation-based reduced-order modeling. For the degree of freedom-based model reduction method, the reduced-order basis becomes a frequency-dependent matrix since the relationship between master and slave degrees of freedom stems from partial equations of a second-order dynamical system. Such frequency-dependency makes the analysis inefficient for investigating the frequency response of the system. Considering that the coverage of a local reduced-order basis at a single frequency varies depending on the frequency, a new frequency sampling algorithm was proposed with a strategy of constructing multiple local reduced-order models (ROMs) at sample frequencies. For adaptive sampling, the frequency range of a local ROM was evaluated, and a new sample was added if there was a gap between two adjacent ROMs. As a result, the accuracy of the local ROM can be estimated, and the efficiency in the online stage was greatly enhanced. The proposed method was verified by performing frequency response analysis with several numerical examples, including a large-scale structural and dynamic system.

Synthesis of quasi-optimal fast filters by the least square criterion

The subject of the article research is special signal processing methods based on the optimal discrete filtering theory. The goal is to increase the efficiency of model-based methods for processing information signals by reducing computational costs and increasing the speed of optimal discrete filtering algorithms. Applied methods: description of dynamic processes in terms of state space using elements of vector-matrix algebra, weighted least squares method, elements of Kalman's theory of optimal discrete filtering, basic concepts of the O'Reilly–Luenberger theory of functional observers, elements of probability theory, statistical modeling by the Monte Carlo method. Results: a new method for reducing computational costs is proposed, which uses the approximation of the Kalman filter transfer matrix time dependence by given piecewise linear functions according to the least squares criterion. The effectiveness of the method was evaluated on the example of a second-order dynamical system. On the basis of a comparative analysis, several acceptable variants of the considered approximation are proposed. The practical significance of the work lies in the further development of methods for the synthesis of quasi-optimal high-speed filters. The operability of the proposed modifications is confirmed by the example of a second-order linear dynamic system. The efficiency of the algorithms was evaluated by the statistical modeling method according to the criterion "accuracy-computational costs". It is shown that the total savings in the number of multiplication and addition operations can reach tens of times due to insignificant losses in the accuracy of the filtering process.

Time Rescaling of a Primal-Dual Dynamical System with Asymptotically Vanishing Damping

In this work, we approach the minimization of a continuously differentiable convex function under linear equality constraints by a second-order dynamical system with an asymptotically vanishing damping term. The system under consideration is a time rescaled version of another system previously found in the literature. We show fast convergence of the primal-dual gap, the feasibility measure, and the objective function value along the generated trajectories. These convergence rates now depend on the rescaling parameter, and thus can be improved by choosing said parameter appropriately. When the objective function has a Lipschitz continuous gradient, we show that the primal-dual trajectory asymptotically converges weakly to a primal-dual optimal solution to the underlying minimization problem. We also exhibit improved rates of convergence of the gradient along the primal trajectories and of the adjoint of the corresponding linear operator along the dual trajectories. We illustrate the theoretical outcomes and also carry out a comparison with other classes of dynamical systems through numerical experiments.

Existence and uniqueness of the solution to a second-order nonlinear dynamical system model with an unbounded variable and a discontinuous input

We consider an initial value problem for a second-order nonlinear dynamical system that allows discontinuous inputs. In this problem, one state variable is restricted in a closed and bounded interval while the other state variable is unlimited. The existence and uniqueness of the solution to this problem is studied in a time interval determined by two adjacent instants at which the input may be discontinuous. Some results are obtained on the maximal interval of existence of the unique solution in this time interval. The results are then extended to the entire time domain. A robot example is given to verify our results.

Stability of nonlinear m-perturbed feedback controls systems

This paper examines the global and local stability of unstable nonlinear first- and second-order dynamical systems and the necessary conditions. More precisely, global and local stability is attained when these systems are controlled by m-perturbed feedback input controls functions. Finally, we obtain a dynamical system with m-perturbed feedback controls, a summation form with a nonlinear function that is exploited, and the stability of the systems supported by some illustrative examples.

On the effect of perturbations in first-order optimization methods with inertia and Hessian driven damping

<p style='text-indent:20px;'>Second-order continuous-time dissipative dynamical systems with viscous and Hessian driven damping have inspired effective first-order algorithms for solving convex optimization problems. While preserving the fast convergence properties of the Nesterov-type acceleration, the Hessian driven damping makes it possible to significantly attenuate the oscillations. To study the stability of these algorithms with respect to perturbations, we analyze the behaviour of the corresponding continuous systems when the gradient computation is subject to exogenous additive errors. We provide a quantitative analysis of the asymptotic behaviour of two types of systems, those with implicit and explicit Hessian driven damping. We consider convex, strongly convex, and non-smooth objective functions defined on a real Hilbert space and show that, depending on the formulation, different integrability conditions on the perturbations are sufficient to maintain the convergence rates of the systems. We highlight the differences between the implicit and explicit Hessian damping, and in particular point out that the assumptions on the objective and perturbations needed in the implicit case are more stringent than in the explicit case.</p>

Construction of the planar vector fields with nonsimple critical point of prescribed topological structure

The problem of constructing n-linear (n 2) plane vector elds with isolated critical point and given separatrices of prescribed types is considered. Such constructions are based on the use of vector algebra, the qualitative theory of second-order dynamic systems and classical methods for investigating their critical points. This problem is essentially an inverse problem of the qualitative theory of ordinary di erential equations, and its solution can be used to synthesize mathematical models of controlled dynamical systems of various physical nature.

Uniform global asymptotic stability for nonautonomous nonlinear dynamical systems

Research on uniform global asymptotic stability has the advantage of being able to predict the asymptotic velocity to the origin for all solutions of equations describing nonlinear phenomena. This study elucidates the sufficient conditions under which the zero solution of a nonautonomous nonlinear dynamical system of second order is uniformly globally asymptotically stable. From the results obtained, a certain first-order nonlinear differential equation associated with the second-order dynamical system plays a vital role in uniform global asymptotic stability. More precisely, under several reasonable assumptions, if the integral from σ to t+σ of a particular solution of the first-order differential equation diverges uniformly with respect to σ, then the zero solution of the dynamical system is guaranteed to be uniformly globally asymptotically stable. For the proof, the behavior of the solutions of the dynamical system is carefully tracked. An example that also includes a linear case is provided to illustrate the main result. Simulations are also presented to facilitate understanding of the example, and a local theorem corresponding to the main result and its application to an ecosystem model are discussed.

Data-Driven Operator Theoretic Methods for Phase Space Learning and Analysis

This paper uses data-driven operator theoretic approaches to explore the global phase space of a dynamical system. We defined conditions for discovering new invariant subsets in the state space of a dynamical system starting from an invariant subset based on the spectral properties of the Koopman operator. When the system evolution is known locally in several invariant subsets in the state space of a dynamical system, a phase space stitching result is derived that yields the global Koopman operator. Additionally, in the case of equivariant systems, a phase space stitching result is developed to identify the global Koopman operator using the symmetry properties between the invariant subsets of the dynamical system and time-series data from any one of the invariant subsets. Finally, these results are extended to topologically conjugate dynamical systems; in particular, the relation between the Koopman tuple of topologically conjugate systems is established. The proposed results are demonstrated on several second-order nonlinear dynamical systems including a bistable toggle switch. Our method elucidates a strategy for designing discovery experiments: experiment execution can be done in many steps, and models from different invariant subsets can be combined to approximate the global Koopman operator.