Modeling Of Nonlinear Dynamic Systems Research Articles

Using scientific machine learning for experimental bifurcation analysis of dynamic systems

Augmenting mechanistic ordinary differential equation (ODE) models with machine-learnable structures is a novel approach to create highly accurate, low-dimensional models of engineering systems incorporating both expert knowledge and reality through measurement data. Our exploratory study focuses on training universal differential equation (UDE) models for physical nonlinear dynamical systems with limit cycles: an aerofoil undergoing flutter oscillations and an electrodynamic nonlinear oscillator. We consider examples where training data is generated by numerical simulations, whereas we also employ the proposed modelling concept to physical experiments allowing us to investigate problems with a wide range of complexity. To collect the training data, the method of control-based continuation is used as it captures not just the stable but also the unstable limit cycles of the observed system. This feature makes it possible to extract more information about the observed system than the open-loop approach (surveying the steady state response by parameter sweeps without using control) would allow. We use both neural networks and Gaussian processes as universal approximators alongside the mechanistic models to give a critical assessment of the accuracy and robustness of the UDE modelling approach. We also highlight the potential issues one may run into during the training procedure indicating the limits of the current modelling framework.

Dynamic Investigation and Resonant Frequency Determination of Shaft Bending Fatigue Testing Machine

The testing machine of shaft bending fatigue is widely used to test the fatigue limit and life of various shaft-type parts in mechanical engineering. Machine dynamic characteristics and system bending resonant frequency are very important for the design, testing, and application of the testing machine, but existing complex finite element calculation and aimless large-range frequency scanning in testing are the available methods to solve these two problems helplessly and inefficiently. A novel analytical model of nonlinear coupling dynamic system is built for the fatigue testing machine in this paper. The nonlinear dynamic model is simulated and investigated numerically, then it is reasonably linearized and reduced as a linear one, and the corresponding error analysis and application range are presented. By using eigenvalue approximation, the shaft bending resonant frequency formula is obtained, then the formula is compared and verified with the results from the nonlinear system simulation and finite element software, and it can improve the machine design and testing efficiency obviously. The analytical dynamic modeling and linearizing, and the given mode resonant frequency approximating, also provide a reference for many similar vibration machines.

Nonlinear Dynamics

The modeling and control of nonlinear dynamic systems is challenging in mathematics and engineering [...]

Combining Androgen Deprivation and Immunotherapy in Prostate Cancer Treatment: A Mechanistic Approach

Due to its initial dependence on testosterone, prostate cancer patients are initially treated with androgen deprivation therapy, a form of chemical castration. However, in many cases, the cancer develops resistance to this treatment. Sipuleucel-T (Provenge), is the first live cell vaccine approved for treating patients with advanced, hormonally refractive prostate cancer. However, it has shown limited survival benefit. Recently, it has been proposed that combining Provenge with androgen deprivation may result in a better treatment outcome. Here, we develop a nonlinear dynamical systems model with a view to predicting the therapeutic potential of such a combination. Our model accounts for the mechanism of action of Provenge and the immune system response elicited by androgen deprivation. We use data from mouse xenograft experiments to calibrate and validate our model. The validated model is then used to explain the limited clinical success of Provenge, and predict optimal scheduling that maximizes the anti-tumor potential of Provenge combined with androgen deprivation. In particular, we demonstrate that the two treatments should be given concurrently, rather than sequentially, as is current practice.

THE SYNTHESIS OF MATHEMATICAL MODELS OF NONLINEAR DYNAMIC SYSTEMS USING VOLTERRA INTEGRAL EQUATION

The problem of creating mathematical models of nonlinear dynamical systems does not have an unambiguous solution and requires the creation of a separate synthesis method for each such object. To develop a method for synthesizing mathematical models of an extensive class of nonlinear dynamical systems with polynomial nonlinearities. The work uses a method based on the solution of the Volterra integral equation in the ideology set forth in Van Trees H.L., according to which the structure of a nonlinear dynamical object present47s a series connection of the linear part, characterizing the inertial properties of the system, and the nonlinear element, given by static characteristic. The difference of the suggested version of the method from the classical one, proposed in the works of Van Trees H.L., is an expansion of their input and output signals into Fourier series and a representation of the inertial part of these systems by their Bode plots, connected into one structure with input and output signals and non-linearity by Volterra integral equation. The algorithm of the proposed method is disclosed by the example of solving the problem of identifying a nonlinear dynamical system which impulse response of the inertial part satisfies the separability requirement, the order of the polynomial nonlinearity is three, and the model of the input signal has the form of a sinusoid "raised" over the time axis on a priori given constant level. A computational experiment was carried out on the example of nonlinear dynamical systems with the third order of the nonlinear characteristic and the first and second orders of the model of the inertial part of these systems with the specified algorithms of their parametric identification. The suggested method allows to synthesis the mathematical model of a nonlinear dynamical system with the polynomial static characteristic to the case when the input signal has an arbitrary number of harmonics, and the model of the inertial part and the nonlinear polynomial function have an arbitrary order.

Evaluation on spectral submanifold based reduced models of a rotor/stator rubbing system with cross-coupling stiffness

The methodology for reduced order models of nonlinear dynamical systems is always of great interest in dynamics community. Recently, spectral submanifolds (SSMs) related to hyperbolic fixed points have been constructed and successfully used to build reduced order models for nonlinear systems with a large number of degrees of freedom. There are, however, few works on the spectral submanifolds associated with periodic orbits in literature. In this paper, SSM associated to the periodic orbit of a nonlinear normal mode (NNM) of a rotor/stator rubbing system with cross-coupling stiffness is first constructed following the parameterization method. The SSM with a two-dimensional slow sub-stable invariant manifold attached to the stable NNM with forward whirl also serves as a reduced order model of the system, whose dynamics on the parameter space conjugates the dynamics of the full system. Unlike the previous works, where reduced order models are mainly used to obtain backbones of conservative autonomous systems or FRCs of light damped forcing systems as the non-autonomous perturbation, the reduced order model derived from autonomous system are used to predict the self-excited dominant forcing responses of the rotor/stator rubbing system. As can be seen, the reduced order model is valid with good accuracy in some range of forcing amplitudes. Moreover, the modification of the reduced order model by considering forcing perturbation is also built and works well even in a larger range of forcing amplitudes.

Recurrent neural networks enable design of multifunctional synthetic human gut microbiome dynamics.

Predicting the dynamics and functions of microbiomes constructed from the bottom-up is a key challenge in exploiting them to our benefit. Current models based on ecological theory fail to capture complex community behaviors due to higher order interactions, do not scale well with increasing complexity and in considering multiple functions. We develop and apply a long short-term memory (LSTM) framework to advance our understanding of community assembly and health-relevant metabolite production using a synthetic human gut community. A mainstay of recurrent neural networks, the LSTM learns a high dimensional data-driven non-linear dynamical system model. We show that the LSTM model can outperform the widely used generalized Lotka-Volterra model based on ecological theory. We build methods to decipher microbe-microbe and microbe-metabolite interactions from an otherwise black-box model. These methods highlight that Actinobacteria, Firmicutes and Proteobacteria are significant drivers of metabolite production whereas Bacteroides shape community dynamics. We use the LSTM model to navigate a large multidimensional functional landscape to design communities with unique health-relevant metabolite profiles and temporal behaviors. In sum, the accuracy of the LSTM model can be exploited for experimental planning and to guide the design of synthetic microbiomes with target dynamic functions.

Fuzzy identification of nonlinear dynamic system based on selection of important input variables

Input variables selection (IVS) is proved to be pivotal in nonlinear dynamic system modeling. In order to optimize the model of the nonlinear dynamic system, a fuzzy modeling method for determining the premise structure by selecting important inputs of the system is studied. Firstly, a simplified two stage fuzzy curves method is proposed, which is employed to sort all possible inputs by their relevance with outputs, select the important input variables of the system and identify the structure. Secondly, in order to reduce the complexity of the model, the standard fuzzy c-means clustering algorithm and the recursive least squares algorithm are used to identify the premise parameters and conclusion parameters, respectively. Then, the effectiveness of IVS is verified by two well-known issues. Finally, the proposed identification method is applied to a realistic variable load pneumatic system. The simulation experiments indi cate that the IVS method in this paper has a positive influence on the approximation performance of the Takagi-Sugeno (T-S) fuzzy modeling.

Physics-guided Deep Markov Models for learning nonlinear dynamical systems with uncertainty

In this paper, we propose a probabilistic physics-guided framework, termed Physics-guided Deep Markov Model (PgDMM). The framework targets the inference of the characteristics and latent structure of nonlinear dynamical systems from measurement data, where exact inference of latent variables is typically intractable. A recently surfaced option pertains to leveraging variational inference to perform approximate inference. In such a scheme, transition and emission functions of the system are parameterized via feed-forward neural networks (deep generative models). However, due to the generalized and highly versatile formulation of neural network functions, the learned latent space often lacks physical interpretation and structured representation. To address this, we bridge physics-based state space models with Deep Markov Models, thus delivering a hybrid modeling framework for unsupervised learning and identification of nonlinear dynamical systems. The proposed framework takes advantage of the expressive power of deep learning, while retaining the driving physics of the dynamical system by imposing physics-driven restrictions on the side of the latent space. We demonstrate the benefits of such a fusion in terms of achieving improved performance on illustrative simulation examples and experimental case studies of nonlinear systems. Our results indicate that the physics-based models involved in the employed transition and emission functions essentially enforce a more structured and physically interpretable latent space, which is essential for enhancing and generalizing the predictive capabilities of deep learning-based models.

Surrogate Modeling of Nonlinear Dynamic Systems: A Comparative Study

Abstract Surrogate models play a vital role in overcoming the computational challenge in designing and analyzing nonlinear dynamic systems, especially in the presence of uncertainty. This paper presents a comparative study of different surrogate modeling techniques for nonlinear dynamic systems. Four surrogate modeling methods, namely, Gaussian process (GP) regression, a long short-term memory (LSTM) network, a convolutional neural network (CNN) with LSTM (CNN-LSTM), and a CNN with bidirectional LSTM (CNN-BLSTM), are studied and compared. All these model types can predict the future behavior of dynamic systems over long periods based on training data from relatively short periods. The multi-dimensional inputs of surrogate models are organized in a nonlinear autoregressive exogenous model (NARX) scheme to enable recursive prediction over long periods, where current predictions replace inputs from the previous time window. Three numerical examples, including one mathematical example and two nonlinear engineering analysis models, are used to compare the performance of the four surrogate modeling techniques. The results show that the GP-NARX surrogate model tends to have more stable performance than the other three deep learning (DL)-based methods for the three particular examples studied. The tuning effort of GP-NARX is also much lower than its deep learning-based counterparts.

Low latency cyberattack detection in smart grids with deep reinforcement learning

This paper focuses on low latency detection of cyberattacks in smart grids with deep reinforcement learning (DRL). The objective of low latency detection is to minimize detection delay while ensuring high detection accuracy. This is different from conventional detection methods that focus mainly on detection accuracy and pay little attention to detection delay. A lower detection delay can reduce recovery time, thus minimizing service interruption or economic losses due to cyberattacks. Since detection delay is the main design metric, the algorithm is developed by using a non-linear dynamic AC system model with an extended Kalman filter (EKF) to capture power grid state transitions in real-time, while many other works in the literature use a simplified linear DC model. The DRL detection algorithm is developed by using a continuous state space deep Q-network (DQN) on the framework of a Markov decision process (MDP). The new DQN design has two main innovations. First, the MDP state is designed as a sliding window of Rao-statistics of the AC dynamic state estimation residues. The proposed state formulation can accurately capture dynamic power state transitions in real-time. Second, a new reward function is designed to allow a flexible trade-off between detection delays and detection accuracy. The delay-accuracy trade-off can be adjusted by tuning a single parameter in the reward function. Simulation results show that the proposed DQN-based DRL detection algorithm can achieve very low detection delays with high detection accuracy.

Nonlinear model predictive control for the space heating system of a university building in Norway

Space heating accounts for a significant share of a building's energy use; moreover, occupants' desires for better thermal comfort may further increase the energy use and peak load of the space heating system, as well as its total energy cost. The goal of this study was to use an optimal control technique: model-based predictive controller, to improve the energy and economic performance of space heating systems with satisfying indoor temperature. The proposed optimal controller incorporated the dynamic energy prices, the disturbances from weather and occupancy, as well as a nonlinear system dynamic model that considered the building thermal mass as thermal energy storage. The model-based predictive controller was tested by simulation on a university building in central Norway, and its control performance was compared to a conventional rule-based control approach, which is currently employed by the case system. The model-based predictive controller's effectiveness was demonstrated by a 2.8% reduction in heat use, 8.8% shaving in peak load, 3.8% saving in heating cost, and a 59% drop in indoor temperature violation. Furthermore, a sensitivity study revealed that the model-based predictive controller still maintained high energy and economic performance even under lightweight building constructions with decreased building thermal mass.

Bayesian updating of failure probability curves with multiple performance functions of nonlinear structural dynamic systems

System failure often involves multiple failure modes which require considering multiple performance functions. Based on measured system response data, Bayesian updating of multiple failure probability curves needs to be performed. In this paper, a new approach based on extending ISS is proposed which can update multiple failure probability curves by simultaneously considering all the multiple performance functions in one run. A new scheme of how the generated samples are used is proposed to improve the estimation of failure probability curves. Discontinuity issues on the failure curves in ISS are resolved by the proposed method. The proposed method is applied to two illustrative examples involving uncertain model parameters and nonlinear structural dynamic systems subjected to future uncertain earthquake excitations. The computational efficiency of the proposed method is compared with direct application of Subset Simulation. Significant computational savings are observed while the coefficient of variation of the failure probability estimators is kept at the same level.

Intelligent modeling of nonlinear dynamical systems by machine learning

An intelligent data-driven method of modeling nonlinear dynamical systems named as LSTM network with output recurrence (OR-LSTM) is proposed, which can learn the inherent characteristics of the dynamical systems from data and predict the states of the systems given external excitation and initial conditions. For the proposed OR-LSTM architecture, the most important key point is output recurrence connections. In order to evaluate it, a teacher-forcing LSTM network without output recurrence connections (TF-LSTM) is designed for comparison. The differences between OR-LSTM and TF-LSTM from the perspective of gradient propagation and parameter updating are analyzed firstly. To verify the capability of the LSTM networks in dynamical systems modeling, the oscillation of a 42-story reinforced concrete frame-core tube building under severe and extreme seismic excitations are modeled. The results show that the OR-LSTM network performs well and maintains the long-term robustness in earthquake responses prediction, even only known displacements of 9 stories. However, the TF-LSTM network fails. In order to evaluate the extensive practicability of the OR-LSTM network, the famous Van der Pol dynamic system and Lorenz dynamic system with strong nonlinearity are also modeled by the proposed method. The results show that OR-LSTM network is competent in modeling most of nonlinear dynamic systems, while has limited in fully chaotic systems, but the predictable length of time sequence and prediction accuracy are improved.

INVESTIGATION OF THE FRACTAL RESPONSE OF A NONLINEAR PACKAGING SYSTEM

The paper aims to obtain the cushion curves of a polymer foam commonly used in the protective packaging of fragile products and its fractal cushioning performance. First, the impact curves are obtained to evaluate the dropping damage of a critical component considering a fractal nonlinear dynamic packaging system model, more specifically a fractal cubic-quintic-heptic Duffing’s equation. Then, the two-scale fractal derivative transform is introduced to determine the packaging material fractal behavior due to the material molecular structure. The findings explain the dynamic response behavior of the porous cushion material, and the proposed fractal dynamic model sheds new light on the design and analysis of cushioning packaging systems.

Compact neural network modeling of nonlinear dynamical systems via the standard nonlinear operator form

Dynamic artificial neural networks (DANNs) have become popular for the data-driven modelling of nonlinear dynamical systems. This article elucidates properties of a compact DANN model structure called the standard normal operator form (SNOF). Sets of nonlinear dynamical systems are characterized for which the SNOF can achieve the same model identification error with fewer neurons than needed by the popular DANN model structures. The results are demonstrated in a case study for a multi-stage bioreactor, which is a highly nonlinear dynamical system in which the cells have a sharp change in dynamics during a change in the feed composition as the bioreactor shifts from growth mode to production mode. The ability of the SNOF to model highly nonlinear dynamical systems with a very small number of neurons suggests its potential for serving as a basis for the design of model-based optimal control systems with theoretical guarantees of closed-loop stability and performance.

Two Filtering Methods of Forecasting Linear and Nonlinear Dynamics of Intensive Longitudinal Data.

With the advent of new data collection technologies, intensive longitudinal data (ILD) are collected more frequently than ever. Along with the increased prevalence of ILD, more methods are being developed to analyze these data. However, relatively few methods have yet been applied for making long- or even short-term predictions from ILD in behavioral settings. Applications of forecasting methods to behavioral ILD are still scant. We first establish a general framework for modeling ILD and then extend that frame to two previously existing forecasting methods: these methods are Kalman prediction and ensemble prediction. After implementing Kalman and ensemble forecasts in free and open-source software, we apply these methods to daily drug and alcohol use data. In doing so, we create a simple, but nonlinear dynamical system model of daily drug and alcohol use and illustrate important differences between the forecasting methods. We further compare the Kalman and ensemble forecasting methods to several simpler forecasts of daily drug and alcohol use. Ensemble forecasts may be more appropriate than Kalman forecasts for nonlinear dynamical systems models, but further forecasting evaluation methods must be put into practice.

Type-2 fuzzy instrumental variable algorithm for evolving neural-fuzzy modeling of nonlinear dynamic systems in noisy environment

In neural-fuzzy modeling, dealing with dynamic systems in a noisy environment is a relevant issue due to uncertainties in the experimental data. In this sense, a type-2 fuzzy instrumental variable based learning algorithm for evolving neural-fuzzy modeling is proposed in this paper. For antecedent adaptation, an Extend Kalman Filter based evolving type-2 fuzzy algorithm is adopted, where the participatory concept is used to mitigate the effects of outliers in the data flow. For consequent estimation, a type-2 fuzzy instrumental variable based subspace identification method is proposed. From the data flow, the instrumental variables are computed via recursive singular spectral analysis and used to estimate the fuzzy Markov parameters. From Markov parameters, the linear state-space matrices are obtained, recursively. Results illustrate the efficiency of the proposed methodology compared to other prominent approaches in the literature for identifying a nonlinear dynamic system with discontinuous function corrupted by colored noise and online identification of a 2DoF helicopter with correlated noise.

Формулы обращения для трехмерного интегрального уравнения Вольтерра I рода с предысторией

The article is devoted to solving one class of Volterra equations of the first kind with variable upper and lower limits. These equations were introduced in connection with the problem of identifying asymmetric kernels for constructing integral models of nonlinear dynamical systems of ”input-output” type in the form of Volterra polynomials. To solve the identification problem, previously introduced test signals with duration h (grid sampling step) are used in the form of a linear combination of Heaviside functions. The article demonstrates a method for obtaining the desired solution, which develops the step method for the one-dimensional case. Matching conditions are established that ensure the desired smoothness of the solution.

Glasdi: Parametric Physics-Informed Greedy Latent Space Dynamics Identification

A parametric adaptive physics-informed greedy Latent Space Dynamics Identification (gLaSDI) method is proposed for accurate, efficient, and robust data-driven reduced-order modeling of high-dimensional nonlinear dynamical systems. In the proposed gLaSDI framework, an autoencoder discovers intrinsic nonlinear latent representations of high-dimensional data, while dynamics identification (DI) models capture local latent-space dynamics. An interactive training algorithm is adopted for the autoencoder and local DI models, which enables identification of simple latent-space dynamics and enhances accuracy and efficiency of data-driven reduced-order modeling. To maximize and accelerate the exploration of the parameter space for the optimal model performance, an adaptive greedy sampling algorithm integrated with a physics-informed residual-based error indicator and random-subset evaluation is introduced to search for the optimal training samples on-the-fly. Further, to exploit local latent-space dynamics captured by the local DI models for an improved modeling accuracy with a minimum number of local DI models in the parameter space, an efficient k-nearest neighbor convex interpolation scheme is employed. The effectiveness of the proposed framework is demonstrated by modeling various nonlinear dynamical problems, including Burgers equations, nonlinear heat conduction, and radial advection. The proposed adaptive greedy sampling outperforms the conventional predefined uniform sampling in terms of accuracy. Compared with the high-fidelity models, gLaSDI achieves 66 to 4,417x speed-up with 1 to 5% relative errors.