Modeling Of Nonlinear Dynamic Systems Research Articles

Grounds for Searching the Best Solution for Controlling the Pressurized Water Reactor in Dynamic Modes when Changing the Controlled Parameters

The article focused on the development of information technology for the optimization of control over complex dynamic systems at the stage of their design that should realize possibilities of modeling of linear and nonlinear dynamic systems, the analysis and synthesis of such systems, their optimization on various quality criteria. The purpose of this article is to develop the structure and elements of information technology to optimize the control of complex dynamic systems, including automated control systems. The general structure and functional model of information technology of the analysis and synthesis of control systems includes modeling of dynamic systems in the form of systems of differential equations and transfer functions, integration of systems of differential equations, calculation of system quality criteria, methods for solving various optimization tasks on solutions of optimization problems of dynamic systems is offered.

Consensus formation in networks with neighbor-dependent synergy and observer effect

In this paper, we study the consensus formation over a directed hypergraph, which is an important generalization of standard graph structure by allowing possible neighbor-dependent synergy. The proposed model is situated in the social dynamics providing key features including social observer effect and bounded confidence. Under the minimal siphon condition of a directed hypergraph (Petri net), we show that global consensus can be reached with the final consensus value residing in the common comfortable range if it is non-empty. To achieved this, we establish an equivalent condition for the commensurate graph of a finite state machine to be strongly connected. Convergence analysis is performed based on the proposed nonlinear dynamic system model and Petri net method. The consensus result holds for any non-negative confidence bound, which distinguishes from traditional bounded confidence opinion models as we measure the difference among neighbors rather than the gap between neighbors and the ego. Numerical studies are conducted to unravel some insights in relation to the influence of observers, hypergraph architecture, and confidence bounds on opinion evolution. The results and methodologies presented here facilitate research of social consensus and also offer a way to make sense of synergy in networked complex systems.

Numerical solution the Volterra dual integral equation of the first kind based on a method of Runge-Kutta type

The paper focuses on the non-classical Volterra dual equation of the first kind, to which the problem of identifying non-symmetric Volterra kernels is reduced. For this equation, we obtained difference analogs based on the application of the Runge-Kutta type method. Such equations arise in the problem of modeling nonlinear dynamical systems of black-box type, which employ the Volterra integro-power series, when the input perturbation is a vector function of time.

Automation, Research and Investment Policies in Firms

In our time automation, in combination with burgeoning fields such as Artificial Intelligence, has grown to be a significant factor, and with it the role of scientific and engineering knowledge in the working of firms has grown too. We present a model-based study of firms in which generation of new knowledge, and the application of accumulated knowledge are integral to business since these determine the range and scope of the firms' products, and also the efficiency of R&D and the production process. In our model the firm is organized as two functionally separate stages in series. Stage 1's activity is R&D which creates new concepts, methods and prototypes of products. Stage 2 deploys labor and capital, in the form of machines'', in production, which transforms selected outputs of Stage 1 into marketable, profitable products. New knowledge is generated from dedicated research in Stage 1 as well as by Learning-by-Doing (LbD) in both stages. Knowledge is subject to obsolescence over time. The firm's investment policy determines the allocation of funds to each stage subject to a budget constraint, operations management controls the admission of the output of Stage 1 to Stage 2, and also the combination of labor and machines in Stage 2. We analyze the interaction of these decisions, and the dynamical evolution of the knowledge stock under two management strategies. The short-term-focused, myopic strategy takes the existing knowledge stock as given, and maximizes the immediate profit. The long-term-focused strategy takes into account the future benefits of generating new knowledge in the investment decision. We use commonly-used production functions to obtain nonlinear dynamical system models, which are analyzed. We show that for both strategies the system converges to a steady-state where the knowledge stock and investment allocation remain constant over time. In numerical studies we compare the system behavior for the two strategies, and characterize their dependencies on various factors, such as the strength of the LbD effect, return on knowledge stock, and the influence of knowledge in expanding the scope and range of the firm's products.

Simulation of variational Gaussian process NARX models with GPGPU

Gaussian processes (GP) regression is a powerful probabilistic tool for modeling nonlinear dynamical systems. The downside of the method is its cubic computational complexity with respect to the training data that can be partially reduced using pseudo-inputs. The dynamics can be represented with an autoregressive model, which simplifies the training to that of the static case. When simulating an autoregressive model, the uncertainty is propagated through a nonlinear function and the simulation cannot be evaluated in closed-form. This paper combines the variational methods of GP approximations with a nonlinear autoregressive model with exogenous inputs (NARX) to form variational GP (VGP-NARX) models. We show how VGP-NARX models, on average, better approximate a full GP-NARX model than more commonly used GP-NARX (FITC) model on 10 chaotic time-series. The modeling capabilities of VGP-NARX models are compared with the existing approaches on two benchmarks for modeling nonlinear dynamical systems. The advantage of general-purpose computing on graphics processing units (GPGPU) for Monte Carlo simulation on large validation data sets is addressed.

Weighted Nonlinear Dynamic System for Deep Extraction of Nonlinear Dynamic Latent Variables and Industrial Application

Soft sensor plays an increasingly important role in modern industrial processes for estimating key quality variables which are hard to measure. With the development of deep learning technologies, soft sensors based on the deep learning methods have drawn great attention. Aiming to predict key quality variables, a supervised weighted nonlinear dynamic system (WNDS) model aided by the maximal information coefficient (MIC) is proposed in this article. The variational autoencoder is employed into the system for extracting nonlinear dynamic features. The supervised WNDS model can simultaneously analyze the correlations between variables and the relationships between historical samples and present samples. Furthermore, the proposed method is extended to a semisupervised form, in order to handle the imbalanced numbers between routinely recorded process data and limited labeled quality data. The prediction performance is validated by an industrial case.

Fault detection and diagnosis of nonlinear dynamical processes through correlation dimension and fractal analysis based dynamic kernel PCA

A novel Dynamic Kernel PCA (DKPCA) method is developed for process monitoring in nonlinear dynamical systems. Classical DKPCA approaches still exhibit vague linearity assumptions to determine the number of principal components and to construct the dynamical structure. The optimal Static PCA (SPCA) and Dynamic PCA (DPCA) structures are constructed herein through the powerful theory of the nonlinear Fractal Dimension (FDim). While DKPCA offers a generic data-driven modelling of nonlinear dynamical systems, the fractal correlation dimension provides an intrinsic measure of the data complexity counting for the nonlinear dynamics and the chaotic behaviour. The proposed Fractal-based DKPCA (FDKPCA) integrates the two strategies to overcome SPCA/DPCA/DKPCA shortcomings, FDim allows verifying the degree of fitting and ensures optimal dimensionality reduction. The novel fault detection and diagnosis method is validated through seven applications using the Process Network Optimization (PRONTO) benchmark with real heterogeneous data, FDKPCA showed superior performance compared to contemporary approaches.

A self-organizing recurrent fuzzy neural network based on multivariate time series analysis

Fuzzy neural networks (FNNs) have attracted considerable interest for modeling nonlinear dynamic systems in recent years. However, the recurrent design and the self-organizing design of FNNs generally lack adaptability, and their analyses on the change rule of networks in continuous time are insufficient. To solve these problems, a self-organizing recurrent fuzzy neural network based on multivariate time series analysis (SORFNN-MTSA) is proposed in this paper. First, a recurrent mechanism, based on wavelet transform fuzzy Markov chain algorithm, is introduced to obtain adaptive recurrent values and accelerate convergence speed of the network. Second, a self-organization mechanism, based on weighted dynamic time warping algorithm and sensitivity analysis algorithm, is presented to optimize the network structure. Third, the convergence of SORFNN-MTSA is theoretically analyzed to show the efficiency in both fixed structure and self-organizing structure cases. Finally, several benchmark nonlinear systems and a real application of wastewater treatment are used to verify the effectiveness of SORFNN-MTSA. Compared with other existing methods, the proposed SORFNN-MTSA performs better in terms of both high accuracy and compact structure.

Adaptive Control Using Neural Networks and Approximate Models for Nonlinear Dynamic Systems

In this research, a comparative study of two recurrent neural networks, nonlinear autoregressive with exogenous input (NARX) neural network and nonlinear autoregressive moving average (NARMA-L2), and a feedforward neural network (FFNN) is performed for their ability to provide adaptive control of nonlinear systems. Three dynamical nonlinear systems of different complexity are considered. The aim of this work is to make the output of the plant follow the desired reference trajectory. The problem becomes more challenging when the dynamics of the plants are assumed to be unknown, and to tackle this problem, a multilayer neural network-based approximate model is set up which will work in parallel to the plant and the control scheme. The network parameters are updated using the dynamic backpropagation (BP) algorithm.

A LSTM based prediction model for nonlinear dynamical systems with chaotic itinerancy

The prediction for chaotic trajectory from the measured data of time history, without prior knowledge of underlying dynamical model, is a challenging task in the data-driven analysis, due to its sensitivity to initial conditions. In this paper, the Long Short-Term Memory Network (LSTM) with the merge layer is proposed to predict the future states of the coupled Morris-Lecar (M-L) system with the chaotic itinerancy responses. Here, the two LSTM models with single-branch and multi-branch are constructed respectively to carry out the predictions in the multivariate loading conditions. By comparison to the network model with single-branch, the multi-branch model with adding merge layer can provide a high utilization of weights to reduce training cost greatly and receive a low prediction error, which make the multi-layer LSTM promising to estimate a high-dimensional complex dynamical behavior like transient chaotic itinerancy.

Adaptive basis construction and improved error estimation for parametric nonlinear dynamical systems

SummaryAn adaptive scheme to generate reduced‐order models for parametric nonlinear dynamical systems is proposed. It aims to automatize the proper orthogonal decomposition (POD)‐Greedy algorithm combined with empirical interpolation. At each iteration, it is able to adaptively determine the number of the reduced basis vectors and the number of the interpolation basis vectors for basis construction. The proposed technique is able to derive a suitable match between the RB and the interpolation basis vectors, making the generation of a stable, compact and reliable ROM possible. This is achieved by adaptively adding new basis vectors or removing unnecessary ones, at each iteration of the greedy algorithm. An efficient output error indicator plays a key role in the adaptive scheme. We also propose an improved output error indicator based on previous work. Upon convergence of the POD‐Greedy algorithm, the new error indicator is shown to be sharper than the existing ones, implicating that a more reliable ROM can be constructed. The proposed method is tested on several nonlinear dynamical systems, namely, the viscous Burgers' equation and two other models from chemical engineering.

Data-driven predictions of a multiscale Lorenz 96 chaotic system using machine-learning methods: reservoir computing, artificial neural network, and long short-term memory network

Abstract. In this paper, the performance of three machine-learning methods for predicting short-term evolution and for reproducing the long-term statistics of a multiscale spatiotemporal Lorenz 96 system is examined. The methods are an echo state network (ESN, which is a type of reservoir computing; hereafter RC–ESN), a deep feed-forward artificial neural network (ANN), and a recurrent neural network (RNN) with long short-term memory (LSTM; hereafter RNN–LSTM). This Lorenz 96 system has three tiers of nonlinearly interacting variables representing slow/large-scale (X), intermediate (Y), and fast/small-scale (Z) processes. For training or testing, only X is available; Y and Z are never known or used. We show that RC–ESN substantially outperforms ANN and RNN–LSTM for short-term predictions, e.g., accurately forecasting the chaotic trajectories for hundreds of numerical solver's time steps equivalent to several Lyapunov timescales. The RNN–LSTM outperforms ANN, and both methods show some prediction skills too. Furthermore, even after losing the trajectory, data predicted by RC–ESN and RNN–LSTM have probability density functions (pdf's) that closely match the true pdf – even at the tails. The pdf of the data predicted using ANN, however, deviates from the true pdf. Implications, caveats, and applications to data-driven and data-assisted surrogate modeling of complex nonlinear dynamical systems, such as weather and climate, are discussed.

On the structure of time-delay embedding in linear models of non-linear dynamical systems.

This work addresses fundamental issues related to the structure and conditioning of linear time-delayed models of non-linear dynamics on an attractor. While this approach has been well-studied in the asymptotic sense (e.g., for an infinite number of delays), the non-asymptotic setting is not well-understood. First, we show that the minimal time-delays required for perfect signal recovery are solely determined by the sparsity in the Fourier spectrum for scalar systems. For the vector case, we provide a rank test and a geometric interpretation for the necessary and sufficient conditions for the existence of an accurate linear time delayed model. Furthermore, we prove that the output controllability index of a linear system induced by the Fourier spectrum serves as a tight upper bound on the minimal number of time delays required. An explicit expression for the exact linear model in the spectral domain is also provided. From a numerical perspective, the effect of the sampling rate and the number of time delays on numerical conditioning is examined. An upper bound on the condition number is derived, with the implication that conditioning can be improved with additional time delays and/or decreasing sampling rates. Moreover, it is explicitly shown that the underlying dynamics can be accurately recovered using only a partial period of the attractor. Our analysis is first validated in simple periodic and quasiperiodic systems, and sensitivity to noise is also investigated. Finally, issues and practical strategies of choosing time delays in large-scale chaotic systems are discussed and demonstrated on 3D turbulent Rayleigh-Bénard convection.

The influence of the controlling delay time on two-degree-of-freedom system with a high-static-low-dynamic-stiffness isolator

In order to research the influence of the control delay time on the vibration isolation system, the dynamic characteristic analysis of a two-degree-of-freedom vibration system, based on high-static-low-dynamic-stiffness vibration isolator (HSLDS-VI) with time-delayed feedback control is presented. The two-degree-of-freedom nonlinear vibration isolation dynamical system model comprising the delay time parameter is also derived. Then, the effects of feedback gain coefficient and delay parameter on the dynamic characteristics are analyzed with multiple scales method. The results demonstrate that the amplitude can be suppressed by the time-delay variation for a fixed feedback gain. The system can exhibit periodic motion or chaotic vibration when the feedback gain coefficient and time-delay parameter are adjusted, which is beneficial to the application of high-static-low-dynamic-stiffness vibration isolator.

Estimation and validation of individualized dynamic brain models with resting state fMRI

A key challenge for neuroscience is to develop generative, causal models of the human nervous system in an individualized, data-driven manner. Previous initiatives have either constructed biologically-plausible models that are not constrained by individual-level human brain activity or used data-driven statistical characterizations of individuals that are not mechanistic. We aim to bridge this gap through the development of a new modeling approach termed Mesoscale Individualized Neurodynamic (MINDy) modeling, wherein we fit nonlinear dynamical systems models directly to human brain imaging data. The MINDy framework is able to produce these data-driven network models for hundreds to thousands of interacting brain regions in just 1–3 ​min per subject. We demonstrate that the models are valid, reliable, and robust. We show that MINDy models are predictive of individualized patterns of resting-state brain dynamical activity. Furthermore, MINDy is better able to uncover the mechanisms underlying individual differences in resting state activity than functional connectivity methods.

A Kriging–NARX Model for Uncertainty Quantification of Nonlinear Stochastic Dynamical Systems in Time Domain

A novel approach, referred to as sparse Kriging–NARX (KNARX), is proposed for the uncertainty quantification of nonlinear stochastic dynamical systems. It combines the nonlinear autoregressive with exogenous (NARX) input model with the high fidelity surrogate model Kriging. The sparsity in the proposed approach is introduced in the NARX model by reducing the number of polynomial bases using the least-angle regression (LARS) algorithm. Sparse KNARX captures the nonlinearity of a problem by the NARX model, whereas the uncertain parameters are propagated using the Kriging surrogate model, and LARS makes the model efficient. The accuracy and the efficiency of the sparse KNARX was measured through uncertainty quantification applied to three nonlinear stochastic dynamical systems. The time-dependent mean and standard deviation were predicted for all the numerical examples. Instantaneous stochastic response characteristics and maximum absolute response were also predicted. All the results were compared with the full scale Monte Carlo simulation (MCS) results and a mean error was calculated for all the numerical problems to measure the accuracy. All the results had excellent agreement with the MCS results at a very limited computational cost. The efficiency of the sparse KNARX also was measured by the CPU time and the required number of surrogate model evaluations. In all instances, sparse KNARX outperformed other state-of-the-art methods, which justifies the applicability of this model for nonlinear stochastic dynamical systems.

Response Characteristics Prediction of Surge Protective Device Based on NARX Neural Network

The nonlinear autoregressive network with exogenous inputs (NARX) is a recurrent dynamic neural architecture, which is commonly used for input–output modeling of nonlinear dynamical systems. Due to the lack of accurate mathematical model for the performance analysis on surge protective devices (SPDs), with the input of fast rising time electromagnetic pulse (FREMP), the NARX neural network is employed to predict the response characteristics of SPD in this paper. In order to verify the feasibility of this method, SPD test system is set up according to IEC 61000-4-24. The results show that the response curve estimated by the proposed model is in good agreement with experimental results, especially the waveform parameters such as response time, pulse peak, and residual voltage. The good forecasting performance of the network suggests that the NARX model used in this paper has good generalization ability. Moreover, with less measured data it can predict the response of the SPD under different voltage levels that were not yet measured.

Dynamic Prediction of the Thermal Nonlinear Process Based on Deep Hybrid Neural Network

Nonlinear system prediction plays an important role in the practical thermal process, and deep learning algorithm is now popular in nonlinear dynamic system modeling because of its powerful learning ability. In this paper, the dynamic artificial neural networks (DANNs), which can be divided into two different types with external dynamic characteristics and internal dynamic characteristics, are analyzed. The mathematical formulations of feedforward deep neural network (DNN), traditional recurrent neural network (RNN) and Long-Short Term Memory network (LSTM) models are given. Furthermore, the structure of deep Hybrid Neural Network (DHNN) is described. Finally, the applicability of the above models in the thermal nonlinear process with different structural features is discussed. Simulation experiments reveal that DANNs with internal dynamic characteristics more suitable for solving thermal nonlinear system modeling problems with unknown order, and DHNN based on LSTM model has performed much better in approximating the dynamics of the thermal process with state parameters.

An Improved Parallel MDBN With AVMD for Nonlinear System Modeling

Nonlinear system modeling using Deep Belief Network (DBN) is currently a research hotspot. However, the training process of DBN needs large amount of data to guarantee accuracy, and the traditional DBN may not meet the requirement of high-precision modeling. In this paper, we first improve the original DBN and Variational Mode Decomposition (VMD) algorithms, and on this basis, we then proposed a parallel Momentum Deep Belief Networks (MDBN) with Adaptive Variational Mode Decomposition (AVMD). Parallel AVMD-MDBN is an improved modeling method based on the deep learning model DBN. Firstly, a single raw dataset is decomposed into a specific number of sub-datasets using AVMD. Then these sub-datasets are distributed among a number of improved MDBNs. A single raw dataset learning model and algorithm is extended to multiple feature extraction nodes to learn the characteristics of multiple sub-datasets in parallel. Finally, the results of the multiple nodes are transmitted to the main feature extraction node to complete the regression calculation. In order to verify the effectiveness of the model, the proposed parallel AVMD-MDBN model is tested on a nonlinear dynamic system modeling, a Mackey-Glass time-series prediction and a financial stock prediction. Our experimental results show that the proposed parallel AVMD-MDBN has better performances in terms of improving feature learning ability than that of other methods.

A Novel Hybrid Data-Driven Modeling Method for Missiles

This paper proposes a novel hybrid data-driven modeling method for missiles. Based on actual flight test data, the missile hybrid model is established by combining neural networks and the mechanism modeling method, considering the uncertainties and nonlinear factors in missiles. This method can avoid the problems in missile mechanism modeling and traditional data-driven modeling, and can also provide a solution for nonlinear dynamic system modeling problems in offline usage scenarios. Finally, the feasibility of the proposed method and the credibility of the established model are verified by simulation experiments and statistical analysis.