Noisy Dynamical Systems Research Articles

TimeTeller: A tool to probe the circadian clock as a multigene dynamical system.

Recent studies have established that the circadian clock influences onset, progression and therapeutic outcomes in a number of diseases including cancer and heart diseases. Therefore, there is a need for tools to measure the functional state of the molecular circadian clock and its downstream targets in patients. Moreover, the clock is a multi-dimensional stochastic oscillator and there are few tools for analysing it as a noisy multigene dynamical system. In this paper we consider the methodology behind TimeTeller, a machine learning tool that analyses the clock as a noisy multigene dynamical system and aims to estimate circadian clock function from a single transcriptome by modelling the multi-dimensional state of the clock. We demonstrate its potential for clock systems assessment by applying it to mouse, baboon and human microarray and RNA-seq data and show how to visualise and quantify the global structure of the clock, quantitatively stratify individual transcriptomic samples by clock dysfunction and globally compare clocks across individuals, conditions and tissues thus highlighting its potential relevance for advancing circadian medicine.

On the topology of cryptocurrency markets

Cryptocurrency markets are characterised by high volatility, high returns and comparative immaturity relative to equity and commodity markets. Topological Data Analysis (TDA) persistence norms are effective tools for the analysis of noisy dynamical systems like the cryptocurrency markets. We show how information from the shape of daily return data adds additional inference on activity within the cryptocurrency markets. TDA persistence norms embed volatility and connectedness between coins as well as incorporating information from uncertainty indexes, financial market performance and commodity returns. Our TDA measures are robust to noise and are consistent across a raft of alternative coin selections. Further, we exposit how persistence norms peak to forewarn of crashes and stay low as markets face exogenous shocks. We demonstrate the clear advantages of TDA for the study of cryptocurrency markets and develop the next steps for exploiting the potential of TDA for application to cryptocurrency markets.

Filtering Theory for a Weakly Coloured Noise Process

The problem of analyzing the Ito stochastic differential system and its filtering has received attention. The classical approach to accomplish filtering for the Ito SDE is the Kushner equation. In contrast to the classical filtering approach, this paper presents filtering for the stochastic differential system affected by weakly coloured noise, i.e., $$\dot{x}_{t} = f\left( {x_{t} } \right) + g\left( {x_{t} } \right)\xi_{t} ,$$ where the input process $$\xi_{t}$$ is a weakly coloured noise process. As a special case, the process $$\xi_{t}$$ can be regarded as the Ornstein–Uhlenbeck (OU) process, i.e., $$d\xi_{t} = - \alpha \xi_{t} dt + \beta dB_{t} ,$$ where $$\alpha > 0.$$ More precisely, the filtering model of this paper can be cast as $$\dot{x}_{t} = f\left( {x_{t} } \right) + g\left( {x_{t} } \right)\xi_{t} ,$$ $$z_{t} = \int\limits_{{t_{0} }}^{t} {h(x_{\tau } )d\tau + \eta_{t} ,}$$ where $$h(x_{t} )$$ is the measurement non-linearity and $$\eta = \{ \eta_{t} ,t_{0} \le t < \infty \}$$ is the Brownian motion process. The former expression describes the structure of a noisy dynamical system, and the latter is the observation equation. The novelties of this paper are two (1) the extension of the filtering theory for the Ito stochastic differential system to the filtering theory for the ‘weakly coloured noise-driven’ stochastic differential system (2) the theory of this paper is based on a pioneering contribution of Ruslan Stratonovich involving the perturbation-theoretic approach to noisy dynamical systems in combination with the notion of the ‘filtering density’ evolution. The stochastic evolution of condition moment is derived by utilizing the filtering density evolution equation. A scalar Duffing system driven by the OU process is employed to test the effectiveness of the filtering theory of the paper. Numerical simulations involving four different sets of initial conditions and system parameters are utilized to examine the efficacy of the filtering algorithm of this paper.

Critical speeding up as an early warning signal of stochastic regime shifts

The use of critical slowing down as an early warning indicator for regime switching in observations from noisy dynamical systems and models has been widely studied and implemented in recent years. Some systems, however, have been shown to avoid critical slowing down prior to a transition between equilibria (Ditlevsen and Johnsen, Geophysical Research Letters, 37(19), 2010; Hastings and Wysham, Ecol Lett 13(4):464–472, 2010). Possible explanations include a non-smooth potential driving the dynamic (Hastings and Wysham, Ecol Lett 13(4):464–472, 2010) or large perturbations driving the system out of the initial basin of attraction (Boettiger and Batt 2020). In this paper, we discuss a phenomenon analogous to critical slowing down, where a slow parameter change leads to a high likelihood of a regime shift and creates signature warning signs in the statistics of the process’s sample paths. This effect, which we dub “critical speeding up,” is demonstrated using a simple population model exhibiting an Allee effect. In short, if a basin of attraction is compressed under a parameter change then the potential well steepens, leading to a drop in the time series’ variance and autocorrelation; precisely the opposite warning signs exhibited by critical slowing down. The fact that either falling or rising variance / autocorrelation can indicate imminent state change should underline the need for reliable modeling of any empirical system where one desires to forecast regime change.

Deep learning algorithm for data-driven simulation of noisy dynamical system

We present a deep learning model, DE-LSTM, for the simulation of a stochastic process with an underlying nonlinear dynamics. The deep learning model aims to approximate the probability density function of a stochastic process via numerical discretization and the underlying nonlinear dynamics is modeled by the Long Short-Term Memory (LSTM) network. It is shown that, when the numerical discretization is used, the function estimation problem can be solved by a multi-label classification problem. A penalized maximum log likelihood method is proposed to impose a smoothness condition in the prediction of the probability distribution. We show that the time evolution of the probability distribution can be computed by a high-dimensional integration of the transition probability of the LSTM internal states. A Monte Carlo algorithm to approximate the high-dimensional integration is outlined. The behavior of DE-LSTM is thoroughly investigated by using the Ornstein–Uhlenbeck process and noisy observations of nonlinear dynamical systems; Mackey–Glass time series and forced Van der Pol oscillator. It is shown that DE-LSTM makes a good prediction of the probability distribution without assuming any distributional properties of the stochastic process. For a multiple-step forecast of the Mackey–Glass time series, the prediction uncertainty, denoted by the 95% confidence interval, first grows, then dynamically adjusts following the evolution of the system, while in the simulation of the forced Van der Pol oscillator, the prediction uncertainty does not grow in time even for a 3,000-step forecast.

Ergodic Theorems for the Transfer Operators of Noisy Dynamical Systems

Ergodic Theorems for the Transfer Operators of Noisy Dynamical Systems

Extended recurrence plot and quantification for noisy continuous dynamical systems.

One main challenge in constructing a reliable recurrence plot (RP) and, hence, its quantification [recurrence quantification analysis (RQA)] of a continuous dynamical system is the induced noise that is commonly found in observation time series. This induced noise is known to cause disrupted and deviated diagonal lines despite the known deterministic features and, hence, biases the diagonal line based RQA measures and can lead to misleading conclusions. Although discontinuous lines can be further connected by increasing the recurrence threshold, such an approach triggers thick lines in the plot. However, thick lines also influence the RQA measures by artificially increasing the number of diagonals and the length of vertical lines [e.g., Determinism ( ) and Laminarity ( ) become artificially higher]. To take on this challenge, an extended RQA approach for accounting disrupted and deviated diagonal lines is proposed. The approach uses the concept of a sliding diagonal window with minimal window size that tolerates the mentioned deviated lines and also considers a specified minimal lag between points as connected. This is meant to derive a similar determinism indicator for noisy signal where conventional RQA fails to capture. Additionally, an extended local minima approach to construct RP is also proposed to further reduce artificial block structures and vertical lines that potentially increase the associated RQA like LAM. The methodology and applicability of the extended local minima approach and equivalent measure are presented and discussed, respectively.

Editorial

The lead article for this issue of EJAM is Prof. Bernard J. Matkowsky's personalized survey-style article on the theory and application of singular perturbation methods to noisy dynamical systems in the limit of small noise. This article is based on his John von Neumann Prize lecture presented at the Society of Industrial and Applied Mathematics (SIAM) Annual Meeting in July 2017. The John von Neumann Lecture is awarded by SIAM for outstanding and distinguished contributions to the field of applied mathematical sciences and for the effective communication of these ideas to the community. From 1990–1996, Matkowsky was an inaugural editorial board member for EJAM.

Singular perturbations in noisy dynamical systems

Consider a deterministic dynamical system in a domain containing a stable equilibrium, e.g., a particle in a potential well. The particle, independent of initial conditions, eventually reaches the bottom of the well. If however, the particle is subjected to white noise, due, e.g., to collisions with a population of smaller, lighter particles comprising the medium through which the particle travels, a dramatic difference in the behaviour of the Brownian particle occurs. The particle will exit the well. The natural questions then are how long will it take for it to exit and from where on the boundary of the domain of attraction of the deterministic equilibrium (the rim of the well) will it exit. We compute the mean first passage time to the boundary and the mean probabilities of the exit positions. When the noise is small each quantity satisfies a singularly perturbed deterministic boundary value problem. We treat the problem by the method of matched asymptotic expansions (MAE) and generalizations thereof. MAE has been used successfully to solve problems in many applications. However, there exist problems for which MAE does not suffice. Among these are problems exhibiting boundary layer resonance, i.e., the problem of ‘spurious solutions’, which led some to conclude that this was ‘the failure of MAE’. We present a physical argument and four mathematical arguments to modify or augment MAE to make it successful. Finally, we discuss applications of the theory.

A parallel time integrator for noisy nonlinear oscillatory systems

In this paper, we adapt a parallel time integration scheme to track the trajectories of noisy non-linear dynamical systems. Specifically, we formulate a parallel algorithm to generate the sample path of nonlinear oscillator defined by stochastic differential equations (SDEs) using the so-called parareal method for ordinary differential equations (ODEs). The presence of Wiener process in SDEs causes difficulties in the direct application of any numerical integration techniques of ODEs including the parareal algorithm. The parallel implementation of the algorithm involves two SDEs solvers, namely a fine-level scheme to integrate the system in parallel and a coarse-level scheme to generate and correct the required initial conditions to start the fine-level integrators. For the numerical illustration, a randomly excited Duffing oscillator is investigated in order to study the performance of the stochastic parallel algorithm with respect to a range of system parameters. The distributed implementation of the algorithm exploits Massage Passing Interface (MPI).

On the covariance matrix of the stationary distribution of a noisy dynamical system

On the covariance matrix of the stationary distribution of a noisy dynamical system

Secure State Estimation Against Sensor Attacks in the Presence of Noise

We consider the problem of estimating the state of a noisy linear dynamical system when an unknown subset of sensors is arbitrarily corrupted by an adversary. We propose a secure state estimation algorithm, and derive (optimal) bounds on the achievable state estimation error given an upper bound on the number of attacked sensors. The proposed state estimator involves Kalman filters operating over subsets of sensors to search for a sensor subset which is reliable for state estimation. To further improve the subset search time, we propose Satisfiability Modulo Theory based techniques to exploit the combinatorial nature of searching over sensor subsets. Finally, as a result of independent interest, we give a coding theoretic view of attack detection and state estimation against sensor attacks in a noiseless dynamical system.

Escaping from a chaotic saddle in the presence of noise

There are certain situations in noisy nonlinear dynamical systems, for which it is required a fast transition between a chaotic and a periodic state. Here, we present a novel procedure to achieve this goal in the context of the partial control method of chaotic systems. The partial control method is a recently developed control procedure, that allows to keep the dynamics of a system showing transient chaos, close to its chaotic saddle. In this kind of systems, and in absence of an external control, trajectories remain chaotic for a while in a certain region of phase space before eventually escaping towards an external attractor. The aim of the control algorithm proposed here, is to maintain the chaotic orbits as much time as we want close to the chaotic saddle before forcing an immediate escape. To do that, we use the safe sets defined in the partial control method in a completely different way. By only using this set, we show how possible is to handle the stabilisation and destabilisation of the chaotic dynamics of the partially controlled system.

Bayesian optimization for likelihood-free inference of simulator-based statistical models

Our paper deals with inferring simulator-based statistical models given some observed data. A simulator-based model is a parametrized mechanism which specifies how data are generated. It is thus also referred to as generative model. We assume that only a finite number of parameters are of interest and allow the generative process to be very general; it may be a noisy nonlinear dynamical system with an unrestricted number of hidden variables. This weak assumption is useful for devising realistic models but it renders statistical inference very difficult. The main challenge is the intractability of the likelihood function. Several likelihood-free inference methods have been proposed which share the basic idea of identifying the parameters by finding values for which the discrepancy between simulated and observed data is small. A major obstacle to using these methods is their computational cost. The cost is largely due to the need to repeatedly simulate data sets and the lack of knowledge about how the parameters affect the discrepancy. We propose a strategy which combines probabilistic modeling of the discrepancy with optimization to facilitate likelihood-free inference. The strategy is implemented using Bayesian optimization and is shown to accelerate the inference through a reduction in the number of required simulations by several orders of magnitude.

An efficient recursive least square-based condition monitoring approach for a rail vehicle suspension system

ABSTRACTA model-based condition monitoring strategy for the railway vehicle suspension is proposed in this paper. This approach is based on recursive least square (RLS) algorithm focusing on the deterministic ‘input–output’ model. RLS has Kalman filtering feature and is able to identify the unknown parameters from a noisy dynamic system by memorising the correlation properties of variables. The identification of suspension parameter is achieved by machine learning of the relationship between excitation and response in a vehicle dynamic system. A fault detection method for the vertical primary suspension is illustrated as an instance of this condition monitoring scheme. Simulation results from the rail vehicle dynamics software ‘ADTreS’ are utilised as ‘virtual measurements’ considering a trailer car of Italian ETR500 high-speed train. The field test data from an E464 locomotive are also employed to validate the feasibility of this strategy for the real application. Results of the parameter identification performed indicate that estimated suspension parameters are consistent or approximate with the reference values. These results provide the supporting evidence that this fault diagnosis technique is capable of paving the way for the future vehicle condition monitoring system.

Wavelet shrinkage of a noisy dynamical system with non-linear noise impact

By filtering wavelet coefficients, it is possible to construct a good estimate of a pure signal from noisy data. Especially, for a simple linear noise influence, Donoho and Johnstone (1994) have already defined an optimal filter design in the sense of a minimization of the error made when estimating the pure signal. We set here a different framework where the influence of the noise is non-linear. In particular, we propose a method to filter the wavelet coefficients of a discrete dynamical system disrupted by a weak noise, in order to construct good estimates of the pure signal, including Bayes’ estimate, minimax estimate, oracular estimate or thresholding estimate. We present the example of a logistic and a Lorenz chaotic dynamical system as well as an adaptation of our technique in order to show empirically the robustness of the thresholding method in presence of leptokurtic noise. Moreover, we test both the hard and the soft thresholding and also another kind of smoother thresholding which seems to have almost the same reconstruction power as the hard thresholding. Finally, besides the tests on an estimated dataset, the method is tested on financial data: oil prices and NOK/USD exchange rate.

Power-law distributions in noisy dynamical systems

We consider a dynamical system which is non-autonomous, has a stable attractor and which is perturbed by an additive noise. We establish that under some quite typical conditions, the intermittent fluctuations from the attractor have a probability distribution with power-law tails. We show that this results from a stochastic cascade of amplification of fluctuations due to transient periods of instability. The exponent of the power-law is interpreted as a negative fractal dimension, and is explicitly determined, using numerics or perturbation expansion, in the case of a model of colloidal particles in one-dimension.

Bayesian inference of interaction properties of noisy dynamical systems with time-varying coupling: capabilities and limitations

We investigate a recently published analysis framework based on Bayesian inference for the time-resolved characterization of interaction properties of noisy, coupled dynamical systems. It promises wide applicability and a better time resolution than well-established methods. At the example of representative model systems, we show that the analysis framework has the same weaknesses as previous methods, particularly when investigating interacting, structurally different non-linear oscillators. We also inspect the tracking of time-varying interaction properties and propose a further modification of the algorithm, which improves the reliability of obtained results. We exemplarily investigate the suitability of this algorithm to infer strength and direction of interactions between various regions of the human brain during an epileptic seizure. Within the limitations of the applicability of this analysis tool, we show that the modified algorithm indeed allows a better time resolution through Bayesian inference when compared to previous methods based on least square fits.

Symplectic geometry spectrum analysis of nonlinear time series

Various time-series decomposition techniques, including wavelet transform, singular spectrum analysis, empirical mode decomposition and independent component analysis, have been developed for non-linear dynamic system analysis. In this paper, we describe a symplectic geometry spectrum analysis (SGSA) method to decompose a time series into a set of independent additive components. SGSA is performed in four steps: embedding, symplectic QR decomposition, grouping and diagonal averaging. The obtained components can be used for de-noising, prediction, control and synchronization. We demonstrate the effectiveness of SGSA in reconstructing and predicting two noisy benchmark nonlinear dynamic systems: the Lorenz and Mackey-Glass attractors. Examples of prediction of a decadal average sunspot number time series and a mechanomyographic signal recorded from human skeletal muscle further demonstrate the applicability of the SGSA method in real-life applications.

Stochastic Stability of Markovianly Switched Systems

This technical note examines the stochastic stability of noisy dynamics in discrete and continuous time. The notion of moment stability in the wide sense (MSWS) is presented as a generalization of ϵ-moment stability. MSWS is intentionally not based on stochastic convergence properties, since in most practically appearing systems convergence to any equilibrium is not present. A sufficient criterion for both MSWS and ergodicity is presented for a class of systems comprising a finite set of noisy dynamical systems among which switching is governed by a Markov chain. Stability/instability properties for each separate subsystem are assumed to be quantified by a Lyapunov function candidate together with an associated growth rate equation. For the set of Lyapunov functions, a compatibility criterion is assumed to be fulfilled, bounding the ratio between pairs of Lyapunov functions.