Nonlinear State Space Research Articles

Discrete Time Approximation of Continuous Time Nonlinear State Space Models

In this paper we discuss the problem of approximating a continuous time nonlinear state space model with an explicit discrete time state space model. Two steps are made to do so. The first one is to replace the classical zero-order-hold (ZOH) excitation signal assumption by a more general concept of low pass (LP) signals. It will be shown that opposed to the ZOH-property, the LP-property is maintained for a wide class of (non)linear systems, including cascaded and closed loop systems. The second step is based on a result from the signal processing theory that states that a BL-signal can be exactly predicted from its past samples by a fixed causal linear filter. This result is generalized to the discrete time integration of LP-signals by introducing an error bound that can be made arbitrarily small by increasing the sample frequency. Both ideas are then combined to bound the approximation errors of an explicit discrete time nonlinear state space approximation for a continuous time nonlinear state space model. The order of the decay of the approximation error as a function of the sample frequency is given. It is shown that eventually the aliasing error is the dominating error, at a cost of a (slightly) increased model complexity. These results are directly applicable to nonlinear system identification, and will be experimentally verified on the identification of a closed loop nonlinear system (the silverbox) using a discrete time nonlinear state space model.

An Unstructured Flexible Nonlinear Model for the Cascaded Water-tanks Benchmark

Many real world systems exhibit a quasi linear or weakly nonlinear behavior during normal operation, and a hard saturation effect for high peaks of the input signal. A typical example of such systems is the cascaded water-tanks benchmark. This benchmark combines soft and hard nonlinearities to be identified based on relatively short data records. In this paper, a methodology to identify an unstructured flexible nonlinear state space model (NLSS) for the cascaded water-tanks benchmark is proposed. The flexibility of the NLSS model structure is demonstrated by introducing two different initialisation schemes. Furthermore the strengths and short-comings of the model structure are discussed with respect to the cascaded water-benchmark identification problem.

Some nonlinear exponential smoothing models are unstable

This paper discusses the instability of eleven nonlinear state space models that underly exponential smoothing. Hyndman et al. (2002) proposed a framework of 24 state space models for exponential smoothing, including the well-known simple exponential smoothing, Holt's linear and Holt-Winters' additive and multiplicative methods. This was extended to 30 models with Taylor's (2003) damped multiplicative methods. We show that eleven of these 30 models are unstable, having infinite forecast variances. The eleven models are those with additive errors and either multiplicative trend or multiplicative seasonality, as well as the models with multiplicative errors, multiplicative trend and additive seasonality. The multiplicative Holt-Winters' model with additive errors is among the eleven unstable models. We conclude that: (1) a model with a multiplicative trend or a multiplicative seasonal component should also have a multiplicative error; and (2) a multiplicative trend should not be mixed with additive seasonality.

Non-linear exponential smoothing and positive data

We consider the properties of nonlinear exponential smoothing state space models under various assumptions about the innovations, or error, process. Our interest is restricted to those models that are used to describe non-negative observations, because many series of practical interest are so constrained. We first demonstrate that when the innovations process is assumed to be Gaussian, the resulting prediction distribution may have an infinite variance beyond a certain forecasting horizon. Further, such processes may converge almost surely to zero; an examination of purely multiplicative models reveals the circumstances under which this condition arises. We then explore effects of using an (invalid) Gaussian distribution to describe the innovations process when the underlying distribution is lognormal. Our results suggest that this approximation causes no serious problems for parameter estimation or for forecasting one or two steps ahead. However, for longer-term forecasts the true prediction intervals become increasingly skewed, whereas those based on the Gaussian approximation may have a progressively larger negative component. In addition, the Gaussian approximation is clearly inappropriate for simulation purposes. The performance of the Gaussian approximation is compared with those of two lognormal models for short-term forecasting using data on the weekly sales of over three hundred items of costume jewelry.

Multilevel models with stochastic volatility for repeated cross-sections: An application to tribal art prices

In this paper, we introduce a multilevel specification with stochastic volatility for repeated cross-sectional data. Modelling the time dynamics in repeated cross sections requires a suitable adaptation of the multilevel framework where the individuals/items are modelled at the first level whereas the time component appears at the second level. We perform maximum likelihood estimation by means of a nonlinear state space approach combined with Gauss–Legendre quadrature methods to approximate the likelihood function. We apply the model to the first database of tribal art items sold in the most important auction houses worldwide. The model allows to account properly for the heteroscedastic and autocorrelated volatility observed and has superior forecasting performance. Also, it provides valuable information on market trends and on predictability of prices that can be used by art markets stakeholders.

A semi-analytical treatment of xenon oscillations

A novel approach is developed to investigate xenon oscillations within a two-group two-region coupled core reactor model incorporating thermal feedback and poison effects. Group-wise neutronic coupling coefficients between the core regions are calculated applying the associated fundamental and first mode eigenvalue separation values. The resultant nonlinear state space representation of the core behavior is quite suitable for evaluation of reactivity induced power transients such as load following operation. The model however comprises a multi-physics coupling of sub-systems with extremely distant relaxation times whose stiffness treatment inquire costly multistep implicit numerical methods. An adiabatic treatment of the sluggish poison dynamics is therefore proposed as a way out. The approach helps further investigate the nonlinear system within a linear time varying (LTV) framework whereby a semi-analytical framework is established. This scheme incorporates a matrix exponential analytical solution of the perturbed system as a quite efficient tool to study load following operation and control purposes. Poison dynamics are updated within larger intervals which exclude the need for specific numerical schemes of stiff systems. Simulation results of the axial offset conducted on a VVER-1000 reactor at the beginning (BOC) and the end of cycle (EOC) display quite acceptable results compared with available benchmarks. The LTV reactor model is further investigated within a stability analysis of the associated time varying systems at these two stages employing the concept of Lyapunov exponent.

A flexible state–space model for learning nonlinear dynamical systems

We consider a nonlinear state–space model with the state transition and observation functions expressed as basis function expansions. The coefficients in the basis function expansions are learned from data. Using a connection to Gaussian processes we also develop priors on the coefficients, for tuning the model flexibility and to prevent overfitting to data, akin to a Gaussian process state–space model. The priors can alternatively be seen as a regularization, and helps the model in generalizing the data without sacrificing the richness offered by the basis function expansion. To learn the coefficients and other unknown parameters efficiently, we tailor an algorithm using state-of-the-art sequential Monte Carlo methods, which comes with theoretical guarantees on the learning. Our approach indicates promising results when evaluated on a classical benchmark as well as real data.

State estimation in fractional-order systems with coloured measurement noise

This paper presents new estimation methods for discrete fractional-order state-space systems with coloured measurement noise. A novel approach is proposed to convert a fractional system with coloured measurement noise to a system with white measurement noise in which the process and measurement noises are correlated with each other. In this paper, two new Kalman filter algorithms for fractional-order linear state-space systems with coloured measurement noise, as well as a new extended Kalman filter algorithm for state estimation in nonlinear fractional-order state-space systems with coloured measurement noise, are proposed. The accuracy of the equations and relations is confirmed in several theorems. The validity and effectiveness of the proposed algorithms are verified by simulation results and compared with previous work. Results show that for linear and nonlinear fractional-order systems with coloured noise, the proposed methods are more accurate than conventional methods regarding estimation error and estimation error covariance. Simulation results demonstrate that the proposed algorithms can accurately perform estimation in fractional-order systems with coloured measurement noise.

Bayesian Analysis in Non-Linear Non-Gaussian State-Space Models Using Particle Gibbs

Bayesian Analysis in Non-Linear Non-Gaussian State-Space Models Using Particle Gibbs

Structured Inference Networks for Nonlinear State Space Models

Gaussian state space models have been used for decades as generative models of sequential data. They admit an intuitive probabilistic interpretation, have a simple functional form, and enjoy widespread adoption. We introduce a unified algorithm to efficiently learn a broad class of linear and non-linear state space models, including variants where the emission and transition distributions are modeled by deep neural networks. Our learning algorithm simultaneously learns a compiled inference network and the generative model, leveraging a structured variational approximation parameterized by recurrent neural networks to mimic the posterior distribution. We apply the learning algorithm to both synthetic and real-world datasets, demonstrating its scalability and versatility. We find that using the structured approximation to the posterior results in models with significantly higher held-out likelihood.

Volatility forecasting using global stochastic financial trends extracted from non-synchronous data

A method based on various linear and nonlinear state space models used to extract global stochastic financial trends (GST) out of non-synchronous financial data is introduced. These models are constructed in order to take advantage of the intraday arrival of closing information coming from different international markets so that volatility description and forecasting is improved. A set of three major asynchronous international stock market indices is considered in order to empirically show that this forecasting scheme is capable of significant performance gains when compared to standard parametric models like the dynamic conditional correlation (DCC) family.

Inflation and Professional Forecast Dynamics: An Evaluation of Stickiness, Persistence, and Volatility

This paper studies the joint dynamics of U.S. inflation and the average inflation predictions of the Survey of Professional Forecasters (SPF) on a sample running from 1968Q4 to 2014Q2. The joint data generating process (DGP) of these data consists of the unobserved components (UC) model of Stock and Watson (2007, Why has US inflation become harder to forecast?, Journal of Money, Credit and Banking 39(S1), 3-33) and the sticky information (SI) forecast updating equation of Mankiw and Reis (2002, Sticky information versus sticky prices: A proposal to replace the New Keynesian Phillips curve, Quarterly Journal of Economics 117, 1295-1328). We introduce timevarying inflation gap persistence into the Stock and Watson (SW)-UC model and a timevarying frequency of forecast updating into the SI forecast updating equating. These models combine to produce a nonlinear state space model. This model is estimated using Bayesian tools grounded in the particle filter, which is an implementation of sequential Monte Carlo methods. The estimates reveal the data prefer the joint DGP of time-varying frequency of SI forecast updating and a SW-UC model with time-varying persistence. The joint DGP produces estimates that indicate the inflation spike of 1974 was explained most by gap inflation, but trend inflation dominates the inflation peak of the early 1980s. We also find the stochastic volatility (SV) of trend inflation exhibits negative co-movement with the time-varying frequency of SI forecast updating while the SV and time-varying persistence of gap inflation often show positive co-movement. Thus, the average SPF respondent is most sensitive to the impact of permanent shocks on the conditional mean of inflation.

Two step transfer entropy – An estimator of delayed directional couplings between multivariate EEG time series

Quantifying delayed directional couplings between electroencephalographic (EEG) time series requires an efficient method of causal network inference. This is especially due to the limited knowledge about the underlying dynamics of the brain activity. Recent methods based on information theoretic measures such as Transfer Entropy (TE) made significant progress on this issue by providing a model-free framework for causality detection. However, TE estimation from observed data is not a trivial task, especially when the number of variables is large which is the case in a highly complex system like human brain. Here we propose a computationally efficient procedure for TE estimation based on using sets of the Most Informative Variables that effectively contribute to resolving the uncertainty of the destination. In the first step of this method, some conditioning sets are determined through a nonlinear state space reconstruction; then in the second step, optimal estimation of TE is done based on these sets. Validation of the proposed method using synthetic data and neurophysiological signals demonstrates computational efficiency in quantifying delayed directional couplings compared with the common TE analysis.

A dynamic environment-sensitive site index model for the prediction of site productivity potential under climate change

Accurate and reliable predictions of the future development of forest site productivity are crucial for the effective management of forest stands. Static models which simply extrapolate productivity into the future are inappropriate under conditions of environmental change since they lack a close link between fundamental environmental drivers and forest growth processes. Here we present a dynamic environment-sensitive site index model formulated in the framework of a nonlinear state space approach based on longitudinal data from long-term experimental plots. Estimation of the model parameters was carried out using the prediction error minimization method. Our aim was to identify dynamic relationships between site index and environmental variables and to make conditional predictions of the future development of site index under climate change scenarios. Nonlinear, interactive, as well as accumulative effects of environmental factors (climate/weather and nitrogen influx) on the growth response were considered in the model. In the study, we estimated the dynamic environment-sensitive site index model using data from 604 Norway spruce (Picea abies [L.] Karst.) long-term experimental plots in southwest Germany with measurement data covering a period of more than 100 years from the end of the 19th century until today. We used the calibrated model to project future site index changes under increasing growing season temperature scenarios. Conventional climate change impact studies usually utilize a gradient approach and apply space-for-time substitution for the parameterization of models that are calibrated using spatial variability in the data. In contrast, the approach presented here utilizes the longitudinal data structure of multiple real growth time series to simultaneously exploit spatial and temporal variation in the data to provide more reliable and robust projections. Limitations of the space-for-time substitution approach in forest growth modelling are discussed.

Nonlinear Directed Interactions Between HRV and EEG Activity in Children With TLE.

Epileptic seizure activity influences the autonomic nervous system (ANS) in different ways. Heart rate variability (HRV) is used as indicator for alterations of the ANS. It was shown that linear, nondirected interactions between HRV and EEG activity before, during, and after epileptic seizure occur. Accordingly, investigations of directed nonlinear interactions are logical steps to provide, e.g., deeper insight into the development of seizure onsets. Convergent cross mapping (CCM) investigates nonlinear, directed interactions between time series by using nonlinear state space reconstruction. CCM is applied to simulated and clinically relevant data, i.e., interactions between HRV and specific EEG components of children with temporal lobe epilepsy (TLE). In addition, time-variant multivariate Autoregressive model (AR)-based estimation of partial directed coherence (PDC) was performed for the same data. Influence of estimation parameters and time-varying behavior of CCM estimation could be demonstrated by means of simulated data. AR-based estimation of PDC failed for the investigation of our clinical data. Time-varying interval-based application of CCM on these data revealed directed interactions between HRV and delta-related EEG activity. Interactions between HRV and alpha-related EEG activity were visible but less pronounced. EEG components mainly drive HRV. The interaction pattern and directionality clearly changed with onset of seizure. Statistical relevant interactions were quantified by bootstrapping and surrogate data approach. In contrast to AR-based estimation of PDC CCM was able to reveal time-courses and frequency-selective views of nonlinear interactions for the further understanding of complex interactions between the epileptic network and the ANS in children with TLE.

Heterogeneous recurrence analysis of heartbeat dynamics for the identification of sleep apnea events

Obstructive sleep apnea (OSA) is a common sleep disorder that affects 24% of adult men and 9% of adult women. It occurs due to the occlusion of the upper airway during sleep, thereby leading to a decrease of blood oxygen level that triggers arousals and sleep fragmentation. OSA significantly impacts the quality of sleep and it is known to be responsible for a number of health complications, such as high blood pressure and type 2 diabetes. Traditional diagnosis of OSA relies on polysomnography, which is expensive, time-consuming and inaccessible to the general population. Recent advancement of sensing provides an unprecedented opportunity for the screening of OSA events using single-channel electrocardiogram (ECG). However, existing approaches are limited in their ability to characterize nonlinear dynamics underlying ECG signals. As such, hidden patterns of OSA-altered cardiac electrical activity cannot be fully revealed and understood. This paper presents a new heterogeneous recurrence model to characterize the heart rate variability for the identification of OSA. A nonlinear state space is firstly reconstructed from a time series of RR intervals that are extracted from single-channel ECGs. Further, the state space is recursively partitioned into a hierarchical structure of local recurrence regions. A new fractal representation is designed to efficiently characterize state transitions among segmented sub-regions. Statistical measures are then developed to quantify heterogeneous recurrence patterns. In addition, we integrate classification models with heterogeneous recurrence features to differentiate healthy subjects from OSA patients. Experimental results show that the proposed approach captures heterogeneous recurrence patterns in the transformed space and provides an effective tool to detect OSA using one-lead ECG signals.

System Entropy Measurement of Stochastic Partial Differential Systems

System entropy describes the dispersal of a system’s energy and is an indication of the disorder of a physical system. Several system entropy measurement methods have been developed for dynamic systems. However, most real physical systems are always modeled using stochastic partial differential dynamic equations in the spatio-temporal domain. No efficient method currently exists that can calculate the system entropy of stochastic partial differential systems (SPDSs) in consideration of the effects of intrinsic random fluctuation and compartment diffusion. In this study, a novel indirect measurement method is proposed for calculating of system entropy of SPDSs using a Hamilton–Jacobi integral inequality (HJII)-constrained optimization method. In other words, we solve a nonlinear HJII-constrained optimization problem for measuring the system entropy of nonlinear stochastic partial differential systems (NSPDSs). To simplify the system entropy measurement of NSPDSs, the global linearization technique and finite difference scheme were employed to approximate the nonlinear stochastic spatial state space system. This allows the nonlinear HJII-constrained optimization problem for the system entropy measurement to be transformed to an equivalent linear matrix inequalities (LMIs)-constrained optimization problem, which can be easily solved using the MATLAB LMI-toolbox (MATLAB R2014a, version 8.3). Finally, several examples are presented to illustrate the system entropy measurement of SPDSs.

A Comparison of Inferential Methods for Highly Nonlinear State Space Models in Ecology and Epidemiology

Highly nonlinear, chaotic or near chaotic, dynamic models are important in fields such as ecology and epidemiology: for example, pest species and diseases often display highly nonlinear dynamics. However, such models are problematic from the point of view of statistical inference. The defining feature of chaotic and near chaotic systems is extreme sensitivity to small changes in system states and parameters, and this can interfere with inference. There are two main classes of methods for circumventing these difficulties: information reduction approaches, such as Approximate Bayesian Computation or Synthetic Likelihood, and state space methods, such as Particle Markov chain Monte Carlo, Iterated Filtering or Parameter Cascading. The purpose of this article is to compare the methods in order to reach conclusions about how to approach inference with such models in practice. We show that neither class of methods is universally superior to the other. We show that state space methods can suffer multimodality problems in settings with low process noise or model misspecification, leading to bias toward stable dynamics and high process noise. Information reduction methods avoid this problem, but, under the correct model and with sufficient process noise, state space methods lead to substantially sharper inference than information reduction methods. More practically, there are also differences in the tuning requirements of different methods. Our overall conclusion is that model development and checking should probably be performed using an information reduction method with low tuning requirements, while for final inference it is likely to be better to switch to a state space method, checking results against the information reduction approach.

Dealing with Transients due to Multiple Experiments in Nonlinear System Identification

Although in many static estimation problems the data are collected from repeated experiments, the default underlying (assumption) setting in most of the system identification tasks is that the data are generated from a single experiment. The rationale for this is that more data can be obtained by increasing the measurement time instead of doing more experiments. As the measurement time tends to infinity, under suitable assumptions it is possible to estimate consistently the model parameters. In most of the real life cases, however, increasing the measurement time is either not possible or it does not cover the whole operating range of the system to be identified, hence data from multiple experiments need to be combined. Furthermore, most of the real life systems are nonlinear and a large variety of nonlinear systems can be described by a nonlinear state space model structure. In this paper, a methodology to deal with the transients arising due to concatenating data from multiple experiments during the identification of Polynomial nonlinear state space (PNLSS) models is described. The methodology is validated on data generated from a laboratory experimental set-up.

A Self-Organizing State Space Type Microstructure Model for Financial Asset Allocation

Among the models that describe the dynamic behaviors of financial market, the discrete time microstructure model stands out because of its efficiency in considering the relationship between the price, excess demand, and liquidity of a market. However, the estimation problem of such a microstructure model is challenging, because the model is essentially a nonlinear state space model. A decent solution is to define a self-organizing state-space model by combining the unknown parameters and the state vector of the original model into a new state vector. Then, the sequential Monte Carlo method can be used to simultaneously estimate the parameters and states. To handle the difficulty in setting the initial distributions of parameters for the self-organizing state space model, we propose to use the results obtained by the Kalman filter on the original microstructure model. Finally, a dynamic asset allocation strategy is designed based on estimated excess demand using the self-organizing state space model. The proposed methodology is evaluated by the China SZSE (ShenZhen Stock Exchange) Composite Index time series, and the results show its effectiveness.