Gaussian State Space Models Research Articles

Improved latent variable models for nonlinear and dynamic process monitoring

The kernel mapping is a popular method for designing nonlinear process monitoring techniques. In most cases, kernel mapping is achieved by using the radial basis kernel function which, in theory, is able to provide an infinite order nonlinear mapping. However, such an infinite order mapping might be redundant and inefficient as the order of nonlinear relationship between process variables of industrial process system is often limited by many physical constraints. To address this issue, an efficient nonlinear mapping method called the constructive polynomial mapping (CPM) is utilized in lieu of the radial basis kernel function to avoid the excessive modeling redundancy. In addition, the CPM is integrated with the dynamic principal component analysis and the linear Gaussian state space model to build improved latent variable models for nonlinear and dynamic process monitoring. The promising performance of the proposed models has been demonstrated through two case studies.

Forecasting compositional time series: A state space approach

A framework for the forecasting of composite time series, such as market shares, is proposed. Based on Gaussian multi-series innovations state space models, it relies on the log-ratio function to transform the observed shares (proportions) onto the real line. The models possess an unrestricted covariance matrix, but also have certain structural elements that are common to all series, which is proved to be both necessary and sufficient to ensure that the predictions of shares are invariant to the choice of base series. The framework includes a computationally efficient maximum likelihood approach to estimation, relying on exponential smoothing methods, which can be adapted to handle series that start late or finish early (new or withdrawn products). Simulated joint prediction distributions provide approximations to the required prediction distributions of individual shares and the associated quantities of interest. The approach is illustrated on US automobile market share data for the period 1961–2013.

Structured Inference Networks for Nonlinear State Space Models

Gaussian state space models have been used for decades as generative models of sequential data. They admit an intuitive probabilistic interpretation, have a simple functional form, and enjoy widespread adoption. We introduce a unified algorithm to efficiently learn a broad class of linear and non-linear state space models, including variants where the emission and transition distributions are modeled by deep neural networks. Our learning algorithm simultaneously learns a compiled inference network and the generative model, leveraging a structured variational approximation parameterized by recurrent neural networks to mimic the posterior distribution. We apply the learning algorithm to both synthetic and real-world datasets, demonstrating its scalability and versatility. We find that using the structured approximation to the posterior results in models with significantly higher held-out likelihood.

Time Series Models with a Common Stochastic Variance for Analysing Economic Time Series

The linear Gaussian state space model for which the common variance is treated as a stochastic time-varying variable is considered for the modelling of economic time series. The focus of this paper is on the simultaneous estimation of parameters related to the stochastic processes of the mean part and the variance part of the model. The estimation method is based on maximum likelihood and it requires the subsequent uses of the Kalman filter to treat the mean part and sampling techniques to treat the variance part. This approach leads to the evaluation of the exact likelihood function of the model subject to simulation error. The standard asymptotic properties of maximum likelihood estimators apply as a result. A Monte Carlo study is carried out to investigate the small-sample properties of the estimation procedure. We present two illustrations which are concerned with the modelling and forecasting of two U.S. macroeconomic time series: Inflation and industrial production.

KFAS: Exponential Family State Space Models in R

State space modelling is an efficient and flexible method for statistical inference of a broad class of time series and other data. This paper describes an R package KFAS for state space modelling with the observations from an exponential family, namely Gaussian, Poisson, binomial, negative binomial and gamma distributions. After introducing the basic theory behind Gaussian and non-Gaussian state space models, an illustrative example of Poisson time series forecasting is provided. Finally, a comparison to alternative R packages suitable for non-Gaussian time series modelling is presented.

A New Process Uncertainty Robust Student’s t Based Kalman Filter for SINS/GPS Integration

Motivated by the problem that the Gaussian assumption of process noise may be violated and the statistics of process noise may be inaccurate when the carrier maneuvers severely, a new process uncertainty robust Student’s t-based Kalman filter is proposed to integrate the strap-down inertial navigation system (SINS) and global positioning system (GPS). To better address the heavy-tailed process noise induced by severe maneuvering, the one-step predicted probability density function is modeled as a Student’s t distribution, and the conjugate prior distributions of inaccurate mean vector, scale matrix, and degrees of freedom (dofs) parameter are, respectively, selected as Gaussian, inverse Wishart, and Gamma distributions, based on which a new Student’s t-based hierarchical Gaussian state-space model for SINS/GPS integration is constructed. The state vector, auxiliary random variable, mean vector, scale matrix, and dof parameter are jointly estimated based on the constructed hierarchical Gaussian state-space model using the variational Bayesian approach. Experimental results illustrate that the proposed method has significantly better robustness for the suppression of the process uncertainty but slightly higher computational complexity than the existing state-of-the-art methods.

Understanding the Ensemble Kalman Filter

ABSTRACTThe ensemble Kalman filter (EnKF) is a computational technique for approximate inference in state-space models. In typical applications, the state vectors are large spatial fields that are observed sequentially over time. The EnKF approximates the Kalman filter by representing the distribution of the state with an ensemble of draws from that distribution. The ensemble members are updated based on newly available data by shifting instead of reweighting, which allows the EnKF to avoid the degeneracy problems of reweighting-based algorithms. Taken together, the ensemble representation and shifting-based updates make the EnKF computationally feasible even for extremely high-dimensional state spaces. The EnKF is successfully used in data-assimilation applications with tens of millions of dimensions. While it implicitly assumes a linear Gaussian state-space model, it has also turned out to be remarkably robust to deviations from these assumptions in many applications. Despite its successes, the EnKF is largely unknown in the statistics community. We aim to change that with the present article, and to entice more statisticians to work on this topic.

Bayesian Learning of Degenerate Linear Gaussian State Space Models Using Markov Chain Monte Carlo

Linear Gaussian state-space models are ubiquitous in signal processing, and an important procedure is that of estimating system parameters from observed data. Rather than making a single point estimate, it is often desirable to conduct Bayesian learning, in which the entire posterior distribution of the unknown parameters is sought. This can be achieved using Markov chain Monte Carlo. On some occasions it is possible to deduce the form of the unknown system matrices in terms of a small number of scalar parameters, by considering the underlying physical processes involved. Here we study the case where this is not possible, and the entire matrices must be treated as unknowns. An efficient Gibbs sampling algorithm exists for the basic formulation of linear model. We extend this to the more challenging situation where the transition model is possibly degenerate, i.e., the transition covariance matrix is singular. Appropriate Markov kernels are devised and demonstrated with simulations.

Bayesian estimation inherent in a Mexican-hat-type neural network.

Brain functions, such as perception, motor control and learning, and decision making, have been explained based on a Bayesian framework, i.e., to decrease the effects of noise inherent in the human nervous system or external environment, our brain integrates sensory and a priori information in a Bayesian optimal manner. However, it remains unclear how Bayesian computations are implemented in the brain. Herein, I address this issue by analyzing a Mexican-hat-type neural network, which was used as a model of the visual cortex, motor cortex, and prefrontal cortex. I analytically demonstrate that the dynamics of an order parameter in the model corresponds exactly to a variational inference of a linear Gaussian state-space model, a Bayesian estimation, when the strength of recurrent synaptic connectivity is appropriately stronger than that of an external stimulus, a plausible condition in the brain. This exact correspondence can reveal the relationship between the parameters in the Bayesian estimation and those in the neural network, providing insight for understanding brain functions.

Dynamic Monitoring and Control of Irreversible Chronic Diseases with Application to Glaucoma

To effectively manage chronic disease patients, clinicians must know (1) how to monitor each patient (i.e., when to schedule the next visit and which tests to take), and (2) how to control the disease (i.e., what levels of controllable risk factors will sufficiently slow progression). Our research addresses these questions simultaneously and provides the optimal solution to a novel linear quadratic Gaussian state space model. For the new objective of minimizing the relative change in state over time (i.e., disease progression), which is necessary for management of irreversible chronic diseases, we show that the classical two-way separation of estimation and control holds, thereby making a previously intractable problem solvable by decomposition into two separate, tractable problems while maintaining optimality. The resulting optimization is applied to the management of glaucoma. Based on data from two large randomized clinical trials, we validate our model and demonstrate how our decision support tool can provide actionable insights to the clinician caring for a patient with glaucoma. This methodology can be applied to a broad range of irreversible chronic diseases to optimally devise patient-specific monitoring and treatment plans.

Fast Kalman-Like Filtering for Large-Dimensional Linear and Gaussian State-Space Models

This article considers the filtering problem for linear and Gaussian state-space models with large dimensions, a setup in which the optimal Kalman Filter (KF) might not be applicable owing to the excessive cost of manipulating huge covariance matrices. Among the most popular alternatives that enable cheaper and reasonable computation is the Ensemble KF (EnKF), a Monte Carlo-based approximation. In this article, we consider a class of a posteriori distributions with diagonal covariance matrices and propose fast approximate deterministic-based algorithms based on the Variational Bayesian (VB) approach. More specifically, we derive two iterative KF-like algorithms that differ in the way they operate between two successive filtering estimates; one involves a smoothing estimate and the other involves a prediction estimate. Despite its iterative nature, the prediction-based algorithm provides a computational cost that is, on the one hand, independent of the number of iterations in the limit of very large state dimensions, and on the other hand, always much smaller than the cost of the EnKF. The cost of the smoothing-based algorithm depends on the number of iterations that may, in some situations, make this algorithm slower than the EnKF. The performances of the proposed filters are studied and compared to those of the KF and EnKF through a numerical example.

On Modelling and Forecasting Predictable Components in European Stock Markets

This study investigates possible existence of predictable components in stock excess returns in eighteen European countries i.e. Austria, Croatia, Denmark, France, Germany, Greece, Hungary, Iceland, Ireland, Italy, Netherlands, Norway, Russia, Slovakia, Spain, Sweden, Switzerland, and United Kingdom. The excess return series for these countries are modeled using non Gaussian state space or unobserved component model that encompasses non normality and time varying volatility that might be present in the series. While statistically significant evidence of non-normality and volatility persistence does exist in most series, statistically significant persistent predictable component also prevails in Austria, France, Germany, Iceland, Ireland, Italy, Netherlands, Norway, Spain, Sweden, and Switzerland excess returns. However, the results on possible existence of predictable components in stock excess returns in Croatia, Denmark, Greece, Hungary, Russia, Slovakia, and United Kingdom are in sharp contrast. The efficiently estimated excess returns range between 0.002 % per month for Slovakia to 0.094 % per month for Russia stock excess returns. The characteristic exponent ranges between of 1.632 for Croatia to 1.917 for France showing non-normal behavior in these series although the characteristic exponent of 1.999 shows a normal behavior in Italian and Russian excess return series.

Price generating process and volatility in the Nigerian agricultural commodities market

The study examined the price generating process and volatility of Nigerian agricultural commodities market using secondary data for price series on meat, cereals, sugar, dairy and food for the period of January 1990 to February 2014. The data were analysed using both descriptive and inferential statistics. The descriptive statistics used the coefficient of variation while the inferential statistics used the linear Gaussian State-Space (SS) model. The results of the descriptive statistics showed that the coefficients of variation for cereals (39.88 %), food (32.65 %) and dairy price (43.08 %) were respectively higher during the overall time period (January 1990 to February 2014) than during the first (January 1990 to January 2002) and second (February 2002 to February 2014) sub-time periods. The results of the inferential statistics showed that authoregressive moving average (ARMA) model is the most selected Nigeria agricultural commodity price generating model for the time periods, and that the final states of their unobserved component of cereals, meat, dairy and sugar prices were 6317.86, 2.06, 34.45 and 10.24 respectively. The prices of cereals, meat, dairy, sugar and food in general were generated and most fitted by the ARMA in Nigeria. Also the prices have been on the increase and have exhibited high volatility. The volatility, process and the determinants of the Nigerian food commodities prices can best be described by the simple (ARMA) model. Keywords : price, volatility, state-space, stationarity, ARMA

Combining particle MCMC with Rao-Blackwellized Monte Carlo data association for parameter estimation in multiple target tracking

We consider state and parameter estimation in multiple target tracking problems with data association uncertainties and unknown number of targets. We show how the problem can be recast into a conditionally linear Gaussian state-space model with unknown parameters and present an algorithm for computationally efficient inference on the resulting model. The proposed algorithm is based on combining the Rao-Blackwellized Monte Carlo data association algorithm with particle Markov chain Monte Carlo algorithms to jointly estimate both parameters and data associations. Both particle marginal Metropolis–Hastings and particle Gibbs variants of particle MCMC are considered. We demonstrate the performance of the method both using simulated data and in a real-data case study of using multiple target tracking to estimate the brown bear population in Finland.

Matrix exponential stochastic volatility with cross leverage

A multivariate stochastic volatility model with the dynamic correlation and the cross leverage effect is described and its estimation using Markov chain Monte Carlo is proposed. The time-varying covariance matrices are guaranteed to be positive definite by using a matrix exponential transformation. Of particular interest is our approach for sampling a set of latent matrix logarithm variables from their conditional posterior distribution, where we construct the proposal density based on an approximating linear Gaussian state space model. The proposed model and its extensions with fat-tailed error distribution are applied to trivariate returns data (daily stocks, bonds, and exchange rates) of Japan. Further, a model comparison is conducted including constant correlation multivariate stochastic volatility models with leverage and diagonal multivariate GARCH models.

Marginalized Predictive Likelihood Comparisons of Linear Gaussian State-Space Models with Applications to DSGE, DSGE-VAR, and VAR Models

The predictive likelihood is of particular relevance in a Bayesian setting when the purpose is to rank models in a forecast comparison exercise. This paper discusses how the predictive likelihood can be estimated for any subset of the observable variables in linear Gaussian state-space models with Bayesian methods, and proposes to utilize a missing observations consistent Kalman filter in the process of achieving this objective. As an empirical application, we analyze euro area data and compare the density forecast performance of a DSGE model to DSGE-VARs and reduced-form linear Gaussian models.

Density-Tempered Marginalized Sequential Monte Carlo Samplers

We propose a density-tempered marginalized sequential Monte Carlo (SMC) sampler, a new class of samplers for full Bayesian inference of general state-space models. The dynamic states are approximately marginalized out using a particle filter, and the parameters are sampled via a sequential Monte Carlo sampler over a density-tempered bridge between the prior and the posterior. Our approach delivers exact draws from the joint posterior of the parameters and the latent states for any given number of state particles and is thus easily parallelizable in implementation. We also build into the proposed method a device that can automatically select a suitable number of state particles. Since the method incorporates sample information in a smooth fashion, it delivers good performance in the presence of outliers. We check the performance of the density-tempered SMC algorithm using simulated data based on a linear Gaussian state-space model with and without misspecification. We also apply it on real stock prices using a GARCH-type model with microstructure noise.

Efficient estimation of conditionally linear and Gaussian state space models

An efficient estimation procedure for conditionally linear and Gaussian state space models is developed. Efficient importance sampling together with a Rao-Blackwellization step are used to construct a highly efficient estimation method that produces continuous approximations to the likelihood function, greatly enhancing simulated maximum likelihood estimation. An application where the unobserved component stochastic volatility model is used to model inflation is proposed and parameter estimates for all G7 countries are shown to be statistically different from calibrated values used in the literature. The estimated model is used to forecast inflation of these countries.

Spatial functional normal mixed effect approach for curve classification

This paper proposes a spatial functional formulation of the normal mixed effect model for the statistical classification of spatially dependent Gaussian curves, in both parametric and state space model frameworks. Fixed effect parameters are represented in terms of a functional multiple regression model whose regression operators can change in space. Local spatial homogeneity of these operators is measured in terms of their Hilbert---Schmidt distances, leading to the classification of fixed effect curves in different groups. Assuming that the Gaussian random effect curves obey a spatial autoregressive dynamics of order one [SARH(1) dynamics], a second functional classification criterion is proposed in order to detect local spatially homogeneous patterns in the mean quadratic functional variation of Gaussian random effect curve increments. Finally, the two criteria are combined to detect local spatially homogeneous patterns in the regression operators and in the functional mean quadratic variation, under a state space approach. A real data example in the financial context is analyzed as an illustration.

Automatic detection of spindles and K-complexes in sleep EEG using switching multiple models

This work investigates the use of switching linear Gaussian state space models for the segmentation and automatic labelling of Stage 2 sleep EEG data characterised by spindles and K-complexes. The advantage of this approach is that it offers a unified framework of detecting multiple transient events within background EEG data. Specifically for the identification of background EEG, spindles and K-complexes, a true positive rate (false positive rate) of 76.04% (33.47%), 83.49% (47.26%) and 52.02% (7.73%) respectively was obtained on a sample by sample basis. A novel semi-supervised model allocation approach is also proposed, allowing new unknown modes to be learnt in real time.