non-Gaussian State-space Models Research Articles

Bayesian multivariate nonlinear state space copula models

A novel flexible class of multivariate nonlinear non-Gaussian state space models, based on copulas, is proposed. Specifically, it is assumed that the observation equation and the state equation are defined by copula families that are not necessarily equal. Inference is performed within the Bayesian framework, using the Hamiltonian Monte Carlo method. Simulation studies show that the proposed copula-based approach is extremely flexible, since it is able to describe a wide range of dependence structures and, at the same time, allows us to deal with missing data. The application to atmospheric pollutant measurement data shows that the approach is suitable for accurate modeling and prediction of data dynamics in the presence of missing values. Comparison to a Gaussian linear state space model and to Bayesian additive regression trees shows the superior performance of the proposed model with respect to predictive accuracy.

Estimation of Yu and Meyer bivariate stochastic volatility model by iterated filtering

In financial applications, understanding the asset correlation structure is crucial to tasks such as asset pricing, portfolio optimisation, risk management, and asset allocation. Thus, modelling the volatilities and correlations of multivariate stock market returns is of great importance. This paper proposes the iterated filtering algorithm for estimating the bivariate stochastic volatility model of Yu and Meyer. The iterated filtering method is a frequentist-based approach that utilises particle filters and can be applied to estimating the parameters of non-linear or non-Gaussian state-space models. The paper presents an empirical example that demonstrates the way in which the proposed estimation method might be used to estimate the correlation between the returns of two assets: Standard and Poor’s 500 index and the price of gold in US dollars. This is accompanied by a simulation study that proves the validity of the approach.

Efficient Data Augmentation Techniques for Some Classes of State Space Models

Data augmentation improves the convergence of iterative algorithms, such as the EM algorithm and Gibbs sampler by introducing carefully designed latent variables. In this article, we first propose a data augmentation scheme for the first-order autoregression plus noise model, where optimal values of working parameters introduced for recentering and rescaling of the latent states, can be derived analytically by minimizing the fraction of missing information in the EM algorithm. The proposed data augmentation scheme is then utilized to design efficient Markov chain Monte Carlo (MCMC) algorithms for Bayesian inference of some non-Gaussian and nonlinear state space models, via a mixture of normals approximation coupled with a block-specific reparametrization strategy. Applications on simulated and benchmark real data sets indicate that the proposed MCMC sampler can yield improvements in simulation efficiency compared with centering, noncentering and even the ancillarity-sufficiency interweaving strategy.

Predictability in Stock Returns in an Emerging Market: Evidence from KSE 100 Stock Price Index

We investigate the persistence in monthly KSE100 excess stock returns over the Treasury bills rates using non-Gaussian state space or unobservable component model with stable distributions and volatility persistence. Results from our non-Gaussian state space model, which is an improvement over Conard and Kaul (1988), show that the conditional distribution has a stable of 1.748 and normality is rejected even after accounting for GARCH. There exists a statistically significant predictable component in the KSE 100 excess stock returns. The optimal predictor in the unconditional expectation of the series is estimated to be 0.18 percent per annum. An evidence of highly nonconstant scales in different periods of time exhibits a tendency towards stock market crashes which invites remedial policy action.

Coupling Techniques for Nonlinear Ensemble Filtering

We consider filtering in high-dimensional non-Gaussian state-space models with intractable transition kernels, nonlinear and possibly chaotic dynamics, and sparse observations in space and time. We propose a novel filtering methodology that harnesses transportation of measures, convex optimization, and ideas from probabilistic graphical models to yield robust ensemble approximations of the filtering distribution in high dimensions. Our approach can be understood as the natural generalization of the ensemble Kalman filter (EnKF) to nonlinear updates, using stochastic or deterministic couplings. The use of nonlinear updates can reduce the intrinsic bias of the EnKF at a marginal increase in computational cost. We avoid any form of importance sampling and introduce non-Gaussian localization approaches for dimension scalability. Our framework achieves state-of-the-art tracking performance on challenging configurations of the Lorenz-96 model in the chaotic regime.

A Bayesian Non-Linear State Space Copula Model for Air Pollution in Beijing

AbstractAir pollution is a serious issue that currently affects many industrial cities in the world and can cause severe illness to the population. In particular, it has been proven that extreme high levels of airborne contaminants have dangerous short-term effects on human health, in terms of increased hospital admissions for cardiovascular and respiratory diseases and increased mortality risk. For these reasons, an accurate estimation of airborne pollutant concentrations is crucial. In this paper, we propose a flexible novel approach to model hourly measurements of fine particulate matter and meteorological data collected in Beijing in 2014. We show that the standard state space model, based on Gaussian assumptions, does not correctly capture the time dynamics of the observations. Therefore, we propose a non-linear non-Gaussian state space model where both the observation and the state equations are defined by copula specifications, and we perform Bayesian inference using the Hamiltonian Monte Carlo method. The proposed copula state space approach is very flexible, since it allows us to separately model the marginal distributions and to accommodate a wide variety of dependence structures in the data dynamics. We show that the proposed approach allows us not only to accurately estimate particulate matter measurements, but also to capture unusual high levels of air pollution, which were not detected by measured effects.

A closed-form filter for binary time series

Non-Gaussian state-space models arise in several applications, and within this framework the binary time series setting provides a relevant example. However, unlike for Gaussian state-space models — where filtering, predictive and smoothing distributions are available in closed form — binary state-space models require approximations or sequential Monte Carlo strategies for inference and prediction. This is due to the apparent absence of conjugacy between the Gaussian states and the likelihood induced by the observation equation for the binary data. In this article we prove that the filtering, predictive and smoothing distributions in dynamic probit models with Gaussian state variables are, in fact, available and belong to a class of unified skew-normals (sun) whose parameters can be updated recursively in time via analytical expressions. Also the key functionals of these distributions are, in principle, available, but their calculation requires the evaluation of multivariate Gaussian cumulative distribution functions. Leveraging sun properties, we address this issue via novel Monte Carlo methods based on independent samples from the smoothing distribution, that can easily be adapted to the filtering and predictive case, thus improving state-of-the-art approximate and sequential Monte Carlo inference in small-to-moderate dimensional studies. Novel sequential Monte Carlo procedures that exploit the sun properties are also developed to deal with online inference in high dimensions. Performance gains over competitors are outlined in a financial application.

Bssm: Bayesian Inference of Non-linear and Non-Gaussian State Space Models in R

We present an R package bssm for Bayesian non-linear/non-Gaussian state space modelling. Unlike the existing packages, bssm allows for easy-to-use approximate inference based on Gaussian approximations such as the Laplace approximation and the extended Kalman filter. The package accommodates also discretely observed latent diffusion processes. The inference is based on fully automatic, adaptive Markov chain Monte Carlo (MCMC) on the hyperparameters, with optional importance sampling post-correction to eliminate any approximation bias. The package implements also a direct pseudo-marginal MCMC and a delayed acceptance pseudo-marginal MCMC using intermediate approximations. The package offers an easy-to-use interface to define models with linear-Gaussian state dynamics with non-Gaussian observation models, and has an Rcpp interface for specifying custom non-linear and diffusion models.

Efficient Likelihood-based Estimation via Annealing for Dynamic Structural Macrofinance Models

Most solved dynamic structural macrofinance models are non-linear and/or non-Gaussian state-space models with high-dimensional and complex structures. We propose an annealed controlled sequential Monte Carlo method that delivers numerically stable and low variance estimators of the likelihood function. The method relies on an annealing procedure to gradually introduce information from observations and constructs globally optimal proposal distributions by solving associated optimal control problems that yield zero variance likelihood estimators. To perform parameter inference, we develop a new adaptive SMC$^2$ algorithm that employs likelihood estimators from annealed controlled sequential Monte Carlo. We provide a theoretical stability analysis that elucidates the advantages of our methodology and asymptotic results concerning the consistency and convergence rates of our SMC$^2$ estimators. We illustrate the strengths of our proposed methodology by estimating two popular macrofinance models: a non-linear new Keynesian dynamic stochastic general equilibrium model and a non-linear non-Gaussian consumption-based long-run risk model.

Controlled sequential Monte Carlo

Sequential Monte Carlo methods, also known as particle methods, are a popular set of techniques for approximating high-dimensional probability distributions and their normalizing constants. These methods have found numerous applications in statistics and related fields; e.g. for inference in non-linear non-Gaussian state space models, and in complex static models. Like many Monte Carlo sampling schemes, they rely on proposal distributions which crucially impact their performance. We introduce here a class of controlled sequential Monte Carlo algorithms, where the proposal distributions are determined by approximating the solution to an associated optimal control problem using an iterative scheme. This method builds upon a number of existing algorithms in econometrics, physics, and statistics for inference in state space models, and generalizes these methods so as to accommodate complex static models. We provide a theoretical analysis concerning the fluctuation and stability of this methodology that also provides insight into the properties of related algorithms. We demonstrate significant gains over state-of-the-art methods at a fixed computational complexity on a variety of applications.

PMCMC for Term Structure of Interest Rates under Markov Regime Switching and Jumps

AbstractA parameter estimation method, called PMCMC in this paper, is proposed to estimate a continuous-time model of the term structure of interests under Markov regime switching and jumps. There is a closed form solution to term structure of interest rates under Markov regime. However, the model is extended to be a CKLS model with non-closed form solutions which is a typical nonlinear and non-Gaussian state-space model(SSM) in the case of adding jumps. Although the difficulty of parameter estimation greatly prevents from researching models, we prove that the nonlinear and non-Gaussian state-space model has better performances in studying volatility. The method proposed in this paper will be implemented in simulation and empirical study for SHIBOR. Empirical results illustrate that the PMCMC algorithm has powerful advantages in tackling the models.

Inference for Nonlinear State Space Models: A Comparison of Different Methods applied to Markov-Switching Multifractal Models

Nonlinear, non-Gaussian state space models have found wide applications in many areas. These models usually do not allow for an analytical representation of their likelihood function and thus, sequential Monte Carlo or particle filter methods are mostly applied to estimate their parameters. Finding the best-fitting parameters of a model is a non-trivial task since stochastic approximations lead to non-smooth likelihood functions. Recently proposed iterative filtering algorithms developed for this purpose are compared with simpler on-line filters and more traditional methods of inference. A highly nonlinear class of Markov-switching models, the so called Markov-switching multifractal model (MSM) is used as illustrative example in the comparison of different optimisation routines. Besides the well-established univariate discrete-time MSM, univariate and multivariate continuous-time versions of MSM are considered. Monte Carlo simulation experiments indicate that across a variety of MSM specifications, the classical Nelder-Mead or simplex algorithm appears still as more efficient and robust compared to a number of online and iterated filters. A very close competitor is one of the recently proposed iterated filters while other alternatives are mostly dominated by these two algorithms. An empirical application of both discrete and continuous-time MSM to seven financial time series shows that both models dominate GARCH and FIGARCH models in terms of in-sample goodness-of-fit. Out-of-sample forecast comparisons show in the majority of cases a clear dominance of the continuous-time MSM under a mean absolute error criterion, and less conclusive results under a mean squared error criterion.

A flexible particle Markov chain Monte Carlo method

Particle Markov Chain Monte Carlo methods are used to carry out inference in nonlinear and non-Gaussian state space models, where the posterior density of the states is approximated using particles. Current approaches usually perform Bayesian inference using either a particle marginal Metropolis–Hastings (PMMH) algorithm or a particle Gibbs (PG) sampler. This paper shows how the two ways of generating variables mentioned above can be combined in a flexible manner to give sampling schemes that converge to a desired target distribution. The advantage of our approach is that the sampling scheme can be tailored to obtain good results for different applications. For example, when some parameters and the states are highly correlated, such parameters can be generated using PMMH, while all other parameters are generated using PG because it is easier to obtain good proposals for the parameters within the PG framework. We derive some convergence properties of our sampling scheme and also investigate its performance empirically by applying it to univariate and multivariate stochastic volatility models and comparing it to other PMCMC methods proposed in the literature.

Importance Densities for Particle Filtering Using Iterated Conditional Expectations

In this letter, we consider Gaussian approximations of the optimal importance density in sequential importance sampling for nonlinear, non-Gaussian state-space models. The proposed method is based on generalized statistical linear regression and posterior linearization using conditional expectations. Simulation results show that the method outperforms the compared methods in terms of the effective sample size and provides a better local approximation of the optimal importance density.

An Adaptive Gaussian Sum Kalman Filter Based on a Partial Variational Bayesian Method

In this article, we address the online state estimation problem of linear discrete-time systems in the presence of inaccurate and slowly time-varying non-Gaussian measurement noise (NGMN). Recently, the variational Bayesian (VB) method has been successfully used to jointly estimate the system state along with the statistics of the unknown Gaussian measurement noise. However, we prove that the original VB method for the non-Gaussian state-space models, modeled by the Gaussian mixture distributions, is analytically intractable. To overcome this problem, we propose a partial VB-based adaptive Gaussian sum Kalman filter, which uses a feedback-based filtering framework to independently calculate the posterior distribution of the state and posterior distribution of the NGMN. Experimental results demonstrate the effectiveness of the proposed filter.

Bayesian Model Comparison with the Hyvärinen Score: Computation and Consistency

ABSTRACT The Bayes factor is a widely used criterion in model comparison and its logarithm is a difference of out-of-sample predictive scores under the logarithmic scoring rule. However, when some of the candidate models involve vague priors on their parameters, the log-Bayes factor features an arbitrary additive constant that hinders its interpretation. As an alternative, we consider model comparison using the Hyvärinen score. We propose a method to consistently estimate this score for parametric models, using sequential Monte Carlo methods. We show that this score can be estimated for models with tractable likelihoods as well as nonlinear non-Gaussian state-space models with intractable likelihoods. We prove the asymptotic consistency of this new model selection criterion under strong regularity assumptions in the case of nonnested models, and we provide qualitative insights for the nested case. We also use existing characterizations of proper scoring rules on discrete spaces to extend the Hyvärinen score to discrete observations. Our numerical illustrations include Lévy-driven stochastic volatility models and diffusion models for population dynamics. Supplementary materials for this article are available online.

The Gibbs sampler with particle efficient importance sampling for state-space models*

We consider Particle Gibbs (PG) for Bayesian analysis of non-linear non-Gaussian state-space models. As a Monte Carlo (MC) approximation of the Gibbs procedure, PG uses sequential MC (SMC) importance sampling inside the Gibbs to update the latent states. We propose to combine PG with the Particle Efficient Importance Sampling (PEIS). By using SMC sampling densities which are approximately globally fully adapted to the targeted density of the states, PEIS can substantially improve the simulation efficiency of the PG relative to existing PG implementations. The efficiency gains are illustrated in PG applications to a non-linear local-level model and stochastic volatility models.

An Efficient Sequential Learning Algorithm in Regime-Switching Environments

We provide a noble approach of estimating a regime-switching nonlinear and non-Gaussian state space model based on a particle learning scheme. Particularly, we extend the particle learning method in Liu and West (2001) by constructing a new proposal distribution for the latent regime index variable that incorporates all available information contained in the current and past observations. The Monte Carlo simulation result implies that our approach categorically outperforms a popular existing algorithm. For an empirical illustration, the proposed algorithm is used to analyze the underlying dynamics of the U.S. excess stock return.

An efficient sequential learning algorithm in regime-switching environments

Abstract We provide a novel approach of estimating a regime-switching nonlinear and non-Gaussian state-space model based on a particle learning scheme. In particular, we extend the particle learning method in Liu, J., and M. West. 2001. “Combined Parameter and State Estimation in Simulation-Based Filtering.” In Sequential Monte Carlo Methods in Practice, 197–223. Springer. by constructing a new proposal distribution for the latent regime index variable that incorporates all available information contained in the current and past observations. The Monte Carlo simulation result implies that our approach categorically outperforms a popular existing algorithm. For empirical illustration, the proposed algorithm is used to analyze the underlying dynamics of US excess stock return.

A method for high-dimensional smoothing

We consider the problem of the computation of smoothed additive functionals, which are some integrals with respect to the joint smoothing distribution. It is a key issue in inference for general state-space models as these quantities appear naturally for maximum likelihood parameter inference. The computation of smoothed additive functionals is very challenging as exact computations are not possible for non-linear non-Gaussian state-space models. It becomes even more difficult when the hidden state lies in a high dimensional space because traditional numerical methods suffer from the curse of dimensionality. We propose a new algorithm to efficiently calculate the smoothed additive functionals in an online manner for a specific family of high-dimensional state-space models in discrete time, which is named the Space–Time Forward Smoothing (STFS) algorithm. The cost of this algorithm is at least O(N2d2T), which is polynomial in d. T and N denote the number of time steps and the number of particles respectively, while d is the dimension of the hidden state space. Its superior performance over other existing methods is illustrated by various simulation studies. Moreover, STFS algorithm is successfully applied to perform Maximum Likelihood estimation for static model parameters both in an online and an offline manner.