Gibbs Sampling for Bayesian Prediction of SARMA Processes

Ayman Amin

doi:10.18187/pjsor.v15i2.2174

Abstract

In this article we present a Bayesian prediction of multiplicative seasonal autoregressive moving average (SARMA) processes using the Gibbs sampling algorithm. First, we estimate the unobserved errors using the nonlinear least squares (NLS) method to approximate the likelihood function. Second, we employ conjugate priors on the model parameters and initial values and assume the model errors are normally distributed to derive the conditional posterior and predictive distributions. In particular, we show that the conditional posterior distribution of the model parameters and the variance are multivariate normal and inverse gamma respectively, and the conditional predictive distribution of the future observations is a multivariate normal. Finally, we use these closed-form conditional posterior and predictive distributions to apply the Gibbs sampling algorithm to approximate empirically the marginal posterior and predictive distributions, enabling us easily to carry out multiple-step ahead predictions. We evaluate our proposed Bayesian method using simulation study and real-world time series datasets.

Highlights

Bayesian analysis of seasonal autoregressive moving average (SARMA) models is difficult since the likelihood function is analytically intractable, which causes problems in the posterior and predictive analysis
Different approaches have been proposed in literature for the Bayesian analysis of SARMA models including analytical approximations and Markov Chain Monte Carlo (MCMC) methods-based approximations
We mean the proposed Bayesian methods that approximate the posterior and predictive densities to be standard closed-form distributions that are analytically tractable, see for example Newbold (1973), Zellner and Reynolds (1978), Broemeling and Shaarawy (1984), and Amin (2018a, 2018b). These methods are conditioning on the initial values leading to waste observations, and treat SARMA model as an additive not a multiplicative model which can increase the number of unnecessary parameters

Summary

Introduction

Bayesian analysis of seasonal autoregressive moving average (SARMA) models is difficult since the likelihood function is analytically intractable, which causes problems in the posterior and predictive analysis. We mean the proposed Bayesian methods that approximate the posterior and predictive densities to be standard closed-form distributions that are analytically tractable, see for example Newbold (1973), Zellner and Reynolds (1978), Broemeling and Shaarawy (1984), and Amin (2018a, 2018b) These methods are conditioning on the initial values leading to waste observations, and treat SARMA model as an additive not a multiplicative model which can increase the number of unnecessary parameters. Vermaak et al (1998) proposed a Bayesian estimation of SAR models, aiming to model speech production for voiced sounds Their estimation is based on the state space formulation and using the Metropolis within Gibbs sampling algorithm, without considering the prediction problem. Amin (2009) and Amin and Ismail (2014,2015) used the Gibbs sampling algorithm to present a Bayesian estimation of multiplicative SARMA and double SAR models, without considering the prediction problem.

Multiplicative SARMA Models and Bayesian Concepts

Posterior Analysis for SARMA Models

Predictive Analysis for SARMA Models

Proposed Gibbs Sampling Algorithm for SARMA Models

Simulation Study

Real-World Application

Findings

Conclusions