Nonstationary Time Series Models Research Articles

Stochastic simulation model for nonstationary time series using an autoregressive wavelet decomposition: Applications to rainfall and temperature

A time series simulation scheme based on wavelet decomposition coupled to an autoregressive model is presented for hydroclimatic series that exhibit band‐limited low‐frequency variability. Many nonlinear dynamical systems generate time series that appear to have amplitude‐ and frequency‐modulated oscillations that may correspond to the recurrence of different solution regimes. The use of wavelet decomposition followed by an autoregressive model of each leading component is explored as a model for such time series. The first example considered is the Lorenz‐84 low‐order model of extratropical circulation, which has been used to illustrate how chaos and intransitivity (multiple stable solutions) can lead to low‐frequency variability. The central England temperature (CET) time series, the NINO3.4 series that is a surrogate for El Nino–Southern Oscillation, and seasonal rainfall from Everglades National Park, Florida, are then modeled with this approach. The proposed simulation model yields better results than a traditional linear autoregressive (AR) time series model in terms of reproducing the time‐frequency properties of the observed rainfall, while preserving the statistics usually reproduced by the AR models.

ON THE IDENTIFICATION AND ESTIMATION OF NONSTATIONARY AND COINTEGRATED ARMAX SYSTEMS

This paper extends current theory on the identification and estimation of vector time series models to nonstationary processes. It examines the structure of dynamic simultaneous equations systems or ARMAX processes that start from a given set of initial conditions and evolve over a given, possibly infinite, future time horizon. The analysis proceeds by deriving the echelon canonical form for such processes. The results are obtained by amalgamating ideas from the theory of stochastic difference equations with adaptations of the Kronecker index theory of dynamic systems. An extension of these results to the analysis of unit-root, partially nonstationary (cointegrated) time series models is also presented, leading to straightforward identification conditions for the error correction, echelon canonical form. An innovations algorithm for the evaluation of the exact Gaussian likelihood is given. The asymptotic properties of the approximate Gaussian estimator and the exact maximum likelihood estimator based upon the algorithm are derived for the cointegrated case. Examples illustrating the theory are discussed, and some experimental evidence is also presented.

The Application of Kernel Smoothing to Time Series Data

There are already a lot of models to fit a set of stationary time series, such as AR, MA, and ARMA models. For the non-stationary data, an ARIMA or seasonal ARIMA models can be used to fit the given data. Moreover, there are also many statistical softwares that can be used to build a stationary or non-stationary time series model for a given set of time series data, such as SAS, SPLUS, etc. However, some statistical softwares wouldn’t work well for small samples with or without missing data, especially for small time series data with seasonal trend. A nonparametric smoothing technique to build a forecasting model for a given small seasonal time series data is carried out in this paper. And then, both the method provided in this paper and that in SAS package are applied to the modeling of international airline passengers data respectively, the comparisons between the two methods are done afterwards. The results of the comparison show us the method provided in this paper has superiority over SAS’s method.

Identifying the time‐effect factors of multiple time series

AbstractThe Peña–Box model is considered for finding the time‐effect factors of a multiple time series. This paper first establishes the connection between the Peña–Box model and the vector ARMA model. According to the Peña–Box model, some series can be ignored while modelling the vector ARMA model. A consistent estimator is then proposed to identify the model for nonlinear and nonstationary time series. Finally, the finite‐sample behaviour of the estimator is illustrated via simulations. Copyright © 2005 John Wiley & Sons, Ltd.

Auto-Regressive Models of Non-Stationary Time Series with Finite Length

To analyze and simulate non-stationary time series with finite length, the statistical characteristics and auto-regressive (AR) models of non-stationary time series with finite length are discussed and studied. A new AR model called the time varying parameter AR model is proposed for solution of non-stationary time series with finite length. The auto-covariances of time series simulated by means of several AR models are analyzed. The result shows that the new AR model can be used to simulate and generate a new time series with the auto-covariance same as the original time series. The size curves of cocoon filaments regarded as non-stationary time series with finite length are experimentally simulated. The simulation results are significantly better than those obtained so far, and illustrate the availability of the time varying parameter AR model. The results are useful for analyzing and simulating non-stationary time series with finite length.

The distance between rival nonstationary fractional processes

Asymptotic inference on nonstationary fractional time series models, including cointegrated ones, is proceeding along two routes, determined by alternative definitions of nonstationary processes. We derive bounds for the mean squared error of the difference between (possibly tapered) discrete Fourier transforms under the two regimes. We apply the results to deduce limit theory for estimates of memory parameters, including ones for cointegrating errors, with mention also of implications for estimates of cointegrating coefficients.

On the cusum of squares test for variance change in nonstationary and nonparametric time series models

In this paper we consider the problem of testing for a variance change in nonstationary and nonparametric time series models. The models under consideration are the unstable AR(q) model and the fixed design nonparametric regression model with a strong mixing error process. In order to perform a test, we employ the cusum of squares test introduced by Inclan and Tiao (1994,J. Amer. Statist. Assoc.,89, 913–923). It is shown that the limiting distribution of the test statistic is the sup of a standard Brownian bridge as seen in iid random samples. Simulation results are provided for illustration.

Recursive estimation of model parameters with sharp discontinuity in non-stationary air quality data

Recursive method of time series filtering and smoothing based on the state–space concept provides a natural approach to the modeling of non-stationary environmental time series. The flexibility of this stochastic formulation allows for a suitable degree of variability in the estimated components, and in this paper we show how it can be extended for handling sharp changes or discontinuities in the model parameters. The approach is based on the time variable parameter version of the well known linear regression model and exploits the suite of recursive Kalman filtering and fixed interval smoothing (FIS) algorithms. The practical utility of the method is demonstrated by an example of modeling of the RSP levels during an episode event.

MINIMUM DISTANCE ESTIMATION OF NONSTATIONARY TIME SERIES MODELS

This paper analyzes the limit distribution of minimum distance (MD) estimators for nonstationary time series models that involve nonlinear parameter restrictions. A rotation for the restricted parameter space is constructed to separate the components of the MD estimator that converge at different rates. We derive regularity conditions for the restriction function that are easier to verify than the stochastic equicontinuity conditions that arise from direct estimation of the restricted parameters. The sequence of matrices that is used to weigh the discrepancy between the unrestricted estimates and the restriction function is allowed to have a stochastic limit. For MD estimators based on unrestricted estimators with a mixed normal asymptotic distribution the optimal weight matrix is derived and a goodness-of-fit test is proposed. Our estimation theory is illustrated in the context of a permanent-income model and a present-value model.

Wavelet packet transfer function modelling of nonstationary time series

This article shows how a non-decimated wavelet packet transform (NWPT) can be used to model a response time series, Yt, in terms of an explanatory time series, Xt. The proposed computational technique transforms the explanatory time series into a NWPT representation and then uses standard statistical modelling methods to identify which wavelet packets are useful for modelling the response time series. We exhibit S-Plus functions from the freeware WaveThresh package that implement our methodology. The proposed modelling methodology is applied to an important problem from the wind energy industry: how to model wind speed at a target location using wind speed and direction from a reference location. Our method improves on existing target site wind speed predictions produced by widely used industry standard techniques. However, of more importance, our NWPT representation produces models to which we can attach physical and scientific interpretations and in the wind example enable us to understand more about the transfer of wind energy from site to site.

Multichannel Electroencephalographic Analyses via Dynamic Regression Models with Time-Varying Lag–Lead Structure

SUMMARY Multiple time series of scalp electrical potential activity are generated routinely in electroencephalographic (EEG) studies. Such recordings provide important non-invasive data about brain function in human neuropsychiatric disorders. Analyses of EEG traces aim to isolate characteristics of their spatiotemporal dynamics that may be useful in diagnosis, or may improve the understanding of the underlying neurophysiology or may improve treatment through identifying predictors and indicators of clinical outcomes. We discuss the development and application of non-stationary time series models for multiple EEG series generated from individual subjects in a clinical neuropsychiatric setting. The subjects are depressed patients experiencing generalized tonic–clonic seizures elicited by electroconvulsive therapy (ECT) as antidepressant treatment. Two varieties of models—dynamic latent factor models and dynamic regression models—are introduced and studied. We discuss model motivation and form, and aspects of statistical analysis including parameter identifiability, posterior inference and implementation of these models via Markov chain Monte Carlo techniques. In an application to the analysis of a typical set of 19 EEG series recorded during an ECT seizure at different locations over a patient’s scalp, these models reveal time-varying features across the series that are strongly related to the placement of the electrodes. We illustrate various model outputs, the exploration of such time-varying spatial structure and its relevance in the ECT study, and in basic EEG research in general.

Evaluation and Comparison of EEG Traces: Latent Structure in Nonstationary Time Series

We explore and illustrate the use of time series decomposition methods for evaluating and comparing latent structure in nonstationary electroencephalographic (EEG) traces obtained from depressed patients during brain seizures induced as part of electroconvulsive therapy (ECT). Analysis of the patterns of change over time in the frequency structure of such EEG data provides insight into the neurophysiological mechanisms of action of this effective but poorly understood antidepressant treatment, and allows clinicians to modify ECT treatments to optimize therapeutic benefits while minimizing associated side effects. Our work has introduced new methods of time-frequency analysis of EEG series that identify the complete pattern of time evolution of frequency structure over the course of a seizure, and usefully assist in these scientific and clinical studies. New methods of decomposition of flexible dynamic models provide time domain decompositions of individual EEG series into collections of latent components in different frequency bands. This allows us to explore ECT seizure characteristics via inferences on the time-varying parameters that characterize these latent components, and to relate differences in such characteristics across seizures to differences in the therapeutic effectiveness and cognitive side effects of those seizures. This article discusses the scientific context and problems, development of nonstationary time series models and new methods of decomposition to explore time-frequency structure, and aspects of model fitting and analysis. We include applied studies on two datasets from recent clinical ECT studies. One is an initial illustrative analysis of a single EEG trace, the second compares the EEG data recorded during two types of ECT treatment that differ in therapeutic effectiveness and cognitive side effects. The uses of these models and time series decomposition methods in extracting and contrasting key features of the seizure underlying the EEG signals are highlighted. Through the use of these models we have quantified, for the first time, decreases in the dominant frequencies of low-frequency EEG components during ECT seizures. We have also identified preliminary evidence that such decreases are enhanced under the more effective ECTs at higher electrical dosages, a finding consistent with prior reports and the hypothesis that more effective forms of ECT are more effective in eliciting neurophysiological inhibitory processes.

New methods of time series analysis of non-stationary EEG data: eigenstructure decompositions of time varying autoregressions

Objective: Those who analyze EEG data require quantitative techniques that can be validly applied to time series exhibiting ranges of non-stationary behavior. Our objective is to introduce a new analysis technique based on formal non-stationary time series models. This novel method provides a decomposition of the time series into a set of ‘latent’ components with time-varying frequency content. The identification of these components can lead to practical insights and quantitative comparisons of changes in frequency structure over time in EEG time series. Methods: The technique begins with the development of time-varying autoregressive models of the EEG time series. Such models have been previously used in EEG analysis but we extend their utility by the introduction of eigenstructure decomposition methods. We review the basis and implementation of this method and report on the analysis of two channel EEG data recorded during 3 generalized tonic–clonic seizures induced in an individual as part of a course of electroconvulsive therapy for major depression. Results: This technique identified EEG patterns consistent with prior reports. In addition, it quantified a decrease in dominant frequency content over the seizures and suggested for the first time that this decrease is continuous across the end of the seizures. The analysis also suggested that the seizure EEG may be best modeled by the combination of multiple processes, whereas post-ictally there appears to be one dominant process. There was also preliminary evidence that these features may differ as a function of ECT therapeutic effectiveness. Conclusions: Eigenanalysis of time-varying autoregressive models has promise for improving the analysis of EEG time series.

Evaluation and Comparison of EEG Traces: Latent Structure in Nonstationary Time Series

We explore and illustrate the use of time series decomposition methods for evaluating and comparing latent structure in nonstationary electroencephalographic (EEG) traces obtained from depressed patients during brain seizures induced as part of electroconvulsive therapy (ECT). Analysis of the patterns of change over time in the frequency structure of such EEG data provides insight into the neurophysiological mechanisms of action of this effective but poorly understood antidepressant treatment, and allows clinicians to modify ECT treatments to optimize therapeutic benefits while minimizing associated side effects. Our work has introduced new methods of time-frequency analysis of EEG series that identify the complete pattern of time evolution of frequency structure over the course of a seizure, and usefully assist in these scientific and clinical studies. New methods of decomposition of flexible dynamic models provide time domain decompositions of individual EEG series into collections of latent components in different frequency bands. This allows us to explore ECT seizure characteristics via inferences on the time-varying parameters that characterize these latent components, and to relate differences in such characteristics across seizures to differences in the therapeutic effectiveness and cognitive side effects of those seizures. This article discusses the scientific context and problems, development of nonstationary time series models and new methods of decomposition to explore time-frequency structure, and aspects of model fitting and analysis. We include applied studies on two datasets from recent clinical ECT studies. One is an initial illustrative analysis of a single EEG trace, the second compares the EEG data recorded during two types of ECT treatment that differ in therapeutic effectiveness and cognitive side effects. The uses of these models and time series decomposition methods in extracting and contrasting key features of the seizure underlying the EEG signals are highlighted. Through the use of these models we have quantified, for the first time, decreases in the dominant frequencies of low-frequency EEG components during ECT seizures. We have also identified preliminary evidence that such decreases are enhanced under the more effective ECTs at higher electrical dosages, a finding consistent with prior reports and the hypothesis that more effective forms of ECT are more effective in eliciting neurophysiological inhibitory processes.

Time-Dependent Spectral Analysis of Nonstationary Time Series

Modeling of nonstationary stochastic time series has found wide applications in speech processing, biomedical signal processing, seismology, and failure detection. Data from these fields have often been modeled as piecewise stationary processes with abrupt changes, and their time-varying spectral features have been studied with the help of spectrograms. A general class of piecewise locally stationary processes is introduced here that allows both abrupt and smooth changes in the spectral characteristics of the nonstationary time series. It is shown that this class of processes behave as approximately piecewise stationary processes and can be used to model various naturally occuring phenomena. An adaptive segmentation method of estimating the time-dependent spectrum is proposed for this class of processes. The segmentation procedure uses binary trees and windowed spectra to nonparametrically and adaptively partition the data into approximately stationary intervals. Results of simulation studies demonstrate that the method has excellent ability to adapt to the rate at which the spectrum is changing. Applications of the method to speech signals and earthquake data are considered.

Time-Dependent Spectral Analysis of Nonstationary Time Series

Abstract Modeling of nonstationary stochastic time series has found wide applications in speech processing, biomedical signal processing, seismology, and failure detection. Data from these fields have often been modeled as piecewise stationary processes with abrupt changes, and their time-varying spectral features have been studied with the help of spectrograms. A general class of piecewise locally stationary processes is introduced here that allows both abrupt and smooth changes in the spectral characteristics of the nonstationary time series. It is shown that this class of processes behave as approximately piecewise stationary processes and can be used to model various naturally occuring phenomena. An adaptive segmentation method of estimating the time-dependent spectrum is proposed for this class of processes. The segmentation procedure uses binary trees and windowed spectra to nonparametrically and adaptively partition the data into approximately stationary intervals. Results of simulation studies demonstrate that the method has excellent ability to adapt to the rate at which the spectrum is changing. Applications of the method to speech signals and earthquake data are considered.

Adaptive AR modeling of nonstationary time series by means of Kalman filtering.

An adaptive on-line procedure is presented for autoregressive (AR) modeling of nonstationary multivariate time series by means of Kalman filtering. The parameters of the estimated time-varying model can be used to calculate instantaneous measures of linear dependence. The usefulness of the procedures in the analysis of physiological signals is discussed in two examples: First, in the analysis of respiratory movement, heart rate fluctuation, and blood pressure, and second, in the analysis of multichannel electroencephalogram (EEG) signals. It was shown for the first time that in intact animals the transition from a normoxic to a hypoxic state requires tremendous short-term readjustment of the autonomic cardiac-respiratory control. An application with experimental EEG data supported observations that the development of coherences among cell assemblies of the brain is a basic element of associative learning or conditioning.

Large Sample Properties of Posterior Densities, Bayesian Information Criterion and the Likelihood Principle in Nonstationary Time Series Models

Asymptotic normality of the posterior is a well understood result for dynamic as well as nondynamic models based on sets of abstract conditions whose actual applicability is hardly known especially for the case of nonstationarity. In this paper we provide a set of conditions by which we can relatively easily prove the asymptotic posterior normality under quite general situations of possible nonstationarity. This result reinforces and generalizes the point of Sims and Uhlig (1991) that inference based on the likelihood principle, explained by Berger and Wolpert (1988), will be unchanged regardless of whether the data are generated by a stationary process or by a unit root process. On the other hand, our conditions allow us to generalize the Bayesian information criterion known as the Schwarz criterion to the case of possible nonstationarity. In addition, we have shown that consistency of the maximum likelihood estimator, not the asymptotic normality of the estimator, with some minor additional assumptions is sufficient for asymptotic posterior normality.

Exact Initial Kalman Filtering and Smoothing for Nonstationary Time Series Models

Exact Initial Kalman Filtering and Smoothing for Nonstationary Time Series Models

Exact Initial Kalman Filtering and Smoothing for Nonstationary Time Series Models

This article presents a new exact solution for the initialization of the Kalman filter for state space models with diffuse initial conditions. For example, the regression model with stochastic trend, seasonal and other nonstationary autoregressive integrated moving average components requires a (partially) diffuse initial state vector. The proposed analytical solution is easy to implement and computationally efficient. The exact solution for smoothing is also given. Missing observations are handled in a straightforward manner. All proofs rely on elementary results.