Parametric Time Series Models Research Articles

Bootstrap Methods for Time Series

SummaryThe bootstrap is a method for estimating the distribution of an estimator or test statistic by resampling one's data or a model estimated from the data. The methods that are available for implementing the bootstrap and the accuracy of bootstrap estimates depend on whether the data are an independent random sample or a time series. This paper is concerned with the application of the bootstrap to time‐series data when one does not have a finite‐dimensional parametric model that reduces the data generation process to independent random sampling. We review the methods that have been proposed for implementing the bootstrap in this situation and discuss the accuracy of these methods relative to that of first‐order asymptotic approximations. We argue that methods for implementing the bootstrap with time‐series data are not as well understood as methods for data that are independent random samples. Although promising bootstrap methods for time series are available, there is a considerable need for further research in the application of the bootstrap to time series. We describe some of the important unsolved problems.RésuméLe bootstrap est une méthode pour estimer la distribution d'un estimateur en rééchantillonnant ses données ou un modéle estiméà partir des données. Les méthodes disponibles pour mettre en oeuvre le bootstrap et la précision des estimateurs de bootstrap dépendent de ce que les données proviennent d'un échantillon aléatoire indépendant ou d'une série temporelle. Cet article concerne l'application du bootstrap aux données des séries temporelles quand on ne dispose pas de modéle paramétrique de dimension finie qui réduise le processus de génération des données à l'échantillonnage aléatoire indépendent. Nous examinons les méthodes qui ont été proposées pour mettre en oeuvre le bootstrap dans cette situation et discutons la precision de ces méthodes comparativement à celle des approximations asymptotiques de premier ordre. Nous montrons que les méthodes pour mettre en oeuvre le bootstrap avec les données des séries temporelles ne sont pas aussi bien comprises que les méthodes pour les données des échantillons aléatoires indépendants. Bien que des méthodes de bootstrap prometteuses pour les séries temporelles soient disponibles, il y a un besoin considérable de recherche supplémental re dans leur application. Nous décrivons quelques problémes importants non résolus.

Adaptive Forecast-Based Monitoring for Dynamic Systems

Several forecast-based monitoring methods have been developed for autocorrelated data. One effective method is to use the forecasts based on the exponentially weighted moving average (EWMA). However, during the transition period of dynamic systems, the forecast-based monitoring procedure becomes inadequate due to its use of constant time series model parameters. In this article we present an adaptive forecast-based monitoring approach that performs well on dynamic systems. We examine two competing procedures: the adaptive time series model and the adaptive EWMA. We use a plastic extrusion process with first-order dynamics to illustrate the application of these two procedures, and we also evaluate the performance of the two procedures via simulation.

Seasonality in the Statistics of Surface Air Temperature and the Pricing of Weather Derivatives

The pricing of weather derivatives motivates the need to build accurate statistical models of daily temperature variability. Current published models are shown to be inaccurate for locations that show strong seasonality in the probability distribution and autocorrelation structure of temperature anomalies. With respect to the first of these problems, we present a new transform that allows seasonally varying non-normal temperature anomaly distributions to be cast into normal distributions. With respect to the second we present a new parametric time-series model that captures both the seasonality and the slow decay of the autocorrelation structure of observed temperature anomalies. This model is valid when the seasonality is slowly varying. Finally we present a simple non-parametric method for the modelling of daily temperatures that is accurate in all cases including extreme non-normality and rapidly varying seasonality.

Adaptive modeling and classification of the secondary settling tank

In biological wastewater treatment plants the biomass is separated from the treated wastewater in the secondary settler; thus, efficient operation of the secondary settler is crucial to achieving satisfactory effluent quality in the wastewater treatment process (WWTP). In the present work, system identification and soft-computing techniques were used to formulate a model for predicting the solid volume index (SVI) and classification of the sludge bulking phenomenon in the settler. An adaptive time series model was applied to predict the SVI of the secondary settler; this model uses the recursive least square (RLS) method to update the model parameters. The method for classifying the current state of the secondary settler is based on the strong correlation that was observed between the settler state and the values of the time series model parameters, which enabled the time series model parameters to be used as effective features for monitoring the secondary settler. To classify the current state of the secondary settler, a neural network (NN) was used to classify the adaptive time series model parameters, where a hybrid Genetic Algorithm (GA) was used to decide the number of hidden nodes of the NN classifier. Application of the proposed method to a full-scale WWTP demonstrated the utility of the method for simultaneously predicting the SVI value of the secondary settler and classifying the current state of the settler.

MODELING ASYMMETRIES AND MOVING EQUILIBRIA IN UNEMPLOYMENT RATES

The paper discusses a simple univariate nonlinear parametric time-series model for unemployment rates, focusing on the asymmetry observed in many OECD unemployment series. The model is based on a standard logistic smooth transition autoregressive model for the first difference of unemployment, but it also includes a lagged level term. This model allows for asymmetric behavior by permitting “local” nonstationarity in a globally stable model. Linearity tests are performed for a number of quarterly, seasonally unadjusted, unemployment series from OECD countries, and linearity is rejected for a number of them. For a number of series, nonlinearity found by testing can be modeled satisfactorily by use of our smooth transition autoregressive model. The properties of the estimated models, including persistence of the shocks according to them, are illustrated in various ways and discussed. Possible existence of moving equilibria in series not showing asymmetry is investigated and modeled with another smooth transition autoregressive model.

Functional Coefficient Autoregressive Models: Estimation and Tests of Hypotheses

In this paper, we study nonparametric estimation and hypothesis testing procedures for the functional coefficient AR (FAR) models of the form Xt=f1(Xt−d)Xt− 1+ ... +fp(Xt−d)Xt−p+εt, first proposed by Chen and Tsay (1993). As a direct generalization of the linear AR model, the FAR model is a rich class of models that includes many useful parametric nonlinear time series models such as the threshold AR models of Tong (1983) and exponential AR models of Haggan and Ozaki (1981). We propose a local linear estimation procedure for estimating the coefficient functions and study its asymptotic properties. In addition, we propose two testing procedures. The first one tests whether all the coefficient functions are constant, i.e. whether the process is linear. The second one tests if all the coefficient functions are continuous, i.e. if any threshold type of nonlinearity presents in the process. The results of some simulation studies as well as a real example are presented.

Fitting of Threshold Models for Time Series

One of the most important parametric nonlinear time series models is the (self exciting) threshold autoregressive model. The TAR model was introduced by Tong/Lim (1980). There are some extensions to this class of models, namely smooth transition threshold autoregressive model. Other nonlinear models like functional auroregressive models can be thought of as generalizations. It is rather often stated that nonlinear models are necessary but there are not such many applications. In fact there exists a gap between the theoretical development of nonlinear time series models and their application. One reason for this gap may be the lack of applicable software and some investigations of the numerical side of these models. The present paper tries to do a step to fill this gap. For this we describe a procedure to implement the fitting process efficiently. Also we report on some difficulties with estimating STAR models and give some advice to overcome the problems. For FAR models it seems to be of help to smooth the observed parametric relationship. Nevertheless, the usefulness of FAR models to recover more complicated functional coefficients from the data is doubted. All programming was done in GAUSS. The progams are available at request from the author.

On Selecting a Power Transformation in Time-Series Analysis

The primary aim of this paper is to select an appropriate power transformation when we use ARMA models for a given time series. We propose a Bayesian procedure for estimating the power transformation as well as other parameters in time series models. The posterior distributions of interest are obtained utilizing the Gibbs sampler, a Markov Chain Monte Carlo (MCMC) method. The proposed methodology is illustrated with two real data sets. The performance of the proposed procedure is compared with other competing procedures. #1997 John Wiley & Sons, Ltd.

PARAMETRIC AND NON-PARAMETRIC MODELLING OF TIME SERIES — AN EMPIRICAL STUDY

Time series modelling methods can be loosely classified as (i) parametric methods and (ii) non-parametric methods. Within a usually quite flexible but well structured family of models, the parametric modelling process typically consists of model identification, parameter estimation, model diagnostic checking, and forecasting. On the other hand, within a much less structured framework, different non-parametric smoothing techniques are usually used to bring out the features of the observed time series, however, few serious and systematic attempts have been made to model time series non-parametrically. We concentrate on a non-parametric method based on multivariate adaptive regression splines (MARS). Parallel to the parametric modelling process, we systemize a non-parametric modelling process as (i) model perception (where a very large spline expansion of a very large family of models is specified), (ii) model search (forward plus backward search to come up with a model), (iii) model diagnostic checking, and (iv) forecasting. The major difference between the MARS and the parametric methods is that the potential models for the MARS method form a family which is much larger than any family of parametric time series models, and the local structures found in the data are used to guide the search for a fitted model. Also, unlike most non-parametric methods, MARS time series models can be analytically written down. In this paper, we present the results of an empirical comparison of parametric (ARIMA) and non-parametric (MARS) time series modelling methods. Eight environmental time series are used for the comparison. © 1997 by John Wiley & Sons Ltd.

Spectral analysis based on the canonical autoregressive decomposition

Statistical inference for mixed spectral problems based on a parametric time series model is studied. The model used is based on the canonical autoregressive decomposition (CARD) and represents the underlying random process as the sum of an autoregressive process and sinusoids. Maximum likelihood estimation of the unknown parameters in the model is considered. The entire estimation problem can be shown to require a numerical maximization with respect to only the sinusoidal frequencies. An iterative algorithm to efficiently implement this maximization is presented. This enables us to examine a host of issues associated with a practical implementation of inferential procedures for mixed spectral problems. Some of the topics are accuracy of parameter estimates, selection of model orders, and sensitivity and robustness of the spectral estimates to modeling inaccuracies. The modeling approach, together with the inferential procedures, overcome many of the difficulties encountered in current spectral estimation techniques.

The moving blocks bootstrap versus parametric time series models

The application of a parametric time series model to a water resources problem involves selecting a model and estimating its parameters, both steps adding uncertainty to the analysis. The moving blocks bootstrap is a simple resampling algorithm which can replace parametric time series models, avoiding model selection and only requiring an estimate of the moving block length. The moving blocks bootstrap resamples the observed time series using approximately independent moving blocks. A Monte Carlo experiment is performed involving the use of a time series model to estimate the storage capacity S of a surface water reservoir. Our results document that the bootstrap always produced storage estimates with lower root‐mean‐square‐error than a parametric alternative, even when no model error is introduced into the parametric scheme. These results suggest that the moving blocks bootstrap can provide a simple and attractive alternative to more complex multivariate ARMA models.

Asymptotically optimal estimation in misspecified time series models

A concept of asymptotically efficient estimation is presented when a misspecified parametric time series model is fitted to a stationary process. Efficiency of several minimum distance estimates is proved and the behavior of the Gaussian maximum likelihood estimate is studied. Furthermore, the behavior of estimates that minimize the h-step prediction error is discussed briefly. The paper answers to some extent the question what happens when a misspecified model is fitted to time series data and one acts as if the model were true.

Asymptotically optimal estimation in misspecified time series models

A concept of asymptotically efficient estimation is presented when a misspecified parametric time series model is fitted to a stationary process. Efficiency of several minimum distance estimates is proved and the behavior of the Gaussian maximum likelihood estimate is studied. Furthermore, the behavior of estimates that minimize the h-step prediction error is discussed briefly. The paper answers to some extent the question what happens when a misspecified model is fitted to time series data and one acts as if the model were true.

An algorithm for estimating parameters of state-space models

We describe an algorithm for estimating the parameters of time-series models expressed in state-space form. The algorithm is based on the EM algorithm, and generalizes an algorithm given by Shumway and Stoffer (1982)

Improved estimates of the parameters of state space time series models

An instrumental variable interpretation of Aoki's (1987, 1990) state space time series estimators is utilized to provide an intuitively appealing improved estimator that is asymptotically efficient within its class. While the analysis shows the new estimator to be more efficient, the gain in efficiency is a complicated function of many variables. To investigate the magnitude of the efficiency gain, data following several representative processes were generated and the estimators' effects on in- and out-of-sample predictions were compared, showing marked improvement. In addition, the new estimator was applied to Canadian macroeconomic data, and again the forecast summary statistics were improved. The estimator can also be applied to other state space procedures, such as those due to Akaike or Rao.

Testing for dependence in the input to a linear time series model

This paper presents a simple test for dependence in the residuals of a linear parametric time series model fitted to non gaussian data. The test statistic is a third order extension of the standard correlation test for whiteness. but the number of lags used in this test is a function of the sample size. The power of this test goes to one as the sample size goes to infinity for any alternative which has non zero bicovariances c e3(r,s)= E[e(t)e(t + r)e(t + s)] for a zero mean stationary random time series. The asymptotic properties of the test statistic are rigorously determined. This test is important for the validation of the sampling properties of the parameter estimates for standard finite parameter linear models when the unobserved input (innovations) process is white but not gaussian. The sizes and power derived from the asymptotic results are checked using artificial data for a number of sample sizes. Theoretical and simulation results presented in this paper support the proposition that the test wi...

Shrinkage estimation of contemporaneous outliers in concurrent time serie

Contemporaneous outlier blocks (additive or reallocation) caused by special events frequently occur in repeated business time series. When the time series have strong inter-series dependence, shrinkage estimation techniques provide improved estimates of the time series model parameters and of the outlier block. A bootstrap estimate of the covariance matrix of the vector of outlier magnitudes enables us to incorporate the dependence and obtain the shrinkage estimates.

A method of estimating the rate of miniature end-plate potential occurrences based on parametric time series modeling

This paper presents a parametric method of estimating the rate of miniature end-plate potential (MEPP) occurrences. We consider the case where the rate of MEPP occurrences is raised by the constant deporalization of presynaptic terminals by using high-concentration potassium solutions. Under such conditions, since MEPP occurrences cannot be identified by eye due to waveform superposition, it is necessary to estimate the rate with the aid of statistical techniques instead of counting the occurrences by eye. In this paper it is assumed according to the literatures that the MEPP data are modeled as a stationary Poisson impulse process filtered by the linear system the impulse response function of which is the sum of two exponentials. Then, the discretized MEPP data are shown to be a second-order autoregressive (AR(2)) process, driven by the sum of 2 first-order moving average (MA(1)) processes (the residual time series). An explicit formula for estimating the rate can be derived by combining the second- and third-order moments of the residual time series. The validity of the proposed estimation method is verified through Monte Carlo simulations in which the rate is varied ranging from 100 to 10,000 s −1. Likewise, the proposed method is applied to estimation of the rate of actual MEPP data, which were observed at the frog's neuromuscular junction under high-concentration potassium solutions.

Safety Stock Determination with Serially Correlated Demand in a Periodic-Review Inventory System

We consider a periodic-review inventory replenishment model with an order-up-to-R operating doctrine for the case of deterministic lead times and a covariance-stationary stochastic demand process. A method is derived for setting the inventory safety stock to achieve an exact desired stockout probability when the autocovariance function for Gaussian demand is known. Because the method does not require that parametric time-series models be fit to the data, it is easily implemented in practice. Moreover, the method is shown to be asymptotically valid when the autocovariance function of demand is estimated from historical data. The effects on the stockout rate of various levels of autocorrelated demand are demonstrated for situations in which autocorrelation in demand goes undetected or is ignored by the inventory manager. Similarly, the changes to the required level of safety stock are demonstrated for varying levels of autocorrelation.

Adaptive online load forecasting via time series modeling

Reliable short-term load forecasting by computer is a topic of considerable interest. In this paper, a reliable online method to accomplish short-term load forecasting is presented. It automatically adapts to changing conditions and requires very little human intervention. It makes use of time series models, automatically adjusting the model parameters, to forecast the load. Unique in this method are (i) the autocorrelation optimization used for incorporating daily and weekly periodicities, and (ii) the identification method used for adapting the time series model parameters, which adapts not only the model parameter values, but also the structure and order of the time series model itself.