ARMA Time Series Research Articles

A single series representation of multiple independent ARMA processes

This article shows that multiple independent time series from the same ARMA process can be represented by a single univariate ARMA time series through an interleaving of the original series. Using this result, existing univariate modelling software can be used to fit a single ARMA time series model simultaneously to multiple independent realizations of the same ARMA process. The interleaving approach and its properties will be presented and compared with alternative estimation options. It will be applied to the modelling of 66 years of daily maximum temperatures for Perth, Western Australia and to other time series models.

Based on Time Series Prediction of Photovoltaic Power Plant Output

With the PV power system capacity continues to expand, PV power generation forecasting techniques can reduce the PV system output power of randomness, it has great impact on power systems. This paper presents a method based on ARMA time series power prediction model. With historical electricity data and meteorological factors, the model gets test and evaluation by Eviews software. Results indicated that the prediction model has high accuracy, it can solve the shortcomings of PV randomness and also can improve the ability of the stable operation of the system.

Least absolute deviation estimation for general ARMA time series models with infinite variance

Least absolute deviation estimation for general ARMA time series models with infinite variance

Stochastic Excitation Model of Strip Rolling Mill

The stochastic excitation power spectral density (PSD) model and ARMA time series model are established based on the stochastic rolling force acquisition data, which is processed into stationary, normal and zero mean from a aluminum hot strip tandem mill. Characteristics of rolling force ARMA time series models are discussed by means of random process theory. The rolling forces PSD function of facilitating engineering application is obtained by utilizing Levenberg-Marquardt combined with generalized global planning algorithm, and the stochastic excitation model is established. It provides the basis for the prediction of rolling force.

Reliability/cost evaluation of a wind power delivery system

When the geographical locations with good wind resources are not close to the main load centers, it becomes extremely important to assess adequate transmission facility to deliver wind power to the power grid. A probabilistic method is presented to evaluate the contribution of a wind power delivery system to the overall system reliability. The basic model incorporates transmission line connecting a remotely located large wind farm to a conventional grid system. The classical generation system adequacy evaluation model is extended to incorporate limited transmission system. The mean Capacity Outage Probability Table (mean-COPT) concept is used to increase the computational efficiency as it allows determination of EUE (expected unserved energy) and LOLE (loss of load expectation) simultaneously in calculation of reliability indices of the generating system. The wind farm generation model is obtained by superimposing simulated wind speed, obtained from developed ARMA time series model, on power curve of WTG. An apportioning method has been used to reduce the number of states in the resulting model, obtaining an equivalent reduced 5-state model. Applying transmission line constraints result in wind generation model ranging from 2- to 5-state models. The study recognizes benefits from fuel offset by wind power, reliability worth and environmental improvement and determines appropriate transmission line capacity based on its contribution to the overall system risk and associated transmission system cost. The paper illustrates results using a real wind farm. The presented methods and discussions should be useful to power system planners and policy makers.

Estimation and Inference with Weak, Semi-Strong, and Strong Identification

This paper analyzes the properties of standard estimators, tests, and confidence sets (CS's) for parameters that are unidentified or weakly identified in some parts of the parameter space. The paper also introduces methods to make the tests and CS's robust to such identification problems. The results apply to a class of extremum estimators and corresponding tests and CS's that are based on criterion functions that satisfy certain asymptotic stochastic quadratic expansions and that depend on the parameter that determines the strength of identification. This covers a class of models estimated using maximum likelihood (ML), least squares (LS), quantile, generalized method of moments (GMM), generalized empirical likelihood (GEL), minimum distance (MD), and semi-parametric estimators. The consistency/lack-of-consistency and asymptotic distributions of the estimators are established under a full range of drifting sequences of true distributions. The asymptotic sizes (in a uniform sense) of standard and identification-robust tests and CS's are established. The results are applied to the ARMA(1, 1) time series model estimated by ML and to the nonlinear regression model estimated by LS. In companion papers the results are applied to a number of other models.

Analysis and modelling of wind speed in New York

In this paper we propose an ARMA time-series model for the wind speed at a single spatial location, and estimate it on in-sample data recorded in three different wind farm regions in New York state. The data have a three-hour granularity, but based on applications to financial wind derivatives contracts, we also consider daily average wind speeds. We demonstrate that there are large discrepancies in the behaviour of daily average and three-hourly wind speed records. The validation procedure based on out-of-sample observations reflects that the proposed model is reliable and can be used for various practical applications, like, for instance, weather prediction, pricing of financial wind contracts, wind generated power, etc. Furthermore, we discuss some striking resemblances with temperature dynamics.

Re-examining the sources of heteroskedasticity: The paradigm of noisy chaotic models

In this paper, we analyze the rich dynamic properties of the noisy chaotic model developed by Kyrtsou [C. Kyrtsou, Evidence for neglected linearity in noisy chaotic models, International Journal of Bifurcation and Chaos 15 (10) (2005)] considering homoskedastic errors, with the aim of deriving information about possible links between noisy chaotic dynamics and ARCH effects. With the joint application of the Engle [R.F. Engle, Autoregressive conditional heteroscedasticity with estimates of the variance of United Kingdom inflation, Econometrica 50 (1982) 987–1007] and McLeod–Li [A.I. McLeod, W.K. Li, Diagnostic checking ARMA time series models using squared-residuals autocorrelations, Journal of Time Series Analysis 4 (1983) 269–273] tests for non-linearity in the second moment, we attempt to show how highly non-linear models can exhibit heteroskedasticity when no heteroskedastic structure is assumed by construction.

Fence methods for mixed model selection

Many model search strategies involve trading off model fit with model complexity in a penalized goodness of fit measure. Asymptotic properties for these types of procedures in settings like linear regression and ARMA time series have been studied, but these do not naturally extend to nonstandard situations such as mixed effects models, where simple definition of the sample size is not meaningful. This paper introduces a new class of strategies, known as fence methods, for mixed model selection, which includes linear and generalized linear mixed models. The idea involves a procedure to isolate a subgroup of what are known as correct models (of which the optimal model is a member). This is accomplished by constructing a statistical fence, or barrier, to carefully eliminate incorrect models. Once the fence is constructed, the optimal model is selected from among those within the fence according to a criterion which can be made flexible. In addition, we propose two variations of the fence. The first is a stepwise procedure to handle situations of many predictors; the second is an adaptive approach for choosing a tuning constant. We give sufficient conditions for consistency of fence and its variations, a desirable property for a good model selection procedure. The methods are illustrated through simulation studies and real data analysis.

Consistent testing for non‐correlation of two cointegrated ARMA time series

AbstractA consistent approach to the problem of testing non‐correlation between two univariate infinite‐order autoregressive models was proposed by Hong (1996). His test is based on a weighted sum of squares of residual cross‐correlations, with weights depending on a kernel function. In this paper, the author follows Hong's approach to test non‐correlation of two cointegrated (or partially non‐stationary) ARMA time series. The test of Pham, Roy & Cédras (2003) may be seen as a special case of his approach, as it corresponds to the choice of a truncated uniform kernel. The proposed procedure remains valid for testing non‐correlation between two stationary invertible multivariate ARMA time series. The author derives the asymptotic distribution of his test statistics under the null hypothesis and proves that his procedures are consistent. He also studies the level and power of his proposed tests in finite samples through simulation. Finally, he presents an illustration based on real data.

On the connection between model selection criteria and quadratic discrimination in ARMA time series models

This article establishes the connection between quadratic discrimination and model selection criterion in the ARMA framework. We show that analyzing model selection in ARMA time series models as a quadratic discrimination problem provides a unifying approach for deriving model selection criteria.

Quasi-maximum-likelihood estimation in conditionally heteroscedastic time series: A stochastic recurrence equations approach

This paper studies the quasi-maximum-likelihood estimator (QMLE) in a general conditionally heteroscedastic time series model of multiplicative form Xt = σt Zt , where the unobservable volatility σt is a parametric function of (Xt−1 ,...,X t−p ,σ t−1 ,...,σ t−q ) for some p, q ≥ 0, and (Zt ) is standardized i.i.d. noise. We assume that these models are solutions to stochastic recurrence equations which satisfy a contraction (random Lipschitz coefficient) property. These assumptions are satisfied for the popular GARCH, asymmetric GARCH and exponential GARCH processes. Exploiting the contraction property, we give conditions for the existence and uniqueness of a strictly stationary solution (Xt ) to the stochastic recurrence equation and establish consistency and asymptotic normality of the QMLE. We also discuss the problem of invertibility of such time series models. 1. Introduction. Gaussian quasi-maximum-likelihood estimation, that is, likelihood estimation under the hypothesis of Gaussian innovations, is a popular method which is widely used for inference in time series models. However, it is often a nontrivial task to establish the consistency and asymptotic normality of the quasi-maximum-likelihood estimator (QMLE) applied to specific models and, therefore, an in-depth analysis of the probabilistic structure generated by the model is called for. A classical example of this kind is the seminal paper by Hannan [18] on estimation in linear ARMA time series. In this paper we study the QMLE for a general class of conditionally heteroscedastic time series models, which includes GARCH, asymmetric GARCH and exponential GARCH. Recall that a GARCH(p, q) [generalized autoregressive conditionally heteroscedastic of order (p, q)] process [4 ]i s def ined by

Singularities Affect Dynamics of Learning in Neuromanifolds

The parameter spaces of hierarchical systems such as multilayer perceptrons include singularities due to the symmetry and degeneration of hidden units. A parameter space forms a geometrical manifold, called the neuromanifold in the case of neural networks. Such a model is identified with a statistical model, and a Riemannian metric is given by the Fisher information matrix. However, the matrix degenerates at singularities. Such a singular structure is ubiquitous not only in multilayer perceptrons but also in the gaussian mixture probability densities, ARMA time-series model, and many other cases. The standard statistical paradigm of the Cramér-Rao theorem does not hold, and the singularity gives rise to strange behaviors in parameter estimation, hypothesis testing, Bayesian inference, model selection, and in particular, the dynamics of learning from examples. Prevailing theories so far have not paid much attention to the problem caused by singularity, relying only on ordinary statistical theories developed for regular (nonsingular) models. Only recently have researchers remarked on the effects of singularity, and theories are now being developed. This article gives an overview of the phenomena caused by the singularities of statistical manifolds related to multilayer perceptrons and gaussian mixtures. We demonstrate our recent results on these problems. Simple toy models are also used to show explicit solutions. We explain that the maximum likelihood estimator is no longer subject to the gaussian distribution even asymptotically, because the Fisher information matrix degenerates, that the model selection criteria such as AIC, BIC, and MDL fail to hold in these models, that a smooth Bayesian prior becomes singular in such models, and that the trajectories of dynamics of learning are strongly affected by the singularity, causing plateaus or slow manifolds in the parameter space. The natural gradient method is shown to perform well because it takes the singular geometrical structure into account. The generalization error and the training error are studied in some examples.

Singularities Affect Dynamics of Learning in Neuromanifolds

The parameter spaces of hierarchical systems such as multilayer perceptrons include singularities due to the symmetry and degeneration of hidden units. A parameter space forms a geometrical manifold, called the neuromanifold in the case of neural networks. Such a model is identified with a statistical model, and a Riemannian metric is given by the Fisher information matrix. However, the matrix degenerates at singularities. Such a singular structure is ubiquitous not only in multilayer perceptrons but also in the gaussian mixture probability densities, ARMA time-series model, and many other cases. The standard statistical paradigm of the Cramér-Rao theorem does not hold, and the singularity gives rise to strange behaviors in parameter estimation, hypothesis testing, Bayesian inference, model selection, and in particular, the dynamics of learning from examples. Prevailing theories so far have not paid much attention to the problem caused by singularity, relying only on ordinary statistical theories developed for regular (nonsingular) models. Only recently have researchers remarked on the effects of singularity, and theories are now being developed. This article gives an overview of the phenomena caused by the singularities of statistical manifolds related to multilayer perceptrons and gaussian mixtures. We demonstrate our recent results on these problems. Simple toy models are also used to show explicit solutions. We explain that the maximum likelihood estimator is no longer subject to the gaussian distribution even asymptotically, because the Fisher information matrix degenerates, that the model selection criteria such as AIC, BIC, and MDL fail to hold in these models, that a smooth Bayesian prior becomes singular in such models, and that the trajectories of dynamics of learning are strongly affected by the singularity, causing plateaus or slow manifolds in the parameter space. The natural gradient method is shown to perform well because it takes the singular geometrical structure into account. The generalization error and the training error are studied in some examples.

A Generalized Space-Time ARMA Model with an Application to Regional Unemployment Analysis in Italy

The STARMA (space-time autoregressive moving average) model class was introduced in the mid-1970s as a spatio-temporal extension to the ARMA time series model class. To enhance the model’s ability in dealing with spatial dependence and heterogeneity of observations, the article extends the STARMA model specification by augmenting the set of explanatory variables with simultaneous spatial lags of the observed process and unobservable shocks and by letting model parameters vary with location in space. Having introduced the extended specification,the space time impulse response function is subsequently presented as a useful tool in addressing structural issues. The article then deals with maximum likelihood estimation and hypothesis testing; in particular, Lagrange multiplier tests are proposed for spatial heterogeneity in the intercept, conditional variance and ARMA coefficients. The article closes with an application to the analysis of the series of the regional unemployment rate in Italy, aimed at evaluating the extent of the spatial propagation of regional specific shocks to unemployment and the degree of spatial heterogeneity in the process parameters.

Some Algorithms for the Conditional Mean Vector and Covariance Matrix

We consider here the problem of computing the mean vector and covariance matrix for a conditional normal distribution, considering especially a sequence of problems where the conditioning variables are changing. The sweep operator provides one simple general approach that is easy to implement and update. A second, more goal-oriented general method avoids explicit computation of the vector and matrix, while enabling easy evaluation of the conditional density for likelihood computation or easy generation from the conditional distribution. The covariance structure that arises from the special case of an ARMA(p, q) time series can be exploited for substantial improvements in computational efficiency.

Testing Non‐Correlation and Non‐Causality between Multivariate ARMA Time Series

Abstract. Haugh [Journal of the American Statistical Association (1976) Vol. 71, pp. 378–85] developed an approach to the problem of testing non‐correlation (at all leads and lags) between two univariate time series. Haugh's tests however have low power against two series which are related over a long distributed lag when individual lag coefficients are relatively small. As a remedy, Koch and Yang [Journal of the American Statistical Association (1986) Vol. 8, pp. 533–44] proposed an alternative method that performs better than Haugh's under such dependencies. A multivariate extension of Haugh's procedure was proposed by El Himdi and Roy [The Canadian Journal of Statistics (1997) Vol. 25, pp. 233–56], but suffers the same weaknesses as the original univariate method. We develop here an asymptotic test generalizing Koch and Yang's method to the multivariate case. Our method includes El Himdi and Roy's as a special case. Based on the same idea, we also suggest a generalization of the El Himdi and Roy procedure for testing causality in the sense of Granger [Econometrica (1969) Vol. 37, pp. 424–38] between two multivariate series. A Monte Carlo study is conducted, which indicates that our approach performs better than El Himdi and Roy's for a wide range of models. Both procedures are applied to the problem of testing the absence of correlation between Canadian and US economic indicators, and to a brief study of causality between money and income in Canada.

Tests for non‐correlation of two cointegrated ARMA time series

In multivariate time series modelling, we are often led to investigate the existence of a relationship between two time series. Here, we generalize the procedure proposed by Haugh (1976) and extended by El Himdi and Roy (1997) for multivariate stationary ARMA time series to the case of cointegrated (or partially nonstationary) ARMA series. The main contribution consists in showing that, in the case of two uncorrelated cointegrated time series, an arbitrary vector of residual cross‐correlation matrices asymptotically follows the same distribution as the corresponding vector of cross‐correlation matrices between the two innovation series. The estimation method from which the residuals are obtained can be the conditional maximum likelihood method as discussed in Yap and Reinsel (1995) or some other which has the same convergence rate. From this result, it follows that the considered test statistics, which are based on residual cross‐correlation matrices, asymptotically follow χ2 distributions. The finite sample properties, under the null hypothesis, of the test statistics are studied by simulation.

Generalized Autoregressive Moving Average Models

A class of generalized autoregressive moving average (GARMA) models is developed that extends the univariate Gaussian ARMA time series model to a flexible observation-driven model for non-Gaussian time series data. The dependent variable is assumed to have a conditional exponential family distribution given the past history of the process. The model estimation is carried out using an iteratively reweighted least squares algorithm. Properties of the model, including stationarity and marginal moments, are either derived explicitly or investigated using Monte Carlo simulation. The relationship of the GARMA model to other models is shown, including the autoregressive models of Zeger and Qaqish, the moving average models of Li, and the reparameterized generalized autoregressive conditional heteroscedastic GARCH model (providing the formula for its fourth marginal moment not previously derived). The model is demonstrated by the application of the GARMA model with a negative binomial conditional distribution to a well-known time series dataset of poliomyelitis counts.

Order identification for gaussian moving averages using the covariation

In the traditional Box-Jenkins procedure for fitting ARMA time series models to data, the first step is order identification. The sample autocorrelation function can be used to identify pure moving average behavior. In this paper we consider using the autocovariation function identify the order of a univariate Gaussian time series. Simulation evidence indicates the suggested method may be a superior order identification tool when at least 100 observations are taken.