Parametric Time Series Models Research Articles

Sampling properties of color Independent Component Analysis

Independent Component Analysis (ICA) offers an effective data-driven approach for blind source extraction encountered in many signal and image processing problems. Although many ICA methods have been developed, they have received relatively little attention in the statistics literature, especially in terms of rigorous theoretical investigation for statistical inference. The current paper aims at narrowing this gap and investigates the statistical sampling properties of the colorICA (cICA) method. The cICA incorporates the correlation structure within sources through parametric time series models in the frequency domain and outperforms several existing ICA alternatives numerically. We establish the consistency and asymptotic normality of the cICA estimates, which then enables statistical inference based on the estimates. These asymptotic properties are further validated using simulation studies.

Asymptotic theory for time series with changing mean and variance

The paper develops point estimation and asymptotic theory with respect to a semiparametric model for time series with moving mean and unconditional heteroscedasticity. These two features are modelled nonparametrically, whereas autocorrelations are described by a short memory stationary parametric time series model. We first study the usual least squares estimate of the coefficient of the first-order autoregressive model based on constant but unknown mean and variance. Allowing for both the latter to vary over time in a general way we establish its probability limit and a central limit theorem for a suitably normed and centred statistic, giving explicit bias and variance formulae. As expected mean variation is the main source of inconsistency and heteroscedasticity the main source of inefficiency, though we discuss circumstances in which the estimate is consistent for, and asymptotically normal about, the autoregressive coefficient, albeit inefficient. We then consider standard implicitly-defined Whittle estimates of a more general class of short memory parametric time series model, under otherwise more restrictive conditions. When the mean is correctly assumed to be constant, estimates that ignore the heteroscedasticity are again found to be asymptotically normal but inefficient. Allowing a slowly time-varying mean we resort to trimming out of low frequencies to achieve the same outcome. Returning to finite order autoregression, nonparametric estimates of the varying mean and variance are given asymptotic justification, and forecasting formulae developed. Finite sample properties are studied by a small Monte Carlo simulation, and an empirical example is also included.

Dynamic displacements from high-rate GNSS: Error modeling and vibration detection

Nowadays, GNSS (Global Navigation Satellite System) receivers can measure with sampling rates up to 100 Hz, and it is possible to provide displacements at a level of precision of a few millimeters. This opened an interesting field of applications, e.g. for disciplines that monitor dynamic movements, like seismology or structural health monitoring. However, the detection of smallest displacements possible requires a thorough analysis of both error types, systematic and stochastic. In view of this, this paper deals with the detection of small vibrations with amplitudes down to the sub-millimeter level, and the error modelling necessary to achieve this. The data under investigation are time series of kinematic GNSS displacement measurements. We will use parametric time series models of the ARIMA (Autoregressive Integrated Moving Average) type to characterize the GNSS errors and to do a pre-filtering based on a calibrated model. With non-parametric spectral estimation methods – namely periodograms – we can then test for the presence of harmonic processes (sinusoids) in the data. Once a sinusoid is detected, its amplitude and phase are estimated by means of harmonic regression. Subsequently, the sinusoid can be removed, and the procedure can be repeated. This algorithm is tested with data collected for a GNSS shake table experiment; it is shown that it is possible resolve and detect vibrations down to a level of tens of microns with few minutes of data being analyzed. We also highlight some theoretical considerations on minimum detectable amplitudes of vibrations for measurements of different quality and record lengths.

Time series central subspace with covariates and its application to forecasting pine sawtimber stumpage prices in the Southern United States

To model and forecast a monthly pine sawtimber (PST) stumpage price, $$y_t$$ , data collected across 11 southern states in the U.S., we adopt a new semiparametric approach where the first phase adopts a nonparametric method called “Time Series Central Subspace with Covariates” (TSCS-C) to extract sufficient information about $$y_t$$ through a univariate time series $$\{d_t\}$$ , which is a linear combination of a set of past values of $$y_t$$ and a high dimensional covariate vector $${{\mathbf {x}}}_t$$ of sale characteristics. Then, $$\{d_t\}$$ alone is used as the predictor series to build a parametric nonlinear time series model for $$y_t$$ . This yields a new semiparametric nonlinear time series model for $$y_t$$ . Assessment in terms of out-of-sample forecasts of monthly PST stumpage prices show that our semiparametric model with the covariate $${{\mathbf {x}}}_t$$ has the smallest average forecasting error compared to another semiparametric nonlinear time series model without $${{\mathbf {x}}}_t$$ and two other parametric counterparts based on multiplicative seasonal autoregressive integrated moving average models with and without $${{\mathbf {x}}}_t$$ . This data underscores the ability of our semiparametric approach to first reduce the dimensionality of $${{\mathbf {x}}}_t$$ and a set of past values of $$y_t$$ significantly using the TSCS-C nonparametric methodology and then to produce a superior nonlinear time series model.

Penalized Averaging of Parametric and Non-Parametric Quantile Forecasts

Abstract We propose a hybrid penalized averaging for combining parametric and non-parametric quantile forecasts when faced with a large number of predictors. This approach goes beyond the usual practice of combining conditional mean forecasts from parametric time series models with only a few predictors. The hybrid methodology adopts the adaptive LASSO regularization to simultaneously reduce predictor dimension and obtain quantile forecasts. Several recent empirical studies have considered a large set of macroeconomic predictors and technical indicators with the goal of forecasting the S&P 500 equity risk premium. To illustrate the merit of the proposed approach, we extend the mean-based equity premium forecasting into the conditional quantile context. The application offers three main findings. First, combining parametric and non-parametric approaches adds quantile forecast accuracy over and above the constituent methods. Second, a handful of macroeconomic predictors are found to have systematic forecasting power. Third, different predictors are identified as important when considering lower, central and upper quantiles of the equity premium distribution.

Identification of parameters of adaptive time series models

The work is devoted to the study of adaptive models of time series and the development of methods for their construction. The most problematic stage in the implementation of algorithms for time series forecasting methods is the identification of unknown parameters, on which the adequacy of the prediction depends. On the basis of economic data, a number of adaptive models have been studied: using exponential smoothing, Brown, Holt-Winters and Theil-Wage models, which take into account seasonality. It was found that the main difficulties arise in the selection of constant smoothing and the choice of coefficients, since no universal algorithm for their task exists.The paper proposes a method for integrating the adaptive method and the method of numerical optimization. As an optimization method, the non-gradient method was chosen - the Particle Swarm Optimization (PSO), the use of which provided less error in the final forecast and convergence in all applications. Algorithms for finding the parameters to be identified, and a predictive model based on real data are presented. It is shown that in some cases the proposed model has an advantage over other models of time series.

Joint Estimation of Gross Recharge, Groundwater Usage, and Hydraulic Properties within HydroSight.

Groundwater management decisions are often founded upon estimates of aquifer hydraulic properties, recharge and the rate of groundwater usage. Too often hydraulic properties are unavailable, recharge estimates are very uncertain, and usage is unmetered or infrequently metered over only recent years or estimated using numerical groundwater models decoupled from the drivers of drawdown. This paper extends the HydroSight groundwater time-series package ( http://peterson-tim-j.github.io/HydroSight/) to allow the joint estimation of gross recharge, transmissivity, storativity, and daily usage at multiple production bores. A genetic evolutionary scheme was extended from estimating time-series model parameters to also estimating time series of usage that honor metered volumes at each production bore and produces (1) the best fit with the observed hydrograph and (2) plausible estimates of actual evapotranspiration and hence recharge. The reliability of the approach was rigorously tested. Repeated calibration of models for four bores produced estimates of transmissivity, storativity, and mean recharge that varied by a factor of 0.22-0.32, 0.13-0.2, and 0.03-0.48, respectively, when recharge boundary effects were low and the error in monthly, quarterly, and biannual metered usage was generally <10%. Application to the 30 observation bores within the Warrion groundwater management area (Australia), produced a coefficient of efficiency of ≥0.80 at 22 bores and ≥0.90 at 12 bores. The aquifer transmissivity and storativity were reasonably estimated, and were consistent with independent estimates, while mean gross recharge may be slightly overestimated. Overall, the approach allows greater insights from the available data and provides opportunity for the exploration of usage and climatic scenarios.

On the Prediction of Upwelling Events at the Colombian Caribbean Coasts from Modis-SST Imagery.

The upwelling cores on the Caribbean Colombian coasts are mainly located at the Peninsula de la Guajira and Cabo de la Aguja. We used monthly averaged Moderate Resolution Imaging Spectroradiometer (MODIS) sea surface temperature as the only information to build up a prediction model for the upwelling events. This comprised two steps: (i) the reduction of the complexity by means of the Karhunen–Loève transform and (ii) a prediction model of time series. Two prediction models were considered: (a) a parametric autoregressive-moving average (ARMA) time series from the Box–Jenkins methodology and (b) a harmonic synthesis model. The harmonic synthesis also comprised of two steps: the maximum entropy spectral analysis and a least-squares harmonic analysis on the set of frequencies. The parametric ARMA time series model failed at the time of prediction with a very narrow range, and it was quite difficult to apply. The harmonic synthesis allowed prediction with a horizon of six months with a correlation of about 0.80. The results can be summarized using the time series of the weights of the different oscillation modes, their spatial structures with the nodal lines, and a high confidence model with a horizon of prediction of about four months.

Complementing the power of deep learning with statistical model fusion: Probabilistic forecasting of influenza in Dallas County, Texas, USA

Influenza is one of the main causes of death, not only in the USA but worldwide. Its significant economic and public health impacts necessitate development of accurate and efficient algorithms for forecasting of any upcoming influenza outbreaks. Most currently available methods for influenza prediction are based on parametric time series and regression models that impose restrictive and often unverifiable assumptions on the data. In turn, more flexible machine learning models and, particularly, deep learning tools whose utility is proven in a wide range of disciplines, remain largely under-explored in epidemiological forecasting. We study the seasonal influenza in Dallas County by evaluating the forecasting ability of deep learning with feedforward neural networks as well as performance of more conventional statistical models, such as beta regression, autoregressive integrated moving average (ARIMA), least absolute shrinkage and selection operators (LASSO), and non-parametric multivariate adaptive regression splines (MARS) models for one week and two weeks ahead forecasting. Furthermore, we assess forecasting utility of Google search queries and meteorological data as exogenous predictors of influenza activity. Finally, we develop a probabilistic forecasting of influenza in Dallas County by fusing all the considered models using Bayesian model averaging.

A New Process Control Chart for Monitoring Short-Range Serially Correlated Data

–Statistical process control (SPC) charts are critically important for quality control and management in manufacturing industries, environmental monitoring, disease surveillance, and many other applications. Conventional SPC charts are designed for cases when process observations are independent at different observation times. In practice, however, serial data correlation almost always exists in sequential data. It has been well demonstrated in the literature that control charts designed for independent data are unstable for monitoring serially correlated data. Thus, it is important to develop control charts specifically for monitoring serially correlated data. To this end, there is some existing discussion in the SPC literature. Most existing methods are based on parametric time series modeling and residual monitoring, where the data are often assumed to be normally distributed. In applications, however, the assumed parametric time series model with a given order and the normality assumption are often invalid, resulting in unstable process monitoring. Although there is some nice discussion on robust design of such residual monitoring control charts, the suggested designs can only handle certain special cases well. In this article, we try to make another effort by proposing a novel control chart that makes use of the restarting mechanism of a CUSUM chart and the related spring length concept. Our proposed chart uses observations within the spring length of the current time point and ignores all history data that are beyond the spring length. It does not require any parametric time series model and/or a parametric process distribution. It only requires the assumption that process observation at a given time point is associated with nearby observations and independent of observations that are far away in observation times, which should be reasonable for many applications. Numerical studies show that it performs well in different cases.

On model selection from a finite family of possibly misspecified time series models

Consider finite parametric time series models. “I have $n$ observations and $k$ models, which model should I choose on the basis of the data alone” is a frequently asked question in many practical situations. This poses the key problem of selecting a model from a collection of candidate models, none of which is necessarily the true data generating process (DGP). Although existing literature on model selection is vast, there is a serious lacuna in that the above problem does not seem to have received much attention. In fact, existing model selection criteria have avoided addressing the above problem directly, either by assuming that the true DGP is included among the candidate models and aiming at choosing this DGP, or by assuming that the true DGP can be asymptotically approximated by an increasing sequence of candidate models and aiming at choosing the candidate having the best predictive capability in some asymptotic sense. In this article, we propose a misspecification-resistant information criterion (MRIC) to address the key problem directly. We first prove the asymptotic efficiency of MRIC whether the true DGP is among the candidates or not, within the fixed-dimensional framework. We then extend this result to the high-dimensional case in which the number of candidate variables is much larger than the sample size. In particular, we show that MRIC can be used in conjunction with a high-dimensional model selection method to select the (asymptotically) best predictive model across several high-dimensional misspecified time series models.

Penalized Averaging of Parametric and Non-Parametric Quantile Forecasts

We propose a hybrid penalized averaging for combining parametric and non-parametric quantile forecasts when faced with a large number of predictors. This approach goes beyond the usual practice of combining conditional mean forecasts from parametric time series models with only a few predictors. The hybrid methodology adopts the adaptive LASSO regularization to simultaneously reduce predictor dimension and obtain quantile forecasts. Several recent empirical studies have considered a large set of macroeconomic predictors and technical indicators with the goal of forecasting the S&P 500 equity risk premium. To illustrate the merit of the proposed approach, we extend the mean-based equity premium forecasting into the conditional quantile context. The application offers three main findings. First, combining parametric and non-parametric approaches adds quantile forecast accuracy over and above the constituent methods. Second, a handful of macroeconomic predictors are found to have systematic forecasting power. Third, different predictors are identified as important when considering lower, central and upper quantiles of the equity premium distribution.

A hidden Markov regime-switching smooth transition model

Abstract In this paper, we develop a new class of parametric nonlinear time series models by combining two important classes of models, namely smooth transition models and hidden Markov regime-switching models. The class of models is general and flexible enough to incorporate two types of switching behavior: smooth state transitions and abrupt changes in hidden states. The estimation of the hidden states and model parameters is performed by applying filtering theory and a filter-based expectation-maximization (EM) algorithm. Applications of the model are illustrated using simulated data and real financial data. Other potential applications are mentioned.

Identifying Dynamical Time Series Model Parameters from Equilibrium Samples, with Application to Gene Regulatory Networks.

Gene regulatory network reconstruction is an essential task of genomics in order to further our understanding of how genes interact dynamically with each other. The most readily available data, however, are from steady state observations. These data are not as informative about the relational dynamics between genes as knockout or over-expression experiments, which attempt to control the expression of individual genes. We develop a new framework for network inference using samples from the equilibrium distribution of a vector autoregressive (VAR) time-series model which can be applied to steady state gene expression data. We explore the theoretical aspects of our method and apply the method to synthetic gene expression data generated using GeneNetWeaver.

Estimation pitfalls when the noise is not i.i.d.

This paper extends Whittle estimation to linear processes with a general stationary ergodic martingale difference noise. We show that such estimation is valid for standard parametric time series models with smooth bounded spectral densities, e.g., ARMA models. Furthermore, we clarify the impact of the hidden dependence in the noise on such estimation. We show that although the asymptotic normality of the Whittle estimates may still hold, the presence of dependence in the noise impacts the limit variance. Hence, the standard errors and confidence intervals valid under i.i.d. noise may not be applicable and thus require correction. The goal of this paper is to raise awareness to the impact of a non-i.i.d. noise in applied work.

Wavelet estimation of functional coefficient regression models

The area of nonlinear time series models has experienced a great development since the 1980s. Although there is a wide range of parametric nonlinear time series models, in general, we do not know if the postulated model is the most appropriated one for a specific data set. This situation highlights the importance of nonparametric models. An interesting nonparametric model to fit nonlinear time series is the well-known functional coefficient regression model. Nonparametric estimations by, e.g., local linear regression and splines, are developed in the literature. In this work, we study the estimation of such a model using wavelets. It is a proposal that takes into account both, classical and warped wavelets. We present the rates of convergence of the proposed estimators and carry out simulation studies to evaluate automatic procedures (among AIC, AICc and BIC) for selecting the coarsest and finest levels to be used during the estimation process. Moreover, we illustrate the methodology with an application to a real data set, where we also calculate multi-step-ahead forecasts and compare the results with other methods known in the literature.

TESTS FOR TAR MODELS VS. STAR MODELS--A SEPARATE FAMILY OF HYPOTHESES APPROACH

The threshold autoregressive (TAR) model and the smooth threshold autoregressive (STAR) model have been popular parametric nonlinear time series models for the past three decades or so. As yet there is no formal statistical test in the literature for one against the other. The two models are fundamentally different in their autoregressive functions, the TAR model being generally discontinuous while the STAR model is smooth (except in the limit of infinitely fast switching for some cases). Following the approach initiated by Cox (1961, 1962), we treat the test problem as one of separate families of hypotheses. The test statistic under a STAR model is shown to follow asymptotically a chi-squared distribution, and the one under a TAR model can be expressed as a functional of a chi-squared process. We present numerical results with both simulated and real data to assess the performance of our procedure.

Modeling baseline conditions of ecological indicators: Marine renewable energy environmental monitoring

Ecological indicators are often collected to detect and monitor environmental change. Statistical models are used to estimate natural variability, pre-existing trends, and environmental predictors of baseline indicator conditions. Establishing standard models for baseline characterization is critical to the effective design and implementation of environmental monitoring programs. An anthropogenic activity that requires monitoring is the development of Marine Renewable Energy sites. Currently, there are no standards for the analysis of environmental monitoring data for these development sites. Marine Renewable Energy monitoring data are used as a case study to develop and apply a model evaluation to establish best practices for characterizing baseline ecological indicator data. We examined a range of models, including six generalized regression models, four time series models, and three nonparametric models. Because monitoring data are not always normally distributed, we evaluated model ability to characterize normal and non-normal data using hydroacoustic metrics that serve as proxies for ecological indicator data. The nonparametric support vector regression and random forest models, and parametric state-space time series models generally were the most accurate in interpolating the normal metric data. Support vector regression and state-space models best interpolated the non-normally distributed data. If parametric results are preferred, then state-space models are the most robust for baseline characterization. Evaluation of a wide range of models provides a comprehensive characterization of the case study data, and highlights advantages of models rarely used in Marine Renewable Energy environmental monitoring. Our model findings are relevant for any ecological indicator data with similar properties, and the evaluation approach is applicable to any monitoring program.

A New Approach for Parameter Estimation of Autoregressive Models Using Adaptive Network-Based Fuzzy Inference System (ANFIS)

Time series modeling plays an important role in different fields of science and engineering such as hydrology and water resources management. The proper estimation of the parameters in time series models is one of the essential steps of modeling. Yule–Walker, least square, Burge and forward–backward approaches are known, and common methods of parameter estimation are used in many time series studies. Recently, intelligent techniques such as adaptive network-based fuzzy inference system (ANFIS) have been used for time series modeling. Review of previous researches, especially in the field of hydrological time series, shows that these systems are often used as intelligent forecasting systems; indeed, they were considered as a black box. In this study, using ANFIS and its basic concepts, a new approach is devised for parameter estimation of autoregressive (AR) models. Performance of this approach is evaluated through the Akaike information criterion; also its application has been surveyed in time series forecasting by naturalized inflow of the Zayandehrud dam located in central Iran. Results show that the proposed approach has a good and effective performance for parameter estimation of AR models which can be depicted as a new “intelligent approach.” In addition, this capability of ANFIS in parameter estimation is a new application of ANFIS that was not addressed in the past. Also, the new driven method from ANFIS shows that this system can be employed as a parameter estimator for time series models such as AR models.

Capturing Context-Related Change in Emotional Dynamics via Fixed Moderated Time Series Analysis

ABSTRACTMuch of recent affect research relies on intensive longitudinal studies to assess daily emotional experiences. The resulting data are analyzed with dynamic models to capture regulatory processes involved in emotional functioning. Daily contexts, however, are commonly ignored. This may not only result in biased parameter estimates and wrong conclusions, but also ignores the opportunity to investigate contextual effects on emotional dynamics. With fixed moderated time series analysis, we present an approach that resolves this problem by estimating context-dependent change in dynamic parameters in single-subject time series models. The approach examines parameter changes of known shape and thus addresses the problem of observed intra-individual heterogeneity (e.g., changes in emotional dynamics due to observed changes in daily stress). In comparison to existing approaches to unobserved heterogeneity, model estimation is facilitated and different forms of change can readily be accommodated. We demonstrate the approach's viability given relatively short time series by means of a simulation study. In addition, we present an empirical application, targeting the joint dynamics of affect and stress and how these co-vary with daily events. We discuss potentials and limitations of the approach and close with an outlook on the broader implications for understanding emotional adaption and development.