Time Series Model Selection Research Articles

Estimation of prediction error in time series

Summary The accurate estimation of prediction errors in time series is an important problem, which has immediate implications for the accuracy of prediction intervals as well as the quality of a number of widely used time series model selection criteria such as the Akaike information criterion. Except for simple cases, however, it is difficult or even impossible to obtain exact analytical expressions for one-step and multi-step predictions. This may be one of the reasons that, unlike in the independent case (see Efron, 2004), up to now there has been no fully established methodology for time series prediction error estimation. Starting from an approximation to the bias-variance decomposition of the squared prediction error, a method for accurate estimation of prediction errors in both univariate and multivariate stationary time series is developed in this article. In particular, several estimates are derived for a general class of predictors that includes most of the popular linear, nonlinear, parametric and nonparametric time series models used in practice, with causal invertible autoregressive moving average and nonparametric autoregressive processes discussed as lead examples. Simulations demonstrate that the proposed estimators perform quite well in finite samples. The estimates may also be used for model selection when the purpose of modelling is prediction.

On consistency for time series model selection

On consistency for time series model selection

An extended Meta Learning Approach for Automating Model Selection in Big Data Environments using Microservice and Container Virtualizationz Technologies

For a given specific machine learning task, very often several machine learning algorithms and their right configurations are tested in a trial-and-error approach, until an adequate solution is found. This wastes human resources for constructing multiple models, requires a data analytics expert and is time-consuming. Meta learning addresses these problems and supports non-expert users by recommending a promising learning algorithm based on meta features computed from a given dataset. In the present paper, a new concept for enhancing the predictive performance of meta learning classification models by generating new meta examples is introduced. Our concept is realized and evaluated in a microservice-based meta learning framework. This framework makes use of a powerful Big Data software stack, container visualization, modern web technologies and a microservice architecture. In this demonstration and for evaluation purpose, time series model selection is taken as a use case for applying meta learning. It is shown that the proposed microservice-based meta learning framework introduces an excellent performance in assigning the adequate forecasting model for the chosen time series datasets. Moreover, our new concept for generating new meta examples enhances the predictive performance of the meta learner up to 16.77% and 27.07% in the case of using the original and encoded representation forms of meta features respectively. The recommendation of the most appropriate forecasting model results in a well acceptable low framework overhead demonstrating that the framework can provide an efficient approach to solve the problem of model selection in the context of Big Data.

Model selection for time series with nonlinear trend

A two-step model selection procedure is proposed for autoregressive and moving-average (ARMA) model class. It is an adaptive least absolute shrinkage and selection operator (adLASSO) type model selection method that simultaneously chooses both the orders and significant lagged variables when the autoregressive and moving-average (ARMA) time series is contaminated with a nonlinear trend. The adLASSO is applied not to the observations, but to the detrended residuals. The proposed two-step adLASSO model selection procedure under some conditions can identify the true model with probability approaching one as the sample size increases, and the asymptotic properties of estimators are not affected by the replacement of observations with detrended residuals. The simulation studies show that the proposed method performs well with sample size as small as 50. The application of the method is illustrated by the annual varve thickness data collected from a location in Massachusetts.

A first order autoregressive process with a change point: A bayesian approach based on model selection

The change points have considerable effects in different areas of applied research. We will use in this work the pseudo-bayes factor in three autoregressive models of order (1); this method permits to analyse the impact of choice between models and allows the use of a simpler technique with model selection in time series. For application, the monthly fluctuations of the DOW-JONES series between January 1999 and September 2009 have been used; we try to detect the financial crisis between 2007 and 2008 to evaluate the model selection method.

Mutual information model selection algorithm for time series

ABSTRACT Time series model selection has been widely studied in recent years. It is of importance to select the best model among candidate models proposed for a series in terms of explaining the procedure that governs the series and providing the most accurate forecast for the future observations. In this study, it is aimed to create an algorithm for order selection in Box–Jenkins models that combines penalized natural logarithm of mutual information among the original series and predictions coming from each candidate. The penalization is achieved by subtracting the number of parameters in each candidate and empirical information the data provide.Simulation studies under various scenarios and applications on real data sets imply that our algorithm offers a promising and satisfactory alternative to its counterparts.

Model selection for time series of count data

Selecting between competing statistical models is a challenging problem especially when the competing models are non-nested. An effective algorithm is developed in a Bayesian framework for selecting between a parameter-driven autoregressive Poisson regression model and an observation-driven integer valued autoregressive model when modelling time series count data. In order to achieve this a particle MCMC algorithm for the autoregressive Poisson regression model is introduced. The particle filter underpinning the particle MCMC algorithm plays a key role in estimating the marginal likelihood of the autoregressive Poisson regression model via importance sampling and is also utilised to estimate the DIC. The performance of the model selection algorithms are assessed via a simulation study. Two real-life data sets, monthly US polio cases (1970–1983) and monthly benefit claims from the logging industry to the British Columbia Workers Compensation Board (1985–1994) are successfully analysed.

Non-nested model selection based on the quantiles and it’s application in time series

ABSTRACTWe consider the problem of model selection based on quantile analysis and with unknown parameters estimated using quantile leasts squares. We propose a model selection test for the null hypothesis that the competing models are equivalent against the alternative hypothesis that one model is closer to the true model. We follow with two applications of the proposed model selection test. The first application is in model selection for time series with non-normal innovations. The second application is in model selection in the NoVas method, short for normalizing and variance stabilizing transformation, forecast. A set of simulation results also lends strong support to the results presented in the paper.

Graphical LASSO based Model Selection for Time Series

We propose a novel graphical model selection scheme for high-dimensional stationary time series or discrete time processes. The method is based on a natural generalization of the graphical LASSO algorithm, introduced originally for the case of i.i.d. samples, and estimates the conditional independence graph of a time series from a finite length observation. The graphical LASSO for time series is defined as the solution of an l <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -regularized maximum (approximate) likelihood problem. We solve this optimization problem using the alternating direction method of multipliers. Our approach is nonparametric as we do not assume a finite dimensional parametric model, but only require the process to be sufficiently smooth in the spectral domain. For Gaussian processes, we characterize the performance of our method theoretically by deriving an upper bound on the probability that our algorithm fails. Numerical experiments demonstrate the ability of our method to recover the correct conditional independence graph from a limited amount of samples.

A selection of time series models for short- to medium-term wind power forecasting

Abstract In this paper we investigate the short- to medium-term prediction performance of several recent wind power forecasting models. In particular, we analyze the Wind Power Prediction Tool (WPPT), which is a successfully employed model in Denmark, its generalization (GWPPT, generalized WPPT), an adaptation of the Mycielski approach, a nonparametric regression model and several univariate time series benchmarks. In the longer forecasting horizon scenario, GWPPT performs best, while the time series models are still strong competitors in the short-term setup. Our findings are in line with the majority of the literature. They support the results by Croonenbroeck and Dahl (2014) . The Mycielski approach is a successfully employed wind speed forecaster and usually returns well results. However, its performance as a wind power forecasting model is somewhat limited, showing that the adaptation to this new operational area leaves an opportunity for additional work in the future.

Model Selection in Time Series of Portuguese Hospital Admissions for Chronic Obstructive Pulmonary Disease

Model Selection in Time Series of Portuguese Hospital Admissions for Chronic Obstructive Pulmonary Disease

Information criteria: How do they behave in different models?

The choice of the best model is crucial in modeling data, and parsimony is one of the principles that must guide this choice. Despite their broad use in model selection, the foundations of the Akaike information criterion (AIC), the corrected Akaike criterion (AICc) and the Bayesian information criterion (BIC) are, in general, poorly understood. The AIC, AICc and BIC penalize the likelihoods in order to select the simplest model. These criteria are based upon concepts of information and entropy, which are explained in this work, by focusing on a statistical approach. The three criteria are compared through Monte Carlo simulations, and the applications of these criteria are investigated in the selection of normal models, the selection of biological growth models and selection of time series models. For the simulation with normal models, all three criteria exhibited poor performance for a small sample size N=100 (particularly, when the variances are slightly different). For biological growth model simulations with a very small sample size N=13 the AIC and AICc showed better performance in comparison to the BIC. The simulation based on time series models produced results similar to the normal model simulations. For these simulations, the BIC exhibited superior performance, in some cases, in comparison to the other two information criteria (AIC and AICc) for a small sample size N=100, but in other cases, the BIC performed poorly, as did the AIC and AICc.

Multiplex Protein Signature for the Detection of Bladder Cancer in Voided Urine Samples

Accurate urine assays for bladder cancer detection would benefit patients and health care systems. Through extensive genomic and proteomic profiling of urine components we previously identified a panel of 8 biomarkers that can facilitate the detection of bladder cancer in voided urine samples. In this study we confirmed this diagnostic molecular signature in a diverse multicenter cohort. We performed a case-control, phase II study in which we analyzed voided urine from 102 subjects with bladder cancer and 206 with varying urological disorders. The urinary concentration of 8 biomarkers (IL-8, MMP-9 and 10, PAI-1, VEGF, ANG, CA9 and APOE) was assessed by enzyme-linked immunosorbent assay. Diagnostic performance of the panel of tested biomarkers was evaluated using ROCs and descriptive statistical values, eg sensitivity and specificity. Seven of the 8 urine biomarkers were increased in subjects with bladder cancer relative to those without bladder cancer. The 7 biomarkers were assessed in a new model, which had an AUROC of 0.88 (95% CI 0.84-0.93), and 74% sensitivity and 90% specificity. In contrast, the sensitivity of voided urine cytology and the UroVysion® cytogenetic test in this cohort was 39% and 54%, respectively. Study limitations include analysis performed on banked urine samples and the lack of voided urine cytology and cytogenetic test data on controls. The study provides further evidence that the reported panel of diagnostic biomarkers can reliably achieve the noninvasive detection of bladder cancer with higher sensitivity than currently available urine based assays.

Quality Quandaries*: Time Series Model Selection and Parsimony

Click to increase image sizeClick to decrease image size ACKNOWLEDGMENTS Søren Bisgaard was supported by the Isenberg Program for Technology Management, the Isenberg School of Management, University of Massachusetts Amherst. The work of Murat Kulahci was supported in part by Danish Agency for Science Technology and Innovation grant 274-08-0280.

On time series model selection involving many candidate ARMA models

We study how to perform model selection for time series data where millions of candidate ARMA models may be eligible for selection. We propose a feasible computing method based on the Gibbs sampler. By this method model selection is performed through a random sample generation algorithm, and given a model of fixed dimension the parameter estimation is done through the maximum likelihood method. Our method takes into account several computing difficulties encountered in estimating ARMA models. The method is found to have probability of 1 in the limit in selecting the best candidate model under some regularity conditions. We then propose several empirical rules to implement our computing method for applications. Finally, a simulation study and an example on modelling China's Consumer Price Index (CPI) data are presented for purpose of illustration and verification.

Accumulative prediction error and the selection of time series models

This article reviews the rationale for using accumulative one-step-ahead prediction error (APE) as a data-driven method for model selection. Theoretically, APE is closely related to Bayesian model selection and the method of minimum description length (MDL). The sole requirement for using APE is that the models under consideration are capable of generating a prediction for the next, unseen data point. This means that APE may be readily applied to selection problems involving very complex models. APE automatically takes the functional form of parameters into account, and the ‘plug-in’ version of APE does not require the specification of priors. APE is particularly easy to compute for data that have a natural ordering, such as time series. Here, we explore the possibility of using APE to discriminate the short-range ARMA(1,1) model from the long-range ARFIMA ( 0 , d , 0 ) model. We also illustrate how APE may be used for model meta-selection, allowing one to choose between different model selection methods.

A Note on the Selection of Time Series Models

AbstractWe consider issues related to the order of an autoregression selected using information criteria. We study the sensitivity of the estimated order to (i) whether the effective number of observations is held fixed when estimating models of different order, (ii) whether the estimate of the variance is adjusted for degrees of freedom, and (iii) how the penalty for overfitting is defined in relation to the total sample size. Simulations show that the lag length selected by both the Akaike and the Schwarz information criteria are sensitive to these parameters in finite samples. The methods that give the most precise estimates are those that hold the effective sample size fixed across models to be compared. Theoretical considerations reveal that this is indeed necessary for valid model comparisons. Guides to robust model selection are provided.

Bayesian Subset Model Selection for Time Series

Abstract. This paper considers the problem of subset model selection for time series. In general, a few lags which are not necessarily continuous, explain lag structure of a time‐series model. Using the reversible jump Markov chain technique, the paper develops a fully Bayesian solution for the problem. The method is illustrated using the self‐exciting threshold autoregressive (SETAR), bilinear and AR models. The Canadian lynx data, the Wolfe's sunspot numbers and Series A of Box and Jenkins (1976) are analysed in detail.

Semiparametric Non-Linear Time Series Model Selection

SummarySemiparametric time series regression is often used without checking its suitability, resulting in an unnecessarily complicated model. In practice, one may encounter computational difficulties caused by the curse of dimensionality. The paper suggests that to provide more precise predictions we need to choose the most significant regressors for both the parametric and the nonparametric time series components. We develop a novel cross-validation-based model selection procedure for the simultaneous choice of both the parametric and the nonparametric time series components, and we establish some asymptotic properties of the model selection procedure proposed. In addition, we demonstrate how to implement it by using both simulated and real examples. Our empirical studies show that the procedure works well.

Regression and Time Series Model Selection

Regression and Time Series Model Selection