Nonparametric Time Series Models Research Articles

Short-term forecasting of German generation-based CO2 emission factors using parametric and non-parametric time series models

This study focuses on forecasting German generation-based CO2 emission factors to develop accurate prediction models, which help to shift flexible loads in time with low emissions. While most existing research relies on point forecasts to predict CO2 emission factors, the presented methods are utilized to perform interval forecasts. In addition, compared to other studies, recent data that extends over a long period is used. The study describes the used data and discusses the concept of walk-forward validation. Further, various models are employed and tuned to forecast the emission factors, including benchmark, parametric (e.g., SARIMAX), and non-parametric (bagging, random forest, gradient boosting, CNN, LSTM, MLP) models. The study reveals that all applied parametric and non-parametric models yield better results than the benchmark models, while the gradient boosting model has the lowest mean absolute error with 40.66 gCO2/kWh, the lowest mean absolute percentage error 8.17%, and the random forest has the lowest root mean square error with 57.61 gCO2/kWh. However, the potential of the deep learning models was not fully exploited. In a live application, the implementation effort should be evaluated against the benefit of better prediction.

Estimation of prediction error in time series

Summary The accurate estimation of prediction errors in time series is an important problem, which has immediate implications for the accuracy of prediction intervals as well as the quality of a number of widely used time series model selection criteria such as the Akaike information criterion. Except for simple cases, however, it is difficult or even impossible to obtain exact analytical expressions for one-step and multi-step predictions. This may be one of the reasons that, unlike in the independent case (see Efron, 2004), up to now there has been no fully established methodology for time series prediction error estimation. Starting from an approximation to the bias-variance decomposition of the squared prediction error, a method for accurate estimation of prediction errors in both univariate and multivariate stationary time series is developed in this article. In particular, several estimates are derived for a general class of predictors that includes most of the popular linear, nonlinear, parametric and nonparametric time series models used in practice, with causal invertible autoregressive moving average and nonparametric autoregressive processes discussed as lead examples. Simulations demonstrate that the proposed estimators perform quite well in finite samples. The estimates may also be used for model selection when the purpose of modelling is prediction.

Multi-Step-Ahead Prediction Intervals for Nonparametric Autoregressions via Bootstrap: Consistency, Debiasing, and Pertinence

To address the difficult problem of the multi-step-ahead prediction of nonparametric autoregressions, we consider a forward bootstrap approach. Employing a local constant estimator, we can analyze a general type of nonparametric time-series model and show that the proposed point predictions are consistent with the true optimal predictor. We construct a quantile prediction interval that is asymptotically valid. Moreover, using a debiasing technique, we can asymptotically approximate the distribution of multi-step-ahead nonparametric estimation by the bootstrap. As a result, we can build bootstrap prediction intervals that are pertinent, i.e., can capture the model estimation variability, thus improving the standard quantile prediction intervals. Simulation studies are presented to illustrate the performance of our point predictions and pertinent prediction intervals for finite samples.

PERBANDINGAN MODEL ARIMA DENGAN MODEL NONPARAMETRIK POLINOMIAL LOKAL DAN SPLINE TRUNCATED UNTUK PERAMALAN HARGA MINYAK MENTAH WEST TEXAS INTERMEDIATE (WTI) DILENGKAPI GUI R

Crude oil as one of the most important natural resources experiences price fluctuations from time to time, even the spot price of West Texas Intermediate (WTI) world crude oil on 20th April 2020 reached -36,98 USD/barrel due to the Covid-19 pandemic. WTI oil price data was modeled using the ARIMA method, local polynomial, and spline truncated nonparametric regression then compared and obtained the best model and formed R Graphical User Interface (GUI). The ARIMA model and nonparametric time series models can be used to model time series data, but in the ARIMA model there are assumptions that must be fulfilled. Nonparametric time series models, which include local polynomial model and truncated spline do not need to fulfill these assumptions. The ARIMA model obtained did not fulfilled the assumptions of normality and residual homoscedasticity, so the modeling was stopped and modeling was only carried out using nonparametric regression methods. Based on the minimum MSE criteria, the best nonparametric model was obtained, namely nonparametric truncated spline model with degrees 3 and 3 knot points which was categorized as a strong model based on R-squared in sample value and having a very good forecasting ability based on MAPE out sample value.

A multi-quantile regression time series model with interquantile lipschitz regularization for wind power probabilistic forecasting

Modern decision-making processes require uncertainty-aware models, especially those relying on non-symmetric costs and risk-averse profiles. The objective of this work is to propose a dynamic model for the conditional non-parametric distribution function (CDF) to generate probabilistic forecasts for a renewable generation time series. To do that, we propose an adaptive non-parametric time-series model driven by a regularized multiple-quantile-regression (MQR) framework. In our approach, all regression models are jointly estimated through a single linear optimization problem that finds the global-optimal parameters in polynomial time. An innovative feature of our work is the consideration of a Lipschitz regularization of the first derivative of coefficients in the quantile space, which imposes coefficient smoothness. The proposed regularization induces a coupling effect among quantiles creating a single non-parametric CDF model with improved out-of-sample performance. A case study with realistic wind-power generation data from the Brazilian system shows: 1) the regularization model is capable to improve the performance of MQR probabilistic forecasts, and 2) our MQR model outperforms five relevant benchmarks: two based on the MQR framework, and three based on parametric models, namely, SARIMA, and GAS with Beta and Weibull CDF.

MLTF: Model less time-series forecasting

Multi-step ahead long term forecasting remains a pertinent challenge in time series literature due to non-stationary behaviour of real-world data. Predominantly most traditional time series models are parametric in nature and they use the predicted values to generate forecast for future time steps. This leads to error accumulation in each step of the forecasting horizon which causes increasingly poorer forecast in long-term. Other than the problem of error accumulation, most parametric algorithms also require significant pre-processing, hyper-parameter tuning, training and post-processing which can often put high computational burden on the system. Therefore, this paper proposes, Model Less Time-series Forecasting (MLTF), a non-parametric approach for forecasting which does not require any pre-processing or traditional training (i.e. Backpropagation). MLTF is a non-parametric method which uses statistical representations such as trend, linearity, entropy etc. to cluster series from a pre-defined repository and the series from same cluster are tagged as similar series. The trajectory of the target series is extracted from these similar series after applying an adaptive re-sampling technique. There is minimal training involved in MLTF, therefore this framework is computationally very efficient. The model-less nature also enables it to not suffer from error accumulation in long-horizon forecast. MLTF is validated empirically with a rich set of experiments involving M1, M3 competition dataset, Electricity, Volatility and COVID-19 data (over 4500 independent uni-variate series of different frequencies i.e. Hourly, Daily, Monthly, Quarterly and Yearly). The experiments demonstrate that, MLTF is significantly faster while being similar (or better) in terms of forecasting accuracy than the state-of-the-art DL methods and other non-parametric time series model.

(2011-6287) A fuzzy non-parametric time series model based on fuzzy data

Parametric time series models typically consists of model identification, parameter estimation, model diagnostic checking, and forecasting. However compared with parametric methods, nonparametric time series models often providea very flexible approach to bring out the features of the observed time series. This paper suggested a novel fuzzy nonparametric method in time series models with fuzzy observations. For this purpose, a fuzzy forward fit kernel-basedsmoothing method was introduced to estimate fuzzy smooth functions corresponding to each observation. A simple optimization algorithm was also suggested to evaluate optimal bandwidths and autoregressive order. Several common goodness-of-fit criteria were also extended to compare the performance of the proposed fuzzy time series method compared to other fuzzy time series model based on fuzzy data. Furthermore, the effectiveness of the proposed method was illustrated through two numerical examples including a simulation study. The results indicate that the proposed model performs better than the previous ones in terms of both scatter plot criteria and goodness-of-fit evaluations.

Statistical guided-waves-based structural health monitoring via stochastic non-parametric time series models

Damage detection in active-sensing, guided-waves-based structural health monitoring (SHM) has evolved through multiple eras of development during the past decades. Nevertheless, there still exist a number of challenges facing the current state-of-the-art approaches, both in the industry as well as in research and development, including low damage sensitivity, lack of robustness to uncertainties, need for user-defined thresholds, and non-uniform response across a sensor network. In this work, a novel statistical framework is proposed for active-sensing SHM based on the use of ultrasonic guided waves. This framework is based on stochastic non-parametric time series models and their corresponding statistical properties in order to readily provide healthy confidence bounds and enable accurate and robust damage detection via the use of appropriate statistical decision-making tests. Three such methods and corresponding statistical quantities (test statistics) along with decision-making schemes are formulated and experimentally assessed via the use of three coupons with different levels of complexity: an Al plate with a growing notch, a carbon fiber-reinforced plastic (CFRP) plate with added weights to simulate local damage, and the CFRP panel used in the Open Guided Waves project, all fitted with piezoelectric transducers under a pitch-catch configuration. The performance of the proposed methods is compared to that of state-of-the-art time-domain damage indices (DIs). The results demonstrate the increased detection sensitivity and robustness of the proposed methods, with better tracking capability of damage evolution compared to conventional approaches, even for damage-non-intersecting actuator–sensor paths. In particular, the Z statistic emerges as the best damage detection metric compared to conventional DIs, as well as the other proposed statistics. Overall, the proposed statistics in this study promise greater damage sensitivity across different components, with enhanced robustness to uncertainties, as well as user-friendly application.

A weighted sieve estimator for nonparametric time series models with nonstationary variables

We study a class of nonparametric regression models that includes deterministic time trends and both stationary and nonstationary stochastic processes (whose shocks are allowed to be mutually correlated). We propose a unified approach to estimation based on the weighted sieve method to tackle the issue of unbounded support of the covariates. This approach improves on the existing technology in terms of some key regularity conditions such as moment conditions and the α-mixing coefficients for the stationary process. We establish self-normalized central limit theorems for the sieve estimator and other related quantities. Monte Carlo simulation confirms the theoretical results. We use our methodology to study the effect of CO2 and solar irradiance on global sea level rise.

Censored Nonparametric Time-Series Analysis with Autoregressive Error Models

This paper focuses on nonparametric regression modeling of time-series observations with data irregularities, such as censoring due to a cutoff value. In general, researchers do not prefer to put up with censored cases in time-series analyses because their results are generally biased. In this paper, we present an imputation algorithm for handling auto-correlated censored data based on a class of autoregressive nonparametric time-series model. The algorithm provides an estimation of the parameters by imputing the censored values with the values from a truncated normal distribution, and it enables unobservable values of the response variable. In this sense, the censored time-series observations are analyzed by nonparametric smoothing techniques instead of the usual parametric methods to reduce modelling bias. Typically, the smoothing methods are updated for estimating the censored time-series observations. We use Monte Carlo simulations based on right-censored data to compare the performances and accuracy of the estimates from the smoothing methods. Finally, the smoothing methods are illustrated using a meteorological time- series and unemployment datasets, where the observations are subject to the detection limit of the recording tool.

Allocation of Driver Attention for Varying In-Vehicle System Modalities.

This paper examines drivers' allocation of attention using response time to a tactile detection response task (TDRT) while interacting with an in-vehicle information system (IVIS) over time. Longer TDRT response time is associated with higher cognitive workload. However, it is not clear what role is assumed by the human and system in response to varying in-vehicle environments over time. A driving simulator study with 24 participants was conducted with a restaurant selection task of two difficulty levels (easy and hard) presented in three modalities (audio only, visual only, hybrid). A linear mixed-effects model was applied to identify factors that affect TDRT response time. A nonparametric time-series model was also used to explore the visual attention allocation under the hybrid mode over time. The visual-only mode significantly increased participants' response time compared with the audio-only mode. Females took longer to respond to the TDRT when engaged with an IVIS. The study showed that participants tend to use the visual component more toward the end of the easy tasks, whereas the visual mode was used more at the beginning of the harder tasks. The visual-only mode of the IVIS increased drivers' cognitive workload when compared with the auditory-only mode. Drivers showed different visual attention allocation during the easy and hard restaurant selection tasks in the hybrid mode. The findings can help guide the design of automotive user interfaces and help manage cognitive workload.

A Nonparametric Bayesian Framework for Short-Term Wind Power Probabilistic Forecast

To improve the energy system resilience and economic efficiency, the wind power as a renewable energy starts to be deeply integrated into smart power grids. However, the wind power forecast uncertainty brings operational challenges. In order to provide a reliable guidance on operational decisions, in this paper, we propose a short-term wind power probabilistic forecast. Specifically, to model the rich dynamic behaviors of underlying physical wind power stochastic process occurring in various meteorological conditions, we first introduce an infinite Markov switching autoregressive model. This nonparametric time series model can capture the important properties in the real-world data to improve the prediction accuracy. Then, given finite historical data, the posterior distribution of flexible forecast model can correctly quantify the model estimation uncertainty. Built on it, we develop the posterior predictive distribution to rigorously quantify the overall forecasting uncertainty accounting for both inherent stochastic uncertainty and model estimation error. Therefore, the proposed approach can provide accurate and reliable short-term wind power probabilistic forecast, which can be used to support smart power grids real-time risk management.

An adaptive truncation method for inference in Bayesian nonparametric models

Many exact Markov chain Monte Carlo algorithms have been developed for posterior inference in Bayesian nonparametric models which involve infinite-dimensional priors. However, these methods are not generic and special methodology must be developed for different classes of prior or different models. Alternatively, the infinite-dimensional prior can be truncated and standard Markov chain Monte Carlo methods used for inference. However, the error in approximating the infinite-dimensional posterior can be hard to control for many models. This paper describes an adaptive truncation method which allows the level of the truncation to be decided by the algorithm and so can avoid large errors in approximating the posterior. A sequence of truncated priors is constructed which are sampled using Markov chain Monte Carlo methods embedded in a sequential Monte Carlo algorithm. Implementational details for infinite mixture models with stick-breaking priors and normalized random measures with independent increments priors are discussed. The methodology is illustrated on infinite mixture models, a semiparametric linear mixed model and a nonparametric time series model.

Bayesian Consistency for Markov Models

We consider sufficient conditions for Bayesian consistency of the transition density of time homogeneous Markov processes. To date, this remains somewhat of an open problem, due to the lack of suitable metrics with which to work. Standard metrics seem inadequate, even for simple autoregressive models. Current results derive from generalizations of the i.i.d. case and additionally require some non-trivial model assumptions. We propose suitable neighborhoods with which to work and derive sufficient conditions for posterior consistency which can be applied in general settings. We illustrate the applicability of our result with some examples; in particular, we apply our result to a general family of nonparametric time series models.

Testing for changes in autocovariances of nonparametric time series models

In the literature on change-point analysis, much attention has been paid to detecting changes in certain marginal characteristics, such as mean, variance, and marginal distribution. For time series data with nonparametric time trend, we study the change-point problem for the autocovariance structure of the unobservable error process. To derive the asymptotic distribution of the cumulative sum test statistic, we develop substantial theory for uniform convergence of weighted partial sums and weighted quadratic forms. Our asymptotic results improve upon existing works in several important aspects. The performance of the test statistic is examined through simulations and an application to interest rates data.

Semiparametric and Nonparametric Methods in Econometrics

The main objective of this workshop was to bring together mathematical statisticians and econometricians who work in the field of nonparametric and semiparametric statistical methods. Nonparametric and semiparametric methods are active fields of research in econometric theory and are becoming increasingly important in applied econometrics. This is because the flexibility of non- and semiparametric modelling provides important new ways to investigate problems in substantive economics. Many of the most important developments in semi- and nonparametric statistical theory now take place in econometrics. Moreover, the development of non- and semiparametric methods that are suitable to the needs of economics presents a variety of mathematical challenges. Econometric research aims at achieving an understanding of the economic processes that generate observed data. This is different from fitting data that may be useful for prediction but that do not capture underlying causes. A large part of economic theory consists of models of equilibria of competing processes. Statistical data are a snapshot of the equilibrium but, by themselves, do not reveal the processes that led to the equilibrium. Consequently a reduced form model (e.g.\ a conditional mean function) does not suffice for much economic research. Achieving an understanding of the economic processes requires a careful combining of economic theory and statistical considerations. This often requires the development of statistical tools that are specific to the problems that arise in economics and are unfamiliar in other statistical specialties. For example, econometric research has focused on developing methods to deal with endogenous covariates (that is, covariates that are correlated with a model's error terms), time series models that fit equilibria as stationary submodels (cointegration), and time series models for volatility processes (conditional variances) in finance. Semi- and nonparametric methods are being used increasingly frequently in applied econometrics. The models are not necessarily of the simple form of classical regression, "response = signal plus independent noise," where the signal can be recovered by nonparametric smoothing of the responses. Rather, the nonparametric functions enter the model in a much more complicated way. Mathematically this has led to challenging problems. Identifiability of a model is much more involved in nonparametric model specifications. In particular, this is the case for nonseparable models where the error terms do not enter additively into the model. Some nonparametric inference problems with endogenous covariates lead to statistical inverse problems and require the study of estimates and solutions of noisy integral equations. The mathematical analysis of nonparametric time-series models and of nonparametric diffusion models is strongly related to research in stochastic processes, Markov processes, stochastic analysis and financial mathematics. Empirical process theory is an essential tool for the understanding of uniform performance and of convergence rates of nonparametric estimates and for efficiency considerations in semiparametric models. All these problems were topics of talks and discussions at the workshop. The mathematical development in econometrics is complimentary to recent statistical applications in biology. There, the focus tends to be on dimension reduction for the statistical analysis of high-dimensional data. The intellectual charm of mathematical research in modern econometrics comes from the interplay between statistical and economic theory.

Non-parametric time series models for hydrological forecasting

To perform hydrological forecasting, time series methods are often employed. In univariate time series, the autoregressive integrated moving average (ARIMA) model, the seasonal autoregressive moving average (SARMA) model, the deseasonalized model and the periodic autoregressive (PAR) model are often used. These models are based on the assumption that the influence of lagged riverflows on the riverflow is linear. In reality the assumption is often questionable. In this paper, the functional-coefficient autoregression (FCAR) model, which is a nonlinear model, is introduced to forecast riverflows. To explore the influence of the inflow on the outflow in a river system and to exploit the internal interaction of the outflows, bivariate time series models are needed. The transfer function (TF) model and the semi-parametric regression (SPR) model are often employed. In this paper, a new model, the non-parametric and functional-coefficient autoregression (NFCAR) model, is proposed. It consists of two parts: the first part, the non-parametric part explains the influences of the inflows on the outflow in a river system; the second part, the functional-coefficient linear part reveals the interactions among the outflows in a river system. By comparing the calibration and forecasting of the models, it is found that the NFCAR model performs very well.

On the cusum of squares test for variance change in nonstationary and nonparametric time series models

In this paper we consider the problem of testing for a variance change in nonstationary and nonparametric time series models. The models under consideration are the unstable AR(q) model and the fixed design nonparametric regression model with a strong mixing error process. In order to perform a test, we employ the cusum of squares test introduced by Inclan and Tiao (1994,J. Amer. Statist. Assoc.,89, 913–923). It is shown that the limiting distribution of the test statistic is the sup of a standard Brownian bridge as seen in iid random samples. Simulation results are provided for illustration.

Bootstrap Methods for Time Series

SummaryThe bootstrap is a method for estimating the distribution of an estimator or test statistic by resampling one's data or a model estimated from the data. The methods that are available for implementing the bootstrap and the accuracy of bootstrap estimates depend on whether the data are an independent random sample or a time series. This paper is concerned with the application of the bootstrap to time‐series data when one does not have a finite‐dimensional parametric model that reduces the data generation process to independent random sampling. We review the methods that have been proposed for implementing the bootstrap in this situation and discuss the accuracy of these methods relative to that of first‐order asymptotic approximations. We argue that methods for implementing the bootstrap with time‐series data are not as well understood as methods for data that are independent random samples. Although promising bootstrap methods for time series are available, there is a considerable need for further research in the application of the bootstrap to time series. We describe some of the important unsolved problems.RésuméLe bootstrap est une méthode pour estimer la distribution d'un estimateur en rééchantillonnant ses données ou un modéle estiméà partir des données. Les méthodes disponibles pour mettre en oeuvre le bootstrap et la précision des estimateurs de bootstrap dépendent de ce que les données proviennent d'un échantillon aléatoire indépendant ou d'une série temporelle. Cet article concerne l'application du bootstrap aux données des séries temporelles quand on ne dispose pas de modéle paramétrique de dimension finie qui réduise le processus de génération des données à l'échantillonnage aléatoire indépendent. Nous examinons les méthodes qui ont été proposées pour mettre en oeuvre le bootstrap dans cette situation et discutons la precision de ces méthodes comparativement à celle des approximations asymptotiques de premier ordre. Nous montrons que les méthodes pour mettre en oeuvre le bootstrap avec les données des séries temporelles ne sont pas aussi bien comprises que les méthodes pour les données des échantillons aléatoires indépendants. Bien que des méthodes de bootstrap prometteuses pour les séries temporelles soient disponibles, il y a un besoin considérable de recherche supplémental re dans leur application. Nous décrivons quelques problémes importants non résolus.

Consistent hypothesis testing in semiparametric and nonparametric models for econometric time series

In this paper we modify the general hypothesis studied by Robinson (1989) for semi-/nonparametric time-series models, and present a consistent testing procedure for the modified hypothesis. As examples, we provide consistent tests for the portfolio conditional mean-variance efficiency hypothesis, for theomitted variables in a multivariate nonparametric time-series regression model, and for the two original examples in Robinson. The asymptotic distributions under the null and Pitman local alternatives are established by invoking central limit theorems for Hilbert-valued-dependent random arrays. To approximate the critical values of the general test, we modify the conditional Monte-Carlo approach of Hansen (1996) and the stationary bootstrap of Politis and Romano (1994a,b), and show that both work asymptotically.