Accumulated Prediction Error Research Articles

Multistep Forecast Averaging with Stochastic and Deterministic Trends

This paper presents a new approach to constructing multistep combination forecasts in a nonstationary framework with stochastic and deterministic trends. Existing forecast combination approaches in the stationary setup typically target the in-sample asymptotic mean squared error (AMSE), relying on its approximate equivalence with the asymptotic forecast risk (AFR). Such equivalence, however, breaks down in a nonstationary setup. This paper develops combination forecasts based on minimizing an accumulated prediction errors (APE) criterion that directly targets the AFR and remains valid whether the time series is stationary or not. We show that the performance of APE-weighted forecasts is close to that of the optimal, infeasible combination forecasts. Simulation experiments are used to demonstrate the finite sample efficacy of the proposed procedure relative to Mallows/Cross-Validation weighting that target the AMSE as well as underscore the importance of accounting for both persistence and lag order uncertainty. An application to forecasting US macroeconomic time series confirms the simulation findings and illustrates the benefits of employing the APE criterion for real as well as nominal variables at both short and long horizons. A practical implication of our analysis is that the degree of persistence can play an important role in the choice of combination weights.

Sampling‐based 3‐D line‐of‐sight PWA model predictive control for autonomous rendezvous and docking with a tumbling target

AbstractThis article introduces a line‐of‐sight (LOS)‐Euler rendezvous and docking (RVD) framework to dock with a tumbling target under several RVD constraints. By a double‐loop control scheme, the chaser's position is controlled to track the target's docking port which is coupled with its rotation. The chaser's attitude is driven to track the target's rotation, while satisfying the field of view constraintwhich is coupled with the LOS range. These complex couplings are linearly described in the proposed framework. To this end, the 6 DoF information interactions among the sensor measurements, states, and RVD constraints need no transformation and linearization. Consider the online piecewise affine (PWA) model predictive controller (MPC) may be unsolvable under complex constraints, which is caused by the accumulated prediction error, a sampling‐based method is proposed. The linear predictions are driven to the closer neighborhoods of the actual nonlinear states by constructing directional sampling intervals. Besides, a singularity free strategy is provided to realize continuous tracking with crossing the singularities of angle states. Detailed numerical simulations illustrate the validity of the proposed methods.

Improving the robustness of recursive consequent parameters learning in evolving neuro-fuzzy systems

During the last 15 to 20 years, evolving (neuro-) fuzzy systems (E(N) FS) have enjoyed more and more attraction in the context of data stream mining and modeling processes. This is because they can be updated on the fly in a single-pass sample-wise manner and are able to perform autonomous changes of the models on structural level in order to react onto process drifts. A wide variety of evolving (neuro-) fuzzy systems approaches have been proposed in order to handle data stream mining and modeling processes by dynamically updating the rule structure and antecedents. The current denominator in the update of the consequent (output weight) parameters is the usage of the recursive (fuzzily weighted) least squares estimator (R(FW) LS), as being applied in almost all E(N) FS approaches. In this paper, we propose and examine alternative variants for consequent parameter updates, namely multi-innovation RFWLS, recursive correntropy and especially recursive weighted total least squares (RWTLS). Multi-innovation RFWLS guarantees more stability in the update whenever structural changes (i.e. changes in the antecedents) in the E(N) FS are performed. This is because rule membership degrees are actualized on (a portion of) past samples and properly integrated in each update step. Recursive correntropy addresses the problematic of outliers by down-weighing the influence of higher errors in the parameter updates. Recursive weighted total least squares also takes into account a possible noise level in the input variables (and not solely in the target variable as done in RFWLS). The approaches are compared with standard RFWLS i.) on three data stream regression problems from practical applications, which are affected by noise levels and where one embeds a known drift, and ii.) on a time-series based forecasting problem. The results based on accumulated prediction error trends over time indicate that RFWLS can be largely outperformed by the proposed alternative variants, and this with even lower sensitivity on various data noise levels. So, the proposed variants could be worth of being further considered as promising and serious alternatives.

Predictive processing in mental illness: Hierarchical circuitry for perception and trauma.

Predictive coding emerged as an explanation for how the brain can efficiently encode sensory stimuli. The hierarchical organization of neural circuits for perception thus passes prediction errors between computational layers. Extensions of this theory have provided a unifying understanding of Bayesian inference within neural circuits and psychiatric disorders. In particular, disorders of perception and belief have been explained as a Bayesian process of weighing prior beliefs (predictions) against new sensory data (prediction errors). The present issue of the Journal of Abnormal Psychology provides further evidence for how psychotic disorders develop and persist and how addiction- and trauma-related disorders may also be conceptualized. Trauma-related disorders in particular have begun to be identified as disorders of excessive accumulated prediction errors (uncertainty) over life. Here we summarize and reconcile recent advances in reinforcement learning momentum models with predictive processing and attempt to point out potential pitfalls for the application of hierarchical prediction models to stress. Future directions for understanding stress through this lens may need to involve updates to a purely hierarchical view or reframing long times cale molecular predictions as higher-order predictions. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Using information theory to optimise epidemic models for real-time prediction and estimation.

The effective reproduction number, Rt, is a key time-varying prognostic for the growth rate of any infectious disease epidemic. Significant changes in Rt can forewarn about new transmissions within a population or predict the efficacy of interventions. Inferring Rt reliably and in real-time from observed time-series of infected (demographic) data is an important problem in population dynamics. The renewal or branching process model is a popular solution that has been applied to Ebola and Zika virus disease outbreaks, among others, and is currently being used to investigate the ongoing COVID-19 pandemic. This model estimates Rt using a heuristically chosen piecewise function. While this facilitates real-time detection of statistically significant Rt changes, inference is highly sensitive to the function choice. Improperly chosen piecewise models might ignore meaningful changes or over-interpret noise-induced ones, yet produce visually reasonable estimates. No principled piecewise selection scheme exists. We develop a practical yet rigorous scheme using the accumulated prediction error (APE) metric from information theory, which deems the model capable of describing the observed data using the fewest bits as most justified. We derive exact posterior prediction distributions for infected population size and integrate these within an APE framework to obtain an exact and reliable method for identifying the piecewise function best supported by available epidemic data. We find that this choice optimises short-term prediction accuracy and can rapidly detect salient fluctuations in Rt, and hence the infected population growth rate, in real-time over the course of an unfolding epidemic. Moreover, we emphasise the need for formal selection by exposing how common heuristic choices, which seem sensible, can be misleading. Our APE-based method is easily computed and broadly applicable to statistically similar models found in phylogenetics and macroevolution, for example. Our results explore the relationships among estimate precision, forecast reliability and model complexity.

Studying the spatial temporal spread of the citrus tristeza virus through ODEs and Bernoulli trials

The Citrus tristeza virus (CTV) is one of the most economically devastating citrus diseases worldwide. The spread of CTV in eastern Spain was studied by Gottwald et al. with the goal of determining the spatio-temporal mechanisms of spread. Since the subjects in this study are individual trees, it is natural to think of infections as Bernoulli trials. This approach is difficult however, due to the spatial and temporal dependence of the observations. Consequently, a system of ordinary differential equations (ODE) was used to model the probabilities of infection as well as the spatial and temporal dependence. Given the parameters in the ODE, the probabilities of infection are treated as conditionally independent. Using the conditional independence we then specify the joint likelihood function as a Poisson binomial distribution. For the purpose of model selection and hypothesis testing we, employed accumulated prediction error (APE) which has connections to both Bayesian and frequentist frameworks. We demonstrated the robustness of our method in accounting for spatio-temporal dependencies in the data by accurately predicting the spatial distribution of the disease through Join Counts.

VAR(1) based models do not always outpredict AR(1) models in typical psychological applications.

In psychology, modeling multivariate dynamical processes within a person is gaining ground. A popular model is the lag-one vector autoregressive or VAR(1) model and its variants, in which each variable is regressed on all variables (including itself) at the previous time point. Many parameters have to be estimated in the VAR(1) model, however. The question thus rises whether the VAR(1) model is not too complex and overfits the data. If the latter is the case, the estimated model will not properly predict new unseen data. As a consequence, it cannot be trusted that the estimated parameters adequately characterize the individual from which the data at hand were sampled. In this article, we evaluate for current psychological applications whether the VAR(1) model outpredicts simpler models, using cross-validation (CV) techniques to determine the predictive accuracy. As it is unclear whether one should use standard CV techniques (leave-one-out CV or K-fold CV) or variants that take time dependence into account (blocked CV, hv-block CV, or accumulated prediction errors), we first compare the relative performance of these five CV techniques in a simulation study. The simulation settings mimic the data characteristics of current psychological VAR(1) applications and show that blocked CV has the best performance in general. Subsequently, we use blocked CV to assess to what extent the VAR(1) models predict unseen data for three recent psychological applications. We show that the VAR(1) based models do not outperform the AR(1) based ones for the three presented psychological applications. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Predictor Selection for Positive Autoregressive Processes

Let observations y1, …, yn be generated from a first-order autoregressive (AR) model with positive errors. In both the stationary and unit root cases, we derive moment bounds and limiting distributions of an extreme value estimator, , of the AR coefficient. These results enable us to provide asymptotic expressions for the mean squared error (MSE) of and the mean squared prediction error (MSPE) of the corresponding predictor, , of yn + 1. Based on these expressions, we compare the relative performance of () and the least-squares predictor (estimator) from the MSPE (MSE) point of view. Our comparison reveals that the better predictor (estimator) is determined not only by whether a unit root exists, but also by the behavior of the underlying error distribution near the origin, and hence is difficult to identify in practice. To circumvent this difficulty, we suggest choosing the predictor (estimator) with the smaller accumulated prediction error and show that the predictor (estimator) chosen in this way is asymptotically equivalent to the better one. Both real and simulated datasets are used to illustrate the proposed method. Supplementary materials for this article are available online.

Information and Prediction Criteria in Selecting the Forecasting Model

The purpose of the paper it to compare the performance of both information and prediction criteria in selecting the forecasting model on empirical data for Poland when the data generating model is unknown. The attention will especially focus on the evolution of information criteria (AIC, BIC) and accumulated prediction error (APE) for increasing sample sizes and rolling windows of different size, and also the impact of initial sample and rolling window sizes on the selection of forecasting model. The best forecasting model will be chosen from the set including three models: autoregressive model, AR (with or without a deterministic trend), ARIMA model and random walk (RW) model.

Choosing a Model and Strategy of Model Selection by Accumulated Prediction Error

The purpose of the paper is to present and apply the accumulative one-step-ahead prediction error (APE) not only as a method (strategy) of model selection, but also as a tool of model selection strategy (meta-selection). The APE method is compared with the information approach to model selection (AIC and BIC information criteria), supported by empirical examples. Obtained results indicated that the APE method may be of considerable practical importance.

Estimating Prediction Error: Cross-Validation vs. Accumulated Prediction Error

We study the validation of prediction rules such as regression models and classification algorithms through two out-of-sample strategies, cross-validation and accumulated prediction error. We use the framework of Efron (1983) where measures of prediction errors are defined as sample averages of expected errors and show through exact finite sample calculations that cross-validation and accumulated prediction error yield different smoothing parameter choices in nonparametric regression. The difference in choice does not vanish as sample size increases.

Toward optimal multistep forecasts in non-stationary autoregressions

This paper investigates multistep prediction errors for non-stationary autoregressive processes with both model order and true parameters unknown. We give asymptotic expressions for the multistep mean squared prediction errors and accumulated prediction errors of two important methods, plug-in and direct prediction. These expressions not only characterize how the prediction errors are influenced by the model orders, prediction methods, values of parameters and unit roots, but also inspire us to construct some new predictor selection criteria that can ultimately choose the best combination of the model order and prediction method with probability 1. Finally, simulation analysis confirms the satisfactory finite sample performance of the newly proposed criteria.

Capacity Requirement for Battery Installed at a Wind Farm -Evaluation of Energy Capacity Required for Scheduled Output Operation-

Wind power provokes increasing concerns on its impacts on a power system as many wind power generators have been integrated to a power system. One of the concerns is a challenge to keep supply-and-demand balance in a power system.Battery energy storage systems (BESS) installed at a wind farm are considered as a promising option for the challenge. Two types of operation strategies were proposed for its operation: fluctuation mitigation and scheduled output types. On the other hand, in order to constitute an economical BESS, selecting battery size as small as possible is vital. However, for the scheduled output operation, no study has been published both on the required battery size and on a methodology to examine the size.This paper proposes a methodology to evaluate the required kWh capacity of battery for the scheduled output operation using accumulated prediction error of a wind farm. The methodology evaluates a standard deviation of accumulated prediction error with autocovariance functions of prediction error. Based upon the statistical characteristics obtained for actual prediction error of wind power, the paper presents results of a sample study to show the effectiveness of the methodology.

Accumulated prediction errors, information criteria and optimal forecasting for autoregressive time series

The predictive capability of a modification of Rissanen's accumulated prediction error (APE) criterion, APEδn, is investigated in infinite-order autoregressive (AR(oo)) models. Instead of accumulating squares of sequential prediction errors from the beginning, APEδn is obtained by summing these squared errors from stage nδ n , where n is the sample size and 1/n < δ n < 1-(1/n) may depend on n. Under certain regularity conditions, an asymptotic expression is derived for the mean-squared prediction error (MSPE) of an AR predictor with order determined by APEδn. This expression shows that the prediction performance of APEδn can vary dramatically depending on the choice of δ n . Another interesting finding is that when δ n approaches 1 at a certain rate, APEδn can achieve asymptotic efficiency in most practical situations. An asymptotic equivalence between APEδn and an information criterion with a suitable penalty term is also established from the MSPE point of view. This offers new perspectives for understanding the information and prediction-based model selection criteria. Finally, we provide the first asymptotic efficiency result for the case when the underlying AR(oo) model is allowed to degenerate to a finite autoregression.

Prequential and Cross-Validated Regression Estimation

Prequential model selection and delete-one cross-validation are data-driven methodologies for choosing between rival models on the basis of their predictive abilities. For a given set of observations, the predictive ability of a model is measured by the model‘s accumulated prediction error and by the model‘s average-out-of-sample prediction error, respectively, for prequential model selection and for cross-validation. In this paper, given i.i.d. observations, we propose nonparametric regression estimators—based on neural networks—that select the number of “hidden units” (or “neurons”) using either prequential model selection or delete-one cross-validation. As our main contributions: (i) we establish rates of convergence for the integrated mean-squared errors in estimating the regression function using “off-line” or “batch” versions of the proposed estimators and (ii) we establish rates of convergence for the time-averaged expected prediction errors in using “on-line” versions of the proposed estimators. We also present computer simulations (i) empirically validating the proposed estimators and (ii) empirically comparing the proposed estimators with certain novel prequential and cross-validated “mixture” regression estimators.

Model selection and prediction: Normal regression

This paper discusses the topic of model selection for finite-dimensional normal regression models. We compare model selection criteria according to prediction errors based upon prediction with refitting, and prediction without refitting. We provide a new lower bound for prediction without refitting, while a lower bound for prediction with refitting was given by Rissanen. Moreover, we specify a set of sufficient conditions for a model selection criterion to achieve these bounds. Then the achievability of the two bounds by the following selection rules are addressed: Rissanen's accumulated prediction error criterion (APE), his stochastic complexity criterion, AIC, BIC and the FPE criteria. In particular, we provide upper bounds on overfitting and underfitting probabilities needed for the achievability. Finally, we offer a brief discussion on the issue of finite-dimensional vs. infinite-dimensional model assumptions.

Variable Selection in Nonparametric Regression with Categorical Covariates

Abstract This article extends the problem of variable selection to a nonparametric regression model with categorical covariates. Two selection criteria are considered: the cross-validation (CV) criterion and the accumulated prediction error (APE) criterion. We find that, asymptotically, the CV criterion performs well only when the true model is infinite-dimensional, while the APE criterion is appropriate when the true model is finite-dimensional. This is very similar to the case of linear regression model. A simulation study reveals some interesting small-sample properties of these criteria. To be more specific, suppose that we have observations (X 1, Y 1), …, (Xn, Yn ) that are iid random vectors and X = (X(1), X(2), …), where the X(i)'s are categorical. We allow Y to be of any type. Now a new observation X has arrived and we want to predict the corresponding Y. Such a framework is more appropriate than regressions with fixed covariates in situations where the covariates are observational rather than bei...

Variable Selection in Nonparametric Regression with Categorical Covariates

Abstract This article extends the problem of variable selection to a nonparametric regression model with categorical covariates. Two selection criteria are considered: the cross-validation (CV) criterion and the accumulated prediction error (APE) criterion. We find that, asymptotically, the CV criterion performs well only when the true model is infinite-dimensional, while the APE criterion is appropriate when the true model is finite-dimensional. This is very similar to the case of linear regression model. A simulation study reveals some interesting small-sample properties of these criteria. To be more specific, suppose that we have observations (X 1, Y 1), …, (Xn, Yn ) that are iid random vectors and X = (X(1), X(2), …), where the X(i)'s are categorical. We allow Y to be of any type. Now a new observation X has arrived and we want to predict the corresponding Y. Such a framework is more appropriate than regressions with fixed covariates in situations where the covariates are observational rather than being controlled. For instance, Y could be the time from HIV infection to developing clinical AIDS, and the covariates (mostly categorical or reducible to categorical) could be observations from blood tests, a physical examination, or further personal information, such as sexual practices obtained from an interview. Take another example: Y could be the premium of an insurance policy with the covariates being the customer's general demographical information. Our goal is to select a subset of covariates that best predict Y. We define the true model dimension as d 0 if the regression function E(Y|X(1), X(2), …) is a d 0-variate function. The main conclusions of the article are: (1) The popular CV criterion performs well only when d 0 = ∞. (2) There exist other criteria that are more appropriate than CV when d 0 < ∞. (3) There is no difference between conditional and unconditional prediction errors, as far as asymptotics are concerned. (4) The selection range has to depend on the sample size. In fact, we argue that, for a given sample size n, we should only select models with the number of covariates not exceeding the order of magnitude of o(log n). (5) Simulation study indicates that the CV criterion has nice small-sample properties.

Recursive identification of lung parameters

Determination of lung capacity (FRC) using insoluble gas washout or equilibrium methods is a common procedure in respiratory tests. The lung model can be extended to include multiple compartments with differing volumes and ventilation fractions. A discrete-time mathematical model of a multi-compartment lung was developed based on mass conservation laws of the gas exchange. To estimate the unknown parameters in the model from experimental data, a recursive prediction error algorithm was implemented. The design of the algorithm provides the capability to track time-varying parameters, such that the system can be monitored on-line. Determination of the model order may be as important as estimation of the parameters, especially in biological systems where little may be known about the structure. Two new criteria to distinguish between models of varying complexity, the ‘minimum description length principle’ and the ‘accumulated prediction error’, are illustrated. Theoretical studies show that these criteria yield consistent order estimates of the process. Simulations and experimental data confirm the feasibility of this approach for the estimation of lung parameters.

Order estimation by accumulated prediction errors

This paper presents a new criterion based on prediction error which allows the estimation of the number of parameters as well as structures in statistical models. The criterion is valid for short and long samples alike. Unlike Akaike's earlier criterion, also based on prediction error, the criterion proposed here appears to produce consistent error estimates in ARMA processes.