Simple Time Series Models Research Articles

Assessing the accuracy of directional forecasts

ABSTRACT Directional predictions are used in a variety of settings, including economics, finance, meteorology and medicine. Since the objectives in each setting can be quite different, appropriate tests for assessing the predictions are necessary. After summarizing existing tests, we propose new tests to assess the accuracy for cases where getting extreme events wrong is particularly costly and where there is serial dependence in small samples. We show in simulations that the latter test has a more favourable power-size trade-off relative to existing tests. The new tests are then applied to the directional EUR/USD forecasts in the ifo-Institute’s World Economic Survey and to the point forecasts in the Philadelphia Fed’s Survey of Professional Forecasters. For the former, the new tests show that while there is profitability and predictive accuracy in some cases, a simple time-series model is not beaten. For the latter, extreme values are not the main driver of the predictive value up to two quarters ahead. This shows that the additional tests allow a more precise determination of predictive accuracy and economic value and where it is coming from.

Long-term impact of COVID-19 pandemic on antibiotic use in primary care: lessons to optimize antimicrobial use

ABSTRACT Background Lockdowns due to the COVID-19 pandemic was associated with changes in the pattern of use of antimicrobials, but persistence of changes after lockdowns has not been described. The objective was to describe the number of patients with dispensed antibiotic treatment and consumption of antibiotics in outpatients from primary care in Catalonia 18 months after the end of the emergency period. Research design and methods Data for the COVID-19 pandemic period was obtained from March 2020 to December 2021. Four high transmission rate (Rt > 100) periods were established. To compare data, a simple Bayesian structural time series model was used. Results The observed number of patients with dispensed antibiotics decreased respect to the estimated, especially during the four high transmission rate periods: April–May 2020 (lockdown period) (−42.57% and −42.68%); December 2020-February 2021 (−41.65%, −49.97% and −43.64%); October 2021 (−16.23%), and December 2021 (−20.16%). Overall antibiotic consumption was reduced by 23.37% (p = 0.002). These differences were mainly observed in those ≤ 15 years. Conclusions We describe the reduction in the number of patients with dispensed antibiotics and antibiotic consumption after the COVID-19 lockdown persisted in a period of recovery of healthcare accessibility. This information may help to improve antimicrobial use at the primary care level.

Dynamic Modelling for Cancer-Associated Venous Thromboembolism

Background: Cancer-associated venous thromboembolism (VTE) is an important source of morbidity and mortality. Current risk prediction models for thrombosis are handicapped by static applicability (i.e. one-time evaluation prior to the first cycle of chemotherapy). We hypothesized that event forecasting using time series modelling could account for important changes in clinical state, variations in treatment and aspects of cancer progression over time. Methods: The cohort consisted of adult patients with a solid tumor enrolled in the MSK-IMPACT study between 2014 and 2019. VTE was defined as any symptomatic or asymptomatic pulmonary embolus or lower extremity deep vein thrombosis. We derived time series mixed-effects logistic regression models in order to predict cancer-associated VTE events using baseline patient characteristics (cancer type, time from diagnosis and age) along with time-dependent covariates including total white blood cell count, hemoglobin, platelet count, albumin, creatinine and parenteral chemotherapy type. Time-periods of one month were used and models were fitted to predict the presence or absence of a VTE episode in the following period (time = t+1) based on predictor values in the current period (time = t). The laboratory test result ascribed to a given period could be either the last value in the interval or the mean of all interval-specific values (both approaches tested). Missing laboratory results were imputed using the previously known value when available. Differences in laboratory values between periods t-1 and t were included as predictors in the initial model along with indicators of missingness (see Figure for flow of information between time-period cells). We hypothesized that absence of laboratory testing for a time-period would be a marker of stable medical state and hence associated with a lower risk of VTE. A previously defined training set comprising 80% of the cohort was used and different modelling approaches were evaluated using 5-fold cross-validation. The final selected model was evaluated in a separate dedicated validation set including the remaining 20% of the cohort. Results: The original cohort included 35,391 patients, from which 26,075 individuals with no cancer-associated VTE episode before observation start and sufficient laboratory test information could be included in the analysis, comprising 276,579 months of observation and 1,463 VTE events. The initial logistic regression model including all available predictors was fitted on the training set, revealing associations between predictors and VTE risk largely consistent with data reported by other groups (see Table). After adjusting for multiple comparisons, significant predictors of cancer-associated VTE identified included albumin, cell count result missingness (decreased risk for missing value), treatment with a platinum analogue (increased risk), hemoglobin, white blood cell count, breast cancer (decreased risk) and high-grade glioma (increased risk). Based on model metrics and prior knowledge from published literature, the set of covariates selected for inclusion in the final model consisted of cell counts along with the corresponding missingness indicator, platelet count change over time, albumin and chemotherapy types. The c-index for the final model using 5-fold cross-validation in the training set was 0.77 (95% CI = 0.75-0.80), compared to 0.77 (95% CI = 0.74-0.79) in the validation set. The latter corresponded to a specificity of 0.83 (95% CI = 0.79-0.86) when fixing sensitivity at 50%. Conclusions: A simple mixed-effects logistic regression time series model demonstrated promising potential to estimate the risk of cancer-associated VTE on a dynamic basis using routine laboratory test results and chemotherapy prescription data widely available in most electronic health record systems. Validation of this dynamic VTE risk model has the potential to optimize outpatient thromboprophylaxis during periods of high thrombotic risk.

Comparing forecasting performance with panel data

We develop new methods for testing equal predictive accuracy for panels of forecasts, exploiting information in both the time-series and cross-sectional dimensions of the data. We examine general tests of equal forecasting performance averaged across all time periods and individual units, along with tests that focus on subsets of time or clusters of units. Properties of our tests are demonstrated through Monte Carlo simulations and in an empirical application that compares International Monetary Fund forecasts of country-level real gross domestic product growth and inflation to private-sector survey forecasts and forecasts from a simple time-series model.

KDRank: Knowledge-driven user-aware POI recommendation

Accurate user modeling is crucial for point-of-interest (POI) recommendation as it can significantly improve user satisfaction with recommended POIs and enrich user experience. However, existing methods typically rely on simple time-series models for user check-in sequences, which ignore similar information of global users and fail to capture the user-preference knowledge hidden in complex social networks. To address this issue, we propose a novel knowledge-driven and user-aware POI recommendation method called KDRank. First, we construct a knowledge graph containing users’ personal attributes for POI recommendation, which can reflect users’ historical check-in preferences. Second, we derive users’ knowledge representations using a cross-embedding method, which facilitates feature interaction by sharing information between segments of knowledge representations to achieve a more precise representation of low-dimensional embedding. Third, we propose a knowledge aggregation module to combine users’ knowledge and historical check-in features to achieve knowledge enhancement of check-in data. Furthermore, to enhance global user awareness of our model, we introduce an attention mechanism that focuses on the most similar and significant users in the global context. It allows KDRank to capture more personalized user preferences and increases the precision of POI recommendations. The effectiveness of the proposed method was evaluated on two real datasets, and the results indicated its ability to increase the POI recommendation accuracy. The code associated with this study is available at https://github.com/itshardtocode/KDRank.

On predictive inference for intractable models via approximate Bayesian computation

Approximate Bayesian computation (ABC) is commonly used for parameter estimation and model comparison for intractable simulator-based statistical models whose likelihood function cannot be evaluated. In this paper we instead investigate the feasibility of ABC as a generic approximate method for predictive inference, in particular, for computing the posterior predictive distribution of future observations or missing data of interest. We consider three complementary ABC approaches for this goal, each based on different assumptions regarding which predictive density of the intractable model can be sampled from. The case where only simulation from the joint density of the observed and future data given the model parameters can be used for inference is given particular attention and it is shown that the ideal summary statistic in this setting is minimal predictive sufficient instead of merely minimal sufficient (in the ordinary sense). An ABC prediction approach that takes advantage of a certain latent variable representation is also investigated. We additionally show how common ABC sampling algorithms can be used in the predictive settings considered. Our main results are first illustrated by using simple time-series models that facilitate analytical treatment, and later by using two common intractable dynamic models.

Predicting daily emergency department visits using machine learning could increase accuracy

ObjectiveAdministrators and clinicians alike have attempted to predict emergency department visits for many years. The ability to predict or “forecast” ED visit volume can allow for more efficient resource allocation, including up-staffing or down-staffing, changing OR schedules, and predicting the need for significant resources. The goal of this study is to examine combinations of variables via machine learning to increase prediction accuracy and determine the factors that are most predictive of overall ED visits. As compared to a simple univariate time series model, we hypothesize that machine learning models will predict St. Joseph Mercy Ann Arbor's patient visit load for the emergency department (ED) with higher accuracy than a simple univariate time series model. MethodsUnivariate time series models for daily ED visits, including ARIMA, Exponential Smoothing (ETS), and Facebook Inc.'s prophet algorithm were estimated as a baseline comparison. Machine learning models, including random forests and gradient boosted machines (GBM), were trained using data from 2017 to 2018. After final models were created, they were applied to the 2019 data to determine how well these models predicted actual ED patient volumes in data not utilized during the model fitting process. The accuracy of the machine learning and time series models were assessed based on out-of-sample predictive accuracy, compared using root mean squared error (RMSE). ResultsUsing root mean squared error (RMSE) to assess out-of-sample predictive accuracy of the models, the results showed that the random forest model was the most accurate at predicting daily ED visits in the 2019 test set, followed by the GBM model. These performed only slightly better than the simple exponential smoothing model predictions. The ARIMA model performed poorly in comparison. The day of the week (likely capturing differences between weekdays and weekends) was found to be the most important predictor of patient volumes. Weather-related features such as maximum temperature and SFC pressure appeared to capture some of the seasonality trends related to changes in patient volumes. ConclusionsMachine learning models perform better at predicting daily patient volumes as compared to simple univariate time series models, though not by a substantial amount. Further research can help confirm these limited initial results. Gathering more training data and additional feature engineering could also be beneficial to training the models and potentially improving predictive accuracy.

Smoothed Bernstein Online Aggregation for Short-Term Load Forecasting in IEEE DataPort Competition on Day-Ahead Electricity Demand Forecasting: Post-COVID Paradigm

We present a winning method of the IEEE DataPort Competition on Day-Ahead Electricity Demand Forecasting: Post-COVID Paradigm. The day-ahead load forecasting approach is based on a novel online forecast combination of multiple point prediction models. It contains four steps: i) data cleaning and preprocessing, ii) a new holiday adjustment procedure, iii) training of individual forecasting models, iv) forecast combination by smoothed Bernstein Online Aggregation (BOA). The approach is flexible and can quickly adjust to new energy system situations as they occurred during and after COVID-19 shutdowns. The ensemble of individual prediction models ranges from simple time series models to sophisticated models like generalized additive models (GAMs) and high-dimensional linear models estimated by lasso. They incorporate autoregressive, calendar, and weather effects efficiently. All steps contain novel concepts that contribute to the excellent forecasting performance of the proposed method. It is especially true for the holiday adjustment procedure and the fully adaptive smoothed BOA approach.

The Pearls and Perils of Google Trends: A Housing Market Application.

This study aims at providing insights into the correct usage of Google search data, which are available through Google Trends. The focus is on the effect of sampling errors, which has not received the attention that it deserves. A housing market application is used to demonstrate the effects. For this purpose, the relationship between online search activity for mortgages and real housing market activity is investigated. A simple time series model, which explains transactions by an online mortgage search, is estimated. The results show that the effect of sampling errors is substantial. Thus, although the application of Google Trends data in research remains promising, far more attention should be given to the limitations of these data.

Modeling Judgment in Macroeconomic Forecasts

Many macroeconomic forecasts are the outcome of a judgmental adjustment to a forecast from an econometric model. The size, direction, and motivation of the adjustment are often unknown as usually only the final forecast is available. This is problematic in case an analyst wishes to learn from forecast errors, which could lead to improving the model, the judgment or both. This paper therefore proposes a formal method to include judgment, which makes the combined forecast reproducible. As an illustration, a forecast from a benchmark simple time series model is only modified when the value of a factor, estimated from a multitude of variables, exceeds a user-specified threshold. Simulations and empirical results for forecasting annual real GDP growth in 52 African countries provide an illustration.

An analysis of electricity congestion price patterns in North America

The present paper illustrates the use of principal component analysis (PCA) on the congestion component of local (i.e. zonal) electricity price data to detect the most salient congestion patterns in electricity transmission grids managed by either a Regional Transmission Organization (RTO) or an Independent System Operator (ISO). Outputs from the PCA along with some data visualization tools are shown to make the identification of such patterns seamless and straightforward. An empirical analysis is conducted for three North American power systems, namely NYISO, ISO New England and PJM. Finally, a simple time series model representing the evolution of PCA scores is proposed.

Forecasting Shares Outstanding

Despite the importance of EPS forecasts as benchmarks, little is known about their denominator: shares outstanding forecasts. Because investing clients have limited demand for accurate share forecasts, we predict that analysts develop share forecasts strategically to facilitate EPS incentives. We divide earnings forecasts by EPS forecasts to infer analysts’ share forecasts and evaluate their properties. Analysts’ forecasts of shares outstanding are significantly less accurate than simple time-series models; however, these same forecasts actually improve EPS forecast accuracy. Additional analysis explains why: analysts use share forecasts to herd EPS toward the consensus. That is, share forecast errors often have the same sign as street earnings forecast errors, moving EPS closer to the consensus and to actual EPS, and significantly reducing EPS dispersion. Analysts also appear to use share forecasts to cater to management. Specifically, bias in share forecasts facilitates firms’ ability to meet or beat (“MB”) EPS benchmarks and is consistent with manager preferences (i.e., deflating EPS forecasts at short horizons and inflating at longer horizons). Much of the MB effect arises because analysts fail to incorporate predictable variation in shares outstanding, such as past repurchases. Interviews with sell-side analysts confirm that clients have limited demand for share forecasts, consistent with the inattention and strategic use of forecasts documented in our study.

Cointegration and control: Assessing the impact of events using time series data

SummaryControl groups can provide counterfactual evidence for assessing the impact of an event or policy change on a target variable. We argue that fitting a multivariate time series model offers potential gains over a direct comparison between the target and a weighted average of controls. More importantly, it highlights the assumptions underlying methods such as difference in differences and synthetic control, suggesting ways to test these assumptions. Gains from simple and transparent time series models are analysed using examples from the literature, including the California smoking law of 1989 and German reunification. We argue that selecting controls using a time series strategy is preferable to existing data‐driven regression methods.

Macroeconomic determinants of unemployment in Nepal

The relationship between unemployment and macroeconomic variables is complex, but it is generally accepted that increasing GDP, trade, and expansionary monetary policy lead to a decrease in the rate of unemployment. This paper tests this hypothesis for Nepal using time series data from 1991 to 2021 and a simple time series model. It is found that the increase in GDP, trade openness, money supply, and the rate of inflation decrease the unemployment rate in Nepal. These results have huge policy implications.

COVID-19: Forecasting confirmed cases and deaths with a simple time series model

Forecasting the outcome of outbreaks as early and as accurately as possible is crucial for decision-making and policy implementations. A significant challenge faced by forecasters is that not all outbreaks and epidemics turn into pandemics, making the prediction of their severity difficult. At the same time, the decisions made to enforce lockdowns and other mitigating interventions versus their socioeconomic consequences are not only hard to make, but also highly uncertain. The majority of modeling approaches to outbreaks, epidemics, and pandemics take an epidemiological approach that considers biological and disease processes. In this paper, we accept the limitations of forecasting to predict the long-term trajectory of an outbreak, and instead, we propose a statistical, time series approach to modelling and predicting the short-term behavior of COVID-19. Our model assumes a multiplicative trend, aiming to capture the continuation of the two variables we predict (global confirmed cases and deaths) as well as their uncertainty. We present the timeline of producing and evaluating 10-day-ahead forecasts over a period of four months. Our simple model offers competitive forecast accuracy and estimates of uncertainty that are useful and practically relevant.

Editorial

Editorial

A Causal Time-Series Model Based on Multilayer Perceptron Regression for Forecasting Taiwan Stock Index

Stock forecasting technology is always a popular research topic because accurate forecasts allow profitable investments and social change. We postulate, based on past research, three major drawbacks for using time series in forecasting stock prices as follows: (1) a simple time-series model provides insufficient explanations for inner and external interactions of the stock market; (2) the variables of a time series behave in strict stationarity, but economic time-series are usually in a nonlinear or nonstationary state and (3) the forecasting factors of multivariable time-series are selected based on researcher’s knowledge, and such a method is a “subjective” way to construct a forecasting model. Therefore, this paper proposes a causal time-series model to select forecasting factors and builds a machine learning forecast model. The “Granger causality test” is utilized first in the proposed model to select the critical factors from technical indicators and market indexes; next, a “multilayer perceptron regression (MLPR)” is employed to construct a forecasting model. This paper collected financial data over a 13-year period (from 2003 to 2015) of the Taiwan stock index (TAIEX) as experimental datasets. Furthermore, the root mean square error (RMSE) was used as a performance indicator, and we use five forecasting models as comparison models. The results reveal that the proposed model outperforms the comparison models in forecasting accuracy and performs well for three key indicators. LAG1, S&P500 and DJIA, are critical factors in all 11 of our time sliding windows (T1–T11). We offer these results to investors to aid in their decision-making processes.

Reproducing Popularity Distribution of YouTube Videos

To provide video streaming of user-generated contents (UGCs) with high quality and at low cost by maximizing the effect of content delivery network (CDN), CDN providers are required to adequately design CDN cache servers by accurately estimating the UGC view-count distribution. To achieve this goal in a practical time frame, we need to construct a simple time-series model that captures the transition of UGC popularity. Therefore, in this paper, we first analyze the daily view count (DVC) of YouTube videos over nine months and find that the DVC of YouTube videos obeys a lognormal distribution. As a simple time-series model of the DVC of each YouTube video, we propose the grouped MPP (gMPP), extending the multiplicative process (MPP) which is widely known as a simple time-series model generating a lognormal distribution. We also propose reproducing the DVC distribution of YouTube videos by using a superposed gMPP (SgMPP) that aggregates multiple gMPPs. The SgMPP can accurately reproduce the DVC distribution of YouTube videos with a low computational overhead, so we can expect to use the SgMPP as the input for computer simulations for designing various network components that require the popularity distribution of UGC, e.g., cache capacities. Through numerical evaluation, we confirm that we can adequately design the storage capacity of a cache server with the average error rate of several percent against the target cache hit ratio.

Comparing Forecasting Performance with Panel Data

Abstract This paper develops new methods for testing equal predictive accuracy in panels of forecasts that exploit information in the time series and cross-sectional dimensions of the data. Using a common factor setup, we establish conditions on cross-sectional dependencies in forecast errors which allow us to conduct inference and compare performance on a single cross-section of forecasts. We consider both unconditional tests of equal predictive accuracy as well as tests that condition on the realization of common factors and show how to decompose forecast errors into exposures to common factors and an idiosyncratic variance component. Our tests are demonstrated in an empirical application that compares IMF forecasts of country-level real GDP growth and inflation to private-sector survey forecasts and forecasts from a simple time-series model

Predictive Performance of Alternative Inflation Forecasting Models: New International Evidence

Inflation rate and its volatility have been at a subdued level for most industrialized and emerging countries since the mid-1990s. The objective of this study is to evaluate the predictive performance of three alternative inflation forecasting models -- univariate time-series (ARIMA) model, Phillips curve model, and naïve model -- for a selected number of inflation-targeting countries and non-inflation targeting countries over the period 1998-2015, a unique period marked by relatively low and stable inflation rate. It is found that out-of-sample inflation forecasts generated by ARIMA model are more accurate than those generated by the other two forecasting models for the majority of these countries. This study concludes, that during the period of low inflation rate, the central bank should weigh inflation forecasts obtained from a simple time-series model, such as ARIMA model, more heavily in its decisionmaking process.