Distributional Forecasts Research Articles

Bayesian Ensembles of Exponentially Smoothed Life-Cycle Forecasts

Problem definition: We study the problem of forecasting an entire demand distribution for a new product before and after its launch. Firms need accurate distributional forecasts of demand to make operational decisions about capacity, inventory, and marketing expenditures. We introduce a unified, robust, and interpretable approach to producing these pre- and postlaunch distributional forecasts. Methodology/results: Our approach is inspired by Bayesian model averaging. Each candidate model in our ensemble is a life-cycle model fitted to the completed life cycle of a comparable product. A prelaunch forecast is an ensemble with equal weights on the candidate models’ forecasts, whereas a postlaunch forecast is an ensemble with weights that evolve according to Bayesian updating. Our approach is part frequentist and part Bayesian, resulting in a novel approach tailored to the demand forecasting challenge. We also introduce a new type of life-cycle or product diffusion model with states that can be updated using exponential smoothing. The trend in this model follows the density of an exponentially tilted Gompertz random variable. For postlaunch forecasting, this model is attractive because it can adapt itself to the most recent changes in a product’s life cycle. We provide closed-form distributional forecasts from our model. In two empirical studies, we show that when the ensemble’s candidate models are all in our new type of exponential smoothing model, this version of the ensemble outperforms several leading approaches in both point and quantile forecasting. Managerial implications: In a data-driven operations environment, our model can produce accurate forecasts frequently and at scale. When quantile forecasts are needed, our model has the potential to provide meaningful economic benefits. In addition, our model’s interpretability should be attractive to managers who already use exponential smoothing and ensemble methods for other forecasting purposes. Supplemental Material: The online appendix is available at https://doi.org/10.1287/msom.2022.0359 .

SpatialWavePredict: a tutorial-based primer and toolbox for forecasting growth trajectories using the ensemble spatial wave sub-epidemic modeling framework

BackgroundDynamical mathematical models defined by a system of differential equations are typically not easily accessible to non-experts. However, forecasts based on these types of models can help gain insights into the mechanisms driving the process and may outcompete simpler phenomenological growth models. Here we introduce a friendly toolbox, SpatialWavePredict, to characterize and forecast the spatial wave sub-epidemic model, which captures diverse wave dynamics by aggregating multiple asynchronous growth processes and has outperformed simpler phenomenological growth models in short-term forecasts of various infectious diseases outbreaks including SARS, Ebola, and the early waves of the COVID-19 pandemic in the US.ResultsThis tutorial-based primer introduces and illustrates a user-friendly MATLAB toolbox for fitting and forecasting time-series trajectories using an ensemble spatial wave sub-epidemic model based on ordinary differential equations. Scientists, policymakers, and students can use the toolbox to conduct real-time short-term forecasts. The five-parameter epidemic wave model in the toolbox aggregates linked overlapping sub-epidemics and captures a rich spectrum of epidemic wave dynamics, including oscillatory wave behavior and plateaus. An ensemble strategy aims to improve forecasting performance by combining the resulting top-ranked models. The toolbox provides a tutorial for forecasting time-series trajectories, including the full uncertainty distribution derived through parametric bootstrapping, which is needed to construct prediction intervals and evaluate their accuracy. Functions are available to assess forecasting performance, estimation methods, error structures in the data, and forecasting horizons. The toolbox also includes functions to quantify forecasting performance using metrics that evaluate point and distributional forecasts, including the weighted interval score.ConclusionsWe have developed the first comprehensive toolbox to characterize and forecast time-series data using an ensemble spatial wave sub-epidemic wave model. As an epidemic situation or contagion occurs, the tools presented in this tutorial can facilitate policymakers to guide the implementation of containment strategies and assess the impact of control interventions. We demonstrate the functionality of the toolbox with examples, including a tutorial video, and is illustrated using daily data on the COVID-19 pandemic in the USA.

GrowthPredict: A toolbox and tutorial-based primer for fitting and forecasting growth trajectories using phenomenological growth models

Simple dynamic modeling tools can help generate real-time short-term forecasts with quantified uncertainty of the trajectory of diverse growth processes unfolding in nature and society, including disease outbreaks. An easy-to-use and flexible toolbox for this purpose is lacking. This tutorial-based primer introduces and illustrates GrowthPredict, a user-friendly MATLAB toolbox for fitting and forecasting time-series trajectories using phenomenological dynamic growth models based on ordinary differential equations. This toolbox is accessible to a broad audience, including students training in mathematical biology, applied statistics, and infectious disease modeling, as well as researchers and policymakers who need to conduct short-term forecasts in real-time. The models included in the toolbox capture exponential and sub-exponential growth patterns that typically follow a rising pattern followed by a decline phase, a common feature of contagion processes. Models include the 1-parameter exponential growth model and the 2-parameter generalized-growth model, which have proven useful in characterizing and forecasting the ascending phase of epidemic outbreaks. It also includes the 2-parameter Gompertz model, the 3-parameter generalized logistic-growth model, and the 3-parameter Richards model, which have demonstrated competitive performance in forecasting single peak outbreaks. We provide detailed guidance on forecasting time-series trajectories and available software (https://github.com/gchowell/forecasting_growthmodels), including the full uncertainty distribution derived through parametric bootstrapping, which is needed to construct prediction intervals and evaluate their accuracy. Functions are available to assess forecasting performance across different models, estimation methods, error structures in the data, and forecasting horizons. The toolbox also includes functions to quantify forecasting performance using metrics that evaluate point and distributional forecasts, including the weighted interval score. This tutorial and toolbox can be broadly applied to characterizing and forecasting time-series data using simple phenomenological growth models. As a contagion process takes off, the tools presented in this tutorial can help create forecasts to guide policy regarding implementing control strategies and assess the impact of interventions. The toolbox functionality is demonstrated through various examples, including a tutorial video, and the examples use publicly available data on the monkeypox (mpox) epidemic in the USA.

Interpretable, non-mechanistic forecasting using empirical dynamic modeling and interactive visualization.

Forecasting methods are notoriously difficult to interpret, particularly when the relationship between the data and the resulting forecasts is not obvious. Interpretability is an important property of a forecasting method because it allows the user to complement the forecasts with their own knowledge, a process which leads to more applicable results. In general, mechanistic methods are more interpretable than non-mechanistic methods, but they require explicit knowledge of the underlying dynamics. In this paper, we introduce EpiForecast, a tool which performs interpretable, non-mechanistic forecasts using interactive visualization and a simple, data-focused forecasting technique based on empirical dynamic modelling. EpiForecast's primary feature is a four-plot interactive dashboard which displays a variety of information to help the user understand how the forecasts are generated. In addition to point forecasts, the tool produces distributional forecasts using a kernel density estimation method-these are visualized using color gradients to produce a quick, intuitive visual summary of the estimated future. To ensure the work is FAIR and privacy is ensured, we have released the tool as an entirely in-browser web-application.

Probabilistic time series forecasts with autoregressive transformation models

Probabilistic forecasting of time series is an important matter in many applications and research fields. In order to draw conclusions from a probabilistic forecast, we must ensure that the model class used to approximate the true forecasting distribution is expressive enough. Yet, characteristics of the model itself, such as its uncertainty or its feature-outcome relationship are not of lesser importance. This paper proposes Autoregressive Transformation Models (ATMs), a model class inspired by various research directions to unite expressive distributional forecasts using a semi-parametric distribution assumption with an interpretable model specification. We demonstrate the properties of ATMs both theoretically and through empirical evaluation on several simulated and real-world forecasting datasets.

You Have a Point – But a Point Is Not Enough: The Case for Distributional Forecasts of Earnings

You Have a Point – But a Point Is Not Enough: The Case for Distributional Forecasts of Earnings

Recalibrating probabilistic forecasts of epidemics.

Distributional forecasts are important for a wide variety of applications, including forecasting epidemics. Often, forecasts are miscalibrated, or unreliable in assigning uncertainty to future events. We present a recalibration method that can be applied to a black-box forecaster given retrospective forecasts and observations, as well as an extension to make this method more effective in recalibrating epidemic forecasts. This method is guaranteed to improve calibration and log score performance when trained and measured in-sample. We also prove that the increase in expected log score of a recalibrated forecaster is equal to the entropy of the PIT distribution. We apply this recalibration method to the 27 influenza forecasters in the FluSight Network and show that recalibration reliably improves forecast accuracy and calibration. This method, available on Github, is effective, robust, and easy to use as a post-processing tool to improve epidemic forecasts.

Reference class selection in similarity‐based forecasting of corporate sales growth

AbstractThis paper proposes a general method to handle forecasts exposed to behavioral bias by finding appropriate outside views, in our case corporate sales forecasts of analysts. The idea is to find reference classes, that is, peer groups, for each analyzed company separately that share similarities to the firm of interest with respect to a specific predictor. The classes are regarded to be optimal if the forecasted sales distributions match the actual distributions as closely as possible. The forecast quality is measured by applying goodness‐of‐fit tests on the estimated probability integral transformations and by comparing the predicted quantiles. The method is out‐of‐sample backtested on a data set consisting of 21,808 US firms over the time period 1950–2019, which is also descriptively analyzed. It appears that, in particular, the past operating margins are good predictors for the distribution of future sales. A case study compares the outside view of our distributional forecasts with actual analysts' forecasts and emphasizes the relevance of our approach in practice.

Real-Time Delivery Time Forecasting and Promising in Online Retailing: When Will Your Package Arrive?

Problem definition: Providing fast and reliable delivery services is key to running a successful online retail business. To achieve a better delivery time guarantee policy, we study how to estimate and promise delivery time for new customer orders in real time. Academic/practical relevance: Delivery time promising is critical to managing customer expectations and improving customer satisfaction. Simply overpromising or underpromising is undesirable because of the negative impacts on short-/long-term sales. To the best of our knowledge, we are the first to develop a data-driven framework to predict the distribution of order delivery time and set promised delivery time to customers in a cost-effective way. Methodology: We apply and extend tree-based models to generate distributional forecasts by exploiting the complicated relationship between delivery time and relevant operational predictors. To account for the cost-sensitive decision-making problem structure, we develop a new split rule for quantile regression forests that incorporates an asymmetric loss function in split point selection. We further propose a cost-sensitive decision rule to decide the promised delivery day from the predicted distribution. Results: Our decision rule is proven to be optimal given certain cost structures. Tested on a real-world data set shared from JD.com, our proposed machine learning–based models deliver superior forecasting performance. In addition, we demonstrate that our framework has the potential to provide better promised delivery time in terms of sales, cost, and accuracy as compared with the conventional promised time set by JD.com. Specifically, our simulation results indicate that the proposed delivery time promise policy can improve the sales volume by 6.1% over the current policy. Managerial implications: Through a more accurate estimation of the delivery time distribution, online retailers can strategically set the promised time to maximize customer satisfaction and boost sales. Our data-driven framework reveals the importance of modeling fulfillment operations in delivery time forecasting and integrating the decision-making problem structure with the forecasting model.

Exploring the Potential of Forecasting Fish Distributions in the North East Atlantic With a Dynamic Earth System Model, Exemplified by the Suitable Spawning Habitat of Blue Whiting

Local oceanographic variability strongly influences the spawning distribution of blue whiting (Micromesistius poutassou). Here, we explore the potential of using a dynamic Earth System Model (ESM) to forecast the suitable spawning habitat of blue whiting to assist management. Retrospective forecasts of temperature and salinity with the Max Planck Institute ESM (MPI-ESM) show significant skill within blue whiting’s spawning region and spawning depth (250–600 m) during the peak months of spawning. While persistence forecasts perform well at shorter lead times (≤2 years), retrospective forecasts with MPI-ESM are clearly more skilful than persistence in predicting salinity at longer lead times. Our results indicate that retrospective forecasts of the suitable spawning habitat of blue whiting based on predicted salinity outperform those based on calibrated species distribution models. In particular, we find high predictive skill for the suitable spawning habitat based on salinity predictions around one year ahead in the area of Rockall-Hatton Plateau. Our approach shows that retrospective forecasts with MPI-ESM show a better ability to differentiate between the presence and absence of suitable habitat over Rockall Plateau compared to persistence. Our study highlights that physical-biological forecasts based on ESMs could be crucial for developing distributional forecasts of marine organisms in the North East Atlantic.

Forecasting Airport Transfer Passenger Flow Using Real-Time Data and Machine Learning

Problem definition: Airports and airlines have been challenged to improve decision making by producing accurate forecasts in real time. We develop a two-phased predictive system that produces forecasts of transfer passenger flows at an airport. In the first phase, the system predicts the distribution of individual transfer passengers’ connection times. In the second phase, the system samples from the distribution of individual connection times and produces distributional forecasts for the number of passengers arriving at the immigration and security areas. Academic/practical relevance: To our knowledge, this work is the first to apply machine learning for predicting real-time distributional forecasts of journeys in an airport using passenger level data. Better forecasts of these journeys can help optimize passenger experience and improve airport resource deployment. Methodology: The predictive system developed is based on a regression tree combined with copula-based simulations. We generalize the tree method to predict distributions, moving beyond point forecasts. We also formulate a newsvendor-based resourcing problem to evaluate decisions made by applying the new predictive system. Results: We show that, when compared with benchmarks, our two-phased approach is more accurate in predicting both connection times and passenger flows. Our approach also has the potential to reduce resourcing costs at the immigration and transfer security areas. Managerial implications: Our predictive system can produce accurate forecasts frequently and in real time. With these forecasts, an airport’s operating team can make data-driven decisions, identify late passengers, and assist them to make their connections. The airport can also update its resourcing plans based on the prediction of passenger flows. Our predictive system can be generalized to other operations management domains, such as hospitals or theme parks, in which customer flows need to be accurately predicted.Funding: This work was funded by Eurocontrol (APOC Business Process Reengineering Big Data Study) [Grant 15-220643-AWP6.3.1].Supplemental Material: The online appendix is available at https://doi.org/10.1287/msom.2021.0975 .

Native range climate is insufficient to predict anuran invasive potential

Niche conservatism explains biological invasions worldwide. However, a plethora of ecological processes may lead invasive species to occupy environments that are different from those found within native ranges. Here, we assess the potential invadable areas of the world’s most pervasive invasive amphibians: the cane toad, Rhinella marina + R. horribilis, and the North American bullfrog, Lithobates catesbeianus. The uncontrolled spread of such voracious, large-bodied, and disease-tolerant anurans has been documented to impact native faunas worldwide. To disentangle their invasion-related niche dynamics, we compared the predictive ability and distributional forecasts of ecological niche models calibrated with information from native, invaded and pooled (native + invaded) ranges. We found that including occurrences from invaded ranges improved model accuracy for both studied species. Non-native occurrences also accounted for 54% and 61% increase in the total area of potential distribution of the cane toad and bullfrog, respectively. Besides, the latter species occupied locations with climatic conditions that are more extreme than those found within its native range. Our results indicate that the occupancy of environments different from those found in native ranges increases the overall potential distribution of the studied invasive anuran species. Therefore, climate information on native ranges alone is insufficient to explain and anticipate the distributional patterns of invasion of cane toads and bullfrogs, underestimating predictions of potential invadable distribution. Moreover, such an observed expansion of realized niches towards occupancy of climates not found within native ranges also has clear implications for invasion risk assessments based on climate modelling worldwide.

Real-Time Delivery Time Forecasting and Promising in Online Retailing: When Will Your Package Arrive?

Providing fast and reliable delivery services is key to running a successful online retail business. To achieve a better delivery time guarantee policy, we study how to estimate and promise delivery time for new customer orders in real time. Delivery time promising is critical to managing customer expectations and improving customer satisfaction. Simply over-promising or under-promising is undesirable due to their negative impacts on short-term/long-term sales. We are the first to develop a data-driven framework to predict the distribution of order delivery time and set promised delivery time to customers in a cost-effective way. We adapt regression tree and quantile regression forests to generate distributional forecasts by exploiting the complicated relationship between delivery time and relevant predictors, which include the queue-length predictors to model the distribution center operations. We further propose a cost-sensitive classification decision rule to decide the promised delivery day from the predicted distribution. Tested on a real-world data set shared from JD.com, our proposed machine learning based models deliver superior forecasting performance. In addition, we demonstrate that our framework has the potential to provide better promised delivery time in terms of both cost and accuracy, as compared to the conventional promised time set by JD.com. Through a more accurate estimation of the delivery time distribution, online retailers can strategically set the promised time to maximize customer satisfaction and boost sales. Our data-driven framework reveals the importance of modeling fulfillment operations in delivery time forecasting, which sheds light on further improvement directions.

When Is Growth at Risk?

This paper empirically evaluates the potentially nonlinear nexus between financial indicators and the distribution of future GDP growth, using a rich set of macroeconomic and financial variables covering thirteen advanced economies. We evaluate the out-of-sample forecast performance of financial variables for GDP growth, including a fully real-time exercise based on a flexible nonparametric model. We also use a parametric model to estimate the moments of the time-varying distribution of GDP and evaluate their in-sample estimation uncertainty. Our overall conclusion is pessimistic: moments other than the conditional mean are poorly estimated, and no predictors we consider provide robust and precise advance warnings of tail risks or indeed about any features of the GDP growth distribution other than the mean. In particular, financial variables contribute little to such distributional forecasts, beyond the information contained in real indicators.

Forecasting Airport Transfer Passenger Flow Using Real-Time Data and Machine Learning

Problem definition: Airports and airlines have been challenged to improve decision-making by producing accurate forecasts in real time. We develop a two-phased predictive system that produces forecasts of transfer passenger flows at an airport. In the first phase, the system predicts the distribution of individual transfer passengers’ connection times. In the second phase, the system samples from the distribution of individual connection times and produces distributional forecasts for the number of passengers arriving at the immigration and security areas. Academic/Practical relevance: Our work is the first to apply machine learning for predicting real-time distributional forecasts of journeys in an airport, using passenger level data. Better forecasts of these journeys can help optimize passenger experience and improve airport resource deployment. Methodology: The predictive system developed is based on a regression tree combined with copula-based simulations. We generalize the tree method to predict distributions, moving beyond point forecasts. We also formulate a newsvendor-based resourcing problem to evaluate decisions made by applying the new predictive system. Results: We show that when compared to benchmarks, our two-phased approach is more accurate in predicting both connection times and passenger flows. Our approach also has the potential to reduce resourcing costs at the immigration and transfer security areas. Managerial implications: Our predictive system can produce accurate forecasts frequently and in realtime. With these forecasts, an airport’s operating team can make data-driven decisions, identify late passengers and assist them to make their connections. The airport can also update its resourcing plans based on the prediction of passenger flows. Our predictive system can be generalized to other operations management domains, such as hospitals or theme parks, in which customer flows need to be accurately predicted.

Construction and Visualization of Confidence Sets for Frequentist Distributional Forecasts

ABSTRACTThe focus of this article is on the quantification of sampling variation in frequentist probabilistic forecasts. We propose a method of constructing confidence sets that respects the functional nature of the forecast distribution, and use animated graphics to visualize the impact of parameter uncertainty on the location, dispersion, and shape of the distribution. The confidence sets are derived via the inversion of a Wald test, and the ellipsoid that defines the boundary of the set computed numerically. A wide range of linear and nonlinear time series models—encompassing long memory, state space, and mixture specifications—is used to demonstrate the procedure, based on artificially generated data. An empirical example in which distributional forecasts of both financial returns and its stochastic volatility are produced is then used to illustrate the practical importance of accommodating sampling variation in the manner proposed.

Nonmechanistic forecasts of seasonal influenza with iterative one-week-ahead distributions.

Accurate and reliable forecasts of seasonal epidemics of infectious disease can assist in the design of countermeasures and increase public awareness and preparedness. This article describes two main contributions we made recently toward this goal: a novel approach to probabilistic modeling of surveillance time series based on “delta densities”, and an optimization scheme for combining output from multiple forecasting methods into an adaptively weighted ensemble. Delta densities describe the probability distribution of the change between one observation and the next, conditioned on available data; chaining together nonparametric estimates of these distributions yields a model for an entire trajectory. Corresponding distributional forecasts cover more observed events than alternatives that treat the whole season as a unit, and improve upon multiple evaluation metrics when extracting key targets of interest to public health officials. Adaptively weighted ensembles integrate the results of multiple forecasting methods, such as delta density, using weights that can change from situation to situation. We treat selection of optimal weightings across forecasting methods as a separate estimation task, and describe an estimation procedure based on optimizing cross-validation performance. We consider some details of the data generation process, including data revisions and holiday effects, both in the construction of these forecasting methods and when performing retrospective evaluation. The delta density method and an adaptively weighted ensemble of other forecasting methods each improve significantly on the next best ensemble component when applied separately, and achieve even better cross-validated performance when used in conjunction. We submitted real-time forecasts based on these contributions as part of CDC’s 2015/2016 FluSight Collaborative Comparison. Among the fourteen submissions that season, this system was ranked by CDC as the most accurate.

Statistical and economic evaluation of time series models for forecasting arrivals at call centers

Call centers' managers are interested in obtaining accurate point and distributional forecasts of call arrivals in order to achieve an optimal balance between service quality and operating costs. We present a strategy for selecting forecast models of call arrivals which is based on three pillars: (i) flexibility of the loss function; (ii) statistical evaluation of forecast accuracy; (iii) economic evaluation of forecast performance using money metrics. We implement fourteen time series models and seven forecast combination schemes on three series of daily call arrivals. Although we focus mainly on point forecasts, we also analyze density forecast evaluation. We show that second moments modeling is important both for point and density forecasting and that the simple Seasonal Random Walk model is always outperformed by more general specifications. Our results suggest that call center managers should invest in the use of forecast models which describe both first and second moments of call arrivals.

How well do experience curves predict technological progress? A method for making distributional forecasts

Experience curves are widely used to predict the cost benefits of increasing the deployment of a technology. But how good are such forecasts? Can one predict their accuracy a priori? In this paper we answer these questions by developing a method to make distributional forecasts for experience curves. We test our method using a dataset with proxies for cost and experience for 51 products and technologies and show that it works reasonably well. The framework that we develop helps clarify why the experience curve method often gives similar results to simply assuming that costs decrease exponentially. To illustrate our method we make a distributional forecast for prices of solar photovoltaic modules.

Forecasting the Distributions of Hourly Electricity Spot Prices - Accounting for Serial Correlation Patterns and Non-Normality of Price Distributions

We present a stochastic modelling approach to describe the dynamics of hourly electricity prices. The suggested methodology is a stepwise combination of several mathematical operations to adequately characterize the distribution of electricity spot prices. The basic idea is to analyze day-ahead prices as panel of 24 cross-sectional hours and to identify principal components of hourly prices to account for the cross correlation between hours. Moreover, non-normality of residuals is addressed by performing a normal quantile transformation and specifying appropriate stochastic processes for time series before fit. We highlight the importance of adequate distributional forecasts and present a framework to evaluate the distribution forecast accuracy. The application for German electricity prices 2015 reveal that: (i) An autoregressive specification of the stochastic component delivers the best distribution but not always the best point forecasting results. (ii) Only a complete evaluation of point, interval and density forecast, including formal statistical tests, can ensure a correct model choice.