Forecasting system's accuracy: A framework for the comparison of different structures

Abstract

AbstractOne of the most challenging aspects for managers when building a forecasting system is choosing how to aggregate the data at different levels. This is frequently done without the manager knowing how these choices can compromise the system's accuracy. This article illustrates these compromises by comparing different structures and aggregation criteria. Our article proposes and empirically tests a framework on how to build a coherent and more accurate forecasting system. The framework's first phase compares different time series forecasting methods, including statistical, “standard” machine learning, and deep learning. Results show that one of the statistical methods (autoregressive integrated moving average, or, for short, ARIMA) outperforms machine and deep learning methods. The second phase compares different combinations of aggregation criteria, structures of the forecasting system, and coherent forecast methods (i.e., adjustments to the forecasts at different levels of aggregation). The results show that using different criteria and structures indeed impacts predictions' accuracy. When it is necessary to disaggregate the forecast, our results show that it is best to add more information in a grouped structure, adjusted by a bottom‐up method. This combination provides the best performance, that is, the lowest mean absolute‐scaled error (MASE) in most nodes, compared to the other structures and coherent forecast methods used. The results also suggest that aggregating the time series further by geographical regions is essential to improve accuracy when forecasting products' and channels' sales.