Forecasting model selection using intermediate classification: Application to MonarchFx corporation

Abstract

Organizations rely on accurate demand forecasts to make production and ordering decisions in a variety of supply chain positions. Significant research in time series forecasting techniques and a variety of forecasting methods are available in the market. However, selecting the most accurate forecasting model for a given time series has become a complicated decision. Prior studies of forecasting methods have used either in-sample or out-of-sample performance as the basis for model selection procedures, but typically fail to incorporate both in their decision-making framework. In this research, we develop an expert system for time series forecasting model selection, using both relative in-sample performance and out-of-sample performance simultaneously to train classifiers. These classifiers are employed to automatically select the best performing forecasting model without the need for decision-maker intervention. The new model selection scheme bridges the gap between using in-sample and out-of-sample performance separately. The best performing model on the validation set is not necessarily selected by the expert system, since both in-sample and out-of-sample information are essential in the selection process. The performance of the proposed expert system is tested using the monthly dataset from the M3-Competition, and the results demonstrate an overall minimum of 20% improvement in the optimality gap comparing to the train/validation method. The new forecasting expert system is also applied to a real case study dataset obtained from MonarchFx (a distributed logistics solutions provider). This result demonstrates a robust predictive capability with lower mean squared errors, which allows organizations to achieve a higher level of accuracy in demand forecasts.