A quantile fusion methodology for deep forecasting

Abstract

Deep learning for forecasting has elicited excitement. Previous deep forecasting methods were usually designed for specific datasets and might hardly generalize well on other datasets. This paper discloses that besides designing a specific deep architecture, if we deploy the loss function more intelligently, we can construct an accurate and robust deep forecasting model. We frame this research from four aspects. First, we propose to transform a hidden layer as quantile layer to generate quantiles as robust distribution representation. Second, we introduce an effective crossing loss which can reduce the frequency of quantile crossing. Third, we investigate the performance of two learning modes, i.e., end-to-end v.s. two-stage, and conclude end-to-end is more advisable for the proposed methodology. Fourth, we incorporate the quantile layer into AutoEncoder to build quantile fusion-based AutoEncoder that achieves lower reconstruction error. We performed rigorous studies and evaluations on eight open UCI datasets using RMSE and MAE metrics. The experimental results substantiate its significance of accuracy and robustness. Our methodology is generic and can be applied to general forecasting problems such as time-series modeling and spatio-temporal prediction.