A novel real-time multi-step forecasting system with a three-stage data preprocessing strategy for containerized freight market

Abstract

The management and decision-making of the shipping market rely heavily on the accurate forecasting of the China Containerized Freight Index (CCFI); however, this is still a challenging task. Furthermore, CCFI is influenced by many factors and exhibits complex nonlinear characteristics, which may cause the evolution of its distributions over time, i.e., the concept drift issue. However, previous studies mainly focused on the simple application of single data preprocessing and machine learning, while ignoring the significance of real-time and multi-step forecasting, which may not effectively deal with the concept drift issue and result in unsatisfactory results. Additionally, some studies disregarded the data leakage problem caused by the one-time decomposition of all datasets. Therefore, a novel real-time multi-step forecasting system with multiple rolling decomposition strategy based on streaming data is designed. It includes data preprocessing, forecasting, and evaluation modules. A three-stage data preprocessing strategy is proposed in the first module to overcome the single preprocessing defect. It can eliminate the adverse effects of outliers, extract main features, and reduce system complexity. Moreover, the forecasting and evaluation modules are designed to enhance the forecasting performance and verify the system’s validity. Considering the obvious concept drift problem of CCFI during the Coronavirus Disease 2019 (COVID-19) epidemic, datasets from the COVID-19 epidemic are selected for experimental verification to verify the system’s ability to solve this problem. Empirical studies demonstrate the superiority of the developed system over the compared models in terms of universality and robustness. Therefore, the proposed system is an effective prediction method for the containerized freight market in the post epidemic period and an alternative for forecasting other streaming data with the concept drift issue.