Augmented support vector regression with an autoregressive process via an iterative procedure

Abstract

The Support Vector Regression (SVR) technique can approximate intricate systems by addressing learning and estimation challenges within a reproducing kernel Hilbert space, devoid of reliance on specific parameter assumptions. However, when dealing with correlated data like time series, the SVR method often falls short in accounting for underlying temporal structures, leading to limited enhancements in prediction efficiency. We introduce an enhanced SVR method that considers temporal correlations (TemporalSVR) to overcome this constraint. Our proposed method extends kernel functions to include additional linear kernels, facilitating learning temporal patterns. Additionally, we develope an iterative training procedure for the augmented regression model. During model training, we estimate the hyper-parameter in the corresponding loss function using a ‘working’ likelihood approach, enhancing the generalization capabilities of the proposed regression. To demonstrate superior forecasting performance, we conduct extensive numerical simulations on both linear and nonlinear systems and the TemporalSVR achieves improvements ranging from 8% to 114% based on the RMSE ratio from the AR-X model. Furthermore, we investigate the forecasting performance of three basic models (NARX-NN, Statistical SVR, and SVR-ARIMA) and four deep learning (DL) techniques (Transformer, Informer, Reformer, Autormer, and Autoformer) by using a WTI forecasting study. Our proposed TemporalSVR achieves the smallest RMSE at 2.22 and attains the highest success ratio of stock direction prediction at 71.30%. All these numerical results highlight the effectiveness and advantages of our TemporalSVR in handling temporal data and making accurate predictions.