Using support vector machines for time series prediction

Abstract

Time series prediction can be a very useful tool in the field of process chemometrics to forecast and to study the behaviour of key process parameters in time. This creates the possibility to give early warnings of possible process malfunctioning. In this paper, time series prediction is performed by support vector machines (SVMs), Elman recurrent neural networks, and autoregressive moving average (ARMA) models. A comparison of these three methods is made based on their predicting ability. In the field of chemometrics, SVMs are hardly used even though they have many theoretical advantages for both classification and regression tasks. These advantages stem from the specific formulation of a (convex) objective function with constraints which is solved using Lagrange Multipliers and has the characteristics that: (1) a global optimal solution exists which will be found, (2) the result is a general solution avoiding overtraining, (3) the solution is sparse and only a limited set of training points contribute to this solution, and (4) nonlinear solutions can be calculated efficiently due to the usage of inner products. The method comparison is performed on two simulated data sets and one real-world industrial data set. The simulated data sets are a data set generated according to the ARMA principles and the Mackey–Glass data set, often used for benchmarking. The first data set is relatively easy whereas the second data set is a more difficult nonlinear chaotic data set. The real-world data set stems from a filtration unit in a yarn spinning process and it contains differential pressure values. These values are a measure of the contamination level of a filter. For practical purposes, it is very important to predict these values accurately because they play a crucial role in maintaining the quality of the process considered. As it is expected, it appears that the ARMA model performs best for the ARMA data set while the SVM and the Elman networks perform similarly. For the more difficult benchmark data set the SVM outperforms the ARMA model and in most of the cases outperforms the best of several Elman neural networks. For the real-world set, the SVM was trained using a training set containing only one tenth of the points of the original training set which was used for the other methods. This was done to test its performance if only few data would be available. Using the same test set for all methods, it appeared that prediction results were equally well for both the SVM and the ARMA model whereas the Elman network could not be used to predict these series.