Recurrent Support Vector Regression Research Articles

Drowsiness Estimation Using Electroencephalogram and Recurrent Support Vector Regression

As a cause of accidents, drowsiness can cause economical and physical damage. A range of drowsiness estimation methods have been proposed in previous studies to aid accident prevention and address this problem. However, none of these methods are able to improve their estimation ability as the length of time or number of trials increases. Thus, in this study, we aim to find an effective drowsiness estimation method that is also able to improve its prediction ability as the subject’s activity increases. We used electroencephalogram (EEG) data to estimate drowsiness, and the Karolinska sleepiness scale (KSS) for drowsiness evaluation. Five parameters (α, β/α, (θ+α)/β, activity, and mobility) from the O1 electrode site were selected. By combining these parameters and KSS, we demonstrate that a typical support vector regression (SVR) algorithm can estimate drowsiness with a correlation coefficient (R2) of up to 0.64 and a root mean square error (RMSE) of up to 0.56. We propose a “recurrent SVR” (RSVR) method with improved estimation performance, as highlighted by an R2 value of up to 0.83, and an RMSE of up to 0.15. These results suggest that in addition to being able to estimate drowsiness based on EEG data, RSVR is able to improve its drowsiness estimation performance.

Recurrent Kernel Extreme Reservoir Machine for Time Series Prediction

This paper proposes a novel recurrent multi-step-ahead prediction model called recurrent kernel extreme reservoir machine (RKERM) with quantum particle swarm optimization (QPSO). This model combines the strengths of recurrent kernel extreme learning machine (RKELM) and modified reservoir computing to overcome the limitations of prediction horizon with increased prediction accuracy based on reservoir computing theory. Furthermore, QPSO is used to optimize the parameters of kernel method and leaking rate of reservoir computing in the RKERM. In the experiment, we apply two synthetic benchmark data sets and five real-world time series data sets, including Malaysia palm oil price, ozone concentration in Toronto, sunspots, Standard & Poor’s 500, and water level at Phra Chulachomklao Fort in Thailand to evaluate the echo state network, recurrent support vector regression, recurrent extreme learning machine, RKELM, and RKERM. The experimental results show that the RKERM with QPSO has superior abilities in the different predicting horizons than others.

Support Vector Regression Based GARCH Model with Application to Forecasting Volatility of Financial Returns

In recent years, support vector regression (SVR), a novel neural network (NN) technique, has been successfully used for financial forecasting. This paper deals with the application of SVR in volatility forecasting. Based on a recurrent SVR, a GARCH method is proposed and is compared with a moving average (MA), a recurrent NN and a parametric GACH in terms of their ability to forecast financial markets volatility. The real data in this study uses British Pound-US Dollar (GBP) daily exchange rates from July 2, 2003 to June 30, 2005 and New York Stock Exchange (NYSE) daily composite index from July 3, 2003 to June 30, 2005. The experiment shows that, under both varying and fixed forecasting schemes, the SVR-based GARCH outperforms the MA, the recurrent NN and the parametric GARCH based on the criteria of mean absolute error (MAE) and directional accuracy (DA). No structured way being available to choose the free parameters of SVR, the sensitivity of performance is also examined to the free parameters.

Recurrent support vector regression for a non-linear ARMA model with applications to forecasting financial returns

Motivated by recurrent neural networks, this paper proposes a recurrent support vector regression (SVR) procedure to forecast nonlinear ARMA model based simulated data and real data of financial returns. The forecasting ability of the recurrent SVR based ARMA model is compared with five competing models (random walk, threshold ARMA model, MLE based ARMA model, recurrent artificial neural network based ARMA model and feed-forward SVR based ARMA model) by using two forecasting accuracy evaluation metrics (NSME and sign) and robust Diebold---Mariano test. The results reveal that for one-step-ahead forecasting, the recurrent SVR model is consistently better than the benchmark models in forecasting both the magnitude and turning points, and statistically improves the forecasting performance as opposed to the usual feed-forward SVR.

Recurrent sparse support vector regression machines trained by active learning in the time-domain

A method for the sparse solution of recurrent support vector regression machines is presented. The proposed method achieves a high accuracy versus complexity and allows the user to adjust the complexity of the resulting model. The sparse representation is guaranteed by limiting the number of training data points for the support vector regression method. Each training data point is selected based on the accuracy of the fully recurrent model using the active learning principle applied to the successive time-domain data. The user can adjust the training time by selecting how often the hyper-parameters of the algorithm should be optimised. The advantages of the proposed method are illustrated on several examples, and the experiments clearly show that it is possible to reduce the number of support vectors and to significantly improve the accuracy versus complexity of recurrent support vector regression machines.

Electric load forecasting by seasonal recurrent SVR (support vector regression) with chaotic artificial bee colony algorithm

Support vector regression (SVR), with hybrid chaotic sequence and evolutionary algorithms to determine suitable values of its three parameters, not only can effectively avoid converging prematurely (i.e., trapping into a local optimum), but also reveals its superior forecasting performance. Electric load sometimes demonstrates a seasonal (cyclic) tendency due to economic activities or climate cyclic nature. The applications of SVR models to deal with seasonal (cyclic) electric load forecasting have not been widely explored. In addition, the concept of recurrent neural networks (RNNs), focused on using past information to capture detailed information, is helpful to be combined into an SVR model. This investigation presents an electric load forecasting model which combines the seasonal recurrent support vector regression model with chaotic artificial bee colony algorithm (namely SRSVRCABC) to improve the forecasting performance. The proposed SRSVRCABC employs the chaotic behavior of honey bees which is with better performance in function optimization to overcome premature local optimum. A numerical example from an existed reference is used to elucidate the forecasting performance of the proposed SRSVRCABC model. The forecasting results indicate that the proposed model yields more accurate forecasting results than ARIMA and TF-ε-SVR-SA models. Therefore, the SRSVRCABC model is a promising alternative for electric load forecasting.

Support vector regression based GARCH model with application to forecasting volatility of financial returns

In recent years, support vector regression (SVR), a novel neural network (NN) technique, has been successfully used for financial forecasting. This paper deals with the application of SVR in volatility forecasting. Based on a recurrent SVR, a GARCH method is proposed and is compared with a moving average (MA), a recurrent NN and a parametric GACH in terms of their ability to forecast financial markets volatility. The real data in this study uses British Pound-US Dollar (GBP) daily exchange rates from July 2, 2003 to June 30, 2005 and New York Stock Exchange (NYSE) daily composite index from July 3, 2003 to June 30, 2005. The experiment shows that, under both varying and fixed forecasting schemes, the SVR-based GARCH outperforms the MA, the recurrent NN and the parametric GARCH based on the criteria of mean absolute error (MAE) and directional accuracy (DA). No structured way being available to choose the free parameters of SVR, the sensitivity of performance is also examined to the free parameters.

Reliability forecasting by recurrent Support Vector Regression

Support Vector Regression (SVR), which employs the structural risk minimisation principle to minimise an upper bound of the generalisation errors instead of minimising the training errors used by ANNs, has been successfully applied to solve nonlinear forecasting and times series problems. However, the application of SVR to reliability forecasting has still not been extensively explored. In general, Recurrent Neural Networks (RNNs) are trained by back-propagation algorithms. In the study, the learning algorithms of SVR are applied to RNNs for forecasting system reliability, and the Immune Algorithm (IA) is applied to the parameter determining the SVR model. A numerical example in the existing literature is employed to demonstrate the prediction performance of the proposed model. Empirical results illustrate that the proposed model outperforms other approaches in the existing literature.

A recurrent support vector regression model in rainfall forecasting

AbstractTo minimize potential loss of life and property caused by rainfall during typhoon seasons, precise rainfall forecasts have been one of the key subjects in hydrological research. However, rainfall forecast is made difficult by some very complicated and unforeseen physical factors associated with rainfall. Recently, support vector regression (SVR) models and recurrent SVR (RSVR) models have been successfully employed to solve time‐series problems in some fields. Nevertheless, the use of RSVR models in rainfall forecasting has not been investigated widely. This study attempts to improve the forecasting accuracy of rainfall by taking advantage of the unique strength of the SVR model, genetic algorithms, and the recurrent network architecture. The performance of genetic algorithms with different mutation rates and crossover rates in SVR parameter selection is examined. Simulation results identify the RSVR with genetic algorithms model as being an effective means of forecasting rainfall amount. Copyright © 2006 John Wiley & Sons, Ltd.