Accurate Online Support Vector Regression Research Articles

Accurate on-line support vector regression incorporated with compensated prior knowledge

When the training data required by the data-driven model is insufficient or difficult to cover the sample space completely, incorporating the prior knowledge and prior knowledge compensation module into the support vector regression (PESVR) can significantly improve the accuracy and generalization performance of the model. However, the optimization problem to be solved is very complex, resulting long training time, and it must be retrained all the data from scratch every time the training set is modified. Comparing to standard support vector regression (SVR), PESVR has multiple input datasets and more complex objective function and constraints, including several coupling constraints, the existing methods cannot effectively solve accurate on-line learning of this nested (i.e. fully coupled) model. In this paper, an accurate on-line support vector regression incorporated with prior knowledge and error compensation is proposed. Under the constraint of Karush–Kuhn–Tucker conditions, the model parameters are updated recursively through the sequential adiabatic incremental adjustments. The error compensation model and the prediction model are updated simultaneously when a real measured sample or prior knowledge sample is added to or removed from the training set. The updated model is identical to the model produced by the batch learning algorithm. Experiments on an artificial dataset and several benchmark datasets show encouraged results for online learning and prediction.

Field data–driven online prediction model for icing load on power transmission lines

Methods for the accurate prediction of icing loads in overhead transmission lines have become an important research topic for electrical power systems as they are necessary for ensuring the safety and stability of power-grid operations. Current machine learning models for the prediction of icing loads on transmission lines are afflicted by the following issues: insufficient prediction accuracy, high randomity in the selection of kernel functions and model parameters, and a lack of generalizability. To address these issues, we propose a field data–driven online prediction model for icing loads on transmission lines. First, the effects of the type of kernel function used in the support vector regression algorithm on the prediction accuracy of the model were analyzed using micrometeorological data and icing data collected by on-site monitoring systems. The particle swarm optimization algorithm was then used to optimize and determine the model parameters such as penalty coefficients. An offline support vector regression prediction model was thus constructed. Using the accurate online support vector regression algorithm, the weighting coefficients of the samples were dynamically adjusted to satisfy the Karush–Kuhn–Tucker conditions, which allowed online updates to be made to the regression function and prediction model. Finally, a simulation analysis was performed using actual icing incidents that occurred in a transmission line of the Yunnan Power Grid, which demonstrated that our model can make online predictions for the icing load on transmission lines in actual applications. Our model proved to be superior to conventional icing-load prediction models with regard to the single-step and multi-step prediction accuracies and generalizability. Hence, our prediction model will improve the decision-making processes regarding the deicing and maintenance of power transmission and transformation systems.

Incremental maximum margin clustering

This paper proposes incremental maximum margin clustering in which one data point at a time is examined to decide which cluster the new data point belongs. The proposed method adopts the off-line iterative maximum margin clustering method's alternating optimization algorithm. Accurate online support vector regression is employed in the alternating optimization. To avoid premature convergence, a sequence of decremental unlearning and incremental learning steps is performed. The proposed method is experimentally argued to (i) be scalable and competitive on training time front when compared with iterative maximum margin clustering and (ii) achieve competitive cluster quality compared to the off-line counterpart.

A Novel Dynamic Financial Conditions Index Approach Based on Accurate Online Support Vector Regression

Abstract In this paper, we construct a novel dynamic financial conditions index (DFCI) for China based on accurate online support vector regression (AOSVR), and the constructed DFCI is evaluated on future inflationary pressures. The research results indicate dynamic effect of financial variables on DFCI in time-varying economic and financial environment, verifying the dynamic nature of the weights in our DFCI. On the whole, in our DFCI exchange rate, stock price, and money supply have the push-down effect on DFCI, taking negative dynamic weights. Housing price has the pull-up effect on DFCI, taking positive dynamic weights. The effect of interest rate on DFCI is erratic, taking sign-changed dynamic weights. The Granger causality test results show the superior performance ability of our DFCI compared with the FCI constructed based on SVR.

Online Support Vector Regression With Varying Parameters for Time-Dependent Data

Support vector regression (SVR) is a machine learning technique that continues to receive interest in several domains, including manufacturing, engineering, and medicine. In order to extend its application to problems in which data sets arrive constantly and in which batch processing of the data sets is infeasible or expensive, an accurate online SVR (AOSVR) technique was proposed. The AOSVR technique efficiently updates a trained SVR function whenever a sample is added to or removed from the training set without retraining the entire training data. However, the AOSVR technique assumes that the new samples and the training samples are of the same characteristics; hence, the same value of SVR parameters is used for training and prediction. This assumption is not applicable to data samples that are inherently noisy and nonstationary, such as sensor data. As a result, we propose AOSVR with varying parameters that uses varying SVR parameters rather than fixed SVR parameters and hence accounts for the variability that may exist in the samples. To accomplish this objective, we also propose a generalized weight function to automatically update the weights of SVR parameters in online monitoring applications. The proposed function allows for lower and upper bounds for SVR parameters. We tested our proposed approach and compared results with the conventional AOSVR approach using two benchmark time-series data and sensor data from a nuclear power plant. The results show that using varying SVR parameters is more applicable to time-dependent data.

Fault tolerance in the framework of support vector machines based model predictive control

Model based predictive control (MBPC) has been extensively investigated and is widely used in industry. Besides this, interest in non-linear systems has motivated the development of MBPC formulations for non-linear systems. Moreover, the importance of security and reliability in industrial processes is in the origin of the fault tolerant strategies developed in the last two decades. In this paper a MBPC based on support vector machines (SVM) able to cope with faults in the plant itself is presented. The fault tolerant capability is achieved by means of the accurate on-line support vector regression (AOSVR) which is capable of training an SVM in an incremental way. Thanks to AOSVR is possible to train a plant model when a fault is detected and to change the nominal model by the new one, that models the faulty plant. Results obtained under simulation are presented.

Forecasting respiratory motion with accurate online support vector regression (SVRpred)

To accurately deliver radiation in image-guided robotic radiosurgery, highly precise prediction algorithms are required. A new prediction method is presented and evaluated. SVRpred, a new prediction method based on support vector regression (SVR), has been developed and tested. Computer-generated data mimicking human respiratory motion with a prediction horizon of 150 ms was used for lab tests. The algorithm was subsequently evaluated on a respiratory motion signal recorded during actual radiosurgical treatment using the CyberKnife. The algorithm's performance was compared to the MULIN prediction methods and Wavelet-based multi scale autoregression (wLMS). The SVRpred algorithm clearly outperformed both the MULIN and the wLMS algorithms on both real (by 15 and 16 percentage points, respectively) and noise-corrupted simulated data (by 13 and 48 percentage points, respectively). Only on noise-free artificial data, the SVRpred algorithm did perform as well as the MULIN algorithms but not as well as the wLMS algorithm. This new algorithm is a feasible tool for the prediction of human respiratory motion signals significantly outperforming previous algorithms. The only drawback is the high computational complexity and the resulting slow prediction speed. High performance computers will be needed to use the algorithm in live prediction of signals sampled at a high resolution.

A new approach for function approximation in boiler combustion optimization based on modified structural AOSVR

In the scheme of boiler combustion optimization, a group of optimal controller settings is found to provide recommendations to balance desired thermal efficiency and lowest emissions limit. Characteristic functions between particular objectives and controlling variables can be approximated based on data sets obtained from field tests. These relationships can change with variations in coal quality, slag/soot deposits and the condition of plant equipment, which can not be sampled on-line. Thus, approximation relationships based on test conditions could have little applicability for on-line optimization of the combustion process. In this paper, a new approach is proposed to adaptively perform function approximation based on a modified accurate on-line support vector regression method. Two modified criteria are proposed for selection of the unwanted trained sample to be removed. A structural matrix is used to process and save the model parameters and training data sets, which can be adaptively regulated by the on-line learning method. The proposed method is illustrated with an example and is also applied to real boiler data successfully. The results reveal their validity in the prediction of NOx emissions and function approximation, which can correctly be adapted to actual variable operating conditions in the boiler.

Online Support Vector Regression Approach for the Monitoring of Motor Shaft Misalignment and Feedwater Flow Rate

Timely and accurate information about incipient faults in online machines will greatly enhance the development of optimal maintenance procedures. The application of support vector regression to machine health monitoring was recently investigated; however, such implementation is based on batch processing of the available data. Therefore, the addition of new sample to the already existing dataset requires that the technique should retrain from scratch. This disadvantage makes the technique unsuitable for online systems that will give real-time information to field engineers so that corrective actions could be taken before there is any damage to the system. This paper presents an application of accurate online support vector regression (AOSVR) approach that efficiently updates a trained predictor whenever a new sample is added to the training set using shaft misalignment and nuclear power plant feedwater flow rate data. The results show that the approach is effective for online machine condition monitoring where it is usually difficult to obtain sufficient training data prior to the installation of the online systems.

Online trained support vector machines-based generalized predictive control of non-linear systems

In this work, an online support vector machines (SVM) training method (Neural Comput. 2003; 15: 2683–2703), referred to as the accurate online support vector regression (AOSVR) algorithm, is embedded in the previously proposed support vector machines-based generalized predictive control (SVM-Based GPC) architecture (Support vector machines based generalized predictive control, under review), thereby obtaining a powerful scheme for controlling non-linear systems adaptively. Starting with an initially empty SVM model of the unknown plant, the proposed online SVM-based GPC method performs the modelling and control tasks simultaneously. At each iteration, if the SVM model is not accurate enough to represent the plant dynamics at the current operating point, it is updated with the training data formed by persistently exciting random input signal applied to the plant, otherwise, if the model is accepted as accurate, a generalized predictive control signal based on the obtained SVM model is applied to the plant. After a short transient time, the model can satisfactorily reflect the behaviour of the plant in the whole phase space or operation region. The incremental algorithm of AOSVR enables the SVM model to learn the new training data pair, while the decremental algorithm allows the SVM model to forget the oldest training point. Thus, the SVM model can adapt the changes in the plant and also in the operating conditions. The simulation results on non-linear systems have revealed that the proposed method provides an excellent control quality. Furthermore, it maintains its performance when a measurement noise is added to the output of the underlying system. Copyright © 2006 John Wiley & Sons, Ltd.

Accurate On-line Support Vector Regression

Batch implementations of support vector regression (SVR) are inefficient when used in an on-line setting because they must be retrained from scratch every time the training set is modified. Following an incremental support vector classification algorithm introduced by Cauwenberghs and Poggio (2001), we have developed an accurate on-line support vector regression (AOSVR) that efficiently updates a trained SVR function whenever a sample is added to or removed from the training set. The updated SVR function is identical to that produced by a batch algorithm. Applications of AOSVR in both on-line and cross-validation scenarios are presented. In both scenarios, numerical experiments indicate that AOSVR is faster than batch SVR algorithms with both cold and warm start.