Support Vector Regression Framework Research Articles

Adaptive EWMA control chart by using support vector regression

AbstractTraditional control charts depend on the process parameters that are used to monitor the shifts in the process. The adaptive control charts are used to adapt a process parameter during the online monitoring. This research introduces a support vector regression (SVR) based adaptive exponentially weighted moving average control chat to enhance the monitoring of the mean in industrial processes. The study systematically investigates the comparative efficiency of linear, radial basis function (RBF), and polynomial functions within the SVR framework. The proposed SVR‐based AEWMA control chart leverages the strengths of the RBF kernel, providing a robust mechanism for detecting shifts in the process mean by adapting the smoothing constant according to the size of the shift. To validate the efficacy of the proposed methodology, a practical application is presented by using real‐life data. The application showcases the adaptability and reliability of the SVR‐based adaptive EWMA control chart in effectively monitoring location shifts.

Hybrid gray wolf optimization method in support vector regression framework for highly precise prediction of remaining useful life of lithium-ion batteries

Hybrid gray wolf optimization method in support vector regression framework for highly precise prediction of remaining useful life of lithium-ion batteries

Support vector regression with asymmetric loss for optimal electric load forecasting

In energy demand forecasting, the objective function is often symmetric, implying that over-prediction errors and under-prediction errors have the same consequences. In practice, these two types of errors generally incur very different costs. To accommodate this, we propose a machine learning algorithm with a cost-oriented asymmetric loss function in the training procedure. Specifically, we develop a new support vector regression incorporating a linear-linear cost function and the insensitivity parameter for sufficient fitting. The electric load data from the state of New South Wales in Australia is used to show the superiority of our proposed framework. Compared with the basic support vector regression, our new asymmetric support vector regression framework for multi-step load forecasting results in a daily economic cost reduction ranging from 42.19% to 57.39%, depending on the actual cost ratio of the two types of errors.

An interpretable regression approach based on bi-sparse optimization

Given the increasing amounts of data and high feature dimensionalities in forecasting problems, it is challenging to build regression models that are both computationally efficient and highly accurate. Moreover, regression models commonly suffer from low interpretability when using a single kernel function or a composite of multi-kernel functions to address nonlinear fitting problems. In this paper, we propose a bi-sparse optimization-based regression (BSOR) model and corresponding algorithm with reconstructed row and column kernel matrices in the framework of support vector regression (SVR). The BSOR model can predict continuous output values for given input points while using the zero-norm regularization method to achieve sparse instance and feature sets. Experiments were run on 16 datasets to compare BSOR to SVR, linear programming SVR (LPSVR), least squares SVR (LSSVR), multi-kernel learning SVR (MKLSVR), least absolute shrinkage and selection operator regression (LASSOR), and relevance vector regression (RVR). BSOR significantly outperformed the other six regression models in predictive accuracy, identification of the fewest representative instances, selection of the fewest important features, and interpretability of results, apart from its slightly high runtime.

Support Vector Regression for the Modeling and Synthesis of Near-Field Focused Antenna Arrays

The powerful support vector regression framework is proposed in a novel method for near-field focusing using antenna arrays. By using this machine-learning method, the set of weights required in the elements of an array can be calculated to achieve an assigned near-field distribution focused on one or more positions. The computational cost is concentrated in an initial training process so that the trained system is fast enough for applications where moving devices are involved. The increased learning capabilities of support vector machines allow using a reduced number of training samples. Thus, these training samples may be generated with a prototype or a convenient electromagnetic analysis tool, and hence realistic effects, such as coupling or the individual radiation patterns of the elements of the arrays, are accounted for. Illustrative examples are presented.

SVR Based Blind Signal Recovery for Convolutive MIMO Systems With High-Order QAM Signals

This paper addresses the blind signal recovery for convolutive multiple-input multiple-output systems with high-order quadrature amplitude modulation (QAM) signals. First, a family of batch blind recovery algorithms is proposed. Concretely, they introduce the error function of multimodulus algorithm and the cross-correlation among different equalizer output vectors into the penalty term of the support vector regression (SVR) framework to recover all sources simultaneously. Then, the corresponding dual-mode blind recovery schemes are constructed to further decrease the interference. The new blind formulation through iterative re-weighted least square achieves low complexity optimization. The SVR framework, in essence, determines that the proposals perform better than the conventional methods in terms of data block size, total interference, and symbol error rate. Moreover, the excellent initialization provided by the first mode and the accurate error expression in the second mode ensure that the SVR-based dual-mode schemes work well with the high-order QAM signals. Finally, the efficiency of the proposals over the classical approaches is evaluated by simulations.

A hybrid support vector regression framework for streamflow forecast

Monthly streamflow time series are highly non-linear. How to improve forecast accuracy is a great challenge in hydrological studies. A lot of research has been conducted to address the streamflow forecasting problem, however, few methods are developed to make a systematic research. The objective of this study is to understand the underlying trend of streamflow so that a regression model can be developed to forecast the flow volume. In this paper, a hybrid streamflow forecast framework is proposed that integrates factor analysis, time series decomposition, data regression, and error suppression. Correlation coefficients between the current streamflow and the streamflow with lags are analyzed using autocorrelation function (ACF), partial autocorrelation function (PACF), and grey correlation analysis (GCA). Support vector regression (SVR) and generalized regression neural network (GRNN) models are integrated with seasonal and trend decomposition to make monthly streamflow forecast. Auto-regression and multi-model combination error correction methods are used to ensure the accuracy. In our experiments, the proposed method is compared with a stochastic autoregressive integrated moving average (ARIMA) streamflow forecast model. Fourteen models are developed, and the monthly streamflow data of Shigu and Xiangjiaba, China from 1961 to 2009 are used to evaluate our proposed method. Our results demonstrate that the integrated model of grey correlation analysis, Seasonal-Trend Decomposition Procedure Based on Loess (STL), Support Vector Regression (GCA-STL-SVR) exhibits an improved performance for monthly streamflow forecast. The average error of the proposed model is reduced to less than one-tenth in contrast to the state-of-the-art method and the standard deviation is also reduced by more than 30%, which implies a greater consistency.

Electricity load forecasting in a smart grid system

Smart grid has recently become a common system in many countries, and smart meter is one of the essential components of this system. The consumption data gathered from smart meters allow electricity companies to better understand electricity usage in the future and monitor electricity supply more e fficiently. An electricity consumption forecasting model established on the framework of support vector regression (SVR) was proposed in this study. Given that various factors affected consumption patterns, apart from historical data, weather variations and features of a particular time were also considered in this study. Based on this information, the stepwise regression analysis was applied for feature selection. However, the accuracy of the SVR model is largely dependent on the selection of the model parameters. The particle swarm optimization (PSO) algorithm was proposed to determine the optimal values of parameter that improves the accuracy and efficiency of the SVR model. The experimental results show that this method can provide electricity forecasts with 0.70% and 2.55% of mean absolute percentage error (MAPE) for 1 and 24 hours ahead, respectively. This result indicates promising performance in forecasting accuracy. This model is also computationally efficient and can be applied to make predictions within 1 second.

Shape Fitting for the Shape Control System of Silicon Single Crystal Growth

Shape fitting, including straight line and ellipse fitting, plays an important role in the (cylinder-) shape control system of silicon single crystal growth, because the straight lines and ellipse in the crystal image contain the important horizontal circle center and diameter information. This information can be used as control variables so that the grown crystal approximates to a perfect cylinder, and thus can be used as high-quality source materials. In this paper, we develop new straight line and ellipse fitting algorithms. The key points are as follows. We formulate the two-dimensional (2-D) binary image into a single-snapshot array signal of a virtual sensor array, and casts the angle estimation problem of straight lines into the direction finding one of virtual incoming sources. Based on the virtual array manifold and potential incoming angles, the relevant over-complete dictionary is constructed, and thus a sparse regression problem is formed. To solve such a regression problem, we introduce the weight vector sparsity term into the conventional linear least-squares support vector regression framework to estimate the angles of these straight lines. Based on the estimated angles and potential offsets, another over-complete dictionary is constructed, and thus the image can be looked upon as the sparse representation of these dictionary atoms. Since the constructed dictionary is of the same size as the image, we use the compressed sensing theory to reduce the relevant dimensionality and then apply the aforementioned sparse regression method to obtain the relevant offsets of these straight lines. We derive a new second-order polynomial of ellipse equation to obtain the ellipse parameters to avoid the trival solution from the conventional polynomial model. Some simulation and experimental examples are given to illustrate the effectiveness of the proposed algorithms.

Fuzzy classifier based support vector regression framework for Poisson ratio determination

Abstract Poisson ratio is considered as one of the most important rock mechanical properties of hydrocarbon reservoirs. Determination of this parameter through laboratory measurement is time, cost, and labor intensive. Furthermore, laboratory measurements do not provide continuous data along the reservoir intervals. Hence, a fast, accurate, and inexpensive way of determining Poisson ratio which produces continuous data over the whole reservoir interval is desirable. For this purpose, support vector regression (SVR) method based on statistical learning theory (SLT) was employed as a supervised learning algorithm to estimate Poisson ratio from conventional well log data. SVR is capable of accurately extracting the implicit knowledge contained in conventional well logs and converting the gained knowledge into Poisson ratio data. Structural risk minimization (SRM) principle which is embedded in the SVR structure in addition to empirical risk minimization (EMR) principle provides a robust model for finding quantitative formulation between conventional well log data and Poisson ratio. Although satisfying results were obtained from an individual SVR model, it had flaws of overestimation in low Poisson ratios and underestimation in high Poisson ratios. These errors were eliminated through implementation of fuzzy classifier based SVR (FCBSVR). The FCBSVR significantly improved accuracy of the final prediction. This strategy was successfully applied to data from carbonate reservoir rocks of an Iranian Oil Field. Results indicated that SVR predicted Poisson ratio values are in good agreement with measured values.

A Bayesian inference based two-stage support vector regression framework for soft sensor development in batch bioprocesses

Inherent process and measurement uncertainty has posed a challenging issue on soft sensor development of batch bioprocesses. In this paper, a new soft sensor modeling framework is proposed by integrating Bayesian inference strategy with two-stage support vector regression (SVR) method. The Bayesian inference procedure is first designed to identify measurement biases and misalignments via posterior probabilities. Then the biased input measurements are calibrated through Bayesian estimation and the first-stage SVR model is thus built for output measurement reconciliation. The inferentially calibrated input and output data can be further used to construct the second-stage SVR model, which serves as the main model of soft sensor to predict new output measurements. The Bayesian inference based two-stage support vector regression (BI-SVR) approach is applied to a fed-batch penicillin cultivation process and the obtained soft sensor performance is compared to that of the conventional SVR method. The results from two test cases with different levels of measurement uncertainty show significant improvement of the BI-SVR approach over the regular SVR method in predicting various output measurements.

Multi-output regression on the output manifold

Multi-output regression aims at learning a mapping from an input feature space to a multivariate output space. Previous algorithms define the loss functions using a fixed global coordinate of the output space, which is equivalent to assuming that the output space is a whole Euclidean space with a dimension equal to the number of the outputs. So the underlying structure of the output space is completely ignored. In this paper, we consider the output space as a Riemannian submanifold to incorporate its geometric structure into the regression process. To this end, we propose a novel mechanism, called locally linear transformation (LLT), to define the loss functions on the output manifold. In this way, currently existing regression algorithms can be improved. In particular, we propose an algorithm under the support vector regression framework. Our experimental results on synthetic and real-life data are satisfactory.

Support Vector Regression for Basis Selection in Laplacian Noise Environment

We demonstrate that the objective function of a basis selection problem in Laplacian noise environment falls into the framework of support vector regression (SVR), and, by iteratively solving a convex quadratic programming (QP) problem that guarantees a globally optimal solution, the sparse solution to the inverse problem can be found. The effectiveness of the proposed algorithm is verified via the application to direction-of-arrival (DOA) estimation. Different from the existing DOA estimation method based on SVR, the proposed algorithm is applicable with single snapshot and does not have to know the number of the sources. Meanwhile, the method does not require a large number of training sets, which in turn decreases the computational complexity.

Multiple Support Vector Regression for Antenna Array Characterization and Synthesis

The support vector regression framework is proposed as the basis for a new antenna array characterization technique able to model radiating structures including all the coupling effects between the elements of the array from experimental data. The obtained model can then be used to solve practical synthesis problems with high accuracy and using an appropriate cost function for template-based specification handling. A design example is presented to show the efficiency of the proposed method.

Robust Support Vector Regression for Biophysical Variable Estimation From Remotely Sensed Images

This letter introduces the epsiv-Huber loss function in the support vector regression (SVR) formulation for the estimation of biophysical parameters extracted from remotely sensed data. This cost function can handle the different types of noise contained in the dataset. The method is successfully compared to other cost functions in the SVR framework, neural networks and classical bio-optical models for the particular case of the estimation of ocean chlorophyll concentration from satellite remote sensing data. The proposed model provides more accurate, less biased, and improved robust estimation results on the considered case study, especially significant when few in situ measurements are available