Ν-support Vector Regression Research Articles

Investigation of Machine Learning Regression Techniques to Predict Critical Heat Flux Over a Large Parameter Space

A unifying and accurate model to predict critical heat flux (CHF) over a wide range of conditions has been elusive since wall boiling research emerged. With the release of the experimental data utilized in the development of the 2006 Groeneveld CHF lookup table (LUT), by far the most extensive public CHF database available to date (nearly 25 000 data points), the development of data-driven prediction models over a large parameter space in simple geometry (vertical, uniformly heated round tubes) can be achieved. Furthermore, the popularization of machine learning techniques to solve regression problems has led to more advanced tools for analyzing large and complex databases. This work compares three machine learning algorithms to predict the CHF database. For each selected regression algorithm (ν-Support vector, Gaussian process, and neural network), a set of optimized hyperparameters is applied. Among the investigated algorithms, the neural network emerges as the most effective, achieving a CHF predicted/measured factor standard deviation of 12.3%, three times less than that of the LUT. In comparison, the Gaussian process regression and the ν-support vector regression achieve a standard deviation of 17.7%, about two times lower than the LUT. Hence, all considered algorithms significantly outperform the LUT prediction performance. The neural network model and training methodology are designed to prevent overfitting, which is confirmed by data analyses of the CHF predictions. Finally, future development directions (including data coverage, transfer learning, and uncertainty quantification) are discussed.

Prediction of high-embankment settlement combining joint denoising technique and enhanced GWO-ν-SVR method

Reliable long-term settlement prediction of a high embankment relates to mountain infrastructure safety. This study developed a novel hybrid model (NHM) that combines a joint denoising technique with an enhanced gray wolf optimizer (EGWO)-ν-support vector regression (ν-SVR) method. High-embankment field measurements were preprocessed using the joint denoising technique, which includes complete ensemble empirical mode decomposition, singular value decomposition, and wavelet packet transform. Furthermore, high-embankment settlements were predicted using the EGWO-ν-SVR method. In this method, the standard gray wolf optimizer (GWO) was improved to obtain the EGWO to better tune the ν-SVR model hyperparameters. The proposed NHM was then tested in two case studies. Finally, the influences of the data division ratio and kernel function on the EGWO-ν-SVR forecasting performance and prediction efficiency were investigated. The results indicate that the NHM suppresses noise and restores details in high-embankment field measurements. Simultaneously, the NHM outperforms other alternative prediction methods in prediction accuracy and robustness. This demonstrates that the proposed NHM is effective in predicting high-embankment settlements with noisy field measurements. Moreover, the appropriate data division ratio and kernel function for EGWO-ν-SVR are 7:3 and radial basis function, respectively.

Universum parametric $$\nu $$-support vector regression for binary classification problems with its applications

AbstractUniversum data sets, a collection of data sets that do not belong to any specific class in a classification problem, give previous information about data in the mathematical problem under consideration to enhance the classifiers’ generalization performance. Recently, some researchers have integrated Universum data into the existing models to propose new models which result in improved classification performance. Inspired by these Universum models, an efficient parametric$$ \nu $$ν-support vector regression with Universum data ($$ {\mathfrak {U}} $$UPar-$$ \nu $$ν-SVR) is proposed in this work. This method, which finds two non-parallel hyperplanes by solving one optimization problem and considers heteroscedastic noise, overcomes some common disadvantages of the previous methods. The$$ {\mathfrak {U}} $$UPar-$$ \nu $$ν-SVR includes unlabeled samples that don’t belong to any class in the training process, which results in a quadratic programming problem. Two approaches are proposed to solve this problem. The first approach derives the dual formulation using the Lagrangian function and KKT conditions. Furthermore, a least squares parametric$$ \nu $$ν-support vector regression with Universum data (named LS-$$ {\mathfrak {U}} $$UPar-$$ \nu $$ν-SVR) is suggested to further increase the generalization performance. The LS-$$ {\mathfrak {U}} $$UPar-$$ \nu $$ν-SVR solves a single system of linear equations, instead of addressing a quadratic programming problem in the dual formulation. Numerical experiments on artificial, UCI, credit card, NDC, and handwritten digit recognition data sets show that the suggested Universum model and its solving methodologies improve prediction accuracy.

A new algorithm for support vector regression with automatic selection of hyperparameters

The hyperparameters in support vector regression (SVR) determine the effectiveness of the support vectors with fitting and predictions. However, the choice of these hyperparameters has always been challenging in both theory and practice. The ν-support vector regression eliminates the need to specify an ϵ value elegantly, but at the cost of specifying or postulating a ν value. We propose an extended primal objective function arising from probability regularization leading to an automatic selection of ϵ, and we can express ν as an explicit function of ϵ. The resultant hyperparameter values can be interpreted as ‘working’ values required only in training but not testing or prediction. This regularized algorithm, namely ϵ*-SVR, automatically provides a data-dependent ϵ and is found to have a close connection to the ν-support vector regression in the sense that ν as a fraction is a sensible function of ϵ. The ϵ*-SVR automatically selects both ν and ϵ values. We illustrate these findings with some public benchmark datasets.

Wheat leaf rust detection at canopy scale under different LAI levels using machine learning techniques

Accurate diagnosis of wheat leaf rust is of high interest for precision farming. Spectral data have been increasingly employed to detect this disease at leaf or canopy scales; however, less attention has been paid to the variations of leaf area index (LAI). Therefore, in this study, identification of wheat leaf rust was investigated at canopy scale and under different LAI levels, namely high, medium and low. Four machine learning (ML) methods including ν-support vector regression (ν-SVR), boosted regression trees (BRT), random forests regression (RFR) and Gaussian process regression (GPR) were built to estimate disease severity (DS) levels at canopy scale, where the reflectance data were measured in field situ by using a spectroradiometer, in which records spectra from 350 to 2500 nm. Results showed that ν-SVR outperformed the other ML methods at all three LAI levels with the R2 measures all being around 0.99. The results, particularly, showed that the performances of the ML methods were improved with increasing LAI value, where RFR reported the worst R2 value of 0.79 (RMSE = 8.5%) at low LAI level. The variable importance obtained using BRT showed three distinct regions of wavelengths that were appropriate across different LAI levels. The results of this research confirmed that hyperspectral signature can be reliably considered to identify wheat leaf rust disease at different LAI levels. Moreover, performances of several spectral vegetation indices (SVIs) were compared with those of the ML techniques. The results showed that the SVIs were consistently outperformed by the ML methods, particularly at low LAI level in which the SVIs were adversely affected. Nevertheless, all the SVIs, except for the RVSI, performed moderately well at high and medium LAI levels.

Multi-Layer Quantization Control for Quality-Constrained H.265/HEVC.

Achieving a stable video quality is an important task in video compression. In this paper, we present a multi-layer quantization control method to compress video to a certain target video quality based on the new hierarchical partition structure of H.265/HEVC. We first model the rate-quality characteristics of different video sequences using ν-support vector regression with a Gaussian radial basis function as the kernel function. Then, we propose a Kalman filter-based multi-layer quantization control method for the quality-constrained video coding. The advantage of our proposed approach in H.265/HEVC is demonstrated through experimental results. Compared with the other H.265/HEVC rate control algorithms and the fixed-QP scheme, the proposed method achieves a more stable video quality.

Prediction of peak ground acceleration using ϵ-SVR, ν-SVR and Ls-SVR algorithm

ABSTRACTIn this paper, a prediction model is developed using support vector machine for forecasting the parameter associated with ground motion of a seismic signal. The prediction model is developed using three learning algorithms, ϵ-support vector regression, ν-support vector regression and least square-support vector regression (Ls-SVR) for forecasting peak ground acceleration (PGA), a parameter associated with ground motion of a seismic signal. The prediction model is developed for each of the algorithms with different kernel functions, namely linear kernel, polynomial kernel and radial basis function kernel. The ground motion parameter is related to four seismic parameters, namely faulting mechanism, average soil shear wave velocity, earthquake magnitude and source to site distance. The database used for modelling is NGA flatfile released by Pacific Earthquake Engineering Research Center. The experimental results show that the optimal prediction model for forecasting PGA is Ls-SVR with RBF kernel. It is observed that the developed prediction model is better compared to the existing conventional models and benchmark models in the same database. This paper further compares the three variations of SVR algorithm for ground motion parameter prediction model. The learning effectiveness of each algorithm is measured in terms of accuracy, testing error and overfitness measure.

Incremental learning for [formula omitted]-Support Vector Regression

The ν-Support Vector Regression (ν-SVR) is an effective regression learning algorithm, which has the advantage of using a parameter ν on controlling the number of support vectors and adjusting the width of the tube automatically. However, compared to ν-Support Vector Classification (ν-SVC) (Schölkopf et al., 2000), ν-SVR introduces an additional linear term into its objective function. Thus, directly applying the accurate on-line ν-SVC algorithm (AONSVM) to ν-SVR will not generate an effective initial solution. It is the main challenge to design an incremental ν-SVR learning algorithm. To overcome this challenge, we propose a special procedure called initial adjustments in this paper. This procedure adjusts the weights of ν-SVC based on the Karush–Kuhn–Tucker (KKT) conditions to prepare an initial solution for the incremental learning. Combining the initial adjustments with the two steps of AONSVM produces an exact and effective incremental ν-SVR learning algorithm (INSVR). Theoretical analysis has proven the existence of the three key inverse matrices, which are the cornerstones of the three steps of INSVR (including the initial adjustments), respectively. The experiments on benchmark datasets demonstrate that INSVR can avoid the infeasible updating paths as far as possible, and successfully converges to the optimal solution. The results also show that INSVR is faster than batch ν-SVR algorithms with both cold and warm starts.

A Soft Sensor Modelling of Biomass Concentration during Fermentation using Accurate Incremental Online V-Support Vector Regression Learning Algorithm

In order to model real fermentation process, a soft sensor modelling of biomass concentration during fermentation using accurate incremental online ν-Support Vector Regression (ν-SVR) learning algorithm was proposed. Firstly, an accurate incremental online ν-SVR learning algorithm was proposed. This algorithm solved the two complications introduced in the dual problem based on the equivalent formulation of ν-SVR. Moreover, it addressed the infeasible updating path problem during the adiabatic incremental process by relaxed adiabatic incremental adjustments and accurate incremental adjustments. Then, the proposed algorithm is used to predict the biomass concentration of glutamic acid fed-batch fermentation process online. The results of simulation experiment showed that the soft sensor modelling of biomass concentration during fermentation using the proposed algorithm was of better generalization ability and cost less training time than that of ν-SVR.

A novel approach to model selection in tourism demand modeling

In many studies on tourism demand modeling, the main conclusion is that none of the considered modeling approaches consistently outperforms the others. We consider Seasonal AutoRegressive Integrated Moving Average, ν-Support Vector Regression, and multi-layer perceptron type Neural Network models and optimize their parameters using different techniques for each and compare their performances on monthly tourist arrival data to Turkey from different countries. Based on these results, this study proposes a novel approach to model selection for a given tourism time series. Our approach is based on identifying the components of the given time series using structural time series modeling. Using the identified components we construct a decision tree and obtain a rule set for model selection.

Intersection traffic flow forecasting based on ν-GSVR with a new hybrid evolutionary algorithm

Abstract To deal well with the normally distributed random error existed in the traffic flow series, this paper introduces the ν-Support Vector Regression (ν-GSVR) model with the Gaussian loss function to the prediction field of short-term traffic flow. A new hybrid evolutionary algorithm (namely CCGA) is established to search the appropriate parameters of the ν-GSVR, coupling the Chaos map, Cloud model and genetic algorithm. Consequently, a new forecasting approach for short-term traffic flow, combining ν-GSVR model and CCGA algorithm, is proposed. The forecasting process considers the traffic flow for the road during the first few time intervals, the traffic flow for the upstream road section and weather conditions. A numerical example from the intersection between Culture Road and Shi-Full Road in Banqiao is used to verify the forecasting performance of the proposed model. The experiment indicates that the model yield more accurate results than the compared models in forecasting the short-term traffic flow at the intersection.

Noise model based [formula omitted]-support vector regression with its application to short-term wind speed forecasting

Support vector regression (SVR) techniques are aimed at discovering a linear or nonlinear structure hidden in sample data. Most existing regression techniques take the assumption that the error distribution is Gaussian. However, it was observed that the noise in some real-world applications, such as wind power forecasting and direction of the arrival estimation problem, does not satisfy Gaussian distribution, but a beta distribution, Laplacian distribution, or other models. In these cases the current regression techniques are not optimal. According to the Bayesian approach, we derive a general loss function and develop a technique of the uniform model of ν-support vector regression for the general noise model (N-SVR). The Augmented Lagrange Multiplier method is introduced to solve N-SVR. Numerical experiments on artificial data sets, UCI data and short-term wind speed prediction are conducted. The results show the effectiveness of the proposed technique.

ν-Nonparallel support vector machine for pattern classification

In this paper, we propose a novel nonparallel hyperplane classifier, named ν-nonparallel support vector machine (ν-NPSVM), for binary classification. Based on our recently proposed method, i.e., nonparallel support vector machine (NPSVM), which has been proved superior to the twin support vector machines, ν-NPSVM is parameterized by the quantity ν to let ones effectively control the number of support vectors. By combining the ν-support vector classification and the ν-support vector regression together to construct the primal problems, ν-NPSVM inherits the advantages of ν-support vector machine so that enables us to eliminate one of the other free parameters of the NPSVM: the accuracy parameter e and the regularization constant C. We describe the algorithm, give some theoretical results concerning the meaning and the choice of ν, and also report the experimental results on lots of data sets to show the effectiveness of our method.

Assessment of Pan Evaporation Modeling Using Bootstrap Resampling and Soft Computing Methods

AbstractThis study evaluated combined bootstrap resampling and neural network models for estimating daily pan evaporation (PE) in the Republic of Korea. Two different support vector machines (SVMs), e-support vector regression (e-SVR) and ν-support vector regression (ν-SVR), were developed for the local implementation of SVMs. Five-input combination models (e-SVR5 and ν-SVR5) were found to be generally the best for the local implementation of SVMs. Optimal SVMs, including e-SVR5 and ν-SVR5, were employed to develop bootstrap-based support vector machines (BSVMs) for two weather stations. The ensemble PE was estimated by averaging the output of 50 individual BSVMs. A Mann-Whitney U test was performed to compare the observed and ensemble of bootstrap resamplings for the training data of the PE. The uncertainty associated with PE estimation using BSVMs was evaluated. Results indicated that BSVMs could improve confidence in PE estimation and that ensemble PE using BSVMs is more stable and reliable than that u...

Prediction of critical heat flux for water flow in vertical round tubes using support vector regression model

This paper deals with ν-support vector regression (ν-SVR) based prediction model of critical heat flux (CHF) for water flow in vertical round tubes. The dataset used in this paper is obtained from available published literature. The dataset is partitioned into two independent sets, a training data set and a test data set, to avoid overfitting problem. To train the ν-SVR models with more informative data, the training data is selected using a subtractive clustering (SC) scheme, and then the remaining data is used as test data to evaluate the performance of the ν-SVR models. Next, the parametric trends of CHF are investigated using the ν-SVR models. The results obtained from the ν-SVR models are compared with those obtained from the radial basis function (RBF) network, which is a kind of artificial neural networks (ANNs). It is found that the results of the ν-SVR models are not only in better agreement with the experimental data than those of the RBF network, but also follow the general understanding. The analysis results indicate that the ν-SVR models can be successfully applied to CHF prediction.

Combination of support vector regression and artificial neural networks for prediction of critical heat flux

This paper presents a hybrid model that couples ν-support vector regression (ν-SVR) with radial basis function networks (RBFNs) for prediction of critical heat flux (CHF). The hybrid model is achieved in two steps. The first step is to determine the initial architecture and initial weights of the hybrid model by an ν-SVR. The second step is to adjust the initial weights using an annealing robust back propagation (ARBP) learning algorithm. Then the hybrid model is used to predict CHF, which is divided into two parts: prediction of CHF for water flow in vertical round tubes and prediction of dryout type CHF for deionized water upflowing through a narrow annular channel with 0.95mm gap. The dataset used in this paper is taken from literature. In the first part, prediction of CHF and analysis of parametric trends of CHF are both carried out based on three conditions, fixed inlet conditions, local conditions and fixed outlet conditions. The predicted results agree better with the corresponding dataset than that of ε-SVR. In the second part, the predicted results are in better agreement with the experimental data than that of back-propagation network (BPN) employed in the literature. Therefore, the hybrid model presented in this paper is a potential tool for predicting CHF and has advantages over other methods.

A ν-support vector regression based approach for predicting imputation quality

BackgroundDecades of genome-wide association studies (GWAS) have accumulated large volumes of genomic data that can potentially be reused to increase statistical power of new studies, but different genotyping platforms with different marker sets have been used as biotechnology has evolved, preventing pooling and comparability of old and new data. For example, to pool together data collected by 550K chips with newer data collected by 900K chips, we will need to impute missing loci. Many imputation algorithms have been developed, but the posteriori probabilities estimated by those algorithms are not a reliable measure the quality of the imputation. Recently, many studies have used an imputation quality score (IQS) to measure the quality of imputation. The IQS requires to know true alleles to estimate. Only when the population and the imputation loci are identical can we reuse the estimated IQS when the true alleles are unknown.MethodsHere, we present a regression model to estimate IQS that learns from imputation of loci with known alleles. We designed a small set of features, such as minor allele frequencies, distance to the nearest known cross-over hotspot, etc., for the prediction of IQS. We evaluated our regression models by estimating IQS of imputations by BEAGLE for a set of GWAS data from the NCBI GEO database collected from samples from different ethnic populations.ResultsWe construct a ν-SVR based approach as our regression model. Our evaluation shows that this regression model can accomplish mean square errors of less than 0.02 and a correlation coefficient close to 0.75 in different imputation scenarios. We also show how the regression results can help remove false positives in association studies.ConclusionReliable estimation of IQS will facilitate integration and reuse of existing genomic data for meta-analysis and secondary analysis. Experiments show that it is possible to use a small number of features to regress the IQS by learning from different training examples of imputation and IQS pairs.

Multiple-[formula omitted] support vector regression based on spectral risk measure minimization

Statistical learning theory provides the justification of the ϵ-insensitive loss in support vector regression, but suggests little guidance on the determination of the critical hyper-parameter ϵ. Instead of predefining ϵ, ν-support vector regression automatically selects ϵ by making the percent of deviations larger than ϵ be asymptotically equal to ν. In stochastic programming terminology, the goal of ν-support vector regression is to minimize the conditional Value-at-Risk measure of deviations, i.e. the expectation of the larger ν-percent deviations. This paper tackles the determination of the critical hyper-parameter ν in ν-support vector regression when the error term follows a complex distribution. Instead of one singleton ν, the paper assumes ν to be a combination of multiple, finite or infinite, candidate choices. Thus, the cost function becomes a weighted sum of component conditional value-at-risk measures associated with these base νs. This paper shows that this cost function can be represented with a spectral risk measure and its minimization can be reformulated to a linear programming problem. Experiments on three artificial data sets show that this multiple-ν support vector regression has great advantage over the classical ν-support vector regression when the error terms follow mixed polynomial distributions. Experiments on 10 real-world data sets also clearly demonstrate that this new method can achieve better performance than ϵ-support vector regression and ν-support vector regression.

Analytical Modeling of Network Throughput Prediction on the Internet

Predicting network throughput is important for network-aware applications. Network throughput depends on a number of factors, and many throughput prediction methods have been proposed. However, many of these methods are suffering from the fact that a distribution of traffic fluctuation is unclear and the scale and the bandwidth of networks are rapidly increasing. Furthermore, virtual machines are used as platforms in many network research and services fields, and they can affect network measurement. A prediction method that uses pairs of differently sized connections has been proposed. This method, which we call connection pair, features a small probe transfer using the TCP that can be used to predict the throughput of a large data transfer. We focus on measurements, analyses, and modeling for precise prediction results. We first clarified that the actual throughput for the connection pair is non-linearly and monotonically changed with noise. Second, we built a previously proposed predictor using the same training data sets as for our proposed method, and it was unsuitable for considering the above characteristics. We propose a throughput prediction method based on the connection pair that uses ν-support vector regression and the polynomial kernel to deal with prediction models represented as a non-linear and continuous monotonic function. The prediction results of our method compared to those of the previous predictor are more accurate. Moreover, under an unstable network state, the drop in accuracy is also smaller than that of the previous predictor.

Comparison of ν-support vector regression and logistic equation for descriptive modeling of Lactobacillus plantarum growth

Due to the complexity and high non-linearity of bioprocess, most simple mathematical models fail to describe the exact behavior of biochemistry systems. As a novel type of learning method, support vector regression (SVR) owns the powerful capability to characterize problems via small sample, nonlinearity, high dimension and local minima. In this paper, we developed a ν-SVR model with genetic algorithms (GA) in the pre-estimate in Lactobacillus plantarum fermentation by comparing the predicting capability of logistic model and SVR model. 5-fold cross validation technique was applied in the SVR train to avoid over-fitting. The information of SVR parameters were obtained in the generation of 150 and the optimal parameters were C= 235.8935, σ= 8.3608, ν=0.7587. Correspondingly, the logistic model parameters μ max and x max were estimated as 0.4791 and 0.3498, respectively. The experimental results demonstrated that, SVR model excelled the logistic model based on the normalized mean square error (NMSE), mean absolute percentage error (MAPE) and the Pearson correlation coefficient R. We found that the ν-SVR model optimized by genetic algorithms could be a potential monitoring method for prediction of biomass. Key words: Support vector regression, genetic algorithm, logistic model, prediction of biomass.