Combination Of Support Vector Regression Research Articles

A Two-phase SVR Model with Information Reduction for Forecasting Electric Quantity at Gateway

With the rapid developments of electricity market it is very important that prediction of electric quantity helps to have insight into the characteristics of power consumption of different customers so as to make a correct decision of power distribution for the optimal resource scheduling and planning. Based on an accurate forecasting, operation center of power grid can effectively decrease the cost of power generation and simultaneously increase the profit of power services. Although there are many techniques available for short-term electric quantity forecasting, mid-term electric quantity forecasting is relatively little. In this paper, we proposed a new two-phase model combining Support Vector Regression (SVR) with information reduction using L1-norm SVR (L1-SVR) in order to improve the accuracy, efficiency and interpretability of prediction. The former is used to obtain the optimal multiplier vectors and forecast electric quantity at gateway with the reduced data set. At the same time, the latter employs a new information reduction approach for regression to find the important variables and remove the less important variables. The results of our experiment and comparison with the single SVR approach showed that the proposed model improved forecasting accuracy, interpretability of each variable or feature and efficiency of electric quantity at gateway on the independent test set.

An integrated simulation and optimization approach for managing human health risks of atmospheric pollutants by coal-fired power plants

This research developed a simulation-aided nonlinear programming model (SNPM). This model incorporated the consideration of pollutant dispersion modeling, and the management of coal blending and the related human health risks within a general modeling framework. In SNPM, the simulation effort (i.e., California puff [CALPUFF]) was used to forecast the fate of air pollutants for quantifying the health risk under various conditions, while the optimization studies were to identify the optimal coal blending strategies from a number of alternatives. To solve the model, a surrogate-based indirect search approach was proposed, where the support vector regression (SVR) was used to create a set of easy-to-use and rapid-response surrogates for identifying the function relationships between coal-blending operating conditions and health risks. Through replacing the CALPUFF and the corresponding hazard quotient equation with the surrogates, the computation efficiency could be improved. The developed SNPM was applied to minimize the human health risk associated with air pollutants discharged from Gaojing and Shijingshan power plants in the west of Beijing. Solution results indicated that it could be used for reducing the health risk of the public in the vicinity of the two power plants, identifying desired coal blending strategies for decision makers, and considering a proper balance between coal purchase cost and human health risk.Implications:A simulation-aided nonlinear programming model (SNPM) is developed. It integrates the advantages of CALPUFF and nonlinear programming model. To solve the model, a surrogate-based indirect search approach based on the combination of support vector regression and genetic algorithm is proposed. SNPM is applied to reduce the health risk caused by air pollutants discharged from Gaojing and Shijingshan power plants in the west of Beijing. Solution results indicate that it is useful for generating coal blending schemes, reducing the health risk of the public, reflecting the trade-off between coal purchase cost and health risk.

Long Term Solar Radiation Forecast Using Computational Intelligence Methods

The point prediction quality is closely related to the model that explains the dynamic of the observed process. Sometimes the model can be obtained by simple algebraic equations but, in the majority of the physical systems, the relevant reality is too hard to model with simple ordinary differential or difference equations. This is the case of systems with nonlinear or nonstationary behaviour which require more complex models. The discrete time-series problem, obtained by sampling the solar radiation, can be framed in this type of situation. By observing the collected data it is possible to distinguish multiple regimes. Additionally, due to atmospheric disturbances such as clouds, the temporal structure between samples is complex and is best described by nonlinear models. This paper reports the solar radiation prediction by using hybrid model that combines support vector regression paradigm and Markov chains. The hybrid model performance is compared with the one obtained by using other methods like autoregressive (AR) filters, Markov AR models, and artificial neural networks. The results obtained suggests an increasing prediction performance of the hybrid model regarding both the prediction error and dynamic behaviour.

Support vector regression with digital band pass filtering for the quantitative analysis of near‐infrared spectra

This paper proposes a novel calibration technique based on combining support vector regression with a digital band pass (DBP) filter for the quantitative analysis of near‐infrared spectra. The efficacy of the proposed method is investigated and validated in the determination of glucose from near‐infrared spectra of a mixture composed of urea, triacetin and glucose. In this paper, the DBP filtering was implemented as a pre‐processing technique in the frequency domain as a Gaussian band pass filter and in the time domain as a Chebyshev filter. The grid‐search optimization method was used to optimize the filter parameters. The results demonstrate that utilization of the optimized DBP filters as a pre‐processing technique improved the performance of the predictive models. Copyright © 2013 John Wiley & Sons, Ltd.

A Grounding Grid Model of Corrosion Factors Based on Clustering and SVR

A structure combined clustering analysis and Support Vector Regression (SVR) was put forward with to establish a grounding grid model of corrosion factors. Firstly, according to the determination of clustering numbers required by K-average algorithm, a new method to determine clustering numbers was proposed with the distance between intra-class and inter-class as the clustering effectiveness measuring; secondly, a training sample set after cluster analysis based upon Kopt-average algorithm was divided into class clusters with similar characteristics; finally, for each cluster after clustering, corresponding models were established respectively. The result shows that the K- average algorithm can determine the best clustering number of a sample set effectively, and the grounding grid SVR model of corrosion factors is not only effective, but also steadily increases the corrosion rate prediction accuracy.

Support vector regression and synthetically mixed training data for quantifying urban land cover

Exploiting imaging spectrometer data with machine learning algorithms has been demonstrated to be an excellent choice for mapping ecologically meaningful land cover categories in spectrally complex urban environments. However, the potential of kernel-based regression techniques for quantitatively analyzing urban composition has not yet been fully explored. To a great extent, this can be explained by difficulties in deriving quantitative training information that reliably represents pairs of spectral signatures with associated land cover fractions needed for empirical modeling. In this paper we present an approach to circumvent this limitation by combining support vector regression (SVR) with synthetically mixed training data to map sub-pixel fractions of single urban land cover categories of interest. This approach was tested on Hyperspectral Mapper (HyMap) data acquired over Berlin, Germany. Fraction estimates were validated with extensive manual mappings and compared to fractions derived from multiple endmember spectral mixture analysis (MESMA). Our regression results demonstrate that the sets of multiple mixtures yielded high accuracies for quantitative estimates for four spectrally complex urban land cover types, i.e., fractions of impervious rooftops and pavements, as well as grass- and tree-covered areas. Despite the extrapolation uncertainty of SVR, which resulted in fraction values below 0% and above 100%, physically meaningful model outputs were reported for a clear majority of pixels, and visual inspection underpinned the quality of produced fraction maps. Statistical accuracy assessment with detailed reference information for 92 urban blocks showed linear relations with R2 values of 0.86, 0.58, 0.81 and 0.85 for the four categories, respectively. Mean absolute errors (MAE) ranged from 6.4 to 12.8% and block-wise sums of the four individually modeled category fractions were always around 100%. Results of MESMA followed similar trends, but with slightly lower accuracies. Our findings demonstrate that the combination of SVR and synthetically mixed training data enable the use of empirical regression for sub-pixel mapping. Thus, the strengths of kernel-based approaches for quantifying urban land cover from imaging spectrometer data can be well utilized. Remaining uncertainties and limitations were related to the known phenomena of spectral similarity or ambiguity of urban materials, the spectral deficiencies in shaded areas, or the dependency on comprehensive and representative spectral libraries. Therefore, the suggested workflow constitutes a new flexible and extendable universal modeling approach to map land cover fractions.

Combination of support vector regression and artificial neural networks for prediction of critical heat flux

This paper presents a hybrid model that couples ν-support vector regression (ν-SVR) with radial basis function networks (RBFNs) for prediction of critical heat flux (CHF). The hybrid model is achieved in two steps. The first step is to determine the initial architecture and initial weights of the hybrid model by an ν-SVR. The second step is to adjust the initial weights using an annealing robust back propagation (ARBP) learning algorithm. Then the hybrid model is used to predict CHF, which is divided into two parts: prediction of CHF for water flow in vertical round tubes and prediction of dryout type CHF for deionized water upflowing through a narrow annular channel with 0.95mm gap. The dataset used in this paper is taken from literature. In the first part, prediction of CHF and analysis of parametric trends of CHF are both carried out based on three conditions, fixed inlet conditions, local conditions and fixed outlet conditions. The predicted results agree better with the corresponding dataset than that of ε-SVR. In the second part, the predicted results are in better agreement with the experimental data than that of back-propagation network (BPN) employed in the literature. Therefore, the hybrid model presented in this paper is a potential tool for predicting CHF and has advantages over other methods.

A hybrid method for imputation of missing values using optimized fuzzy c-means with support vector regression and a genetic algorithm

Missing values in datasets should be extracted from the datasets or should be estimated before they are used for classification, association rules or clustering in the preprocessing stage of data mining. In this study, we utilize a fuzzy c-means clustering hybrid approach that combines support vector regression and a genetic algorithm. In this method, the fuzzy clustering parameters, cluster size and weighting factor are optimized and missing values are estimated. The proposed novel hybrid method yields sufficient and sensible imputation performance results. The results are compared with those of fuzzy c-means genetic algorithm imputation, support vector regression genetic algorithm imputation and zero imputation.

Swarm Intelligence and Its Applications

Swarm Intelligence and Its Applications

Hybrid Support Vector Regression and Autoregressive Integrated Moving Average Models Improved by Particle Swarm Optimization for Property Crime Rates Forecasting with Economic Indicators

Crimes forecasting is an important area in the field of criminology. Linear models, such as regression and econometric models, are commonly applied in crime forecasting. However, in real crimes data, it is common that the data consists of both linear and nonlinear components. A single model may not be sufficient to identify all the characteristics of the data. The purpose of this study is to introduce a hybrid model that combines support vector regression (SVR) and autoregressive integrated moving average (ARIMA) to be applied in crime rates forecasting. SVR is very robust with small training data and high-dimensional problem. Meanwhile, ARIMA has the ability to model several types of time series. However, the accuracy of the SVR model depends on values of its parameters, while ARIMA is not robust to be applied to small data sets. Therefore, to overcome this problem, particle swarm optimization is used to estimate the parameters of the SVR and ARIMA models. The proposed hybrid model is used to forecast the property crime rates of the United State based on economic indicators. The experimental results show that the proposed hybrid model is able to produce more accurate forecasting results as compared to the individual models.

Combined shifted-excitation Raman difference spectroscopy and support vector regression for monitoring the algal production of complex polysaccharides

The applicability of shifted-excitation Raman difference spectroscopy (SERDS) in combination with signal regression analysis as an alternative and non-invasive approach for monitoring the cultivation of phototrophic microorganisms producing complex molecules of pharmaceutical relevance in a bioreactor is demonstrated. As a model system, the cultivation of the red unicellular algae Porphyridium purpureum is used for focusing on the segregation of sulfated exopolysaccharides (EPS) which exhibit antiviral activity. The spectroscopic results obtained by partial linear least squares regression (PLSR) and by nonlinear support vector regression (SVR) are discussed against the corresponding results from the reference analytics based on the phenol-sulfuric acid assay. The SERDS-approach turns out to have strong potential as a non-invasive tool for online-monitoring of biotechnological processes.

Nonlinear regression in environmental sciences by support vector machines combined with evolutionary strategy

A hybrid algorithm combining support vector regression with evolutionary strategy (SVR-ES) is proposed for predictive models in the environmental sciences. SVR-ES uses uncorrelated mutation with p step sizes to find the optimal SVR hyper-parameters. Three environmental forecast datasets used in the WCCI-2006 contest – surface air temperature, precipitation and sulphur dioxide concentration – were tested. We used multiple linear regression (MLR) as benchmark and a variety of machine learning techniques including bootstrap-aggregated ensemble artificial neural network (ANN), SVR-ES, SVR with hyper-parameters given by the Cherkassky–Ma estimate, the M5 regression tree, and random forest (RF). We also tested all techniques using stepwise linear regression (SLR) first to screen out irrelevant predictors. We concluded that SVR-ES is an attractive approach because it tends to outperform the other techniques and can also be implemented in an almost automatic way. The Cherkassky–Ma estimate is a useful approach for minimizing the mean absolute error and saving computational time related to the hyper-parameter search. The ANN and RF are also good options to outperform multiple linear regression (MLR). Finally, the use of SLR for predictor selection can dramatically reduce computational time and often help to enhance accuracy.

Data Mining-Based Operation Optimization of Large Coal-Fired Power Plants

Large coal-fired power generation is a complex process characterized as nonlinear and coupling correlation between the levels of equipment, subsystems and function modules. It is therefore difficult to describe the energy-consumption behaviour and optimize the operation parameters under different operation conditions and boundary conditions with conventional methods. With data mining methods such as Support Vector Regression (SVR) and Genetic Algorithm (GA), a huge amount of practical operation data stored in the plant-level Supervisory Information System (SIS) were used to model the energy consumption and optimize the operation parameters for less coal consumption. The results show that the power coal rate reduced significantly under the combination of SVR and GA. The optimal operation program has a practical feasibility, and the whole optimizing process can supply model basis for large coal-fired power units.

Fault Feature Extraction Based on Improved EEMD and Hilbert Transform

Ensemble Empirical Mode Decomposition (EEMD) can overcome the mode mixing problem in Empirical Mode Decomposition (EMD) effectively. The Hilbert-Huang transform still exists end effect in applications, in order to improve the end effect, this paper put forward a method of fault feature extraction based on improved EEMD and Hilbert transform which combines support vector regression (SVR) machine with mirror extension to continue the signal. The analysis on simulation experiments results show that the method can restrain the end effect effectively, get a more accurate instantaneous frequency and instantaneous amplitude.

HPLC Retention time prediction for metabolome analysis

Liquid Chromatography Time-of-Flight Mass Spectrometry (LC-TOF-MS) is widely used for profiling metabolite compounds. LC-TOF-MS is a chemical analysis technique that combines the physical separation capabilities of high-pressure liquid chromatography (HPLC) with the mass analysis capabilities of Time-of-Flight Mass Spectrometry (TOF-MS) which utilizes the difference in the flight time of ions due to difference in the mass-to-charge ratio. Since metabolite compounds have various chemical characteristics, their precise identification is a crucial problem of metabolomics research. Contemporaneously analyzed reference standards are commonly required for mass spectral matching and retention time matching, but there are far fewer reference standards than there are compounds in the organism. We therefore developed a retention time prediction method for HPLC to improve the accuracy of identification of metabolite compounds. This method uses a combination of Support Vector Regression and Multiple Linear Regression adaptively to the measured retention time. We achieved a strong correlation (correlation coefficient = 0.974) between measured and predicted retention times for our experimental data. We also demonstrated a successful identification of an E. coli metabolite compound that cannot be identified by precise mass alone.

A yield forecast model for pilot products using support vector regression and manufacturing experience–the case of large-size polariser

To build up a manufacturing management model for a newly developed product is fundamentally a difficult problem, because the collected data in the early manufacturing stages is usually insufficient when data size is small. There are several researches on this topic, and most of them focus on the original data analysis such as building up virtual samples to increase the data number. As to other approaches, the usage of old or similar manufacturing experience may be an alternative approach to help in modelling a small data set, by taking advantage of the fact that the new product's manufacturing process could be based on the experience of the old one. This research proposes a combination of support vector regression (SVR) and the manufacturing experience to build up the manufacturing knowledge model for a new product. A real-problem of a new product yield forecast model in a polariser manufacturing company is demonstrated, where two approaches are proposed, and the results show that the presented approach is superior to the performance of a linear regression and back-propagation neural network. The case study shows that the input of the old or similar manufacturing experience into the forecast model can reduce the error rate and enhance the model forecasting ability.

Combining support vector regression and cellular genetic algorithm for multi-objective optimization of coal-fired utility boilers

Support vector regression (SVR) was employed to establish mathematical models for the NO x emissions and carbon burnout of a 300 MW coal-fired utility boiler. Combined with the SVR models, the cellular genetic algorithm for multi-objective optimization (MOCell) was used for multi-objective optimization of the boiler combustion. Meanwhile, the comparison between MOCell and the improved non-dominated sorting genetic algorithm (NSGA-II) shows that MOCell has superior performance to NSGA-II regarding the problem. The field experiments were carried out to verify the accuracy of the results obtained by MOCell, the results were in good agreement with the measurement data. The proposed approach provides an effective tool for multi-objective optimization of coal combustion performance, whose feasibility and validity are experimental validated. A time period of less than 4 s was required for a run of optimization under a PC system, which is suitable for the online application.

Combining support vector regression with feature selection for multivariate calibration

Multivariate calibration is a classic problem in the analytical chemistry field and frequently solved by partial least squares (PLS) and artificial neural networks (ANNs) in the previous works. The spaciality of multivariate calibration is high dimensionality with small sample. Here, we apply support vector regression (SVR) as well as ANNs, and PLS to the multivariate calibration problem in the determination of the three aromatic amino acids (phenylalanine, tyrosine and tryptophan) in their mixtures by fluorescence spectroscopy. The results of the leave-one-out method show that SVR performs better than other methods, and appear to be one good method for this task. Furthermore, feature selection is performed for SVR to remove redundant features and a novel algorithm named Prediction RIsk based FEature selection for support vector Regression (PRIFER) is proposed. Results on the above multivariate calibration data set show that PRIFER is a powerful tool for solving the multivariate calibration problems.

Combining Support Vector Regression and Ant Colony Optimization to Reduce NOx Emissions in Coal-Fired Utility Boilers

Combustion optimization has recently demonstrated its potential to reduce NOx emissions in high capacity coal-fired utility boilers. In the present study, support vector regression (SVR), as well as artificial neural networks (ANN), was proposed to model the relationship between NOx emissions and operating parameters of a 300 MW coal-fired utility boiler. The predicted NOx emissions from the SVR model, by comparing with that of the ANN-based model, showed better agreement with the values obtained in the experimental tests on this boiler operated at different loads and various other operating parameters. The mean modeling error and the correlation factor were 1.58% and 0.94, respectively. Then, the combination of the SVR model with ant colony optimization (ACO) to reduce NOx emissions was presented in detail. The experimental results showed that the proposed approach can effectively reduce NOx emissions from the coal-fired utility boiler by about 18.69% (65 ppm). A time period of less than 6 min was required for NOx emissions modeling, and 2 min was required for a run of optimization under a PC system. The computing times are suitable for the online application of the proposed method to actual power plants.

Learning recurrent neural models with minimal complexity from neural tuning data

A learning algorithm for the estimation of the structure of nonlinear recurrent neural models from neural tuning data is presented. The proposed method combines support vector regression with additional constraints that result from a stability analysis of the dynamics of the fitted network model. The optimal solution can be determined from a single convex optimization problem that can be solved with semidefinite programming techniques. The method successfully estimates the feed-forward and the recurrent connectivity structure of neural field models using as data only examples of stable stationary solutions of the neural dynamics. The class of neural models that can be learned is quite general. The only a priori assumptions are the translation invariance and the smoothness of the feed-forward and recurrent spatial connectivity profile. The efficiency of the method is illustrated by comparing it with estimates based on radial basis function networks and support vector regression.