Classical Support Vector Regression Research Articles

A Method of Extracting the SWH Based on a Constituted Wave Slope Feature Vector (WSFV) from X-Band Marine Radar Images

The shadow statistical method (SSM) used for extracting the significant wave height (SWH) from X-band marine radar images was further investigated because of its advantage of not requiring an external reference for calibration. Currently, a fixed shadow segmentation threshold is utilized to extract the SWH from a radar image based on the SSM. However, the retrieval accuracy of the SWH is not ideal for low wind speeds since the echo intensity of sea waves rapidly decays over distance. In order to solve this problem, an adaptive shadow threshold, which varies with echo intensity over distance and can accurately divide the radar image into shadow and nonshadow areas, is adopted to calculate the wave slope (WS) based on the texture feature of the edge image. Instead of using the averaged WS, the wave slope feature vector (WSFV) is constructed for retrieving the SWH since the illumination ratio and the calculated WS in the azimuth are different for shore-based radar images. In this paper, the SWH is calculated based on the constructed WSFV and classical support vector regression (SVR) technology. The collected 222 sets of X-band marine radar images with an SWH range of 1.0∼3.5 m and an average wind speed range of 5∼10 m/s were utilized to verify the performance of the proposed approach. The buoy record, which was deployed during the experiment, was used as the ground truth. For the proposed approach, the mean bias (BIAS) and the mean absolute error (MAE) were 0.03 m and 0.14 m when the ratio of the training set to the test set was 1:1. Compared to the traditional SSM, the correlation coefficient (CC) of the proposed approach increased by 0.27, and the root mean square error (RMSE) decreased by 0.28 m.

Deformation prediction model of concrete face rockfill dams based on an improved random forest model

The unique structure and complex deformation characteristics of concrete face rockfill dams (CFRDs) create safety monitoring challenges. This study developed an improved random forest (IRF) model for dam health monitoring modeling by replacing the decision tree in the random forest (RF) model with a novel M5' model tree algorithm. The factors affecting dam deformation were preliminarily selected using the statistical model, and the grey relational degree theory was utilized to reduce the dimensions of model input variables. Finally, a deformation prediction model of CFRDs was established using the IRF model. The ten-fold cross-validation method was used to quantitatively analyze the parameters affecting the IRF algorithm. The performance of the established model was verified using data from three specific measurement points on the Jishixia dam and compared with other dam deformation prediction models. At point ES-10, the performance evaluation indices of the IRF model were superior to those of the M5' model tree and RF models and the classical support vector regression (SVR) and back propagation (BP) neural network models, indicating the satisfactory performance of the IRF model. The IRF model also outperformed the SVR and BP models in settlement prediction at points ES2-8 and ES4-10, demonstrating its strong anti-interference and generalization capabilities. This study has developed a novel method for forecasting and analyzing dam settlements with practical significance. Moreover, the established IRF model can also provide guidance for modeling health monitoring of other structures.

Newton-based approach to solving K-SVCR and Twin-KSVC multi-class classification in the primal space

Multi-class classification is an important problem in machine learning, which often occurs in the real world and is an ongoing research issue. Support vector classification-regression machine for k-class classification (K-SVCR) and twin k-class support vector classification (Twin-KSVC) are two novel machine learning methods for multi-class classification problems. This paper presents novel methods to solve the primal problems of K-SVCR and Twin-KSVC, known as NK-SVCR and NTW-KSVC, respectively. The proposed methods evaluate all training data into a “1-versus-1-versus-rest” structure, so it generates ternary outputs {−1,0,+1}. The primal problems are reformulated as unconstrained optimization problems so that the objective functions are only once differentiable, not twice, therefore an extension of the Newton-Armijo algorithm is adopted for finding their solution. To test the efficiency and validity of the proposed methods, we compare the classification accuracy and learning time of these methods with K-SVCR and Twin-KSVC on the United States Postal Service (USPS) handwriting digital data sets and several University of California Irvine (UCI) benchmark data sets. To analyze more in aspect of training time, we also compared all methods on the multi-class version of the Normally Distributed Clustered (NDC) database. To further analyze classification accuracy and learning time differences between the classifiers, the statistical Friedman’s test is used.

Automatic Estimation of Self-Reported Pain by Trajectory Analysis in the Manifold of Fixed Rank Positive Semi-Definite Matrices.

We propose an automatic method to estimate self-reported pain based on facial landmarks extracted from videos. For each video sequence, we decompose the face into four different regions and the pain intensity is measured by modeling the dynamics of facial movement using the landmarks of these regions. A formulation based on Gram matrices is used for representing the trajectory of landmarks on the Riemannian manifold of symmetric positive semi-definite matrices of fixed rank. A curve fitting algorithm is used to smooth the trajectories and temporal alignment is performed to compute the similarity between the trajectories on the manifold. A Support Vector Regression classifier is then trained to encode extracted trajectories into pain intensity levels consistent with self-reported pain intensity measurement. Finally, a late fusion of the estimation for each region is performed to obtain the final predicted pain level. The proposed approach is evaluated on two publicly available datasets, the UNBCMcMaster Shoulder Pain Archive and the Biovid Heat Pain dataset. We compared our method to the state-of-the-art on both datasets using different testing protocols, showing the competitiveness of the proposed approach.

The potential of a novel support vector machine trained with modified mayfly optimization algorithm for streamflow prediction

ABSTRACT This paper focuses on the development of a robust accurate streamflow prediction model by balancing the abilities of exploitation and exploration to find the best parameters of a machine learning model. To do so, the simulated annealing (SA) algorithm is integrated with the mayfly optimization algorithm (MOA) as SAMOA to determine the optimal hyper-parameters of support vector regression (SVR) to overcome the exploration weakness of the MOA method. The proposed method is compared with the classical SVR and hybrid SVR-MOA. To examine the accuracy of the selected methods, monthly hydroclimatic data from Jhelum River Basin is used to predict the monthly streamflow on the basis of RMSE, MAE, NSE, and R2 indices. Test results show that the SVR-SAMOA outperformed the SVR-MOA and SVR models. SVR-SAMOA reduced the prediction errors of the SVR-MOA and SVR models by decreasing the RMSE and the MSE from 21.4% to 14.7% and from 21.7% to 15.1%, respectively, in the test stage.

Enterprise financial management and trend prediction based on time series analytics and edge computing

In order to improve the prediction accuracy and reliability of long‐term financial trend, a financial time series prediction framework is proposed by combining empirical mode decomposition (EMD) and reduced support vector regression (SVR). Through the empirical mode decomposition, the framework can eliminate the disturbance caused by the multi band information of error sequence. The financial time series processed by EMD are used to train a reduced support vector regression model. Compared with classical support vector regression, the reduced support vector regression can discarded the samples, which would not become support vectors, to reduce the scale of problem. Therefore, the reduced support vector regression is much faster than support vector regression and is more suitable for edge computing. The experiments on benchmark dataset show that empirical mode decomposition plus reduced support vector regression can reach the close performance of empirical mode decomposition plus support vector regression, meanwhile running time only costs less than one fortieth of empirical mode decomposition plus support vector regression's.

Diagnosis of Sensor Faults in Hypersonic Vehicles Using Wavelet Packet Translation Based Support Vector Regressive Classifier

In this article, a fault diagnosis scheme based on the data-driven approach of hypersonic vehicles (HVs) is studied. First, the fault features are obtained by wavelet packet translation (WPT) processing. Second, an improved distance evaluation technique (DET) based on Spearman correlation analysis is used to select features and reduce dimensions. The designed enhanced WPT-DET (EWPT-DET) method can extract sensitive features based on self-defined attention coefficients. Third, the fault pattern recognition process is achieved by support vector regression (SVR) with genetic algorithm optimization. The SVR classifier with high dimensional linear fitting ability is very suitable for HVs' reaction control system with sensor faults. The method is further used in locating and diagnosing multisensor fusion faults. Moreover, the fault occurrence time of single sensor timing faults is judged. Finally, simulation studies are provided to illustrate the enhanced performance of the proposed approach.

Application of bagging ensemble model for predicting compressive strength of hollow concrete masonry prism

In the current research, a newly developed ensemble intelligent predictive model called Bagging Regression (BGR) is proposed to predict the compressive strength of a hollow concrete masonry prism ( f p ). A matrix of input combinations is constructed based on several predictive variables, including mortar compressive strength ( f m ), concrete block compressive strength ( f b ), and height to thickness ratio ( h/t ). Three modeling scenarios based on the different data divisions (i.e., 80–20%, 75–25%, and 70–30%) for training-testing phases are evaluated. The proposed model is validated against classical support vector regression (SVR) and decision tree regression (DTR) models using statistical indicators and graphical presentations. Results indicate the superiority of the BGR over the other models. In quantitative terms, BGR attains minimum root mean square error (RMSE = 1.51 MPa) using the data division scenario of 80–20% in the testing phase, while DTR and standalone SVR models offer RMSE = 2.55 and 2.33 MPa, respectively.

An Efficient Feature Weighting Method for Support Vector Regression

Support vector regression (SVR) is a powerful kernel-based method which has been successfully applied in regression problems. Regarding the feature-weighted SVR algorithms, its contribution to model output has been taken into account. However, the performance of the model is subject to the feature weights and the time consumption on training. In the paper, an efficient feature-weighted SVR is proposed. Firstly, the value constraint of each weight is obtained according to the maximal information coefficient which reveals the relationship between each input feature and output. Then, the constrained particle swarm optimization (PSO) algorithm is employed to optimize the feature weights and the hyperparameters simultaneously. Finally, the optimal weights are used to modify the kernel function. Simulation experiments were conducted on four synthetic datasets and seven real datasets by using the proposed model, classical SVR, and some state-of-the-art feature-weighted SVR models. The results show that the proposed method has the superior generalization ability within acceptable time.

Application of an artificial intelligence technique enhanced with intelligent water drops for monthly reference evapotranspiration estimation

Reference evapotranspiration (ET0) is one of the most important parameters, which is required in many fields such as hydrological, agricultural, and climatological studies. Therefore, its estimation via reliable and accurate techniques is a necessity. The present study aims to estimate the monthly ET0 time series of six stations located in Iran. To achieve this objective, gene expression programming (GEP) and support vector regression (SVR) were used as standalone models. A novel hybrid model was then introduced through coupling the classical SVR with an optimization algorithm, namely intelligent water drops (IWD) (i.e., SVR−IWD). Two various types of scenarios were considered, including the climatic data- and antecedent ET0 data-based patterns. In the climatic data-based models, the effective climatic parameters were recognized by using two pre-processing techniques consisting of τ Kendall and entropy. It is worthy to mention that developing the hybrid SVR-IWD model as well as utilizing the τ Kendall and entropy approaches to discern the most influential weather parameters on ET0 are the innovations of current research. The results illustrated that the applied pre-processing methods introduced different climatic inputs to feed the models. The overall results of present study revealed that the proposed hybrid SVR-IWD model outperformed the standalone SVR one under both the considered scenarios when estimating the monthly ET0. In addition to the mentioned models, two types of empirical equations were also used including the Hargreaves−Samani (H−S) and Priestley−Taylor (P−T) in their original and calibrated versions. It was concluded that the calibrated versions showed superior performances compared to their original ones.

A Comparative Chemometric Study for Quantitative Determination of Duloxetine Hydrochloride in the Presence of its Toxic Impurity 1-Naphthol

Background: Duloxetine hydrochloride (DUL) is a serotonin-norepinephrine reuptake inhibitor. It is used for treating depression and anxiety. It is available in the market as a capsule called Cymbatex®, which is used for the treatment of depression. 1-naphthol is a potential impurity of DUL. It is hepatotoxic to humans and has potential toxicity to freshwater fish. Objective: Duloxetine hydrochloride was determined in the presence of its toxic impurity 1-naphthol in raw material and in pharmaceutical dosage forms using three multivariate calibration chemometric methods. Methods: Classical Least Squares (CLS), Partial Least-Squares (PLS) and linear support vector regression (SVR) were developed using UV spectral data. The three methods were compared among each other and the advantages and disadvantages were discussed. For good results, a two-factor four-level experimental design was used, resulting in a training set of 16 mixtures containing different ratios of each component. The test set consisting of nine mixtures was necessary to test the ability of the proposed methods to predict DUL in the presence of its impurity, 1-naphthol. Results: The results show the success of the three developed methods to determine DUL in the presence of small levels of its toxic impurity with good accuracy and selectivity. The results of the dosage form were compared statistically to that of the reported HPLC method, with no significant difference in accuracy and precision. Conclusion: The suggested calibration models are suitable for routine analysis of the drug in bulk and pharmaceutical dosage forms. Compared to the CLS and PLS models, the SVR model gives the best results regarding the accuracy with a lower prediction error and better generalization ability. However, the CLS and PLS models are found to be simpler and faster in usage and management.

A New Diagnostic Model and Improved Prediction Algorithm for the Heights of Evaporation Ducts

This paper develops a new method for the diagnosis and prediction of the evaporation duct heights on the sea, which has certain reference significance for the study of the evaporation ducts. Based on traditional diagnostic and predictive models of evaporation duct heights, a new diagnostic model is proposed. By determining the overall Richardson number Rib, the Monin-Obukhov (M-O) length L and the wind speed characteristic parameter u*, temperature characteristic parameter θ* and humidity characteristic parameters q* are calculated, and then the evaporation duct height is diagnosed. Taking the diagnosed heights as a time series, and using the support vector regression (SVR) algorithm improved by a simulated annealing operator, then the time series is analyzed by taking three consecutive sample steps as input and the next sample step as output in order to develop an algorithm for predicting future heights. Finally, the prediction results are compared with those from the traditional auto-regressive (AR) algorithm and classical SVR algorithm to identify the advantages and disadvantages of the improved SVR algorithm. The results show that the root-mean-square error (RMSE) of the traditional AR, the classical SVR and the improved SVR algorithms is 0.60, 0.45 and 0.38, and the mean absolute percentage error (MAPE) of the three alorithms is 7.79%, 6.10% and 4.78%, respectively. The prediction error of the improved SVR algorithm is 37% less than that of the traditional AR algorithm and 15% less than that of the classical SVR algorithm, signifying an improvement in its prediction capability.

An Efficient SMO Algorithm for Solving Non-smooth Problem Arising in $$\varepsilon $$ ε -Insensitive Support Vector Regression

Classical support vector regression (C-SVR) is a powerful function approximation method, which is robust against noise and performs a good generalization, since it is formulated by a regularized error function employing the $$\varepsilon $$ -insensitiveness property. To exploit the kernel trick, C-SVR generally solves the Lagrangian dual problem. In this paper, an efficient sequential minimal optimization (SMO) algorithm with a novel easy to compute working set selection (WSS) based on the minimization of an upper bound on the difference between consecutive loss function values for solving a convex non-smooth dual optimization problem obtained by reformulating the dual problem of C-SVR with $$l_2$$ error loss function which is equivalent to the $$\varepsilon $$ -insensitive version of the LSSVR, is proposed. The asymptotic convergence to the optimum of the proposed SMO algorithm is also proved. This proposed SMO algorithm for solving non-smooth problem comprises both SMO algorithms for solving LSSVR and C-SVR. Indeed, it becomes equivalent to the SMO algorithm with second-order WSS for solving LSSVR when $$\varepsilon =0$$ . The proposed algorithm has the advantage of dealing with the optimization variables half the number of the ones in C-SVR, which results in lesser number of kernel related matrix evaluations than the standard SMO algorithm developed for C-SVR and improves the probability of the matrix outputs to have been precomputed and cached. Therefore, the proposed SMO algorithm results better training time than the standard SMO algorithm for solving C-SVR, especially with caching process. Moreover, the superiority of the proposed WSS over its first-order counterpart for solving the non-smooth optimization problem is presented.

Application of support vector regression integrated with firefly optimization algorithm for predicting global solar radiation

A fundamental factor for proficient designing of solar energy systems is providing precise estimations of the solar radiation. Global solar radiation (GSR) is a vital parameter for designing and operating solar energy systems. Because records of GSR are not available in many places, especially in developing countries, this research aims to model the GSR using support vector regression (SVR) in a hybrid manner that is integrated with the firefly Optimization algorithm (SVR-FFA). For this purpose, the daily meteorological parameters and GSR measured from beginning of 2011 to the end of 2013 at Tabriz synoptic station were utilized. For assessing the performance of the mentioned methods, different statistical indicators were implemented. For all of the defined predictive models with different combinations of meteorological parameters, the performance of the SVR-FFA hybrid model is better than the classical SVR, evidenced by the higher value of R (~0892-0.982 relative to ~0.891-0.977) and lower values of RMSE and MAE (~1.551-3.725vs.1.748-4.067 and ~0.911-2.862vs.1.103-2.742). As a remarkable point studied empirical equations had higher prediction errors comparing with the developed SVR-FFA models. Conclusively, the obtained results proved the high proficiencies of SVR-FFA method for predicting global solar radiation.

Notice of Retraction: Intelligent Transportation System in Macao Based on Deep Self-Coding Learning

As the basic content of the construction of the intelligent city, the application of intelligent transportation system plays an important role in improving the quality and safety of modern urban traffic operation. Intelligent transportation system combines information technology, data communication technology, electronic sensing technology, global positioning technology, geographic information system technology, computer processing technology, and system engineering technology together reasonably. The intelligent transportation system is a kind of real-time, accurate, efficient, and intelligent transportation management system. In this paper, we introduce the deep code learning technique and apply it to the Macao intelligent system. At the same time, we combine the deep belief network model and support vector regression classifier as the prediction model, and use the deep belief network model to learn traffic flow characteristics.

SamEn‐SVR: using sample entropy and support vector regression for bug number prediction

Monitoring and predicting the trend of bug number time series of a software system is crucial for both software project managers and software end-users. For software managers, accurate prediction of bug number of a software system will assist them in making timely decisions, such as effort investment and resource allocation. For software end-users, knowing possible bug number of their systems ahead will enable them to adopt timely actions in coping with the loss caused by possible system failures. This study proposes an approach called SamEn-SVR to combine sample entropy and support vector regression (SVR) to predict software bug number using time series analysis. The basic idea is to use template vectors with the smallest complexity as input vectors for SVR classifiers to ensure predictability of time series. By using Mozilla Firefox bug data, we conduct extensive experiments to compare the proposed approach and state-of-the-art techniques including auto-regressive integrated moving average (ARIMA), X12 enhanced ARIMA and polynomial regression to predict bug number time series. Experimental results demonstrate that the proposed SamEn-SVR approach outperforms state-of-the-art techniques in bug number prediction.

A Feature-Weighted SVR Method Based on Kernel Space Feature

Support Vector Regression (SVR), which converts the original low-dimensional problem to a high-dimensional kernel space linear problem by introducing kernel functions, has been successfully applied in system modeling. Regarding the classical SVR algorithm, the value of the features has been taken into account, while its contribution to the model output is omitted. Therefore, the construction of the kernel space may not be reasonable. In the paper, a Feature-Weighted SVR (FW-SVR) is presented. The range of the feature is matched with its contribution by properly assigning the weight of the input features in data pre-processing. FW-SVR optimizes the distribution of the sample points in the kernel space to make the minimizing of the structural risk more reasonable. Four synthetic datasets and seven real datasets are applied. A superior generalization ability is obtained by the proposed method.

PTSVRs: Regression models via projection twin support vector machine

Taking motivation from projection twin support vector machine (PTSVM) formulation for recognition, this paper proposes two novel projection twin support vector regression (PTSVR) models, called pair-shifted PTSVR (PPTSVR) and single-shifted PTSVR (SPTSVR), respectively. PTSVRs construct indirectly the target regressor by two functions (hyperplanes) obtained from two smaller-sized quadratic programming problems (QPPs), in which each normal direction makes the within-class variance of the projection of shifted set (or original set) be minimized and the projected center be at a distance of at least 1 from the projection of the other shifted set. As other twin support vector machine (TWSVM) models, the learning speed of PTSVRs is faster than classical support vector regression (SVR) since each of their QPP has only half size. Experimental results on several synthetic as well as benchmark datasets indicate the significant advantage of PPTSVR and SPTSVR in the generalization performance.

Adaptive Morphological Feature Extraction and Support Vector Regressive Classification for Bearing Fault Diagnosis

Numerous studies on fault diagnosis have been conducted in recent years because the timely and correct detection of machine fault effectively minimizes the damage resulting in the unexpected breakdown of machineries. The mathematical morphological analysis has been performed to denoise raw signal. However, the improper choice of the length of the structure element (SE) will substantially influence the effectiveness of fault feature extraction. Moreover, the classification of fault type is a significant step in intelligent fault diagnosis, and many techniques have already been developed, such as support vector machine (SVM). This study proposes an intelligent fault diagnosis strategy that combines the extraction of morphological feature and support vector regression (SVR) classifier. The vibration signal is first processed using various scales of morphological analysis, where the length of SE is determined adaptively. Thereafter, nine statistical features are extracted from the processed signal. Lastly, an SVR classifier is used to identify the health condition of the machinery. The effectiveness of the proposed scheme is validated using the data set from a bearing test rig. Results show the high accuracy of the proposed method despite the influence of noise.

Pairing support vector algorithm for data regression

This study introduces a novel and efficient pairing support vector algorithm for data regression, called PSVR. The introduced PSVR approach aims at estimating an insensitive zone of flexible shape to tightly fit the training samples. Motivated by the twin support vector regression machine (TSVR), the up- and down-bound functions are estimated by solving two smaller quadratic programming problems (QPPs) rather than a single larger QPP; and thus its training speed is more rapid than the classical support vector regression machine (ε-SVR). The benefits of the introduced PSVR over TSVR are the enhanced sparsity and generalization capability. By introducing the concept of the insensitive zone, the number of basis functions required for determining the regression model is significantly reduced, resulting in better sparsity and more rapid prediction speed than TSVR. Besides, the introduced approach embodies the essence of statistical learning theory by adopting the principle of structural risk minimization, resulting in better generalization capability than TSVR. Moreover, the up- and down-bound functions of the insensitive zone successfully reflect the characteristics of data distribution, thus facilitating automatic estimation of the conditional mean and predictive variance simultaneously. The experimental results verify the introduced algorithm not only has more rapid learning speed, but also more preferable generalization capability and a more rapid prediction speed. On the average, PSVR yields satisfactory regression performance with RMSE of 0.2549 on benchmark data sets, surpassing existing methods. The training speed of PSVR is over four times faster than standard ε-SVR. Further, PSVR achieves significant improvements in sparsity of 59.95% over the TSVR.