Support Vector Machine Regression Model Research Articles

Wavelet-Based Estimation of Generalized Discriminant Functions

In this work we propose a wavelet-based classifier method for binary classification. Basically, based on a training data set, we provide a classifier rule with minimum mean square error. Under mild assumptions, we present asymptotic results that provide the rates of convergence of our method compared to the Bayes classifier, ensuring universal consistency and strong universal consistency. Furthermore, in order to evaluate the performance of the proposed methodology for finite samples, we illustrate the approach using Monte Carlo simulations and real data set applications. The performance of the proposed methodology is compared with other classification methods widely used in the literature: support vector machine and logistic regression model. Numerical results showed a very competitive performance of the new wavelet-based classifier.

Estimating significant wave height from SAR imagery based on an SVM regression model

A new method for estimating significant wave height (SWH) from advanced synthetic aperture radar (ASAR) wave mode data based on a support vector machine (SVM) regression model is presented. The model is established based on a nonlinear relationship between σ0, the variance of the normalized SAR image, SAR image spectrum spectral decomposition parameters and ocean wave SWH. The feature parameters of the SAR images are the input parameters of the SVM regression model, and the SWH provided by the European Centre for Medium-range Weather Forecasts (ECMWF) is the output parameter. On the basis of ASAR matching data set, a particle swarm optimization (PSO) algorithm is used to optimize the input kernel parameters of the SVM regression model and to establish the SVM model. The SWH estimation results yielded by this model are compared with the ECMWF reanalysis data and the buoy data. The RMSE values of the SWH are 0.34 and 0.48 m, and the correlation coefficient is 0.94 and 0.81, respectively. The results show that the SVM regression model is an effective method for estimating the SWH from the SAR data. The advantage of this model is that SAR data may serve as an independent data source for retrieving the SWH, which can avoid the complicated solution process associated with wave spectra.

Random Forest Approach to QSPR Study of Fluorescence Properties Combining Quantum Chemical Descriptors and Solvent Conditions.

The Quantitative Structure - Property Relationship (QSPR) approach was performed to study the fluorescence absorption wavelengths and emission wavelengths of 413 fluorescent dyes in different solvent conditions. The dyes included the chromophore derivatives of cyanine, xanthene, coumarin, pyrene, naphthalene, anthracene and etc., with the wavelength ranging from 250nm to 800nm. An ensemble method, random forest (RF), was employed to construct nonlinear prediction models compared with the results of linear partial least squares and nonlinear support vector machine regression models. Quantum chemical descriptors derived from density functional theory method and solvent information were also used by constructing models. The best prediction results were obtained from RF model, with the squared correlation coefficients [Formula: see text] of 0.940 and 0.905 for λabs and λem, respectively. The descriptors used in the models were discussed in detail in this report by comparing the feature importance of RF.

Development and Implementation of Support Vector Machine Regression Surrogate Models for Predicting Groundwater Pumping-Induced Saltwater Intrusion into Coastal Aquifers

Predicting the extent of saltwater intrusion (SWI) into coastal aquifers in response to changing pumping patterns is a prerequisite of any groundwater management framework. This study investigates the feasibility of using support vector machine regression (SVMr), an innovative artificial intelligence-based machine learning algorithm for predicting salinity concentrations at selected monitoring wells in an illustrative aquifer under variable groundwater pumping conditions. For evaluation purpose, the prediction results of SVMr are compared with well-established genetic programming (GP) based surrogate models. SVMr and GP models are trained and validated using identical sets of input (pumping) and output (salinity concentration) datasets. The trained and validated models are then used to predict salinity concentrations at specified monitoring wells in response to new pumping datasets. Prediction capabilities of the two learning machines are evaluated using different proficiency measures to ensure their practicality and generalisation ability. The performance evaluation results suggest that the prediction capability of SVMr is superior to GP models. Also, a sensitivity analysis methodology is proposed for assessing the impact of pumping rates on salt concentrations at monitoring locations. This sensitivity analysis provides a subset of most influential pumping rates, which is used to construct new SVMr surrogate models with improved predictive capabilities. The improved prediction capability and the generalisation ability of the SVMr models together with the ability to improve the accuracy of prediction by refining the input set for training makes the use of proposed SVMr models more attractive. Prediction models with more accurate prediction capability makes it potentially very useful for designing large scale coastal aquifer management strategies.

Nondestructive quality evaluation of banana slices during microwave vacuum drying using spectral and imaging techniques

ABSTRACTMicrowave vacuum drying of banana slices was investigated in this study. Near-infrared hyperspectral imaging (NIR-HSI, 950–1650 nm with 7 nm increments) and chemometrics were used to predict moisture content, hardness, and fracturability of banana slices during the entire drying process. A computer vision system was also used to evaluate browning index (BI) of slices at different drying stages; dark brown spots observed in slices were subjected to over 21-min total microwave heating. In addition, thermal imaging results showed that higher temperature occurred in the central part of banana slices. Calibration models were developed based on partial least square regression and support vector machine regression (SVM-R) using mean spectra with standard normal variate pretreatment. The results showed that SVM-R models performed better in the prediction of the three quality attributes with of 0.996, 0.927, and 0.961, respectively. The moisture distribution maps were generated using NIR-HSI; uniformly distributed moisture patterns were observed in each drying batch with small variations observed within individual slices. This study demonstrated the potential of spectral and imaging techniques for nondestructive quality evaluation of food products during drying processes.

Least-squares support vector machine and successive projection algorithm for quantitative analysis of cotton-polyester textile by near infrared spectroscopy

The application of near infrared spectroscopy for quantitative analysis of cotton-polyester textile was investigated in the present work. A total of 214 cotton-polyester fabric samples, covering the range from 0% to 100% cotton were measured and analyzed. Partial least squares and least-squares support vector machine models with all variables as input data were established. Furthermore, successive projection algorithm was used to select effective wavelengths and establish the successive projection algorithm-least-squares support vector machine models, with the comparison of two other effective wavelength selection methods: loading weights analysis and regression coefficient analysis. The calibration and validation results show that the successive projection algorithm-least-squares support vector machine model outperformed not only the partial least squares and least-squares support vector machine models with all variables as inputs, but also the least-squares support vector machine models with loading weights analysis and regression coefficient analysis effective wavelength selection. The root mean squared error of calibration and root mean squared error of prediction values of the successive projection algorithm-least-squares support vector machine regression model with the optimal performance were 0.77% and 1.17%, respectively. The overall results demonstrated that near infrared spectroscopy combined with least-squares support vector machine and successive projection algorithm could provide a simple, rapid, economical and non-destructive method for determining the composition of cotton-polyester textiles.

Predicting post-stroke activities of daily living through a machine learning-based approach on initiating rehabilitation

ObjectivesPrediction of activities of daily living (ADL) is crucial for optimized care of post-stroke patients. However, no suitably-validated and practical models are currently available in clinical practice. MethodsParticipants of a Post-acute Care-Cerebrovascular Diseases (PAC-CVD) program from a reference hospital in Taiwan between 2014 and 2016 were enrolled in this study. Based on 15 rehabilitation assessments, machine learning (ML) methods, namely logistic regression (LR), support vector machine (SVM), and random forest (RF), were used to predict the Barthel index (BI) status at discharge. Furthermore, SVM and linear regression were used to predict the actual BI scores at discharge. ResultsA total of 313 individuals (men: 208; women: 105) were enrolled in the study. All the classification models outperformed single assessments in predicting the BI statuses of the patients at discharge. The performance of the LR and RF algorithms was higher (area under ROC curve (AUC): 0.79) than that of SVM algorithm (AUC: 0.77). In addition, the mean absolute errors of both SVM and linear regression models in predicting the actual BI score at discharge were 9.86 and 9.95, respectively. ConclusionsThe proposed ML-based method provides a promising and practical computer-assisted decision making tool for predicting ADL in clinical practice.

Predicting Pork Color Scores Using Computer Vision and Support Vector Machine Technology

The objective of this study was to investigate the ability of image color features to predict subjective pork color scores. Subjective and instrumental color were assessed on the bloomed, cross-sectional surface of pork longissimus thoracis et lumborum chops. Images of pork chop samples were acquired using a computer vision system, and 18 image color features (mean and standard deviation of R, G, B, H, S, I, L*, a*, b*) were extracted for inclusion in partial least squares (PLS) and support vector machine (SVM) regression models. For color scores 2, 3, 4, and 5, the accuracies were 50.4, 75.9, 72.4, and 47.3% classified correctly by PLS, respectively, and 70.7, 72.8, 76.7, and 69.7% by SVM, respectively. The overall prediction accuracies of 2 models for pork color scores were 68.3% for PLS and 73.4% for SVM. There was minimal major misclassification of samples (< 0.5%). Image color features isolated through the development of PLS and SVM models, particularly SVM, show potential as a method to predict pork color scores.

Study on Driving Decision-Making Mechanism of Autonomous Vehicle Based on an Optimized Support Vector Machine Regression

Driving Decision-making Mechanism (DDM) is identified as the key technology to ensure the driving safety of autonomous vehicle, which is mainly influenced by vehicle states and road conditions. However, previous studies have seldom considered road conditions and their coupled effects on driving decisions. Therefore, road conditions are introduced into DDM in this paper, and are based on a Support Vector Machine Regression (SVR) model, which is optimized by a weighted hybrid kernel function and a Particle Swarm Optimization (PSO) algorithm, this study designs a DDM for autonomous vehicle. Then, the SVR model with RBF (Radial Basis Function) kernel function and BP (Back Propagation) neural network model are tested to validate the accuracy of the optimized SVR model. The results show that the optimized SVR model has the best performance than other two models. Finally, the effects of road conditions on driving decisions are analyzed quantitatively by comparing the reasoning results of DDM with different reference index combinations, and by the sensitivity analysis of DDM with added road conditions. The results demonstrate the significant improvement in the performance of DDM with added road conditions. It also shows that road conditions have the greatest influence on driving decisions at low traffic density, among those, the most influential is road visibility, then followed by adhesion coefficient, road curvature and road slope, while at high traffic density, they have almost no influence on driving decisions.

Learning epistatic interactions from sequence-activity data to predict enantioselectivity.

Enzymes with a high selectivity are desirable for improving economics of chemical synthesis of enantiopure compounds. To improve enzyme selectivity mutations are often introduced near the catalytic active site. In this compact environment epistatic interactions between residues, where contributions to selectivity are non-additive, play a significant role in determining the degree of selectivity. Using support vector machine regression models we map mutations to the experimentally characterised enantioselectivities for a set of 136 variants of the epoxide hydrolase from the fungus Aspergillus niger (AnEH). We investigate whether the influence a mutation has on enzyme selectivity can be accurately predicted through linear models, and whether prediction accuracy can be improved using higher-order counterparts. Comparing linear and polynomial degree = 2 models, mean Pearson coefficients (r) from [Formula: see text]-fold cross-validation increase from 0.84 to 0.91 respectively. Equivalent models tested on interaction-minimised sequences achieve values of [Formula: see text] and [Formula: see text]. As expected, testing on a simulated control data set with no interactions results in no significant improvements from higher-order models. Additional experimentally derived AnEH mutants are tested with linear and polynomial degree = 2 models, with values increasing from [Formula: see text] to [Formula: see text] respectively. The study demonstrates that linear models perform well, however the representation of epistatic interactions in predictive models improves identification of selectivity-enhancing mutations. The improvement is attributed to higher-order kernel functions that represent epistatic interactions between residues.

Automatic Prediction of Health Status Using Smartphone-Derived Behavior Profiles.

Current methods of assessing the affect patients' health has on their daily lives are extremely limited. The aim of this paper is to develop a sensor-based approach to health status measurement in order to objectively measure health status. Techniques to generate human behavior profiles, derived from the smartphone accelerometer and gyroscope sensors, are proposed. Experiments, using SVM regression models, are then conducted in order to evaluate the use of the proposed behavior profiles as a predictor of health status. Experiments were conducted on data from 171 participants, with an average of 114h of data per participant. Regression models were trained and tested on the 10 SF-36 self-ratings. Results showed that the eight individual SF-36 scales and two component scores could be predicted with an average correlation of 0.683 and 0.698, respectively. General health was predicted with an average correlation of 0.752. Research shows that the clinically important difference for SF-36 self-ratings are approximately 10 points. Health status prediction errors in this study were 11.7 points on average. While the problem has not been fully solved, this paper presents a hugely promising direction for health status prediction. Using the proposed techniques, health status could be measured using unobtrusive, inexpensive, and already available hardware. It could provide a means for clinicians to accurately and objectively assess the daily life benefits of treatments on an individual patient basis.

Supervised Machine Learning for Estimation of Target Aspect Angle From Bistatic Acoustic Scattering

When an aspect-dependent target is insonified by an acoustic source, distinct features are produced in the resulting bistatic scattered field. These features change as the aspect between the source and the target is varied. This paper describes the use of these features for estimation of the target aspect angle using data collected by an autonomous underwater vehicle (AUV). An experiment was conducted in November 2014 in Massachusetts Bay to collect data using a ship-based acoustic source producing 7–9-kHz linear frequency modulation (LFM) chirps insonifying a steel pipe. The true target orientation was unknown, as the target was dropped from the ship with no rotation control. The AUV Unicorn , fitted with a 16-element nose array, was deployed in data collection behaviors around the target, and the ship was moved to create two target aspects. A support vector machine regression model was trained using simulated scattering bistatic field data. This model was then used to estimate the target aspect angle from the data collected during the experiment. The difference between the estimates was consistent with experimental observations of relative source positioning. The simulation-based model appeared successful in estimating the target aspect angle despite uncertainties in target and source location and mismatch between true environment and simulation parameters.

Variability aware transistor stack based regression surrogate models for accurate and efficient statistical leakage estimation

In this paper, we present highly accurate and very efficient stack based surrogate models for standby (idle-time) statistical leakage estimation of CMOS circuits using sampling based methods. Our aim is to replace SPICE simulation with our proposed surrogate models, which can be used to evaluate samples generated by sampling methods in variation space. Our methodology initiates by first characterizing the leakage of basic transistor stacks and then provides estimate of the leakage of basic gates based on these stacks. Transistor stacks are extracted across various standard cells, which are used to estimate leakage of CMOS gates with different input vector combinations. We develop Support Vector Machine (SVM) regression surrogate models and characterize the transistor stacks of CMOS gates, while accounting the combined effect of process variations in transistor length (L), threshold voltage (Vth), oxide thickness (Tox), supply voltage (0.6V−1.2V), temperature (0°C−100°C) and width (28nm−200nm), all scalable at the same time. For gates containing parallel transistor stacks, we merge the parallel transistors having identical inputs, which in turn allows us to use precharacterized basic stacks for leakage calculation, thus avoiding generation of new models. Our experiments illustrate that we only require 30 and 26 stack models to predict the subthreshold and gate tunneling leakage of 20 different gates across 176 input combinations, instead of characterizing leakage model for each input vector separately. SVM regression models generated in our approach have the ability to predict the leakage with maximum average error of 2.7% in mean (μ) and maximum average error of 3.1% in standard deviation (σ), both for OAI22 gate. Our results establish that there is on an average 10× improvement in runtime while estimating the μ and σ of leakage of a gate within 10000 Monte Carlo simulation loop. Our approaches also result into a maximum of 221× runtime improvement for C6288 ISCAS'85 benchmark circuit. We further develop Sparse SVM models using Support Vector spectrum pruning method, which reduces the runtime of the regression models with negligible increase in the error. Runtime efficiency of 17× and 323× is achieved on standard cell library and C6288 benchmark circuit respectively using Sparse SVM models. Our models outperform previous models based on analytical equations and Artificial Neural Network in terms of accuracy.

EMPIRICAL EVALUATION OF MACHINE LEARNING ALGORITHMS FOR AUTOMATIC DOCUMENT CLASSIFICATION

Automatic document classification process is the important area of research in the field of Text Mining(TM). Text mining is the process of discovering the interesting pattern or knowledge from huge amount of data. The document classification process used in many domains. Here, to take the classification process is apply SMS spam classification. The bench marked dataset is used and the same data set is processed in various ML algorithms of Naive Bayes, Support Vector Machine, Decision Tree and Logistic Regression model. In this paper evaluates the results of various machine learning algorithms for automatic document classification in SMS spam classification.

Prediction of Mind-Wandering with Electroencephalogram and Non-linear Regression Modeling.

Mind-wandering (MW), task-unrelated thought, has been examined by researchers in an increasing number of articles using models to predict whether subjects are in MW, using numerous physiological variables. However, these models are not applicable in general situations. Moreover, they output only binary classification. The current study suggests that the combination of electroencephalogram (EEG) variables and non-linear regression modeling can be a good indicator of MW intensity. We recorded EEGs of 50 subjects during the performance of a Sustained Attention to Response Task, including a thought sampling probe that inquired the focus of attention. We calculated the power and coherence value and prepared 35 patterns of variable combinations and applied Support Vector machine Regression (SVR) to them. Finally, we chose four SVR models: two of them non-linear models and the others linear models; two of the four models are composed of a limited number of electrodes to satisfy model usefulness. Examination using the held-out data indicated that all models had robust predictive precision and provided significantly better estimations than a linear regression model using single electrode EEG variables. Furthermore, in limited electrode condition, non-linear SVR model showed significantly better precision than linear SVR model. The method proposed in this study helps investigations into MW in various little-examined situations. Further, by measuring MW with a high temporal resolution EEG, unclear aspects of MW, such as time series variation, are expected to be revealed. Furthermore, our suggestion that a few electrodes can also predict MW contributes to the development of neuro-feedback studies.

Modeling and Prediction of the Air Permeability of Fabrics Based on the Support Vector Machine

Abstract Air permeability is an important index of textiles and has a significant effect on the quality of the fabric. Thus, the air permeability measured by the air permeability tester plays an important role in the textile industry. However, the accuracy of the tester is determined by prediction model precision. A new prediction model for air permeability based on the support vector machine (SVM) was presented in the paper, which can improve the measurement precision and stability of the tester. Three groups of measured open data were used to verify the validity of the model. For each group, 27 samples were used as training data to create the support vector machine regression (SVR) model. The mean square error (MSE) and the correlation coefficient (R) were introduced to evaluate the model. For the selected 3#, 4#, and 6# nozzles, the R values of the SVM regression model (Model 1) were 0.9988, 0.9992, and 0.9995, respectively, whereas for the traditional power function (Model 2), those were 0.9938, 0.9951, and 0.9898, respectively, indicating Model 1 has better correlation and agreement. In addition, the MSE values of Model 1 were 2.3978, 1.5186, and 0.9314, respectively, whereas for Model 2, those were 11.2558, 9.2485, and 36.5991, respectively. As a result, the performance of Model 1 is superior to Model 2, and Model 1 has higher stability and generalization ability, meanwhile demonstrating that Model 1 is practical.

Influence Analysis and Prediction of ESDD and NSDD Based on Random Forests

Equivalent salt deposit density (ESDD) and non-soluble deposit density (NSDD) measurements are a basic requirement of power systems. In order to predict the site pollution severity (SPS) of insulators, a new method based on random forests (RFs) is proposed. Using mutual information (MI) theory and RFs, the weights of factors related to the SPS of insulators are analyzed. The samples of contaminated insulators are extracted from the transmission lines of high voltage alternating current (HVAC) and high voltage direct current transmission (HVDC). The regression models of RFs and support vector machines (SVM) are constructed and compared, which helps to support the lack of information in predicting NSDD in previous works. The results are as follows: according to the mean decrease accuracy (MDA), mean decrease Gini, (MDG), and MI, the types of the insulators (including surface area, surface orientation, and total length) as well as the hydrophobicity are the main factors affecting both ESDD and NSDD. Compared with NSDD, the electrical parameters have a significant effect on ESDD. For the influence factors of ESDD, the weights of the insulator type, hydrophobicity, and meteorological factors are 52.94%, 6.35%, and 21.88%, respectively. For the influence factors of NSDD, the weights of the insulator type, hydrophobicity, and meteorological factors are 55.37%, 11.04%, and 14.26%, respectively. The influence voltage level (vl), voltage type (vt), polarity/phases (pp) exerted on ESDD are 1.5 times, 3 times, and 4.5 times of NSDD, respectively. The influence that distance from the coastline (d), wind velocity (wv), and rainfall (rf) exert on NSDD are 1.5 times, 2 times, and 2.5 times that of ESDD, respectively. Compared with the natural contamination test and the SVM regression model, the RFs regression model can effectively predict the contamination degree of insulators, and the relative error of the predicted ESDD and NSDD is 8.31% and 9.62%, respectively.

ADMET Evaluation in Drug Discovery. Part 17: Development of Quantitative and Qualitative Prediction Models for Chemical-Induced Respiratory Toxicity.

As a dangerous end point, respiratory toxicity can cause serious adverse health effects and even death. Meanwhile, it is a common and traditional issue in occupational and environmental protection. Pharmaceutical and chemical industries have a strong urge to develop precise and convenient computational tools to evaluate the respiratory toxicity of compounds as early as possible. Most of the reported theoretical models were developed based on the respiratory toxicity data sets with one single symptom, such as respiratory sensitization, and therefore these models may not afford reliable predictions for toxic compounds with other respiratory symptoms, such as pneumonia or rhinitis. Here, based on a diverse data set of mouse intraperitoneal respiratory toxicity characterized by multiple symptoms, a number of quantitative and qualitative predictions models with high reliability were developed by machine learning approaches. First, a four-tier dimension reduction strategy was employed to find an optimal set of 20 molecular descriptors for model building. Then, six machine learning approaches were used to develop the prediction models, including relevance vector machine (RVM), support vector machine (SVM), regularized random forest (RRF), extreme gradient boosting (XGBoost), naïve Bayes (NB), and linear discriminant analysis (LDA). Among all of the models, the SVM regression model shows the most accurate quantitative predictions for the test set (q2ext = 0.707), and the XGBoost classification model achieves the most accurate qualitative predictions for the test set (MCC of 0.644, AUC of 0.893, and global accuracy of 82.62%). The application domains were analyzed, and all of the tested compounds fall within the application domain coverage. We also examined the structural features of the compounds and important fragments with large prediction errors. In conclusion, the SVM regression model and the XGBoost classification model can be employed as accurate prediction tools for respiratory toxicity.

Predicting the Enzymatic Hydrolysis Half-lives of New Chemicals Using Support Vector Regression Models Based on Stepwise Feature Elimination.

The enzymatic hydrolysis of chemicals, which is important for in vitro drug metabolism assays, is an important indicator of drug stability profiles during drug discovery and development. Herein, we employed a stepwise feature elimination (SFE) method with nonlinear support vector machine regression (SVR) models to predict the in vitro half-lives in human plasma/blood of various esters. The SVR model was developed using public databases and literature-reported data on the half-lives of esters in human plasma/blood. In particular, the SFE method was developed to prevent over fitting and under fitting in the nonlinear model, and it provided a novel and efficient method of realizing feature combinations and selections to enhance the prediction accuracy. Our final developed model with 24 features effectively predicted an external validation set using the time-split method and presented reasonably good R2 values (0.6) and also predicted two completely independent validation datasets with R2 values of 0.62 and 0.54; thus, this model performed much better than other prediction models.

Quantitative Analysis of Peanut Oil Adulteration Based on Data Fusion of Multi-source Spectra

This study aimed to analyze peanut oil adulteration based on chemometrics combined Raman and near-infrared (NIR) spectrum. The spectral data of 134 adulterated oil samples were collected by laser Raman and NIR spectrometer. The Raman spectra data and NIR were preprocessed respectively by different preprocessing approaches. The backward interval partial least squares (BiPLS) method was applied to extract the featured wavelengths. Based on the full spectrum and the characteristic wavelengths, adulteration quantity prediction models were established by the support vector machine regression (SVR) method. According to the analysis, the SVR model could predict the adulteration content in the peanut oil. Furthermore, the correlation coefficient R was greater than 0.97, and the mean square error (MSE) was smaller than 3.2E-4. The SVR model had advantageous properties such as strong generalization ability and good prediction accuracy. The results showed that Raman and NIR fusion analysis was effective in the quantitative analysis of the adulteration in peanut oil. Multi-spectral analysis for edible oil adulteration is an important field of study.