Nonlinear Support Vector Machine Model Research Articles

Rapid monitoring of the spatial distribution of soil organic matter using unmanned aerial vehicle imaging spectroscopy

ABSTRACT An existing technique for quickly obtaining the spatial distribution data of soil organic matter (SOM) involves combining an unmanned aerial vehicle (UAV) platform and imaging spectroscopy technology; this technique is suitable for precision agricultural management and real-time monitoring. In this study, the average spectral data were first extracted via circular regions of interest (ROIs) of different radii and modelled using the measured SOM to determine the optimal extraction range for the spectral data. Partial least-squares regression (PLSR) and support vector machine (SVM) models were constructed based on the raw spectra, standard normal transform spectra, multiple scattering-corrected spectra, and first-order differential spectra. The difference index, ratio index, and normalized index were calculated and analysed for their correlation with SOM content, and the spectral indices were screened to establish the SOM prediction model. Finally, the accuracy of the model was evaluated, and we selected the optimal model to map the spatial distribution of SOM. The results of our study show that 1) the radius of the ROI significantly affects the model’s accuracy; 2) the accuracy of the nonlinear SVM model is much higher than that of the linear PLSR model; 3) the prediction model based on spectral indices is better than full-band model. This study can serve as a model for the application of UAV imaging spectroscopy technology to field-scale SOM estimations and rapid monitoring, as well as provide technical support for precision agriculture and climate change studies.

ML-based regionalization of climate variables to forecast seasonal precipitation for water resources management

Numerous dams and reservoirs have been constructed in South Korea, considering the distribution of seasonal precipitation which highly deviates from the actual one with high precipitation amount in summer and very low amount in other seasons. These water-related structures should be properly managed in order to meet seasonal demands of water resources wherein the forecasting of seasonal precipitation plays a critical role. However, owing to the impact of diverse complex weather systems, seasonal precipitation forecasting has been a challenging task. The current study proposes a novel procedure for forecasting seasonal precipitation by: (1) regionalizing the influential climate variables to the seasonal precipitation with k-means clustering; (2) extracting the features from the regionalized climate variables with machine learning-based algorithms such as principal component analysis (PCA), independent component analysis (ICA), and Autoencoder; and (3) finally regressing the extracted features with one linear model of generalized linear model (GLM) and another nonlinear model of support vector machine (SVM). Two globally gridded climate variables-mean sea level pressure (MSLP) and sea surface temperature (SST)-were teleconnected with the seasonal precipitation of South Korea, denoted as accumulated seasonal precipitation (ASP). Results indicated that k-means clustering successfully regionalized the highly correlated climate variables with the ASP, and all three extraction algorithms-PCA, ICA, and Autoencoder-combined with the GLM and SVM models presented their superiority in different seasons. In particular, the PCA combined with the linear GLM model performed better, and the Autoencoder combined with the nonlinear SVM model did better. It can be concluded that the proposed forecasting procedure of the seasonal precipitation, combined with several ML-based algorithms, can be a good alternative.

DESIGN OF KIP KULIAH SELECTION SYSTEM AND RECIPIENT DETERMINATION USING SUPPORT VECTOR MACHINE (SVM)

KIP Kuliah is tuition assistance from the government for high school graduates or equivalent with good academic potential but has economic limitations. In recent years it has been seen that the Indonesian government has always tried to increase the quota for KIP Kuliah recipients. In this study, the Support Vector Machine (SVM) method was applied to create a system for selecting and determining KIP Kuliah recipients. To obtain the best model to be used in the system, the training and testing data are divided into three data distribution schemes, namely 60/40, 70/30, and 80/20. After the training and testing process was carried out using the SVM method with various parameter variations, then the best accuracy rate of 94.59% is obtained in the 80/20 data sharing scheme for the nonlinear SVM model with the RBF kernel. With this system, it is hoped that the KIP Kuliah selection process at the tertiary level can run effectively, efficiently and the results of the determination are more targeted.

Real time Drowsy Driver Detection using Polynomial Kernel based Support Vector Machine

Abstract: Casualty increases from road accidents day by day. There are so many reasons that accident causes and mostly due to human errors. Driver drowsiness is one of them. A small drowsiness may turn it into a big accident that resulted heavy casualties. If any of the system automatically detects the driver’s drowsiness and alert at real time may secure many lives. Drowsiness can be recognized by different situations such as by opening full mouth, by closing both the eyes and a combination of both. This may advised not to drive at drowsy state. There are various techniques through which drowsiness can be detected at real time but accuracy matters. OpenCV is a highly utilized open source computer vision library through which facial features can be recognized effectively. Polynomial kernel based support vector machine (SVM) is an advanced classification technique through which drowsiness can be classified from face. SVM is advanced machine learning approach through which linear and non-linear data can be classified with higher level of accuracy. System pertained 96.17 % of accuracy. Polynomial kernel is useful for non-linear data separation. Here system classifies the expressional features of face and result accordingly for drowsiness detection. Keywords: Support Vector Machine (SVM), OpenCV, Machine Learning, Non-Linear SVM Model, Drowsiness Detection, Face Detection, Computer Vision.

High accuracy analysis of fiber-optic laser-induced breakdown spectroscopy by using multivariate regression analytical methods

Fiber-optic laser-induced breakdown spectroscopy (FO-LIBS), which delivers the laser energy through an optical fiber cable, is more suitable for remote analysis and applying in complex environment than traditional laser-induced breakdown spectroscopy (LIBS). However, since the laser fluence is limited by optical fiber loss and attenuation, FO-LIBS suffers more from spectral interference, matrix effect, and self-absorption effect. In this work, three multivariate quantitative analytical methods, including two linear models partial least squares (PLS) and sparse partial least squares (SPLS), one nonlinear model support vector machine (SVM), were utilized to carry out the quantitative analysis of four trace metal elements (Manganese (Mn), Chromium (Cr), Nickel (Ni), and Titanium (Ti)) in pig iron. And the quantitative analytical ability of linear model and nonlinear model was studied and compared. The results show that nonlinear SVM model has the best performance. Using the SVM model, the coefficients of determination (R2) of Mn, Cr, Ni, and Ti were 0.9705, 0.9849, 0.9882, and 0.9837, respectively, and the root mean squared errors of prediction (RMSEP) of Mn, Cr, Ni, and Ti were 0.0982 wt%, 0.0185 wt%, 0.0179 wt%, and 0.0178 wt%, respectively. This work demonstrates that nonlinear quantitative analytical methods can effectively overcome those nonlinear effects and improve the quantitative analytical accuracy of FO-LIBS

Research on Semiconductor Quality Prediction Based on Nonlinear Support Vector Machine

Using a large amount of semiconductor signal data collected by sensors, effective data can be obtained by data cleaning and data mining. With the analysis of intelligent semiconductor detection model, the effective data can find out the problems in the production process of semiconductor manufacturing process in time, so as to improve the process yield and reduce the unit production cost. Firstly, this paper completes the data preprocessing, and then through the comparison of Pearson correlation coefficients, four variables with the largest Pearson correlation coefficients are obtained, which have the greatest influence on the results, so these four variables are taken as key factors. Finally, the nonlinear support vector machine model is established, and the hypersurface model in the input space corresponds to the hyperplane model in the feature space by using Gaussian function as the kernel function, so that the model has stronger robustness and better generalization ability.

Coal Discrimination Analysis Using Tandem Laser-Induced Breakdown Spectroscopy and Laser Ablation Inductively Coupled Plasma Time-of-Flight Mass Spectrometry.

The contribution and impact of combined laser ablation inductively coupled plasma time-of-flight mass spectrometry (LA-ICP-TOF-MS) and laser-induced breakdown spectroscopy (LIBS) were evaluated for the discrimination analysis of different coal samples. This tandem approach allows simultaneous determination of major and minor elements (C, H, Si, Ca, Al, Mg, etc.) and trace elements (V, Ba, Pb, U, etc.) in the coal. The research focused on coal-classification strategies based on principle component analysis (PCA) combined with K-means clustering, partial least-squares discrimination analysis (PLS-DA), and support vector machine (SVM) for analytical performance. Correlation analyses performed from TOF mass and LIBS emission spectra from the coal samples showed that most major, minor, and trace element emissions had negative correlation with the volatile content. Suitable variables for the classification models were determined from these data. The individual TOF data, LIBS data, and combined data of TOF and LIBS as the inputs for different models were analyzed and compared. In all cases, the results obtained with the combined TOF and LIBS data were found to be superior to those obtained with the individual TOF or LIBS data. The nonlinear SVM model combined with TOF and LIBS data provided the best coal-classification performance, with a classification accuracy of up to 98%.

Linear and non-linear SVM prediction for fresh properties and compressive strength of high volume fly ash self-compacting concrete

Support vector machines (SVMs) have recently been used to model the properties of low volume fly ash self-compacting concrete (LVF-SCC) by means of kernel functions to minimize the experimental work. Appropriate linear and non-linear SVM models with different kernels (linear, polynomial, radial basis and sigmoid) were proposed in this paper to predict the fresh properties and compressive strength of high volume fly ash SCC (HVF-SCC) from various volume fly ash SCC (VVF-SCC). Since most available data contain relative low volume fly ash, new SVM models were trained on VVF-SCC mixture proportions adapted from literature, and the prediction results were compared to those of previous models for HVF-SCC and LVF-SCC. Moreover, an experimental plan containing six different SCC mixtures was established and the proposed SVM models were validated by experimental results, including compressive strength, L-box, slump, U-box, and V-funnel. Results showed that new SVM models provide better outcomes if considering appropriate input vector and wide range data to obtain the proper kernel function coefficient to predict the various properties of HVF-SCC. Among the kernel functions, prediction results of the SVM – RBF model were more accurate compared to other kernels.

Automatic Learning Algorithms for Local Support Vector Machines

In this paper, we propose automatic learning algorithms (called auto-kSVM-1p, autokSVM-dp) being possible to automatically tune hyper-parameters of k local support vector machines (SVM) for classifying large data sets. The autokSVM algorithms are able to determine the number of clusters k to partition the large training data, followed which they auto-learn the non-linear SVM model in each cluster to classify the data locally in the parallel way on multi-core computers. The autokSVM algorithms combine the grid search, the .632 bootstrap estimator, the hill climbing heuristic to optimize hyper-parameters in the local non-linear SVM training. The numerical test results on four data sets from UCI repository and three benchmarks of handwritten letters recognition showed that our autokSVM algorithms give very competitive classification results compared to the standard LibSVM and the original kSVM. An example of autokSVM-1p’s effectiveness is given with an accuracy of 96.74% obtained in the classification of Forest cover-type data set having 581,012 datapoints in 54-dimensional input space and 7 classes in 334.45 seconds using a PC Intel(R) Core i7-4790 CPU, 3.6 GHz, 4 cores.

Integration of Local and Global Support Vector Machines to Improve Urban Growth Modelling

The use of local information for the classification and modelling of spatial variables has increased with the application of statistical and machine learning algorithms, such as support vector machines (SVMs). This study presents a new local SVM (LSVM) model that was developed to model the probability of urban development and simulate urban growth in a subregion in the southwestern suburb of the Tehran metropolitan area, Iran, for the periods of 1992–1996 and 1996–2002. Based on the focal training sample, the model was calibrated using the cross-validation method, and the optimal bandwidth was determined. The results were compared with those of a nonlinear global SVM (GSVM) model that was calibrated based on the ten-fold cross-validation method. This study then evaluated an integrated SVM model (LGSVM) obtained based on a weighted combination of the local and global urban development probabilities. A comparison of the probability maps showed a higher accuracy for the LGSVM than for either the LSVM or GSVM model. To assess the performance of the LSVM, GSVM and LGSVM models in the simulation of urban growth, probability maps were employed as the transition rules for urban cellular automata. The results show that a trade-off between local and global SVM models can enhance the performance of urban growth modelling.

Month ahead average daily electricity price profile forecasting based on a hybrid nonlinear regression and SVM model: an ERCOT case study

With the deregulation of the electric power industry, electricity price forecasting plays an increasingly important role in electricity markets, especially for retailors and investment decision making. Month ahead average daily electricity price profile forecasting is proposed for the first time in this paper. A hybrid nonlinear regression and support vector machine (SVM) model is proposed. Off-peak hours, peak hours in peak months and peak hours in off-peak months are distinguished and different methods are designed to improve the forecast accuracy. A nonlinear regression model with deviation compensation is proposed to forecast the prices of off-peak hours and peak hours in off-peak months. SVM is adopted to forecast the prices of peak hours in peak months. Case studies based on data from ERCOT validate the effectiveness of the proposed hybrid method.

Non-linear models for the detection of impaired cerebral blood flow autoregulation.

The ability to discriminate between normal and impaired dynamic cerebral autoregulation (CA), based on measurements of spontaneous fluctuations in arterial blood pressure (BP) and cerebral blood flow (CBF), has considerable clinical relevance. We studied 45 normal subjects at rest and under hypercapnia induced by breathing a mixture of carbon dioxide and air. Non-linear models with BP as input and CBF velocity (CBFV) as output, were implemented with support vector machines (SVM) using separate recordings for learning and validation. Dynamic SVM implementations used either moving average or autoregressive structures. The efficiency of dynamic CA was estimated from the model’s derived CBFV response to a step change in BP as an autoregulation index for both linear and non-linear models. Non-linear models with recurrences (autoregressive) showed the best results, with CA indexes of 5.9 ± 1.5 in normocapnia, and 2.5 ± 1.2 for hypercapnia with an area under the receiver-operator curve of 0.955. The high performance achieved by non-linear SVM models to detect deterioration of dynamic CA should encourage further assessment of its applicability to clinical conditions where CA might be impaired.

Latent‐lSVM classification of very high‐dimensional and large‐scale multi‐class datasets

SummaryWe propose a new parallel learning algorithm of latent local support vector machines (SVM), called latent‐lSVM for effectively classifying very high‐dimensional and large‐scale multi‐class datasets. The common framework of texts/images classification tasks using the Bag‐Of‐(visual)‐Words model for the data representation leads to hard classification problem with thousands of dimensions and hundreds of classes. Our latent‐lSVM algorithm performs these complex tasks into two main steps. The first one is to use latent Dirichlet allocation for assigning the datapoint (text/image) to some topics (clusters) with the corresponding probabilities. This aims at reducing the number of classes and the number of datapoints in the cluster compared to the full dataset, followed by the second one: to learn in a parallel way nonlinear SVM models to classify data clusters locally. The numerical test results on nine real datasets show that the latent‐lSVM algorithm achieves very high accuracy compared to state‐of‐the‐art algorithms. An example of its effectiveness is given with an accuracy of 70.14% obtained in the classification of Book dataset having 100 000 individuals in 89 821 dimensional input space and 661 classes in 11.2 minutes using a PC Intel(R) Core i7‐4790 CPU, 3.6 GHz, 4 cores.

A fast classification strategy for SVM on the large-scale high-dimensional datasets

The challenges of the classification for the large-scale and high-dimensional datasets are: (1) It requires huge computational burden in the training phase and in the classification phase; (2) it needs large storage requirement to save many training data; and (3) it is difficult to determine decision rules in the high-dimensional data. Nonlinear support vector machine (SVM) is a popular classifier, and it performs well on a high-dimensional dataset. However, it easily leads overfitting problem especially when the data are not evenly distributed. Recently, profile support vector machine (PSVM) is proposed to solve this problem. Because local learning is superior to global learning, multiple linear SVM models are trained to get similar performance to a nonlinear SVM model. However, it is inefficient in the training phase. In this paper, we proposed a fast classification strategy for PSVM to speed up the training time and the classification time. We first choose border samples near the decision boundary from training samples. Then, the reduced training samples are clustered to several local subsets through MagKmeans algorithm. In the paper, we proposed a fast search method to find the optimal solution for MagKmeans algorithm. Each cluster is used to learn multiple linear SVM models. Both artificial datasets and real datasets are used to evaluate the performance of the proposed method. In the experimental result, the proposed method prevents overfitting and underfitting problems. Moreover, the proposed strategy is effective and efficient.

A nonlinear support vector machine model with hard penalty function based on glowworm swarm optimization for forecasting daily global solar radiation

For a portion of the power which is generated by grid connected photovoltaic installations, an effective solar irradiation forecasting approach must be crucial to ensure the quality and the security of power grid. This paper develops and investigates a novel model to forecast 30 daily global solar radiation at four given locations of the United States. Eclat data mining algorithm is first presented to discover association rules between solar radiation and several meteorological factors laying a theoretical foundation for these correlative factors as input vectors. An effective and innovative intelligent optimization model based on nonlinear support vector machine and hard penalty function is proposed to forecast solar radiation by converting support vector machine into a regularization problem with ridge penalty, adding a hard penalty function to select the number of radial basis functions, and using glowworm swarm optimization algorithm to determine the optimal parameters of the model. In order to illustrate our validity of the proposed method, the datasets at four sites of the United States are split to into training data and test data, separately. The experiment results reveal that the proposed model delivers the best forecasting performances comparing with other competitors.

Building comprehensible customer churn prediction models with advanced rule induction techniques

Customer churn prediction models aim to detect customers with a high propensity to attrite. Predictive accuracy, comprehensibility, and justifiability are three key aspects of a churn prediction model. An accurate model permits to correctly target future churners in a retention marketing campaign, while a comprehensible and intuitive rule-set allows to identify the main drivers for customers to churn, and to develop an effective retention strategy in accordance with domain knowledge. This paper provides an extended overview of the literature on the use of data mining in customer churn prediction modeling. It is shown that only limited attention has been paid to the comprehensibility and the intuitiveness of churn prediction models. Therefore, two novel data mining techniques are applied to churn prediction modeling, and benchmarked to traditional rule induction techniques such as C4.5 and RIPPER. Both AntMiner+ and ALBA are shown to induce accurate as well as comprehensible classification rule-sets. AntMiner+ is a high performing data mining technique based on the principles of Ant Colony Optimization that allows to include domain knowledge by imposing monotonicity constraints on the final rule-set. ALBA on the other hand combines the high predictive accuracy of a non-linear support vector machine model with the comprehensibility of the rule-set format. The results of the benchmarking experiments show that ALBA improves learning of classification techniques, resulting in comprehensible models with increased performance. AntMiner+ results in accurate, comprehensible, but most importantly justifiable models, unlike the other modeling techniques included in this study.

Computational chemistry study of 3D‐structure‐function relationships for enzymes based on Markov models for protein electrostatic, HINT, and van der Waals potentials

In a significant work, Dobson and Doig (J Mol Biol 2003, 330, 771) illustrated protein prediction as enzymatic or not from spatial structure without resorting to alignments. They used 52 protein features and a nonlinear support vector machine model to classify more than 1000 proteins collected from the PDB with a 77% overall accuracy. The most useful features were: the secondary-structure content, the amino acid frequencies, the number of disulphide bonds, and the largest cleft size. Working on the same dataset used by D&D, in this article we reported a good and simple model, based on the Markov chain models (MCM), to classify protein 3D structures as enzymatic or not, taking into consideration the spatial structure without resorting to alignments. Here we define, for the first time, a general MCM to calculate the electrostatic potential, molecular vibrations, van der Waals (vdw) interactions, and hydrophobic interactions (HINT) and use them in comparative studies of potential fields and/or protein function prediction. The dataset is composed of 1371 proteins divided into 689 enzymes and 682 nonenzymes, all proteins were collected from the PDB. The best model we found was a linear model carried out with the linear discriminant analysis; it was able to classify 74.18% of the proteins using only two electrostatic potentials. In the work described here, we define 3D-HINT potentials (mu(k)) and use them for the first time to derive a classifier for protein enzymes. We analyzed ROC curves, domain of applicability, parametric assumptions, desirability maps, and also tested other nonlinear artificial neural network models which did not improve the linear model. In closing, this MCM allows a fast calculation and comparison of different potentials deriving into accurate protein 3D structure-function relationships, notably simpler than the previous.

Quantitative Structure–Activity Relationship Study on a Series of Novel Ligands Binding to Central Benzodiazepine Receptor By Using the Combination of Heuristic Method and Support Vector Machines

AbstractIn this study, Heuristic Method (HM) and Support Vector Machines (SVMs) were used for Quantitative Structure–Activity Relationship (QSAR) study of a series of 2‐aryl (heteroaryl)‐2,5‐dihydropyrazolo [4,3‐c] quinolin‐3‐(3H)‐ones, which have high affinity with central Benzodiazepine Receptor (BzR). Seven molecular descriptors selected by the HM in CODESSA were used as inputs for SVM. The prediction results are in good agreement with the experimental values. The correlation coefficients R2 of the nonlinear SVM model were 0.93 and 0.96 for the training and testing sets, respectively. This paper proposes a new and effective method to design new ligands of BzR based on QSAR study.

QSAR Models for the Prediction of Binding Affinities to Human Serum Albumin Using the Heuristic Method and a Support Vector Machine.

The binding affinities to human serum albumin for 94 diverse drugs and drug-like compounds were modeled with the descriptors calculated from the molecular structure alone using a quantitative structure−activity relationship (QSAR) technique. The heuristic method (HM) and support vector machine (SVM) were utilized to construct the linear and nonlinear prediction models, leading to a good correlation coefficient (R2) of 0.86 and 0.94 and root-mean-square errors (rms) of 0.212 and 0.134 albumin drug binding affinity units, respectively. Furthermore, the models were evaluated by a 10 compound external test set, yielding R2 of 0.71 and 0.89 and rms error of 0.430 and 0.222. The specific information described by the heuristic linear model could give some insights into the factors that are likely to govern the binding affinity of the compounds and be used as an aid to the drug design process; however, the prediction results of the nonlinear SVM model seem to be better than that of the HM.

QSAR models for the prediction of binding affinities to human serum albumin using the heuristic method and a support vector machine.

The binding affinities to human serum albumin for 94 diverse drugs and drug-like compounds were modeled with the descriptors calculated from the molecular structure alone using a quantitative structure-activity relationship (QSAR) technique. The heuristic method (HM) and support vector machine (SVM) were utilized to construct the linear and nonlinear prediction models, leading to a good correlation coefficient (R2) of 0.86 and 0.94 and root-mean-square errors (rms) of 0.212 and 0.134 albumin drug binding affinity units, respectively. Furthermore, the models were evaluated by a 10 compound external test set, yielding R2 of 0.71 and 0.89 and rms error of 0.430 and 0.222. The specific information described by the heuristic linear model could give some insights into the factors that are likely to govern the binding affinity of the compounds and be used as an aid to the drug design process; however, the prediction results of the nonlinear SVM model seem to be better than that of the HM.