Gaussian Kernel Support Vector Machine Research Articles

Fuzzy quadratic surface support vector machine based on fisher discriminant analysis

In this paper, using the concept of Fisher discriminant analysis and a new fuzzy membership function, a kernel-free fuzzy quadratic surface support vector machine model is proposed for binary classification. The membership function is specially designed to consider not only the ``quadratic-margin distance'' between a training point and its related ``quadratic center surface'' but also the affinity among training points. A decomposition algorithm is designed to solve the proposed model. Computational results on artificial and four real-world classifying data sets indicate that the proposed model outperforms fuzzy support vector machine models with Gaussian or Quadratic kernel and soft quadratic surface support vector machine model, especially, when the data sets contain a large amount of outliers and noises.

Alternative Alphas from Hedge Fund ETF Speculation

Alternative alpha represents risk-adjust absolute return of an alternative investing instrument regressed on alternative risk factors. Over the years, the definition is extended to the absolute return generated from alternative asset speculation-long-only or long-short strategy on alternative assets to generate additional return on top of existence alpha. In this article, we examine and propose a model with state-dependent stochastic differential equations based on Gaussian mixture model and multi-class Gaussian-kernel support vector machine to analyze hedge fund ETF alpha. We provide a new type of long-short speculation which trades on hedge fund strategies. This long-short alternative portfolio is build based on a Sharpe-ratio-like alpha ratio optimization program, and the historical performance from the portfolio shows statistically significant improvement adding to existing alphas. For passive investors, the portfolio also yields a simple portable alpha strategy which outperforms the S&P 500 benchmark return by 7.8% since 2012.

Alternative Alphas from Hedge Fund ETF Speculation

Alternative alpha represents risk-adjust absolute return of an alternative investing instrument regressed on alternative risk factors. Over the years, the definition is extended to the absolute return generated from alternative asset speculation -- long-only or long-short strategy on alternative assets to generate additional return on top of existence alpha. In this article, we examine and propose a model with state-dependent stochastic differential equations based on Gaussian mixture model and multi-class Gaussian-kernel support vector machine to analyze hedge fund ETF alpha. We provide a new type of long-short speculation which trades on hedge fund strategies. This long-short alternative portfolio is build based on a Sharpe-ratio-like alpha ratio optimization program, and the historical performance from the portfolio shows statistically significant improvement adding to existing alphas. For passive investors, the portfolio also yields a simple portable alpha strategy which outperforms the S&P 500 benchmark return by 7.8% since 2012.

Super-parameter selection for Gaussian-Kernel SVM based on outlier-resisting

The learning ability and generalizing performance of the support vector machine (SVM) mainly relies on the reasonable selection of super-parameters. When the scale of the training sample set is large and the parameter space is huge, the existing popular super-parameter selection methods are impractical due to high computational complexity. In this paper, a novel super-parameter selection method for SVM with a Gaussian kernel is proposed, which can be divided into the following two stages. The first one is choosing the kernel parameter to ensure a sufficiently large number of potential support vectors retained in the training sample set. The second one is screening out outliers from the training sample set by assigning a special value to the penalty factor, and training out the optimal penalty factor from the remained training sample set without outliers. The whole process of super-parameter selection only needs two train-validate cycles. Therefore, the computational complexity of our method is low. The comparative experimental results concerning 8 benchmark datasets show that our method possesses high classification accuracy and desirable training time.

Development of Vision‐Based Navigation System for Wheeled Agent

AbstractIn this paper, a vision‐based navigation system including support vector machine (SVM) path planner and fuzzy sliding‐mode controlled (SMC) path follower is developed for a wheeled agent. The developed system, comprising image acquisition, map formatting, path planning, label assignment, and path tracking inference mechanisms, aims to gracefully follow a planned smooth path. To obtain accurate coordinate values, the captured images are first adjusted by calibration processing. As a roadmap method for path generation, the Voronoi diagram is employed as a preprocessor and the Gaussian kernel SVM postprocessor is applied consecutively. To deal with the uncertainties, a path follower based on fuzzy SMC is embedded to track the planned path on line. In this study, a practical framework is implemented to assess the performance. With the real devised system, a series of experiments are carried out and analyzed to confirm the expected performance. The experiments show a robust capability of the system for both path planning and path tracking under various obstacle layouts.

Ensemble of data mining methods for gene ranking

Abstract The paper presents the ensemble of data mining methods for discovering the most important genes and gene sequences generated by the gene expression arrays, responsible for the recognition of a particular type of cancer. The analyzed methods include the correlation of the feature with a class, application of the statistical hypotheses, the Fisher measure of discrimination and application of the linear Support Vector Machine for characterization of the discrimination ability of the features. In the first step of ranking we apply each method individually, choosing the genes most often selected in the cross validation of the available data set. In the next step we combine the results of different selection methods together and once again choose the genes most frequently appearing in the selected sets. On the basis of this we form the final ranking of the genes. The most important genes form the input information delivered to the Support Vector Machine (SVM) classifier, responsible for the final recognition of tumor from non-tumor data. Different forms of checking the correctness of the proposed ranking procedure have been applied. The first one is relied on mapping the distribution of selected genes on the two-coordinate system formed by two most important principal components of the PCA transformation and applying the cluster quality measures. The other one depicts the results in the graphical form by presenting the gene expressions in the form of pixel intensity for the available data. The final confirmation of the quality of the proposed ranking method are the classification results of recognition of the cancer cases from the non-cancer (normal) ones, performed using the Gaussian kernel SVM. The results of selection of the most significant genes used by the SVM for recognition of the prostate cancer cases from normal cases have confirmed a good accuracy of results. The presented methodology is of potential use for practical application in bioinformatics.

Abstract 4061: Computer-assisted Gleason grading of prostate cancer: Two novel approaches using nuclear shape and texture feature to classify pathologic Gleason grade patterns 3 and 4

Abstract Introduction: We used two novel computer-assisted imaging applications that incorporate nuclear shape and texture analysis to differentiate Gleason grade patterns 3 and 4 of H&Ein a tissue microarray (TMA). The two approaches utilized a variational adaptive segmentation scheme (AdACM) (Rutgers) and the cardinal multiridgelet transform (CMRT) based texture analysis (University of Pittsburgh). Methods: The TMA include 0.6 mm cores from blocks that included Gleason patterns of CaP radical prostatectomy cases from 40 patients. The TMA comprised 13 Gleason patterns 6 (3+3), 8 pattern 7 (3+4), 7 pattern 8 (4+4) and 5 pattern 9 (4+5) Gleason grade (GG) patterns. The AdACM uses 200 × 200 pixel images extracted for nuclear & architectural textural features. All features were based on mathematical energy terms with a shape prior in a multi-level set formulation. The CMRT is an image transform that is translation- and rotation-invariant, and provides a means of displaying texture contents for selection of discriminative features. With a 3-scale decomposition, it gives 8 ridgelet packets and 16 statistical texture features in a 256 x 256 pixel image patch that are averaged over 9 non-overlapping patches in a given image for separation of GG patterns 3 and 4. Results: The CMRT study utilized 3 of 16 texture features extracted from images and trained a Gaussian kernel support vector machine (SVM), a leave-one-out cross-validation showed an accuracy of 94% and AUC of 0.97. For AdACM, nuclear features and architectural features were extracted. Linear Discriminant Analysis (LDA) was employed for feature selection and best features from all three classes could discriminate Gleason grade 3 vs. 4 with an accuracy of 86% via a SVM classifier. Conclusions: Unique quantitative imaging of tissue nuclear shape and texture features can accurately differentiate Gleason patterns 3 and 4. Our research will next evaluate CaP outcomes. Citation Format: {Authors}. {Abstract title} [abstract]. In: Proceedings of the 103rd Annual Meeting of the American Association for Cancer Research; 2012 Mar 31-Apr 4; Chicago, IL. Philadelphia (PA): AACR; Cancer Res 2012;72(8 Suppl):Abstract nr 4061. doi:1538-7445.AM2012-4061

Wavelet Support Vector Machines for Forecasting Precipitation in Tropical Cyclones: Comparisons with GSVM, Regression, and MM5

Abstract This study presents two support vector machine (SVM) based models for forecasting hourly precipitation during tropical cyclone (typhoon) events. The two SVM-based models are the traditional Gaussian kernel SVMs (GSVMs) and the advanced wavelet kernel SVMs (WSVMs). A comparison between the fifth-generation Pennsylvania State University–National Center for Atmospheric Research (PSU–NCAR) Mesoscale Model (MM5) and statistical models, including SVM-based models and linear regressions (regression), was made in terms of performance of rainfall prediction at the Shihmen Reservoir watershed in Taiwan. Data from 73 typhoons affecting the Shihmen Reservoir watershed were included in the analysis. This study designed six attribute combinations with different lag times for the forecast target. The modified RMSE, bias, and estimated threat score (ETS) results were employed to assess the predicted outcomes. Results show that better attribute combinations for typhoon climatologic characteristics and typhoon precipitation predictions occurred at 0-h lag time with modified RMSE values of 0.288, 0.257, and 0.296 in GSVM, WSVM, and the regression, respectively. Moreover, WSVM having average bias and ETS values close to 1.0 gave better predictions than did the GSVM and regression models. In addition, Typhoons Zeb (1998) and Nari (2001) were selected for comparison between the MM5 model output and the developed statistical models. Results showed that the MM5 tended to overestimate the peak and cumulative rainfall amounts while the statistical models were inclined to yield underestimations.

Predicting Antitumor Activity of Peptides by Consensus of Regression Models Trained on a Small Data Sample

Predicting antitumor activity of compounds using regression models trained on a small number of compounds with measured biological activity is an ill-posed inverse problem. Yet, it occurs very often within the academic community. To counteract, up to some extent, overfitting problems caused by a small training data, we propose to use consensus of six regression models for prediction of biological activity of virtual library of compounds. The QSAR descriptors of 22 compounds related to the opioid growth factor (OGF, Tyr-Gly-Gly-Phe-Met) with known antitumor activity were used to train regression models: the feed-forward artificial neural network, the k-nearest neighbor, sparseness constrained linear regression, the linear and nonlinear (with polynomial and Gaussian kernel) support vector machine. Regression models were applied on a virtual library of 429 compounds that resulted in six lists with candidate compounds ranked by predicted antitumor activity. The highly ranked candidate compounds were synthesized, characterized and tested for an antiproliferative activity. Some of prepared peptides showed more pronounced activity compared with the native OGF; however, they were less active than highly ranked compounds selected previously by the radial basis function support vector machine (RBF SVM) regression model. The ill-posedness of the related inverse problem causes unstable behavior of trained regression models on test data. These results point to high complexity of prediction based on the regression models trained on a small data sample.

On the Design and Analysis of the Privacy-Preserving SVM Classifier

The support vector machine (SVM) is a widely used tool in classification problems. The SVM trains a classifier by solving an optimization problem to decide which instances of the training data set are support vectors, which are the necessarily informative instances to form the SVM classifier. Since support vectors are intact tuples taken from the training data set, releasing the SVM classifier for public use or shipping the SVM classifier to clients will disclose the private content of support vectors. This violates the privacy-preserving requirements for some legal or commercial reasons. The problem is that the classifier learned by the SVM inherently violates the privacy. This privacy violation problem will restrict the applicability of the SVM. To the best of our knowledge, there has not been work extending the notion of privacy preservation to tackle this inherent privacy violation problem of the SVM classifier. In this paper, we exploit this privacy violation problem, and propose an approach to postprocess the SVM classifier to transform it to a privacy-preserving classifier which does not disclose the private content of support vectors. The postprocessed SVM classifier without exposing the private content of training data is called Privacy-Preserving SVM Classifier (abbreviated as PPSVC). The PPSVC is designed for the commonly used Gaussian kernel function. It precisely approximates the decision function of the Gaussian kernel SVM classifier without exposing the sensitive attribute values possessed by support vectors. By applying the PPSVC, the SVM classifier is able to be publicly released while preserving privacy. We prove that the PPSVC is robust against adversarial attacks. The experiments on real data sets show that the classification accuracy of the PPSVC is comparable to the original SVM classifier.

An incremental support vector machine-trained TS-type fuzzy system for online classification problems

This paper proposes an incremental support vector machine-trained TS-type fuzzy classifier (ISVM-FC). The ISVM-FC is a fuzzy system that consists of Takagi–Sugeno (TS)-type fuzzy rules. Structure and parameters in the ISVM-FC are trained incrementally from one subset of training data at a time. This incremental training approach avoids the use of large amounts of memory required for storing training data in batch learning, reduces training time, and adapts the classifier to time-dependent classification systems where training data are available sequentially. Initially, there are no fuzzy rules for structure learning with the ISVM-FC. It generates all rules according to the distribution of the training data. An incremental linear support vector machine (SVM) is used to tune the resulting rule parameters to give the classifier better generalization performance. The use of incremental learning discards past training data adaptively according to its distance to the linear hyperplane, thereby improving learning efficiency. Three simulations are conducted to verify the performance of the ISVM-FC. Comparisons with fuzzy classifiers and Gaussian-kernel SVM with batch and incremental learning modes demonstrate that the ISVM-FC improves training and test times, and reduces memory consumption for classifier storage without deteriorating the generalization ability.

Transmission line distance relaying using machine intelligence technique

A new approach for distance relaying of transmission line using machine intelligence technique such as support vector machine (SVM) is presented. The proposed SVM technique is used for faulty phase selection and ground detection in different fault situations that occur on large power transmission line. Post-fault current and voltage samples for one-fourth cycle (five samples) are used as inputs to SVM 1, which provide output for faulty phase selection. SVM 2 is trained and tested with zero-sequence components of fundamental, third and fifth harmonic components of the post-fault current signal to provides the involvement of ground in the fault process. The polynomial and Gaussian kernel SVMs are designed to provide the most optimised boundary for classification. The total time taken for faulty phase selection and ground detection is 10 ms (half cycle) from the inception of fault. Also the proposed technique is tested on experimental set-up with different fault situations. The test results are compared with those of the radial basis function neural network and were found to be superior with respect to efficiency and speed. The classification test results from SVMs are accurate for simulation model and experimental set-up, and thus provide fast and robust protection scheme for distance relaying in transmission line.

Fuzzy System Learned Through Fuzzy Clustering and Support Vector Machine for Human Skin Color Segmentation

This paper proposes a Fuzzy System learned through Fuzzy Clustering and Support Vector Machine (FS-FCSVM). The FS-FCSVM is a fuzzy system constructed by fuzzy if-then rules with fuzzy singletons in the consequence. The structure of FS-FCSVM is constructed by fuzzy clustering on the input data, which helps to reduce the number of rules. Parameters in FS-FCSVM are learned through a support vector machine (SVM) for the purpose of achieving higher generalization ability. In contrast to nonlinear kernel-based SVM or some other fuzzy systems with a support vector learning mechanism, both the number of parameters/rules in FS-FCSVM and the computation time are much smaller. FS-FCSVM is applied to skin color segmentation. For color information representation, different types of features based on scaled hue and saturation color space are used. Comparisons with a fuzzy neural network, the Gaussian kernel SVM, and mixture of Gaussian classifiers are performed to show the advantage of FS-FCSVM.

Road-Sign Detection and Recognition Based on Support Vector Machines

This paper presents an automatic road-sign detection and recognition system based on support vector machines (SVMs). In automatic traffic-sign maintenance and in a visual driver-assistance system, road-sign detection and recognition are two of the most important functions. Our system is able to detect and recognize circular, rectangular, triangular, and octagonal signs and, hence, covers all existing Spanish traffic-sign shapes. Road signs provide drivers important information and help them to drive more safely and more easily by guiding and warning them and thus regulating their actions. The proposed recognition system is based on the generalization properties of SVMs. The system consists of three stages: 1) segmentation according to the color of the pixel; 2) traffic-sign detection by shape classification using linear SVMs; and 3) content recognition based on Gaussian-kernel SVMs. Because of the used segmentation stage by red, blue, yellow, white, or combinations of these colors, all traffic signs can be detected, and some of them can be detected by several colors. Results show a high success rate and a very low amount of false positives in the final recognition stage. From these results, we can conclude that the proposed algorithm is invariant to translation, rotation, scale, and, in many situations, even to partial occlusions

A comparison of classification techniques for the P300 Speller

This study assesses the relative performance characteristics of five established classification techniques on data collected using the P300 Speller paradigm, originally described by Farwell and Donchin (1988 Electroenceph. Clin. Neurophysiol. 70 510). Four linear methods: Pearson's correlation method (PCM), Fisher's linear discriminant (FLD), stepwise linear discriminant analysis (SWLDA) and a linear support vector machine (LSVM); and one nonlinear method: Gaussian kernel support vector machine (GSVM), are compared for classifying offline data from eight users. The relative performance of the classifiers is evaluated, along with the practical concerns regarding the implementation of the respective methods. The results indicate that while all methods attained acceptable performance levels, SWLDA and FLD provide the best overall performance and implementation characteristics for practical classification of P300 Speller data.

Parameters selection in gene selection using Gaussian kernel support vector machines by genetic algorithm

In microarray-based cancer classification, gene selection is an important issue owing to the large number of variables and small number of samples as well as its non-linearity. It is difficult to get satisfying results by using conventional linear statistical methods. Recursive feature elimination based on support vector machine (SVM RFE) is an effective algorithm for gene selection and cancer classification, which are integrated into a consistent framework. In this paper, we propose a new method to select parameters of the aforementioned algorithm implemented with Gaussian kernel SVMs as better alternatives to the common practice of selecting the apparently best parameters by using a genetic algorithm to search for a couple of optimal parameter. Fast implementation issues for this method are also discussed for pragmatic reasons. The proposed method was tested on two representative hereditary breast cancer and acute leukaemia datasets. The experimental results indicate that the proposed method performs well in selecting genes and achieves high classification accuracies with these genes.

Collaboration as a gateway to nursing research in the ED

This paper presents an intelligent mechanism for Short Term Load Forecasting (STLF) models, which allows self-adaptation with respect to the load operational conditions. Specifically, a knowledge-based FeedBack Tunning Fuzzy System (FBTFS) is proposed to instantaneously correlate the information about the demand profile and its operational conditions to make decisions for controlling the model’s forecasting error rate. To maintain minimum forecasting error under various operational scenarios, the FBTFS adaptation was optimised using a Multi-Layer Perceptron Artificial Neural Network (MLPANN), which was trained using Backpropagation algorithm, based on the information about the amount of error and the operational conditions at time of forecasting. For the sake of comparison and performance testing, this mechanism was added to the conventional forecasting methods, i.e. Nonlinear AutoRegressive eXogenous-Artificial Neural Network (NARXANN), Fuzzy Subtractive Clustering Method-based Adaptive Neuro Fuzzy Inference System (FSCMANFIS) and Gaussian-kernel Support Vector Machine (GSVM), and the measured forecasting error reduction average in a 12 month simulation period was 7.83%, 8.5% and 8.32% respectively. The 3.5 MW variable load profile of Edith Cowan University (ECU) in Joondalup, Australia, was used in the modelling and simulations of this model, and the data was provided by Western Power, the transmission and distribution company of the state of Western Australia.

An Accurate QSPR Study of O—H Bond Dissociation Energy in Substituted Phenols Based on Support Vector Machines.

The support vector machine (SVM), as a novel type of learning machine, was used to develop a Quantitative Structure−Property Relationship (QSPR) model of the O−H bond dissociation energy (BDE) of 78 substituted phenols. The six descriptors calculated solely from the molecular structures of compounds selected by forward stepwise regression were used as inputs for the SVM model. The root-mean-square (rms) errors in BDE predictions for the training, test, and overall data sets were 3.808, 3.320, and 3.713 BDE units (kJ mol-1), respectively. The results obtained by Gaussian-kernel SVM were much better than those obtained by multiple linear regression, radial basis function neural networks, linear-kernel SVM, and other QSPR approaches.

An accurate QSPR study of O-H bond dissociation energy in substituted phenols based on support vector machines.

The support vector machine (SVM), as a novel type of learning machine, was used to develop a Quantitative Structure-Property Relationship (QSPR) model of the O-H bond dissociation energy (BDE) of 78 substituted phenols. The six descriptors calculated solely from the molecular structures of compounds selected by forward stepwise regression were used as inputs for the SVM model. The root-mean-square (rms) errors in BDE predictions for the training, test, and overall data sets were 3.808, 3.320, and 3.713 BDE units (kJ mol(-1)), respectively. The results obtained by Gaussian-kernel SVM were much better than those obtained by multiple linear regression, radial basis function neural networks, linear-kernel SVM, and other QSPR approaches.

Development of a personal-computer-based intelligent tutoring system

Gaussian kernel Support Vector Machines (SVMs) deliver state-of-the-art generalization performance for non-linear classification, but the time complexity of their training process is at least quadratic w.r.t. the size of training set, preventing them from scaling up on large datasets. To address this issue, we propose a novel approach to large-scale kernel SVMs in sublinear time w.r.t. the size of training set, which combines three well-known and efficient techniques with theoretical guarantees. First, we subsample massive samples to reduce the sample size. Then, we use the random Fourier feature mapping on the subsamples to construct explicit random feature space where we can train a linear SVM to approximate the corresponding Gaussian kernel SVM. Finally, we use parallel algorithms to make our approach more scalable. Deriving the upper bounds of kernel matrix approximation error, hypothesis error and excess risk w.r.t. the size of training set and the dimension of random feature space, we establish the theoretical foundation of our proposed approach. In this way, we can reduce the time complexity of training kernel SVMs without sacrificing much accuracy. Our proposed approach achieves high accuracy and has sublinear training time complexity, which exhibits good scalability theoretically and empirically.