Support Vector Machine Learning Approach Research Articles

Using Domain Adaptation for Incremental SVM Classification of Drift Data

A common assumption in machine learning is that training data is complete, and the data distribution is fixed. However, in many practical applications, this assumption does not hold. Incremental learning was proposed to compensate for this problem. Common approaches include retraining models and incremental learning to compensate for the shortage of training data. Retraining models is time-consuming and computationally expensive, while incremental learning can save time and computational costs. However, the concept drift may affect the performance. Two crucial issues should be considered to address concept drift in incremental learning: gaining new knowledge without forgetting previously acquired knowledge and forgetting obsolete information without corrupting valid information. This paper proposes an incremental support vector machine learning approach with domain adaptation, considering both crucial issues. Firstly, a small amount of new data is used to fine-tune the previous model to generate a model that is sensitive to the new data but retains the previous data information by transferring parameters. Secondly, an ensemble and model selection mechanism based on Bayesian theory is proposed to keep the valid information. The computational experiments indicate that the performance of the proposed model improved as new data was acquired. In addition, the influence of the degree of data drift on the algorithm is also explored. A gain in performance on four out of five industrial datasets and four synthetic datasets has been demonstrated over the support vector machine and incremental support vector machine algorithms.

Modeling and Theoretical Analysis of GNSS-R Soil Moisture Retrieval Based on the Random Forest and Support Vector Machine Learning Approach

Global Navigation Satellite System-Reflectometry (GNSS-R) as a microwave remote sensing technique can retrieve the Earth’s surface parameters using the GNSS reflected signal from the surface. These reflected signals convey the surface features and therefore can be utilized to detect certain physical properties of the reflecting surface such as soil moisture content (SMC). Up to now, a serial of electromagnetic models (e.g., bistatic radar and Fresnel equations, etc.) are employed and solved for SMC retrieval. However, due to the uncertainty of the physical characteristics of the sites, complexity, and nonlinearity of the inversion process, etc., it is still challenging to accurately retrieve the soil moisture. The popular machine learning (ML) methods are flexible and able to handle nonlinear problems. It can dig out and model the complex interactions between input and output and ultimately make good predictions. In this paper, two typical ML methods, specifically, random forest (RF) and support vector machine (SVM), are employed for SMC retrieval from GNSS-R data of self-designed experiments (in situ and airborne). A comprehensive simulated dataset involving different types of soil is constructed firstly to represent the complex interactions between the variables (reflectivity, elevation angle, dielectric constant, and SMC) for the requirement of training ML regression models. Correspondingly, the main task of soil moisture retrieval (regression) is addressed. Specifically, the post-processed data (reflectivity and elevation angle) from sensor acquisitions are used to make predictions by these two adopted ML methods and compared with the commonly used GNSS-R retrieval method (electromagnetic models). The results show that the RF outperforms the SVM method, and it is more suitable for handling the inversion problem. Moreover, the RF regression model built by the comprehensive dataset demonstrates satisfactory accuracy and strong universality, especially when the soil type is not uniform or unknown. Furthermore, the typical task of detecting water/soil (classification) is discussed. The ML algorithms demonstrate a high potential and efficiency in SMC retrieval from GNSS-R data.

Development of species specific putative miRNA and its target prediction tool in wheat (Triticum aestivum L.)

MicroRNA are 20–24 nt, non-coding, single stranded molecule regulating traits and stress response. Tissue and time specific expression limits its detection, thus is major challenge in their discovery. Wheat has limited 119 miRNAs in MiRBase due to limitation of conservation based methodology where old and new miRNA genes gets excluded. This is due to origin of hexaploid wheat by three successive hybridization, older AA, BB and younger DD subgenome. Species specific miRNA prediction (SMIRP concept) based on 152 thermodynamic features of training dataset using support vector machine learning approach has improved prediction accuracy to 97.7%. This has been implemented in TamiRPred (http://webtom.cabgrid.res.in/tamirpred). We also report highest number of putative miRNA genes (4464) of wheat from whole genome sequence populated in database developed in PHP and MySQL. TamiRPred has predicted 2092 (>45.10%) additional miRNA which was not predicted by miRLocator. Predicted miRNAs have been validated by miRBase, small RNA libraries, secondary structure, degradome dataset, star miRNA and binding sites in wheat coding region. This tool can accelerate miRNA polymorphism discovery to be used in wheat trait improvement. Since it predicts chromosome-wise miRNA genes with their respective physical location thus can be transferred using linked SSR markers. This prediction approach can be used as model even in other polyploid crops.

A novel method combining fuzzy SVM and sampling for imbalanced classification

The class imbalance problem has been reported to reduce performance of many existing learning algorithms in intrusion detection. However, the detection rates for minority classes still need to be improved. Thus, the novel hybrid method FSVMs is proposed to solve the problem in the paper, which integrates the prevailing sampling method SMOTE with fuzzy semi-supervised SVM learning approach to class imbalanced intrusion detection data. The basic KDD Cup 1999 dataset, NSLKDD dataset and imbalanced dataset from UCI are used to evaluate the performance of proposed model. Experiment results show that the proposed method outperforms other state-of-the-art classifiers including support vector machine (SVM), back propagation neural network (BPNN), Bayes, k-nearest neighbour (KNN), decision tree (DT), random forest (RF) and four sampling methods in the aspects of detection rate and false alarm rate, and has better robustness for imbalanced classification.

A Novel Method Combining Fuzzy SVM and Sampling for Imbalanced Classification

The class imbalance problem has been reported to reduce performance of many existing learning algorithms in intrusion detection. However, the detection rates for minority classes still need to be improved. Thus, the novel hybrid method FSVMs is proposed to solve the problem in the paper, which integrates the prevailing sampling method SMOTE with fuzzy semi-supervised SVM learning approach to class imbalanced intrusion detection data. The basic KDD Cup 1999 dataset, NSLKDD dataset and imbalanced dataset from UCI are used to evaluate the performance of proposed model. Experiment results show that the proposed method outperforms other state-of-the-art classifiers including support vector machine (SVM), back propagation neural network (BPNN), Bayes, k-nearest neighbour (KNN), decision tree (DT), random forest (RF) and four sampling methods in the aspects of detection rate and false alarm rate, and has better robustness for imbalanced classification.

Promoter Prediction in Bacterial DNA Sequences Using Expectation Maximization and Support Vector Machine Learning Approach

Promoter is a part of the DNA sequence that comes before the gene and is key as a regulator of genes. Promoter prediction helps determine gene position and analyze gene expression. Hence, it is of great importance in the field of bioinformatics. In bioinformatics research, a number of machine learning approaches are applied to discover new meaningful knowledge from biological databases. In this study, two learning approaches, expectation maximization clustering and support vector machine classifier (EMSVM) are used to perform promoter detection. Expectation maximization (EM) algorithm is used to identify groups of samples that behave similarly and dissimilarly, such as the activity of promoters and non-promoters in the first stage, while the support vector machine (SVM) is used in the second stage to classify all the data into the correct class category. We have applied this method to datasets corresponding to σ24, σ32, σ38, σ70 promoters and its effectiveness was demonstrated on a range of different promoter regions. Furthermore, it was compared with other classification algorithms to indicate the appropriate performance of the proposed algorithm. Test results show that EMSVM performs better than other methods.

Multivariate neuroanatomical classification of cognitive subtypes in schizophrenia: a support vector machine learning approach.

Heterogeneity in the structural brain abnormalities associated with schizophrenia has made identification of reliable neuroanatomical markers of the disease difficult. The use of more homogenous clinical phenotypes may improve the accuracy of predicting psychotic disorder/s on the basis of observable brain disturbances. Here we investigate the utility of cognitive subtypes of schizophrenia – ‘cognitive deficit’ and ‘cognitively spared’ – in determining whether multivariate patterns of volumetric brain differences can accurately discriminate these clinical subtypes from healthy controls, and from each other. We applied support vector machine classification to grey- and white-matter volume data from 126 schizophrenia patients previously allocated to the cognitive spared subtype, 74 cognitive deficit schizophrenia patients, and 134 healthy controls. Using this method, cognitive subtypes were distinguished from healthy controls with up to 72% accuracy. Cross-validation analyses between subtypes achieved an accuracy of 71%, suggesting that some common neuroanatomical patterns distinguish both subtypes from healthy controls. Notably, cognitive subtypes were best distinguished from one another when the sample was stratified by sex prior to classification analysis: cognitive subtype classification accuracy was relatively low (<60%) without stratification, and increased to 83% for females with sex stratification. Distinct neuroanatomical patterns predicted cognitive subtype status in each sex: sex-specific multivariate patterns did not predict cognitive subtype status in the other sex above chance, and weight map analyses demonstrated negative correlations between the spatial patterns of weights underlying classification for each sex. These results suggest that in typical mixed-sex samples of schizophrenia patients, the volumetric brain differences between cognitive subtypes are relatively minor in contrast to the large common disease-associated changes. Volumetric differences that distinguish between cognitive subtypes on a case-by-case basis appear to occur in a sex-specific manner that is consistent with previous evidence of disrupted relationships between brain structure and cognition in male, but not female, schizophrenia patients. Consideration of sex-specific differences in brain organization is thus likely to assist future attempts to distinguish subgroups of schizophrenia patients on the basis of neuroanatomical features.

Margin-Maximizing Feature Elimination Methods for Linear and Nonlinear Kernel-Based Discriminant Functions

Feature selection for classification in high-dimensional spaces can improve generalization, reduce classifier complexity, and identify important, discriminating feature "markers." For support vector machine (SVM) classification, a widely used technique is recursive feature elimination (RFE). We demonstrate that RFE is not consistent with margin maximization, central to the SVM learning approach. We thus propose explicit margin-based feature elimination (MFE) for SVMs and demonstrate both improved margin and improved generalization, compared with RFE. Moreover, for the case of a nonlinear kernel, we show that RFE assumes that the squared weight vector 2-norm is strictly decreasing as features are eliminated. We demonstrate this is not true for the Gaussian kernel and, consequently, RFE may give poor results in this case. MFE for nonlinear kernels gives better margin and generalization. We also present an extension which achieves further margin gains, by optimizing only two degrees of freedom--the hyperplane's intercept and its squared 2-norm--with the weight vector orientation fixed. We finally introduce an extension that allows margin slackness. We compare against several alternatives, including RFE and a linear programming method that embeds feature selection within the classifier design. On high-dimensional gene microarray data sets, University of California at Irvine (UCI) repository data sets, and Alzheimer's disease brain image data, MFE methods give promising results.

Biometric classifier update using online learning: A case study in near infrared face verification

The performance of a large scale biometric system may deteriorate over time as new individuals are continually enrolled. To maintain an acceptable level of performance, the classifier has to be re-trained offline in batch mode using both existing and new data. The process of re-training can be computationally expensive and time consuming. This paper presents a new biometric classifier update algorithm that incrementally re-trains the classifier using online learning and progressively establishes a decision hyperplane for improved classification. The proposed algorithm incorporates soft labels and granular computing in the formulation of a 2 ν -Online Granular Soft Support Vector Machine (SVM) to re-train the classifier using only the new data. Granular computing makes it adaptive to local and global variations in data distribution, while soft labels provide resilience to noise. Each time data is acquired, new support vectors that are linearly independent are added and existing support vectors that do not improve the classifier performance are removed. This constrains the size of the support vectors and significantly reduces the training time without compromising the classification accuracy. The efficacy of the proposed online learning strategy is validated in a near infrared face verification application involving different covariates. The results obtained on a heterogeneous near infrared face database of 328 subjects show that in all experiments using different feature extraction and classification algorithms the proposed online 2 ν -Granular Soft Support Vector Machine learning approach is 2–3 times faster while achieving a high level of accuracy similar to offline training using all data.

Prediction of Candidate Primary Immunodeficiency Disease Genes Using a Support Vector Machine Learning Approach

Screening and early identification of primary immunodeficiency disease (PID) genes is a major challenge for physicians. Many resources have catalogued molecular alterations in known PID genes along with their associated clinical and immunological phenotypes. However, these resources do not assist in identifying candidate PID genes. We have recently developed a platform designated Resource of Asian PDIs, which hosts information pertaining to molecular alterations, protein–protein interaction networks, mouse studies and microarray gene expression profiling of all known PID genes. Using this resource as a discovery tool, we describe the development of an algorithm for prediction of candidate PID genes. Using a support vector machine learning approach, we have predicted 1442 candidate PID genes using 69 binary features of 148 known PID genes and 3162 non-PID genes as a training data set. The power of this approach is illustrated by the fact that six of the predicted genes have recently been experimentally confirmed to be PID genes. The remaining genes in this predicted data set represent attractive candidates for testing in patients where the etiology cannot be ascribed to any of the known PID genes.

Classification of clear-cell sarcoma as a subtype of melanoma by genomic profiling.

To develop a genome-based classification scheme for clear-cell sarcoma (CCS), also known as melanoma of soft parts (MSP), which would have implications for diagnosis and treatment. This tumor displays characteristic features of soft tissue sarcoma (STS), including deep soft tissue primary location and a characteristic translocation, t(12;22)(q13;q12), involving EWS and ATF1 genes. CCS/MSP also has typical melanoma features, including immunoreactivity for S100 and HMB45, pigmentation, MITF-M expression, and a propensity for regional lymph node metastases. RNA samples from 21 cell lines and 60 pathologically confirmed cases of STS, melanoma, and CCS/MSP were examined using the U95A GeneChip (Affymetrix, Santa Clara, CA). Hierarchical cluster analysis, principal component analysis, and support vector machine (SVM) analysis exploited genomic correlations within the data to classify CCS/MSP. Unsupervised analyses demonstrated a clear distinction between STS and melanoma and, furthermore, showed that CCS/MSP cluster with the melanomas as a distinct group. A supervised SVM learning approach further validated this finding and provided a user-independent approach to diagnosis. Genes of interest that discriminate CCS/MSP included those encoding melanocyte differentiation antigens, MITF, SOX10, ERBB3, and FGFR1. Gene expression profiles support the classification of CCS/MSP as a distinct genomic subtype of melanoma. Analysis of these gene profiles using the SVM may be an important diagnostic tool. Genomic analysis identified potential targets for the development of therapeutic strategies in the treatment of this disease.