Selection Support Vector Machine Research Articles

Cost-sensitive Feature Selection for Support Vector Machines

Feature Selection is a crucial procedure in Data Science tasks such as Classification, since it identifies the relevant variables, making thus the classification procedures more interpretable, cheaper in terms of measurement and more effective by reducing noise and data overfit. The relevance of features in a classification procedure is linked to the fact that misclassifications costs are frequently asymmetric, since false positive and false negative cases may have very different consequences. However, off-the-shelf Feature Selection procedures seldom take into account such cost-sensitivity of errors.In this paper we propose a mathematical-optimization-based Feature Selection procedure embedded in one of the most popular classification procedures, namely, Support Vector Machines, accommodating asymmetric misclassification costs. The key idea is to replace the traditional margin maximization by minimizing the number of features selected, but imposing upper bounds on the false positive and negative rates. The problem is written as an integer linear problem plus a quadratic convex problem for Support Vector Machines with both linear and radial kernels.The reported numerical experience demonstrates the usefulness of the proposed Feature Selection procedure. Indeed, our results on benchmark data sets show that a substantial decrease of the number of features is obtained, whilst the desired trade-off between false positive and false negative rates is achieved.

Intrusion detection mechanism with machine learning process A case study with FMIFSSVM, FLCFSSVM, misuses SVM, anomaly SVM and Bayesian methods

Today, there is a far reaching of Internet benefits everywhere throughout the world, numerous sorts and vast number of security dangers are expanding. Since it isn't in fact possible to assemble a framework without any vulnerability, Intrusion Detection System (IDS), which can successfully distinguish Intrusion, gets to have pulled in consideration. Intrusion detection can be characterized as the way toward distinguishing irregular, unauthorized or unapproved action that objective is to target a system and its assets. IDS plays a very important role for analyzing the network passage, also it assumes a key part to analyze the system activity log and each log is portrayed by huge arrangement of highlights and it requires tremendous computational preparing force and time for the characterization procedure. This work proposes filter based feature selection methods to predict intrusion with Feature based Mutual Information Feature Selection Support Vector Machine (FMIFSSVM), Feature based Liner Correlation Feature Selection Support Vector Machine (FLCFSSVM), misuses SVM, anomaly SVM and Bayesian methods. The performances of these methods are considered by using the intrusion detection calculation data set called Knowledge Discovery in Databases (KDD) cup 99. Detection Rate (DR), False Alarm Rate (FAR) and Percentage of Successful Prediction (PSP) are the major performance measures studied in this work.

Radiomics Analysis: Complemental But Unessential for Routine Clinical Practice in Predicting Microvascular Invasion of Hepatocellular Carcinoma

Background: Microvascular invasion (MVI) is a prognostic factor of patients with hepatocellular carcinoma (HCC). Preoperative prediction of MVI is challenging. Use of Radiomics as clinical biomarker necessitates amelioration. We therefore estimated whether Radiomics can provide complementary value for predicting MVI risk in HCC patients as compared with a liver imaging score (LI-score) system derived from radiologists' interpretation. Methods: 495 patients (300 in training, 50 in validation and 145 in testing data set) with histologically confirmed HCC between Jan 2009 and Aug 2017 were retrospectively reviewed. All the patients have intact preoperative clinicoradiological records. 93 radiomic features were extracted from each patient and radiomics signature were built by using recursive feature selection support vector machine (RFE-SVM) method. Odds-ratio (OR) regression model was performed to determine risk factors of MVI and construct the predictive nomogram. A decision curve analysis was applied to teste the incremental value of Radiomics score (RS) to LI-score for predicting MVI. Findings: 9 radiomics features were selected to build a radiomics signature that was significantly associated with MVI (p < 0.001). An MVI risk estimation nomogram, which incorporated higher aspartate aminotransferase (AST), higher α-fetoprotein (AFP), non-smooth tumor margin, extrahepatic growth pattern, ill-defined pseudo-capsule, peritumoral arterial enhancement, positive radio-genomic venous invasion (RVI) score and higher RS, was established with an area under curve (AUC) of 0.894 (95%CI, 0.864-0.920). Adding RS to LI-score scheme did not achieve significant improvement in predictive performance and did not add additional net benefits in a decision curve analysis. Interpretation: Radiomics is still a complemental tool that offer limited additional benefit to conventional radiologic method for predicting MVI risk of HCC. The use of Radiomics as clinical biomarkers still needs further study. Funding: China Postdoctoral Fund. A Key Social Development Program for the Ministry of Science and Technology of Jiangsu Province. Declaration of Interest: The authors have declared no conflicts of interest. Ethical Approval: This retrospective study involved routine at a single medical institution. Ethics committee approval was granted by local institutional ethics review board with a waiver of written informed consent. All procedures performed in studies involving human participants were in accordance with the 1964 Helsinki declaration and its later amendments.

A multi-objective evolutionary approach to training set selection for support vector machine

The Support Vector Machine (SVM) is one of the most powerful algorithms for machine learning and data mining in numerous and heterogenous application domains. However, in spite of its competitiveness, SVM suffers from scalability problems which drastically worsens its performance in terms of memory requirements and execution time. As a consequence, there is a strong emergence of approaches for supporting SVM in efficiently addressing the aforementioned problems without affecting its classification capabilities. In this scenario, methods for Training Set Selection (TSS) represent a suitable and consolidated pre-processing technique to compute a reduced but representative training dataset, and improve SVM’s scalability without deprecating its classification accuracy. Recently, TSS has been formulated as an optimization problem characterized by two objectives (the classification accuracy and the reduction rate) and solved through the application of evolutionary algorithms. However, so far, all the evolutionary approaches for TSS have been based on a so-called multi-objective a priori technique, where multiple objectives are aggregated together into a single objective through a weighted combination. This paper proposes to apply, for the first time, a Pareto-based multi-objective optimization approach to the TSS problem in order to explicitly deal with both its objectives and offer a better trade-off between SVM’s classification and reduction performance. The benefits of the proposed approach are validated by a set of experiments involving well-known datasets taken from the UCI Machine Learning Database Repository. As shown by statistical tests, the application of a Pareto-based multi-objective optimization approach improves on state-of-the-art TSS techniques and enhances SVM efficiency.

Naive Bayes Bearing Fault Diagnosis Based on Enhanced Independence of Data.

The bearing is the key component of rotating machinery, and its performance directly determines the reliability and safety of the system. Data-based bearing fault diagnosis has become a research hotspot. Naive Bayes (NB), which is based on independent presumption, is widely used in fault diagnosis. However, the bearing data are not completely independent, which reduces the performance of NB algorithms. In order to solve this problem, we propose a NB bearing fault diagnosis method based on enhanced independence of data. The method deals with data vector from two aspects: the attribute feature and the sample dimension. After processing, the classification limitation of NB is reduced by the independence hypothesis. First, we extract the statistical characteristics of the original signal of the bearings effectively. Then, the Decision Tree algorithm is used to select the important features of the time domain signal, and the low correlation features is selected. Next, the Selective Support Vector Machine (SSVM) is used to prune the dimension data and remove redundant vectors. Finally, we use NB to diagnose the fault with the low correlation data. The experimental results show that the independent enhancement of data is effective for bearing fault diagnosis.

An Evolutionary Multiobjective Model and Instance Selection for Support Vector Machines With Pareto-Based Ensembles

Support vector machines (SVMs) are among the most powerful learning algorithms for classification tasks. However, these algorithms require a high computational cost during the training phase, which can limit their application on large-scale datasets. Moreover, it is known that their effectiveness highly depends on the hyper-parameters used to train the model. With the intention of dealing with these, this paper introduces an evolutionary multiobjective model and instance selection (IS) approach for SVMs with Pareto-based ensemble, whose goals are, precisely, to optimize the size of the training set and the classification performance attained by the selection of the instances, which can be done using either a wrapper or a filter approach. Due to the nature of multiobjective evolutionary algorithms, several Pareto optimal solutions can be found. We study several ways of using such information to perform a classification task. To accomplish this, our proposal performs a processing over the Pareto solutions in order to combine them into a single ensemble. This is done in five different ways, which are based on: 1) a global Pareto ensemble; 2) error reduction; 3) a complementary error reduction; 4) maximized margin distance; and 5) boosting. Through a comprehensive experimental study we evaluate the suitability of the proposed approach and the Pareto processing, and we show its advantages over a single-objective formulation, traditional IS techniques, and learning algorithms.

Hybrid Method Based on Information Gain and Support Vector Machine for Gene Selection in Cancer Classification

It remains a great challenge to achieve sufficient cancer classification accuracy with the entire set of genes, due to the high dimensions, small sample size, and big noise of gene expression data. We thus proposed a hybrid gene selection method, Information Gain-Support Vector Machine (IG-SVM) in this study. IG was initially employed to filter irrelevant and redundant genes. Then, further removal of redundant genes was performed using SVM to eliminate the noise in the datasets more effectively. Finally, the informative genes selected by IG-SVM served as the input for the LIBSVM classifier. Compared to other related algorithms, IG-SVM showed the highest classification accuracy and superior performance as evaluated using five cancer gene expression datasets based on a few selected genes. As an example, IG-SVM achieved a classification accuracy of 90.32% for colon cancer, which is difficult to be accurately classified, only based on three genes including CSRP1, MYL9, and GUCA2B.

Hierarchical Agglomerative Clustering Based Combined Feature Selection and Multi-Class Support Vector Machine for Brain Tumour Classification

Hierarchical Agglomerative Clustering Based Combined Feature Selection and Multi-Class Support Vector Machine for Brain Tumour Classification

Pathway-based Analysis with Support Vector Machine (SVM-LASSO) for Gene Selection and Classification

Genomic knowledge has become a popular research field in bioinformatics biological process that providing further biological process information. Many methods have been done to address the issues of high data throughput due to increased use of microarray technology. However, it is still not able to determine the appropriate diseases accurately. This is because of existing non-informative genes that could be included in the analysis of context specific data like cancer gene expression data, which affect the classification performance. This study proposed a pathway-based analysis for gene classification. Pathway-based analysis enable handling microarray data in order to improved biological interpretation of the analysis outcome. Secondly, Support Vector Machine with Least Absolute Shrinkage and Selection Operator algorithm (SVM-LASSO) is proposed, which to find informative genes for each pathway to ensure efficient gene selection and classification in every pathway. Experiments are done using lung cancer dataset and breast cancer dataset that widely used in cancer classification area. A stratified 10-fold cross validation is implement to evaluate the performance of the proposed method in terms of accuracy, specificity and sensitivity. Moreover, biological validation have been done on the selected genes based on biological literatures and biological databases. Next, the results from the proposed methods are compared with the previous study throughout all the data sets in terms of performance. As conclusion, this research finding can contribute in biology area especially in cancer classification area.

Synchronized feature selection for Support Vector Machines with twin hyperplanes

In this work, a novel feature selection method for twin Support Vector Machine (SVM) is presented. The main idea is to combine two regularizers, namely the Euclidean and infinite norm to perform twin classification and variable selection simultaneously. This latter task is performed in a coordinated fashion, enabling that the same attributes are selected in each twin classifiers. A single optimization problem is used to solve both subproblems, leading to a sparse final classification rule. Experiments on low- and high-dimensional datasets indicate that our approaches present the best average performance compared to well-known feature selection strategies, also achieving a synchronized feature elimination in the two twin classifiers. Our approaches are also able to improve the performance of the twin classifier, demonstrating the importance of feature selection in high-dimensional tasks.

Cost-based feature selection for Support Vector Machines: An application in credit scoring

In this work we propose two formulations based on Support Vector Machines for simultaneous classification and feature selection that explicitly incorporate attribute acquisition costs. This is a challenging task for two main reasons: the estimation of the acquisition costs is not straightforward and may depend on multivariate factors, and the inter-dependence between variables must be taken into account for the modelling process since companies usually acquire groups of related variables rather than acquiring them individually. Mixed-integer linear programming models are proposed for constructing classifiers that constrain acquisition costs while classifying adequately. Experimental results using credit scoring datasets demonstrate the effectiveness of our methods in terms of predictive performance at a low cost compared to well-known feature selection approaches.

A multi-objective genetic algorithm for simultaneous model and feature selection for support vector machines

The Support Vector Machines (SVM) constitute a very powerful technique for pattern classification problems. However, its efficiency in practice depends highly on the selection of the kernel function type and relevant parameter values. Selecting relevant features is another factor that can also impact the performance of SVM. The identification of the best set of parameters values for a classification model such as SVM is considered as an optimization problem. Thus, in this paper, we aim to simultaneously optimize SVMs parameters and feature subset using different kernel functions. We cast this problem as a multi-objective optimization problem, where the classification accuracy, the number of support vectors, the margin and the number of selected features define our objective functions. To solve this optimization problem, a method based on multi-objective genetic algorithm NSGA-II is suggested. A multi-criteria selection operator for our NSGA-II is also introduced. The proposed method is tested on some benchmark data-sets. The experimental results show the efficiency of the proposed method where features were reduced and the classification accuracy has been improved.

Optimal Parameter Selection for Support Vector Machine Based on Artificial Bee Colony Algorithm: A Case Study of Grid-Connected PV System Power Prediction.

Predicting the output power of photovoltaic system with nonstationarity and randomness, an output power prediction model for grid-connected PV systems is proposed based on empirical mode decomposition (EMD) and support vector machine (SVM) optimized with an artificial bee colony (ABC) algorithm. First, according to the weather forecast data sets on the prediction date, the time series data of output power on a similar day with 15-minute intervals are built. Second, the time series data of the output power are decomposed into a series of components, including some intrinsic mode components IMFn and a trend component Res, at different scales using EMD. The corresponding SVM prediction model is established for each IMF component and trend component, and the SVM model parameters are optimized with the artificial bee colony algorithm. Finally, the prediction results of each model are reconstructed, and the predicted values of the output power of the grid-connected PV system can be obtained. The prediction model is tested with actual data, and the results show that the power prediction model based on the EMD and ABC-SVM has a faster calculation speed and higher prediction accuracy than do the single SVM prediction model and the EMD-SVM prediction model without optimization.

Grey wolf optimization based parameter selection for support vector machines

Purpose The purpose of this paper is to tune support vector machine (SVM) classifier using grey wolf optimizer (GWO). Design/methodology/approach The schema of the work aims at extracting the features from the collected data followed by a SVM classifier and metaheuristic optimization to tune the classifier parameters. Findings The optimal tuning of classifier parameters lowers errors due to manual elucidation and decreases the risk in human perceptions and repeated visual dignosis. Originality/value A novel, GWO based tuning algorithm is used for SVM classifier, which is implemented in classifying the complex and nonlinear biomedical signals like intracranial electroencephalogram.

Land Usage Analysis: A Machine Learning Approach

In this article, machine learning based land usage analysis has been investigated. The objective is twofold: Firstly, the analysis and usage of simple pixel based features from the more complex Hyper Spectral images to land cover recognition. Secondly, an investigation into the parametric and non-parametric machine learning algorithms for the pixel based land cover analysis. For an experimental evaluation, we use the SPOT-5 satellite imagery having resolution of 2.5m. From the machine learning set, we select Support Vector Machine (SVM), Maximum Likelihood Estimator (MLE) and Artificial Neural Network (ANN). These algorithms are selected based on their superior performance in pattern recognition tasks. We distribute the feature space in seven classes i.e. Roads, Settled Areas, Tobacco, Sparse Vegetation, Sugar Cane, Barren Land and water. From the extensive experimentation, and in the current setup, it is concluded that SVM is best suited to the land cover analysis.

Differential evolution-based parameters optimisation and feature selection for support vector machine

This paper addresses the problem of SVM classification optimisation. For this purpose, the authors propose an SVM classification system based on differential evolution DE to improve the generalisation performance of the SVM classifier. In the classification system, a method of simultaneous parameters optimisation and feature selection for support vector machine is put forward. The experiments are conducted on the basis of benchmark dataset. The obtained results clearly confirm the superiority of the DE-SVM-FS approach compared to default SVM classifier and DE-SVM algorithm; this suggests that further substantial improvements in terms of classification accuracy can be achieved by the proposed DE-SVM-FS classification system.

Differential evolution-based parameters optimisation and feature selection for support vector machine

Differential evolution-based parameters optimisation and feature selection for support vector machine

IMPROVING CLASSIFICATION ACCURACY FOR NON-COMMUNICABLE DISEASE PREDICTION MODEL BASED ON SUPPORT VECTOR MACHINE

Over recent years, Non-communicable Disease (NCDs) is the high mortality rate in worldwide likely diabetes mellitus, cardiovascular diseases, liver and cancers. NCDs prediction model have problems such as redundant data, missing data, imbalance dataset and irrelevant attribute. This paper proposes a novel NCDs prediction model to improve accuracy. Our model comprisesk-means as clustering technique, Weight by SVM as feature selection technique and Support Vector Machine as classifier technique. The result shows that k-means + weight SVM + SVM improved the classification accuracy on most of all NCDs dataset (accuracy; AUC), likely Pima Indian Dataset (99.52; 0.999), Breast Cancer Diagnosis Dataset (98.85; 1.000), Breast Cancer Biopsy Dataset (97.71; 0.998), Colon Cancer (99.41; 1.000), ECG (98.33; 1.000), Liver Disorder (99.13; 0.998).The significant different performed by k-means + weight by SVM + SVM. In the time to come, we are expecting to better accuracy rate with another classifier such as Neural Network.

Improved Variable Importance Measure of Random Forest via Combining of Proximity Measure and Support Vector Machine for Stable Feature Selection

Improved Variable Importance Measure of Random Forest via Combining of Proximity Measure and Support Vector Machine for Stable Feature Selection

A memetic algorithm with support vector machine for feature selection and classification

The memetic algorithm (MA) is an evolutionary metaheuristic that can be viewed as a hybrid genetic algorithm combined with some kinds of local search. In this paper, we propose a memetic algorithm combined with a support vector machine (SVM) for feature selection and classification in Data mining. The proposed approach tries to find a subset of features that maximizes the classification accuracy rate of SVM. In addition, another hybrid algorithm of MA and SVM with optimized parameters is also developed. The two versions of our proposed method are evaluated on some datasets and compared with some well-known classifiers for data classification. The computational experiments show that the hybrid method MA + SVM with optimized parameters provides competitive results and finds high quality solutions.