Training Time Of Support Vector Machine Research Articles

Choosing Decision Tree-Based Boundary Patterns in The Intrusion Detection Systems with Large Data Sets

Today, due to the growing use of computer networks, the issue of security of these networks and the use of intrusion detection systems has received serious attention. A major challenge in intrusion detection systems is the enormous amount of data. The generalization capability of support vector machine has attracted the attention of intrusion detection systems in the last years. The main drawbacks of a support vector machine occur during its training phase, which is computationally expensive and dependent on the size of the input dataset. In this study, a new algorithm to speed up support vector machine training time is presented. In proposed method, First, Ant Colony Optimization (ACO) is used to find prototype samples, then a number of prototype samples is randomly selected and the approximate boundary is determined using support vector machine. Based on the approximate boundary obtained, boundary samples are determined using decision tree. Using these boundary samples, final model is obtained. To demonstrate the effectiveness of the proposed method, standard publicly available datasets have been used. The experiment results show that despite the data reduction, the proposed model produces results with similar accuracy and in a faster way than state-of-the art and the current Support Vector Machine (SVM) implementations.

Selection of Support Vector Candidates Using Relative Support Distance for Sustainability in Large-Scale Support Vector Machines

Support vector machines (SVMs) are a well-known classifier due to their superior classification performance. They are defined by a hyperplane, which separates two classes with the largest margin. In the computation of the hyperplane, however, it is necessary to solve a quadratic programming problem. The storage cost of a quadratic programming problem grows with the square of the number of training sample points, and the time complexity is proportional to the cube of the number in general. Thus, it is worth studying how to reduce the training time of SVMs without compromising the performance to prepare for sustainability in large-scale SVM problems. In this paper, we proposed a novel data reduction method for reducing the training time by combining decision trees and relative support distance. We applied a new concept, relative support distance, to select good support vector candidates in each partition generated by the decision trees. The selected support vector candidates improved the training speed for large-scale SVM problems. In experiments, we demonstrated that our approach significantly reduced the training time while maintaining good classification performance in comparison with existing approaches.

Facial micro-expression recognition: A machine learning approach

Micro-expression recognition is a growing research area owing to its application in revealing subtle intention of humans, especially while under high stake conditions. With the rapid increase in security issues all over the world, the use of micro-expressions to understand one's state of mind has received major interest. Micro-expressions are characterized by short duration and low intensity, hence, efforts to train humans in recognizing them have resulted in very low performances. Automatic recognition of micro-expressions using machine learning techniques thus promises a more effective result and saves time and resources. In this study, we explore the use of Extreme Learning Machine (ELM) for micro-expression recognition because of its fast learning ability and higher performance when compared with other models. Support Vector Machine (SVM) is used as a baseline model and its recognition performance and its training time compared with ELM training time. Feature extraction is performed on apex micro-expression frames using Local Binary Pattern (LBP) and on micro-expression videos divided into image sequences using a spatiotemporal feature extraction technique called Local Binary Pattern on Three Orthogonal Planes (LBP-TOP). Evaluation of the two models is performed on spontaneous facial micro-expression samples acquired from Chinese Academy of Sciences (CASME II). Results obtained from the experiments show that ELM produces a higher recognition performance than SVM in terms of accuracy, precision, recall and F-score when temporal features are used. Comparison between SVM and ELM training time also shows that ELM learns faster than SVM. An average training time of 0.3405 seconds is achieved for SVM while an average training time of 0.0409 seconds is achieved for ELM for the five selected micro-expression classes. This study shows that automatic recognition of micro-expressions is produces a better result when temporal features and a machine learning algorithm with fast learning speed are used.

CBCH (clustering-based convex hull) for reducing training time of support vector machine

Support vector machine (SVM) is an efficient machine learning technique widely applied to various classification problems due to its robustness. However, the training time grows dramatically as the number of training data increases. As a result, the applicability of SVM to large-scale datasets is somewhat limited. In SVM, only a few training samples called support vectors (SVs) affect the construction of hyperplane. Therefore, removing training data irrelevant to the SVs does not degrade the performance of SVM. In this paper the clustering-based convex hull (CBCH) scheme is introduced which allows to efficiently remove insignificant data and thereby reduce the training time of SVM. The CBCH scheme initially applies k-mean clustering algorithm to the given training data points, and then, the convex hull of each cluster is obtained. Only the vertices of the convex hulls and the data points relevant to the SVs are included as training data points. Computer simulation over various sizes and types of datasets reveals that the proposed scheme is considerably faster and more accurate than the existing SVM classifiers. The proposed algorithm is based on geometric interpretation of the SVM and applicable to both linearly separable and linearly inseparable datasets.

A multiseed-based SVM classification technique for training sample reduction

A support vector machine (SVM) is not a popular method for a very large dataset classification because the training and testing time for such data are computationally expensive. Many researchers try to reduce the training time of SVMs by applying sample reduction methods. Many methods reduced the training samples by using a clustering technique. To reduce its high computational complexity, several data reduction methods were proposed in previous studies. However, such methods are not effective to extract informative patterns. This paper demonstrates a new supervised classification method, multiseed-based SVM (MSB-SVM), which is particularly intended to deal with very large datasets for multiclass classification. The main contributions of the paper are (i) an efficient multiseed technique for selection of seed points from circular/elongated class training samples, (ii) adjacent class pair selection from the set of multiseeds by using the minimum spanning tree, and (iii) extraction of support vectors from class pair seed equivalent regions to manage multiclass classification problems without being computationally expensive. Experimental results on a variety of datasets showed better performance compared to other sample-reducing methods in terms of training and testing time. Traditional support vector machine (SVM) solution suffers from $O(n^{2})$ time complexity, which makes it impractical for very large datasets. Here, multiseed point technique depends on the estimated density of each data, and the order of computation is $O(n$ log $n)$. Using the estimated density, the computational cost of the seed selection algorithm is $O(n)$. So, this is the only burden for reducing the sample. However, reducing the sample takes less time with the proposed algorithm compared to the clustering methods. At the same time, the number of support vectors has been abruptly reduced, which takes less time to find the decision surface. Apart from this, the classification accuracy of the proposed technique is significantly better than other existing sample reduction methods especially for large datasets.

A Classification Prediction Analysis of Flight Cancellation Based on Spark

Nowadays the phenomenon of flight delays and cancellations is becoming more and more serious. Flight delays and cancellations not only waste transportation resources, but also affect passengers’ travel plans, which cause an increase in passenger discontent and complaint rates[2]. The passengers’ dissatisfaction and distrust of airlines seriously damage the airlines’ corporate reputation and then affect passengers’ loyalty. Therefore, based on the information of 5 million flight data in the United States in 2016, we predicted the flight cancellations based on four classification algorithms: logistics regression, support vector machine (SVM), naive bayes and decision tree. We compared the training time of these algorithms by changing the number of nodes in the Spark cluster[9], and we also compared the classification accuracy, AUC (Area Under Curve) and PR(Precision-Recall). Experimental results show that: (1) When the number of nodes was 1, the average training time of these algorithms was 460 seconds. When the number of nodes was 3,5,7,9, the corresponding average training time were 207.25 seconds, 89.75 seconds, 44 seconds, 42 seconds, and when the number of nodes was 11, the average training time was 43.5 seconds; (2) Starting from 7 nodes, as the number of nodes increased while the decline rate of the average training time slowed down; (3) When the number of nodes was 1 and 3, the training time of naive bayes was the shortest, which were 386 seconds and 185 seconds respectively. When the number of nodes was 5,7,9 and11, the training time of SVM was the shortest, which were 72 seconds, 21seconds, 19 seconds and 18 seconds respectively; (4) the classification accuracy of SVM and decision tree was almost 90%, but naive bayes was just about 50.8% and logistics regression was only about 62.4%; (5) Both the AUC and PR of the decision tree algorithm were the highest: AUC was 0.558 and PR was 0.439. Therefore, the decision tree algorithm was the most suitable for predicting whether the flight would be cancelled or not, and there was a 90% probability that the prediction would be successful.

Efficient Decision Tree Based Data Selection and Support Vector Machine Classification

Today’ real world data bases witnessed significant increase in the amount of data in digital format, due to the widespread use of datasets and storage system. There is a need to developing fast and highly accurate algorithms to automatically classify large data. It becomes a vital part of the machine learning and knowledge discovery. The main intention of this paper is however data sizes increases, our proposed method make faster computation and scalable machine learning algorithm is used to learn faster from the labelled training data. Due to its strong mathematical background and theoretical foundation and good generalization performance, Support Vector Machine (SVM) Classification becomes more feasible options for large datasets. A major research goal of SVM is to improve the speed in training and testing phase. In this paper We introduce a proposed algorithm to speed up the training time of SVM is presented. It is highly accurate classification method. However SVM classifiers suffer from slow processing, when training with a large set of data tuples. Our novel approach selects a small representative amount of data from large datasets to enhance training time of SVM. This method uses an induction tree to reduce the training dataset for SVM classification, it generate faster results with improving accuracy rates than the current SVM implementations.

A faster cutting plane algorithm with accelerated line search for linear SVM

Cutting plane algorithm (CPA) is a generalization of iterative first-order gradient method, in which the objective function is approximated successively by supporting hyperplanes. CPA has been tailored to solve regularized loss minimization in machine learning by exploiting the regularization structure. In particular, for linear Support Vector Machine (SVM) embedding a line search procedure effectively remedies the fluctuations of function value and speeds up the convergence in practical issue. However, the existing line search strategy based on sorting algorithm takes O(mlog m) time. In this paper, we propose a more effective line search solver which spends only linear time. It can be extended to multiclass SVM in which an optimized explicit piecewise linear function finding algorithm is prearranged. The total SVM training time is proved to reduce theoretically and experiments consistently confirm the effectiveness of the proposed algorithms.

Improved Instance Selection Methods for Support Vector Machine Speed Optimization

Support vector machine (SVM) is one of the top picks in pattern recognition and classification related tasks. It has been used successfully to classify linearly separable and nonlinearly separable data with high accuracy. However, in terms of classification speed, SVMs are outperformed by many machine learning algorithms, especially, when massive datasets are involved. SVM classification speed scales linearly with number of support vectors, and support vectors increase with increase in dataset size. Hence, SVM classification speed can be enormously reduced if it is trained on a reduced dataset. Instance selection techniques are one of the most effective techniques suitable for minimizing SVM training time. In this study, two instance selection techniques suitable for identifying relevant training instances are proposed. The techniques are evaluated on a dataset containing 4000 emails and results obtained compared to other existing techniques. Result reveals excellent improvement in SVM classification speed.

Fault diagnosis method based on FFT-RPCA-SVM for Cascaded-Multilevel Inverter

Thanks to reduced switch stress, high quality of load wave, easy packaging and good extensibility, the cascaded H-bridge multilevel inverter is widely used in wind power system. To guarantee stable operation of system, a new fault diagnosis method, based on Fast Fourier Transform (FFT), Relative Principle Component Analysis (RPCA) and Support Vector Machine (SVM), is proposed for H-bridge multilevel inverter. To avoid the influence of load variation on fault diagnosis, the output voltages of the inverter is chosen as the fault characteristic signals. To shorten the time of diagnosis and improve the diagnostic accuracy, the main features of the fault characteristic signals are extracted by FFT. To further reduce the training time of SVM, the feature vector is reduced based on RPCA that can get a lower dimensional feature space. The fault classifier is constructed via SVM. An experimental prototype of the inverter is built to test the proposed method. Compared to other fault diagnosis methods, the experimental results demonstrate the high accuracy and efficiency of the proposed method.

Data selection based on decision tree for SVM classification on large data sets

Support Vector Machine (SVM) has important properties such as a strong mathematical background and a better generalization capability with respect to other classification methods. On the other hand, the major drawback of SVM occurs in its training phase, which is computationally expensive and highly dependent on the size of input data set. In this study, a new algorithm to speed up the training time of SVM is presented; this method selects a small and representative amount of data from data sets to improve training time of SVM. The novel method uses an induction tree to reduce the training data set for SVM, producing a very fast and high-accuracy algorithm. According to the results, the proposed algorithm produces results with similar accuracy and in a faster way than the current SVM implementations.

Comparison of classifiers for lip reading with CUAVE and TULIPS database

Automatic lip reading is a technique of understanding the uttered speech by visually interpreting the lip movement of the speaker. The two major parts, which play crucial role in lip reading system, are feature extraction followed by the classifier. For automatic lip reading, there are many competing methods published by researchers for feature extraction and classifiers. In this paper, we compare some of these leading methods. We have compared Support Vector Machine (SVM), Back Propagation Neural Network (BPNN), K-Nearest Neighborhood (KNN), Random Forest Method (RFM) and Naive Bayes (NB) classifiers, on the basis of recognition performance and training time. Discrete Cosine Transform (DCT) and Discrete Wavelet Transform (DWT) are studied to extract feature vectors. The CUAVE and Tulips database are used for experimentation and comparison. It is observed that SVM outperforms the rest for CUAVE database. Training time of SVM is also less than others.

The Computer Interlocking Software System Reliability Test Based on the Monte Carlo

The railway switch failure prediction for railway signal equipment maintenance plays an important role. The paper put forward railway switch failure prediction algorithm based on least squares support vector machine, and chose five characteristic indexes composed of railway switch failure prediction models characteristic input vectors. It reduces the dimension of input vectors, shorten the least squares support vector machine training time, and use a pruning algorithm to accelerate the computing speed maintaining a good regression performance at the same time. The experiment proved that railway switch failure prediction algorithm has strong self-learning ability and higher prediction accuracy based on least squares support vector machine. And it can accelerate the speed of switch failure prediction and improve the accuracy and reliability of railway switch failure prediction.

An Approach for Predicting Clinical EEG Signal Based on SVM

Accurate modeling of Electroencephalography (EEG) signals is an important problem in clinical diagnosis of brain diseases. The method using support vectors machine (SVM) based on the structure risk minimization provides us an effective way of learning machine. But solving the quadratic programming problem for training SVM becomes a bottle-neck of using SVM because of the long time of SVM training. In this paper, a local-SVM method is proposed for modeling EEG signals. The local method is presented for improving the speed of the prediction of EEG signals. Furthermore, this proposed model is used to detect epilepsy from EEG signals in which dynamical characteristics are difference between normal and epilepsy EEG signals. The experimental results show that the training of the local-SVM obtains a good behavior. In addition, the local SVM method significantly improves the prediction and detection precision.

Electroencephalogram and Physiological Signal Analysis for Assessing Flow in Games

Passive brain-computer interaction (BCI) can provide useful information to understand a user's state and anticipate intentions, which is needed to support adaptivity and personalization. Given the huge variety of audiences, a game's capability of adapting to different user profiles-in particular to keep the player in flow-is crucial to make it ever more enjoyable and satisfying. We have performed a user experiment exploiting a four-electrode electroencephalogram (EEG) tool similar to the ones that are soon likely to appear in the market for game control. We have performed a spectral characterization of the video-gaming experience, also in comparison with other tasks. Results show that the most informative frequency bands for discriminating among gaming conditions are around low beta. Simple signals from the peripheral nervous system add marginal information. Classification of three levels of user states is possible, with good accuracy, using a support vector machine (SVM) classifier. A user-independent classification performs worse than a user-dependent approach (50.1% versus 66.4% rate). Personalized SVM training and validation time is reasonable (7-8 min). Thus, we argue that a personalized system could be implemented in a consumer context and research should aim at improving classifiers that can be trained online by end users.

Network Based Anomaly Intrusion Detection System Using SVM

The security and integrity of a computer system is compromised when an intrusion occurs. It becomes impossible for legitimate users to access different network services when network-based attacks purposely occupy or sabotage network resources and services. Our proposed method is a scalable detection method for network based anomalies. We use Support Vector Machines (SVM) for classification. This paper presents a method for enhancing the training time of SVM, particularly when dealing with large data sets, using hierarchical clustering technique. We use the Dynamically Growing Self-Organizing Tree (DGSOT) algorithm for clustering because it has proved to overcome the problems of traditional hierarchical clustering algorithms (e.g., hierarchical agglomerative clustering). Clustering analysis helps to find the boundary points, which are the most qualified data points to train SVM, between any two classes. We present a new approach of combination of SVM and DGSOT, which begins with an initial training set and expands it gradually using the clustering structure produced by the DGSOT algorithm. We show that our proposed variations contribute significantly in improving the training process of SVM with high percentage of detection accuracy.

Network based anomaly intrusion detection system using SVM

The security and integrity of a computer system is compromised when an intrusion occurs. It becomes impossible for legitimate users to access different network services when network-based attacks purposely occupy or sabotage network resources and services. Our proposed method is a scalable detection method for network based anomalies. We use Support Vector Machines (SVM) for classification. This paper presents a method for enhancing the training time of SVM, particularly when dealing with large data sets, using hierarchical clustering technique. We use the Dynamically Growing Self-Organizing Tree (DGSOT) algorithm for clustering because it has proved to overcome the problems of traditional hierarchical clustering algorithms (e.g., hierarchical agglomerative clustering). Clustering analysis helps to find the boundary points, which are the most qualified data points to train SVM, between any two classes. We present a new approach of combination of SVM and DGSOT, which begins with an initial training set and expands it gradually using the clustering structure produced by the DGSOT algorithm. We show that our proposed variations contribute significantly in improving the training process of SVM with high percentage of detection accuracy. Keywords: SVM, classification, Intrusion detection, Intrusion detection System, Network Security

Application of various kernel function based SVM in spam filtering

The Support Vector Machine(SVM)based spam filter was summarized briefly.The mail vector was constructed on TF-IDF model and Bernoulli model.The effect to mail classification of CHI method to descend dimension was tested in detail.Kernel based SVM was introduced into spam filtering.The classification accuracy and training time of SVM based on linear kernel,polynomial kernel and radius basis function kernel were compared and analyzed.It was proposed and analyzed that the imbalance of training samples has great affect on the classification accuracy and the false positive ratio.

A new intrusion detection system using support vector machines and hierarchical clustering

Whenever an intrusion occurs, the security and value of a computer system is compromised. Network-based attacks make it difficult for legitimate users to access various network services by purposely occupying or sabotaging network resources and services. This can be done by sending large amounts of network traffic, exploiting well-known faults in networking services, and by overloading network hosts. Intrusion Detection attempts to detect computer attacks by examining various data records observed in processes on the network and it is split into two groups, anomaly detection systems and misuse detection systems. Anomaly detection is an attempt to search for malicious behavior that deviates from established normal patterns. Misuse detection is used to identify intrusions that match known attack scenarios. Our interest here is in anomaly detection and our proposed method is a scalable solution for detecting network-based anomalies. We use Support Vector Machines (SVM) for classification. The SVM is one of the most successful classification algorithms in the data mining area, but its long training time limits its use. This paper presents a study for enhancing the training time of SVM, specifically when dealing with large data sets, using hierarchical clustering analysis. We use the Dynamically Growing Self-Organizing Tree (DGSOT) algorithm for clustering because it has proved to overcome the drawbacks of traditional hierarchical clustering algorithms (e.g., hierarchical agglomerative clustering). Clustering analysis helps find the boundary points, which are the most qualified data points to train SVM, between two classes. We present a new approach of combination of SVM and DGSOT, which starts with an initial training set and expands it gradually using the clustering structure produced by the DGSOT algorithm. We compare our approach with the Rocchio Bundling technique and random selection in terms of accuracy loss and training time gain using a single benchmark real data set. We show that our proposed variations contribute significantly in improving the training process of SVM with high generalization accuracy and outperform the Rocchio Bundling technique.

Best terms: an efficient feature-selection algorithm for text categorization

In this paper, we propose a new feature-selection algorithm for text classification, called best terms (BT). The complexity of BT is linear in respect to the number of the training-set documents and is independent from both the vocabulary size and the number of categories. We evaluate BT on two benchmark document collections, Reuters-21578 and 20-Newsgroups, using two classification algorithms, naive Bayes (NB) and support vector machines (SVM). Our experimental results, comparing BT with an extensive and representative list of feature-selection algorithms, show that (1) BT is faster than the existing feature-selection algorithms; (2) BT leads to a considerable increase in the classification accuracy of NB and SVM as measured by the F1 measure; (3) BT leads to a considerable improvement in the speed of NB and SVM; in most cases, the training time of SVM has dropped by an order of magnitude; (4) in most cases, the combination of BT with the simple, but very fast, NB algorithm leads to classification accuracy comparable with SVM while sometimes it is even more accurate.