Support Vector Candidates Research Articles

Selection of Support Vector Candidates Using Relative Support Distance for Sustainability in Large-Scale Support Vector Machines

Support vector machines (SVMs) are a well-known classifier due to their superior classification performance. They are defined by a hyperplane, which separates two classes with the largest margin. In the computation of the hyperplane, however, it is necessary to solve a quadratic programming problem. The storage cost of a quadratic programming problem grows with the square of the number of training sample points, and the time complexity is proportional to the cube of the number in general. Thus, it is worth studying how to reduce the training time of SVMs without compromising the performance to prepare for sustainability in large-scale SVM problems. In this paper, we proposed a novel data reduction method for reducing the training time by combining decision trees and relative support distance. We applied a new concept, relative support distance, to select good support vector candidates in each partition generated by the decision trees. The selected support vector candidates improved the training speed for large-scale SVM problems. In experiments, we demonstrated that our approach significantly reduced the training time while maintaining good classification performance in comparison with existing approaches.

Deep Fault Diagnosis for Rotating Machinery with Scarce Labeled Samples

Early and accurately detecting faults is crucial for the modern manufacturing system. We proposed a novel Deep fault diagnosis (DFD) method for rotating machinery with scarce labeled samples. A spectrogram of the raw vibration signal is calculated by applying a Short-time Fourier transform (STFT). Several candidate Support vector machine (SVM) models are trained with different combinations of features in the feature pool with scarce labeled samples. By evaluating the pretrained SVM models on the validation set, the most discriminative features and best-performed SVM models can be selected, which are used to make predictions on the unlabeled samples. The predicted labels reserve the expert knowledge originally carried by the SVM model. They are combined together with the scarce fine labeled samples to form an Augmented training set (ATS). Finally, a novel 2D deep Convolutional neural network (CNN) model is trained on the ATS to learn more discriminative features and a better classifier. Experimental results on two fault diagnosis datasets demonstrate the effectiveness of the proposed DFD.

Between-subclass piece-wise linear solutions in large scale kernel SVM learning

The paper proposes a novel approach for learning kernel Support Vector Machines (SVM) from large scale data with reduced computation time. The proposed approach, termed as Subclass Reduced Set SVM (SRS-SVM), utilizes the subclass structure of data to effectively estimate the candidate support vector set. Since the candidate support vector set cardinality is only a fraction of the training set cardinality, learning SVM from the former requires less time without significantly changing the decision boundary. SRS-SVM depends on a domain knowledge related input parameter, i.e., number of subclasses. To reduce the domain knowledge dependency and to make the approach less sensitive to the subclass parameter, we extend the proposed SRS-SVM to create a robust and improved hierarchical model termed as the Hierarchical Subclass Reduced Set SVM (HSRS-SVM). Since SRS-SVM and HSRS-SVM splits non-linear optimization problem into multiple (smaller) linear optimization problems, both of them are amenable to parallelization. The effectiveness of the proposed approaches is evaluated on four synthetic and six real-world datasets. The performance is also compared with traditional solver (LibSVM) and state-of-the-art approaches such as divide-and-conquer SVM, FastFood, and LLSVM. The experimental results demonstrate that the proposed approach achieves similar classification accuracies while requiring fewer folds of reduced computation time as compared to existing solvers. We further demonstrate the suitability and improved performance of the proposed HSRS-SVM with deep learning features for face recognition using Labeled Faces in the Wild (LFW) dataset.

Fixed least squares support vector machines for flashover modelling of outdoor insulators

Many theoretical and experimental studies have been performed to optimize the maintenance times of outdoor insulators. But most of the proposed methods are both time consuming and difficult to apply in practice and also they rely mainly on pollution charts and they are not reliable to weather variations. Artificial intelligence has been extensively applied to solve a large number of electrical and high voltage engineering modeling and optimization problems. In this paper, a methodology based on modified LS-SVM strategy using a fixed set of support vectors is proposed to evaluate the flashover performance of outdoor insulators under contaminated conditions, where the candidate support vectors are selected from the training set according to a quadratic Renyi criterion. The obtained results are promising and ensure that the presented technique can help high voltage engineers to assess insulator performance including such important factors as the flashover behavior, the insulator geometrical parameters, aging, and contamination accumulation. Further comparative analysis of the estimated results with the measured data collected from the site measurement demonstrate the effectiveness of the use of fixed-size LS-SVMs models for flashover prediction.

Health Evaluation of MVB Based on SVDD and Sample Reduction

Multifunction vehicle bus (MVB) is the most widely used train communication network whose performance degradation and anomaly will heavily affect the train’s safe and stable operation. However, current scheduled maintenance and post-failure maintenance of MVB cannot detect the early anomaly and evaluate the health condition of the network in time. This paper provides a method to detect the anomaly and evaluate the health condition of MVB based on a one-class classification (OCC) algorithm called density-based sample reduction for support vector data description (DBSRSVDD). First, network features are extracted from physical layer waveform parameters. In order to reduce the computational complexity of SVDD, a sample reduction operation is conducted to screen out the edge samples as support vector candidates. Then, the SVDD models representing the normal patterns of a single MVB node are trained based on the support vector candidates. Performance degradation of the node is quantified by the distance between the tested sample and the trained hyper sphere. The whole network’s health condition is the linear weighted sum of the nodes’ scores based on their bandwidth occupancy. The experimental results show that the proposed method can detect the anomaly and degradation of MVB successfully, improve accuracy, and reduce training time compared with the existing methods.

Intelligent recognition algorithm for social network sensitive information based on classification technology

In the social network, there is the problem of network sensitive information with low accuracy rate of information recognition. To effectively improve the accuracy of intelligent identification of sensitive information, an intelligent recognition algorithm for sensitive information based on improved fuzzy support vector machine is proposed in this paper. The information is collected. The trajectory of the best movement of the information node is found in the low energy cache. In the limited time, the performance of information acquisition is improved by using the mobility of information nodes. According to DFS criterion, the features are added into the feature subset or eliminate the sensitive information. The feature selection algorithm based on multi-label is applied to feature selection of the collected information, so that the information gain between information feature and label set can be used to measure the importance. The improved support vector machine classification algorithm is used to classify the information selected by feature selection, and select effective candidate support vector, reduce the number of training samples, and improve the training speed. The new membership function is defined to enhance the effect of support vector on the construction of fuzzy support vector machine. Finally, the nearest neighbor sample density is applied to the design of membership function to reduce the noise, and achieve intelligent recognition of the sensitive information in the social network. Experimental results show that the accuracy rate of sensitive information intelligent recognition can be effectively improved by using the proposed algorithm.

Support vector candidates selection via Delaunay graph and convex-hull for large and high-dimensional datasets

We propose a method to pre-select support vector (SV) candidates for training support vector machines (SVM) with a large-scale dataset. The technique creates a support vector candidates set to feed the SVM training phase, where this set is built by rescaling the dataset to three dimensions, if necessary, and creating a Delaunay Graph and a convex-hull (CH) for each class. The SV candidates set is formed by picking the points from all CHs, and its neighbors from the Delaunay graph, even in a reduced dimension. By testing the technique in four datasets with different size and feature number, we demonstrate that the proposed method accelerates SVM training process without degrading accuracy proportionally to the difference between original dataset and SV candidates set dimensions.

Support vector machine incremental learning triggered by wrongly predicted samples

According to the classic Karush-Kuhn-Tucker (KKT) theorem, at every step of incremental support vector machine (SVM) learning, the newly adding sample which violates the KKT conditions will be a new support vector (SV) and migrate the old samples between SV set and non-support vector (NSV) set, and at the same time the learning model should be updated based on the SVs. However, it is not exactly clear at this moment that which of the old samples would change between SVs and NSVs. Additionally, the learning model will be unnecessarily updated, which will not greatly increase its accuracy but decrease the training speed. Therefore, how to choose the new SVs from old sets during the incremental stages and when to process incremental steps will greatly influence the accuracy and efficiency of incremental SVM learning. In this work, a new algorithm is proposed to select candidate SVs and use the wrongly predicted sample to trigger the incremental processing simultaneously. Experimental results show that the proposed algorithm can achieve good performance with high efficiency, high speed and good accuracy.

Two-phase optimization for support vectors and parameter selection of support vector machines: Two-class classification

Support vector machines (SVMs) are one of the most popular classification tools and show the most potential to address under-sampled noisy data (a large number of features and a relatively small number of samples). However, the computational cost is too expensive, even for modern-scale samples, and the performance largely depends on the proper setting of parameters. As the data scale increases, the improvement in speed becomes increasingly challenging. As the dimension (feature number) largely increases while the sample size remains small, the avoidance of overfitting becomes a significant challenge. In this study, we propose a two-phase sequential minimal optimization (TSMO) to largely reduce the training cost for large-scale data (tested with 3186–70,000-sample datasets) and a two-phased-in differential-learning particle swarm optimization (tDPSO) to ensure the accuracy for under-sampled data (tested with 2000–24481-feature datasets). Because the purpose of training SVMs is to identify support vectors that denote a hyperplane, TSMO is developed to quickly select support vector candidates from the entire dataset and then identify support vectors from those candidates. In this manner, the computational burden is largely reduced (a 29.4%–65.3% reduction rate). The proposed tDPSO uses topology variation and differential learning to solve PSO’s premature convergence issue. Population diversity is ensured through dynamic topology until a ring connection is achieved (topology-variation phases). Further, particles initiate chemo-type simulated-annealing operations, and the global-best particle takes a two-turn diversion in response to stagnation (event-induced phases). The proposed tDPSO-embedded SVMs were tested with several under-sampled noisy cancer datasets and showed superior performance over various methods, even those methods with feature selection for the preprocessing of data.

Fast data selection for SVM training using ensemble margin

Support Vector Machine (SVM) is a powerful classification method. However, it suffers a major drawback: the high memory and time complexity of the training which constrains the application of SVM to large size classification tasks. To accelerate the SVM training, a new ensemble margin-based data selection approach is proposed. It relies on a simple and efficient heuristic to provide support vector candidates: selecting lowest margin instances. This technique significantly reduces the SVM training task complexity while maintaining the accuracy of the SVM classification. A fast alternative of our approach we called SVIS (Small Votes Instance Selection) with great potential for large data problem is also introduced.

Selection of Candidate Support Vectors in incremental SVM for network intrusion detection

In an Incremental Support Vector Machine classification, the data objects labelled as non-support vectors by the previous classification are re-used as training data in the next classification along with new data samples verified by Karush–Kuhn–Tucker (KKT) condition. This paper proposes Half-partition strategy of selecting and retaining non-support vectors of the current increment of classification – named as Candidate Support Vectors (CSV) – which are likely to become support vectors in the next increment of classification. This research work also designs an algorithm named the Candidate Support Vector based Incremental SVM (CSV-ISVM) algorithm that implements the proposed strategy and materializes the whole process of incremental SVM classification. This work also suggests modifications to the previously proposed concentric-ring method and reserved set strategy. Performance of the proposed method is evaluated with experiments and also by comparing it with other ISVM techniques. Experimental results and performance analyses show that the proposed algorithm CSV-ISVM is better than general ISVM classifications for real-time network intrusion detection.

Small sample-oriented case-based kernel predictive modeling and its economic forecasting applications under n-splits-k-times hold-out assessment

In most situations, we can only collect small samples for modeling specific economic forecasting problems. Thus, the availability of accurate predictive model for small samples is vital for solving these problems. This research makes an early investigation on the small sample-oriented case-based kernel predictive method (SSOCBKPM) by integrating support vector machine in case reuse of case-based reasoning, and conducts an early application of SSOCBKPM in binary economic forecasting under the n-splits-k-times hold-out method. After business cases consisting of small samples are represented, the most similar cases from small samples to the current problem are retrieved from small case base. The most similar cases retrieved from small samples are then mapped into a higher dimensional space by kernel function to be candidate support vectors, in which dimension a hyper-plane of support vector machine is constructed by reusing the most similar cases. Two datasets for firm failure prediction and one dataset for loan failure prediction were used to test performance of SSOCBKPM. 100 times' random selection of each of the 20%, 35%, 50%, 65%, and 80% of the total samples are respectively used in training to simulate the availability of samples. The results indicate that SSOCBKPM improves accuracy, stability and sensitivity of the classical CBR significantly; and improves performance of SVM significantly when the volume of the training samples becomes smaller. The SSOCBKPM is more useful in economic forecasting than case-based reasoning and support vector machine since the proportion of available samples is commonly small and less than 50% of the population.

Novel method of mining classification information for SVM training

Support vector machine (SVM) is an important classification tool in the pattern recognition and machine learning community, but its training is a time-consuming process. To deal with this problem, we propose a novel method to mine the useful information about classification hidden in the training sample for improving the training algorithm, and every training point is assigned to a value that represents the classification information, respectively, where training points with the higher values are chosen as candidate support vectors for SVM training. The classification information value for a training point is computed based on the classification accuracy of an appropriate hyperplane for the training sample, where the hyperplane goes through the mapped target of the training point in feature space defined by a kernel function. Experimental results on various benchmark datasets show the effectiveness of our algorithm.

FAST FUZZY MODELING USING A NEW INCREMENTAL SVR

In this paper, we propose a novel approach to fast fuzzy modeling based on a new incremental support vector regression (SVR). Firstly a candidate support vectors selection strategy based on kernel Mahalanobis distance measurement is proposed. This strategy is further used to develop a new incremental learning algorithm to speed up the training process of SVR. Then a hybrid kernel function is utilized to represent an SVR model as a TS fuzzy model. Finally a set of fuzzy rules can be directly extracted from the learning results of SVR. Experimental results of two benchmark examples show that the proposed model not only possesses satisfactory accuracy and generalization ability but also costs less computational time.