Binary Support Vector Machine Research Articles

HYBRID DECISION TREE ARCHITECTURE UTILIZING LOCAL SVMs FOR EFFICIENT MULTI-LABEL LEARNING

Multi-label learning (MLL) problems abound in many areas, including text categorization, protein function classification, and semantic annotation of multimedia. Issues that severely limit the applicability of many current machine learning approaches to MLL are the large-scale problem, which have a strong impact on the computational complexity of learning. These problems are especially pronounced for approaches that transform MLL problems into a set of binary classification problems for which Support Vector Machines (SVMs) are used. On the other hand, the most efficient approaches to MLL, based on decision trees, have clearly lower predictive performance. We propose a hybrid decision tree architecture, where the leaves do not give multi-label predictions directly, but rather utilize local SVM-based classifiers giving multi-label predictions. A binary relevance architecture is employed in the leaves, where a binary SVM classifier is built for each of the labels relevant to that particular leaf. We use a broad range of multi-label datasets with a variety of evaluation measures to evaluate the proposed method against related and state-of-the-art methods, both in terms of predictive performance and time complexity. Our hybrid architecture on almost every large classification problem outperforms the competing approaches in terms of the predictive performance, while its computational efficiency is significantly improved as a result of the integrated decision tree.

A Novel Approach of Analog Circuit Sensor Fault Diagnosis Using Fuzzy Integrated Binary Support Vector Machines

A Novel Approach of Analog Circuit Sensor Fault Diagnosis Using Fuzzy Integrated Binary Support Vector Machines

Application of multi-class classification algorithm based on twin support vector machine in intrusion detection

The multi-class classification algorithms based on traditional Support Vector Machine(SVM) are weak on training speed when dealing with large-scale data.To solve the problem,this paper proposed a multi-class classification algorithm based on Twin Support Vector Machine(TWSVM).It combined binary tree multi-class classification and constructed classifiers based on TWSVM on the nodes of binary tree.To reduce the error accumulation of Binary Tree SVM(BT-SVM),it firstly got clustering centers through the clustering algorithm,and then compared the distances between them to determine the separation sequence of classes.Finally,it was applied to network intrusion detection.The experimental results show that the proposed algorithm has higher detection accuracy and certain advantages on training speed especially for large-scale data.The training speed is approximately two times faster than that of the traditional BT-SVM algorithm.It is valuable for large-scale data processing in the field of network intrusion detection.

A sequence labeling approach to link medications and their attributes in clinical notes and clinical trial announcements for information extraction

ObjectiveThe goal of this work was to evaluate machine learning methods, binary classification and sequence labeling, for medication–attribute linkage detection in two clinical corpora.Data and methodsWe double annotated 3000 clinical trial announcements (CTA) and 1655 clinical notes (CN) for medication named entities and their attributes. A binary support vector machine (SVM) classification method with parsimonious feature sets, and a conditional random fields (CRF)-based multi-layered sequence labeling (MLSL) model were proposed to identify the linkages between the entities and their corresponding attributes. We evaluated the system's performance against the human-generated gold standard.ResultsThe experiments showed that the two machine learning approaches performed statistically significantly better than the baseline rule-based approach. The binary SVM classification achieved 0.94 F-measure with individual tokens as features. The SVM model trained on a parsimonious feature set achieved 0.81 F-measure for CN and 0.87 for CTA. The CRF MLSL method achieved 0.80 F-measure on both corpora.Discussion and conclusionsWe compared the novel MLSL method with a binary classification and a rule-based method. The MLSL method performed statistically significantly better than the rule-based method. However, the SVM-based binary classification method was statistically significantly better than the MLSL method for both the CTA and CN corpora. Using parsimonious feature sets both the SVM-based binary classification and CRF-based MLSL methods achieved high performance in detecting medication name and attribute linkages in CTA and CN.

Transient Stability Assessment of Power System Using Support Vector Machine with Generator Phase Plane Trajectory Inputs

Transient stability assessment (TSA) is part of dynamic stability assessment of power systems, which involves the assessment of the systems ability to remain synchronism under credible disturbances. Recent research shows that transient stability status of a power system following a large disturbance such as a fault can be early predicted based on phase plane trajectories of generator variables. Based on this, a binary support vector machine (SVM) classifier with generator phase plane trajectory inputs was trained to predict the transient stability status. In order to find the best trajectory inputs, three different types of phase plane trajectories were designed. By investigating effectiveness of the three trajectories with New England 39-bus test system, classifiers with phase plane trajectories of electromagnetic power as inputs achieved better predictions than other two types of trajectories. The highest accuracy achieved by the classifier with inputs of electromagnetic power phase plane trajectories is 99.336% which can meet requirements of practical application.

Emotion detection in suicide notes

The success of suicide prevention, a major public health concern worldwide, hinges on adequate suicide risk assessment. Online platforms are increasingly used for expressing suicidal thoughts, but manual monitoring is unfeasible given the information overload experts are confronted with. We investigate whether the recent advances in natural language processing, and more specifically in sentiment mining, can be used to accurately pinpoint 15 different emotions, which might be indicative of suicidal behavior.A system for automatic emotion detection was built using binary support vector machine classifiers. We hypothesized that lexical and semantic features could be an adequate way to represent the data, as emotions seemed to be lexicalized consistently. The optimal feature combination for each of the different emotions was determined using bootstrap resampling. Spelling correction was applied to the input data, in order to reduce lexical variation.Classification performance varied between emotions, with scores up to 68.86% F-score. F-scores above 40% were achieved for six of the seven most frequent emotions: thankfulness, guilt, love, information, hopelessness and instructions. The most salient features are trigram and lemma bags-of-words and subjectivity clues. Spelling correction had a slightly positive effect on classification performance.We showed that fine-grained automatic emotion detection benefits from classifier optimization and a combined lexico-semantic feature representation. The modest performance improvements obtained through spelling correction might indicate the robustness of the system to noisy input text. We conclude that natural language processing techniques have future application potential for suicide prevention.

Automatic detection of sleep apnea and hypopnea events from single channel measurement of respiration signal employing ensemble binary SVM classifiers

This paper presents a novel method for automatic detection of apnea and hypopnea events as well as mean duration of events from the recording of single channel oronasal airflow signal, moreover the automated algorithm has been implemented with PC based low cost Data Acquisition System (DAS). The method divides the respiration signal into overlapping segments of typical 8s duration and then categorize the segments with the help of ensemble binary Support Vector Machine (SVM) classifiers, according to the origin of the segments, i.e. ‘N’ if the segment originates from normal respiration signal during sleep, ‘A’ if it originates from apnea and ‘H’ for hypopnea event related breathing signal. Finally, it uses a heuristically derived rule based system to identify the apnea or hypopnea events by combining the time sequenced decisions of the classifiers. Automatic identification of events helps to provide the direct estimation of Apnea Hypopnea Index (AHI) and thus severity. The overall correlation coefficients between the automatic model predicted indexes and the PSG based manual indexes were 0.970, 0.986 and 0.982 for HI, AI, and AHI respectively.

The one-against-all partition based binary tree support vector machine algorithms for multi-class classification

The binary tree support vector machine (SVM) algorithm is one of the mainstream algorithms for multi-class classification in the fields of pattern recognition and machine learning. In order to reduce the training and testing time of one-against-all SVM (OAA-SVM) and reduced OAA-SVM (R-OAA-SVM), in this study, two OAA partition based binary tree SVM algorithms are proposed for multi-class classification. One is the single-space-mapped binary tree SVM (SBT-SVM) and the other is the multi-space-mapped binary tree SVM (MBT-SVM). In the proposed two algorithms, the best OAA partition is determined for each non-leaf node and the k-fold cross validation strategy is adopted to obtain the optimal classifiers. A set of experiments is conducted on nine UCI datasets and two face recognition datasets to demonstrate their performances. The results show that in term of testing accuracy, MBT-SVM is comparable with one-against-one SVM (OAO-SVM), R-OAA-SVM and OAA-SVM and superior to SBT-SVM. In term of testing time, MBT-SVM is superior to OAO-SVM, binary tree of SVM (BTS), R-OAA-SVM and OAA-SVM and slightly longer than SBT-SVM. In term of training time, MBT-SVM is superior to BTS, R-OAA-SVM and OAA-SVM and comparable with SBT-SVM. For the datasets with smaller class number and training sample number, the training time of MBT-SVM is comparable with that of OAO-SVM. For the datasets with larger class number or training sample number, in most cases, the training time of MBT-SVM is longer than that of OAO-SVM.

Automated 2D detection of construction equipment and workers from site video streams using histograms of oriented gradients and colors

This paper presents a computer vision based algorithm for automated 2D detection of construction workers and equipment from site video streams. The state-of-the-art research proposes semi-automated detection methods for tracking of construction workers and equipment. Considering the number of active equipment and workers on jobsites and their frequency of appearance in a camera's field of view, application of semi-automated techniques can be time-consuming. To address this limitation, a new algorithm based on Histograms of Oriented Gradients and Colors (HOG + C) is proposed. Our proposed detector uses a single sliding window at multiple scales to identify the potential candidates for the location of equipment and workers in 2D. Each detection window is first divided into small spatial regions and then the gradient orientations and hue–saturation colors are locally histogrammed and concatenated to form the HOG + C descriptors. Tiling the sliding detection window with a dense and overlapping grid of formed descriptors and using a binary Support Vector Machine (SVM) classifier for each resource enables automated 2D detection of workers and equipment. A new comprehensive benchmark dataset containing over 8000 annotated video frames including equipment and workers from different construction projects is introduced. This dataset contains a large range of pose, scale, background, illumination, and occlusion variation. Our preliminary results on detection of standing workers, excavators and dump trucks with an average accuracy of 98.83%, 82.10%, and 84.88% respectively indicate the applicability of the proposed method for automated activity analysis of workers and equipment from single video cameras. Unlike other state-of-the-art algorithms in automated resource tracking, this method particularly detects idle resources and does not need manual or semi-automated initialization of the resource locations in 2D video frames. The experimental results and the perceived benefits of the proposed method are discussed in detail.

Twin support vector hypersphere (TSVH) classifier for pattern recognition

Motivated by the support vector data description, a classical one-class support vector machine, and the twin support vector machine classifier, this paper formulates a twin support vector hypersphere (TSVH) classifier, a novel binary support vector machine (SVM) classifier that determines a pair of hyperspheres by solving two related SVM-type quadratic programming problems, each of which is smaller than that of a conventional SVM, which means that this TSVH is more efficient than the classical SVM. In addition, the TSVH successfully avoids matrix inversion compared with the twin support vector machine, which indicates learning algorithms of the SVM can be easily extended to this TSVH. Computational results on several synthetic as well as benchmark data sets indicate that the proposed TSVH is not only faster, but also obtains better generalization.

Fault Diagnosis of Transmission Rolling Bearing Based on Wavelet Analysis and Binary Tree Support Vector Machine

Fault Diagnosis of Transmission Rolling Bearing Based on Wavelet Analysis and Binary Tree Support Vector Machine

Tree Species Classification and Input Data Evaluation

Semantic world models require detailed classifications that allow the generation of entities that represent real world objects. This study combines advanced binary support vector machine classifiers in the hierarchical structure of a binary decision tree. This approach fulfills the requirements of a subsequent semantic model generation approach as it is an object based approach that works on the single tree level. It is fast and can be applied easily without expert knowledge. The algorithm can be trained on existing tree sample inventory data and additional samples from other data sources can be imported or placed. Classification results were calculated on a variety of input data sources and spatial resolutions. The achieved accuracies were analyzed and provide information for the decision support regarding input data choice for future applications.

A study on output normalization in multiclass SVMs

The use of binary support vector machines (SVMs) in multi-classification is addressed in this paper. Margins associated to the bi-classifiers, since they depend on the geometrical disposition of the classes being separated, are, in general, of various magnitudes. In order to overcome this scaling problem, a normalization process should be applied on the SVMs’ outputs. Thus, a new normalization approach is presented based on the convex hulls that contain the classes to be separated. Furthermore, a theoretical study is developed which justifies the proposed approach, and an interpretation is provided. An empirical study is also carried out to compare this normalization with others found in the literature.

GA Optimized Binary Tree SVM for Analog Circuit Fault Diagnosis

This paper presents a novel method of analog circuit fault diagnosis based on genetic algorithm (GA) optimized binary tree support vector machine (SVM). The real-valued coding genetic algorithm is used to optimize the binary tree structure. In optimization algorithm, we use roulette wheel selection operator, partially mapped crossover operator, inversion mutation operator. In simulation experiment, we use Monte-carlo analysis for 40kHz Sallen-Key bandpass filter and get transient response of ten faults. Then we extract feature vector by db3 wavelet packet transform and principal component analysis (PCA), and diagnose circuit faults by different SVM methods. Experiment results show the proposed method has the better classification accuracy than one-against-one (o-a-o), one-against-rest (o-a-r), Directed Acyclic Graph SVM (DAGSVM) and binary tree SVM (BT-SVM). It is suitable for practical use.

Transient stability assessment of power system using support vector machine with generator combinatorial trajectories inputs

Transient stability assessment (TSA) is part of dynamic stability assessment of power systems, which involves the assessment of the system’s ability to remain synchronism under credible disturbances. By qualitative analysis, this paper shows that the transient stability status of a power system following a large disturbance such as a fault can be early predicted based on dynamic response trajectories of rotor angle, speed, voltage, electromagnetic power and imbalance power. Based on this, a binary support vector machine (SVM) classifier with combinatorial trajectories inputs was trained to predict the transient stability status. Besides, a credible area and an incredible area of the classifier were given to improve the practicality of the classifier and a revised strategy was proposed to improve the performance of the SVM classifier in incredible area. The proposed approach was implemented and tested on New England 39-bus test system. Results show that the proposed approach can achieve 99.759% accuracy in credible area and 93.611% accuracy in incredible area.

Wind turbine fault diagnosis method based on diagonal spectrum and clustering binary tree SVM

Abstract Renewable energy sources like wind energy are copiously available without any limitation. Reliability of wind turbine is critical to extract maximum amount of energy from the wind. The vibration signals in wind turbine's rotation parts are of universal non-Gasussian and nonstationarity and the fault samples are usually very limited. Aiming at these problems, this paper proposed a wind turbine fault diagnosis method based on diagonal spectrum and clustering binary tree Support Vector Machines (SVM). Firstly, the diagonal spectrum is calculated from vibration rotating machine as the input feature vector. Secondly, self-organizing feature map neural network is introduced to cluster the fault feature samples and construct a cluster binary tree. Then the multiple fault classifiers are designed to train and test samples. The wind turbine gear-box fault experiment results proved that this method can effectively extract features from nonstationary signals, and can obtain excellent results despite of less training samples.

Classification of Alzheimer's Disease from 3D Structural MRI Data

With nearly 100.000 cases in Algeria, Alzheimer's disease (AD) represents a major public health problem. Therefore, several different automated methods have been developed to assist clinicians in their diagnosis. We propose here a method based on binary support vector machines (SVM) to distinguish between patients with Alzheimer disease (AD), patients with mild cognitive impairment (MCI) and elderly control subjects (CS) from magnetic resonance imaging (MRI) data. In order to reduce the total computation time, we used the JADE (Java Agent DEvelopement Framework) multi-agent platform. The results obtained show the efficiency of our method and the significant advantages of the parallelization. General Terms Dementia Diagnosis, Neuroimaging, Supervised Learning, Multi-Agent Technology.

Topic Classification for Suicidology

Computational techniques for topic classification can support qualitative research by automatically applying labels in preparation for qualitative analyses. This paper presents an evaluation of supervised learning techniques applied to one such use case, namely, that of labeling emotions, instructions and information in suicide notes. We train a collection of one-versus-all binary support vector machine classifiers, using cost-sensitive learning to deal with class imbalance. The features investigated range from a simple bag-of-words and n-grams over stems, to information drawn from syntactic dependency analysis and WordNet synonym sets. The experimental results are complemented by an analysis of systematic errors in both the output of our system and the gold-standard annotations. Category: Smart and intelligent computing

Automatic sleep staging using fMRI functional connectivity data

Recent EEG-fMRI studies have shown that different stages of sleep are associated with changes in both brain activity and functional connectivity. These results raise the concern that lack of vigilance measures in resting state experiments may introduce confounds and contamination due to subjects falling asleep inside the scanner. In this study we present a method to perform automatic sleep staging using only fMRI functional connectivity data, thus providing vigilance information while circumventing the technical demands of simultaneous recording of EEG, the gold standard for sleep scoring. The features to classify are the linear correlation values between 20 cortical regions identified using independent component analysis and two regions in the bilateral thalamus. The method is based on the construction of binary support vector machine classifiers discriminating between all pairs of sleep stages and the subsequent combination of them into multiclass classifiers. Different multiclass schemes and kernels are explored. After parameter optimization through 5-fold cross validation we achieve accuracies over 0.8 in the binary problem with functional connectivities obtained for epochs as short as 60s. The multiclass classifier generalizes well to two independent datasets (accuracies over 0.8 in both sets) and can be efficiently applied to any dataset using a sliding window procedure. Modeling vigilance states in resting state analysis will avoid confounded inferences and facilitate the study of vigilance states themselves. We thus consider the method introduced in this study a novel and practical contribution for monitoring vigilance levels inside an MRI scanner without the need of extra recordings other than fMRI BOLD signals.

Feature Vector Selection Method Using Mahalanobis Distance for Diagnostics of Analog Circuits Based on LS-SVM

Multi-fault diagnosis for analog circuits based on support vector machine (SVM) usually used a single feature vector to train all binary SVM classifier. In fact, each binary SVM classifier has different classification accuracy for different feature vectors. However, no one has discussed the optimal or near-optimal feature vector selection problem. Based on Mahalanobis distance, a near-optimal feature vector selection method has been proposed for diagnostics of analog circuits using the least squares SVM (LS-SVM). The selection problems of wavelet types, wavelet decomposition level, and normalization methods have been also discussed. Two filters with parametric faults and a nonlinear half-wave rectifier with hard and parametric faults were used as circuits under test (CUTs). The simulation results showed the following: (1) the accuracies using the feature vector with the maximum MD were better than the average accuracies using all the feature vectors, and were better than most accuracies using a single feature vector. But the computation time using the MD method was an order of magnitude larger than that using a single feature vector; (2) Most the diagnostic accuracies using the maximum MD method were near to the optimal accuracies using the exhaustive method while the computation time was reduced about 20–50 % in comparision to the exhaustive method; (3) the Haar wavelet was the best choice among Daubechie’s wavelet family for all CUTs’ diagnosis; (4) only non-normalization, all-normalization, and part-normalization methods are necessary to be considered for feature vector normalization. The proposed method can obtain a near-optimal diagnostic accuracy in a reasonable time, which is beneficial for analog IC or circuits testing and diagnosis.