Two-class Support Vector Machine Research Articles

Whisper Intelligibility Enhancement Using a Supervised Learning Approach

Whispered speech can be effectively used for quiet and private communications over mobile phones. It is also the communication means of laryngectomized patients under a regime of voice rest. However, little progress has been made on the enhancement of whispered speech because of its special acoustic characteristics. Recent studies with normal-hearing listeners have reported large gains in speech intelligibility with the binary mask approach. This method retains the time-frequency (T-F) units of the mixture signal that are stronger than the interfering noise (masker) and removes the T-F units where the interfering noise dominates. In this paper, a supervised learning method to enhance whispered speech is introduced. A binary mask estimated by a two-class SVM classifier is used to synthesize the enhanced whisper. Amplitude modulation spectrum (AMS) and frequency modulation spectrum (FMS) are extracted as input to SVM. Speech corrupted at low signal to noise (SNR) levels with different types of maskers is enhanced by this method and presented to normal-hearing listeners for word identification. Experimental evidence from the listening tests indicated substantial improvements in intelligibility over that attained by human listeners with unprocessed stimuli.

On the SVMpath initialization

This paper presents a novel algorithm for the initial configuration to the model selection problem in the two-class support vector machine (SVM) classification when fitting the entire path of SVM solutions for every value of the regularization parameter. Instead of using quadratic programming for initialization in the conventional two-class SVM regularization path fitting methods, we propose a piecewise linear method which reduces the computational cost significantly. Furthermore, an efficient treatment is provided to deal with the singular case where the data set contains linearly dependent points, duplicate points or nearly duplicate points. The performance of the proposed algorithm in terms of computational complexity and the ability to handle singular cases are backed by strict mathematical analysis and proof, and verified by the experimental results.

Mine Fan Fault Diagnosis Based on EMD and SVM

In this research, a new method based on EMD and SVM for mine fan fault diagnosis is introduced. With EMD, fault feature can be extracted quickly and accurately, and taken as the input samples for SVM with the outstanding non-linear pattern classification performances. 5 two-class SVM classifiers are designed in order to classify and diagnosis 5 typical fault patterns of mine fan. The result of this research shows that the integrative method of EMD and SVM is very fit for the intelligent diagnosis and fault patterns recognition, and it will lead to the possible development of an automated and on-line mine fan condition monitoring and diagnostic system.

Image Classification for Steel Strip Surface Defects Based on Support Vector Machines

In order to realize less time consuming and on-line image classification for steel strip surface defects, an improved multiclass support vector machine (SVM) was proposed. The SVM used a novel algorithm and only constructed (k-1) two-class SVMs where K is the number of classes. In the testing phase, to identify the surface defects it used a new unidirectional acyclic graph which had internal (k-1) nodes and k leaves. Its testing time is less than traditional multiclass SVM method. The experiment results shows that this method is simple and less time consuming while preserving generalization ability and recognition accuracy toward steel strip surface defects.

Model Selection in SVMs using Differential Evolution

AbstractTo improve the performance of classification using Support Vector Machines (SVMs) while reducing the model selection time, this paper introduces Differential Evolution, a heuristic method for model selection in two-class SVMs with a RBF kernel. The model selection method and related tuning algorithm are both presented. Experimental results from application to a selection of benchmark datasets for SVMs show that this method can produce an optimized classification in less time and with higher accuracy than a classical grid search. Comparison with a Particle Swarm Optimization (PSO) based alternative is also included.

PPI_SVM: prediction of protein-protein interactions using machine learning, domain-domain affinities and frequency tables.

Protein-protein interactions (PPI) control most of the biological processes in a living cell. In order to fully understand protein functions, a knowledge of protein-protein interactions is necessary. Prediction of PPI is challenging, especially when the three-dimensional structure of interacting partners is not known. Recently, a novel prediction method was proposed by exploiting physical interactions of constituent domains. We propose here a novel knowledge-based prediction method, namely PPI_SVM, which predicts interactions between two protein sequences by exploiting their domain information. We trained a two-class support vector machine on the benchmarking set of pairs of interacting proteins extracted from the Database of Interacting Proteins (DIP). The method considers all possible combinations of constituent domains between two protein sequences, unlike most of the existing approaches. Moreover, it deals with both single-domain proteins and multi domain proteins; therefore it can be applied to the whole proteome in high-throughput studies. Our machine learning classifier, following a brainstorming approach, achieves accuracy of 86%, with specificity of 95%, and sensitivity of 75%, which are better results than most previous methods that sacrifice recall values in order to boost the overall precision. Our method has on average better sensitivity combined with good selectivity on the benchmarking dataset. The PPI_SVM source code, train/test datasets and supplementary files are available freely in the public domain at: http://code.google.com/p/cmater-bioinfo/. Electronic Supplementary MaterialThe online version of this article (doi: 10.2478/s11658-011-0008-x contains supplementary material, which is available to authorized users.

Characterizing sparse preconditioner performance for the support vector machine kernel

Abstract A two-class Support Vector Machine (SVM) classifier finds a hyperplane that separates two classes of data with the maximum margin. In a first learning phase, SVM involves the construction and solution of a primal-dual interiorpoint optimization problem. Each iteration of the interior-point method (IPM) requires a sparse linear system solution, which dominates the execution time of SVM learning. Solving this linear system can often be computationally expensive depending on the conditioning of the matrix. Consequently, preconditioned linear systems can lead to improved SVM performance while maintaining the classification accuracy. In this paper, we seek to characterize the role of preconditioning schemes for enhancing the SVM classifier performance. We compare and report on the solution time, convergence, and number of Newton iterations of the iterior-point method and classification accuracy of the SVM for 6 well-accepted preconditioning schemes and datasets chosen from well-known machine learning repositories. In particular, we introduce Δ -IPM that sparsifies the linear system at each iteration of the IPM. Our results indicate that on average the Jacobi and SSOR preconditioners perform 10.01 times better than other preconditioning schemes for IPM and 8.83 times better for Δ -IPM. Also, across all datasets Jacobi and SSOR perform between 2 to 30 times better than other schemes in both IPM and Δ -IPM. Moreover, Δ -IPM obtains a speedup over IPM performance on average by 1.25 and as much as 2 times speedup in the best case.

The Accurate Prediction of Protein Family from Amino Acid Sequence by Measuring Features of Sequence Fragments

The rapid advances in proteomic analyses coupled with the completion of multiple genomes have led to an increased demand for determining protein functions. The first step is classification or prediction into families. A method was developed for the prediction of protein family based only on protein sequence using support vector machine (SVM) models. In these models, the amino acids were classified into three categories (apolar, polar, and charged). Consecutive fragments ranging from one to five were annotated by amino acid type to define the protein features of each protein. SVM models were constructed based on the protein features of a training set of proteins and then examined with an independent set of proteins. The approach was tested for 20 protein families from the iProClass database of Protein Information Resources (PIR). For two-class SVM models, an average prediction accuracy of 0.9985 was achieved, while for multi-class SVM models an accuracy of 0.9941 was achieved. This study demonstrates that SVM based methods can accurately recognize and predict the protein family to which a sequence belongs based solely on its primary amino acid sequence.

Classification of breast tissues using Getis-Ord statistics and support vector machine

Female breast cancer is the major cause of cancer-related deaths in western countries. Efforts in computer vision have been made in order to help improving the diagnostic accuracy by radiologists. In this paper, we present a methodology that intends to use Getis Index spatial texture measures in order to distinguish mass and non-mass tissues extracted from mammograms. The computed measures are classified through a One-Class and a Two-Class Support Vector Machine (SVM). The proposed method reaches 99.33% of accuracy using One-Class SVM and 94.21% of accuracy using Two-Class SVM.

On the Performance of Kernel Methods for Skin Color Segmentation

Human skin detection in color images is a key preprocessing stage in many image processing applications. Though kernel-based methods have been recently pointed out as advantageous for this setting, there is still few evidence on their actual superiority. Specifically, binary Support Vector Classifier (two-class SVM) and one-class Novelty Detection (SVND) have been only tested in some example images or in limited databases. We hypothesize that comparative performance evaluation on a representative application-oriented database will allow us to determine whether proposed kernel methods exhibit significant better performance than conventional skin segmentation methods. Two image databases were acquired for a webcam-based face recognition application, under controlled and uncontrolled lighting and background conditions. Three different chromaticity spaces (YCbCr, , and normalized RGB) were used to compare kernel methods (two-class SVM, SVND) with conventional algorithms (Gaussian Mixture Models and Neural Networks). Our results show that two-class SVM outperforms conventional classifiers and also one-class SVM (SVND) detectors, specially for uncontrolled lighting conditions, with an acceptably low complexity.

Combining different biometric traits with one-class classification

It has been demonstrated in the literature that the combining of different biometric traits is a powerful tool to overcome the limitations imposed by a single biometric system. The fusion of different systems can be approached in different ways. In this work, we consider the pattern classification approach, where the scores of the various systems are used as features to feed the classifiers. More specifically, we are interested in one-class classifiers, and we show that one-class classification could be considered as an alternative to biometric fusion, especially when the data are highly unbalanced or when data from only a single class are available. The results reported for one-class classification on two different databases compares with the standard two-class SVM and surpasses all the conventional classifier combination rules tested.

Fire detection based on vision sensor and support vector machines

This paper proposes a new vision sensor-based fire-detection method for an early-warning fire-monitoring system. First, candidate fire regions are detected using modified versions of previous related methods, such as the detection of moving regions and fire-colored pixels. Next, since fire regions generally have a higher luminance contrast than neighboring regions, a luminance map is made and used to remove non-fire pixels. Thereafter, a temporal fire model with wavelet coefficients is created and applied to a two-class support vector machines (SVM) classifier with a radial basis function (RBF) kernel. The SVM classifier is then used for the final fire-pixel verification. Experimental results showed that the proposed approach was more robust to noise, such as smoke, and subtle differences between consecutive frames when compared with the other method.

EFFECTIVENESS OF SUPPORT VECTOR MACHINE FOR CRIME HOT-SPOTS PREDICTION

Crime hot-spot location prediction is important for public safety. The output from the prediction can provide useful information to improve the activities aimed at detecting and preventing safety and security problems. Location prediction is a special case of spatial data mining classification. For instance, in the public safety domain, it may be interesting to predict location(s) of crime hot spots. In this study, we present a support vector machine (SVM)-based approach to predict the location as an alternative to existing modeling approaches. Support vector machine forms the new generation of machine-learning techniques used to find optimal separability between classes within datasets. We compare the performance of two types of SVMs techniques: two-class SVMs and one-class SVMs. We also compared SVM with a neural network-based approach and spatial auto-regression-based approach. Experiments on two different spatial datasets demonstrate that the former approach performs slightly better and the latter one gives reasonable results. Furthermore, in this study, we provide a general framework to customize the spatial data classification task for other spatial domains that have datasets similar to the analyzed crime datasets.

Landslide susceptibility mapping based on Support Vector Machine: A case study on natural slopes of Hong Kong, China

The Support Vector Machine (SVM) is an increasingly popular learning procedure based on statistical learning theory, and involves a training phase in which the model is trained by a training dataset of associated input and target output values. The trained model is then used to evaluate a separate set of testing data. There are two main ideas underlying the SVM for discriminant-type problems. The first is an optimum linear separating hyperplane that separates the data patterns. The second is the use of kernel functions to convert the original non-linear data patterns into the format that is linearly separable in a high-dimensional feature space. In this paper, an overview of the SVM, both one-class and two-class SVM methods, is first presented followed by its use in landslide susceptibility mapping. A study area was selected from the natural terrain of Hong Kong, and slope angle, slope aspect, elevation, profile curvature of slope, lithology, vegetation cover and topographic wetness index (TWI) were used as environmental parameters which influence the occurrence of landslides. One-class and two-class SVM models were trained and then used to map landslide susceptibility respectively. The resulting susceptibility maps obtained by the methods were compared to that obtained by the logistic regression (LR) method. It is concluded that two-class SVM possesses better prediction efficiency than logistic regression and one-class SVM. However, one-class SVM, which only requires failed cases, has an advantage over the other two methods as only “failed” case information is usually available in landslide susceptibility mapping.

Support Vector Machine Learning for Region-Based Image Retrieval with Relevance Feedback

We present a relevance feedback approach based on multi-class support vector machine (SVM) learning and cluster-merging which can significantly improve the retrieval performance in region-based image retrieval. Semantically relevant images may exhibit various visual characteristics and may be scattered in several classes in the feature space due to the semantic gap between low-level features and high-level semantics in the user's mind. To find the semantic classes through relevance feedback, the proposed method reduces the burden of completely re-clustering the classes at iterations and classifies multiple classes. Experimental results show that the proposed method is more effective and efficient than the two-class SVM and multi-class relevance feedback methods.

An Iterative Algorithm for KLDA Classifier

Abstract — The Linear discriminant analysis (LDA) can be generalized into a nonlinear form ─ kernel LDA (KLDA) expedientlyby using the kernel functions. But KLDA is often referred to a generaleigenvalue problem in singular case. To avoid this complication, this paper proposes an iterative algorithm for the two-class KLDA. Theproposed KLDA is used as a nonlinear discriminant classifier, and the experiments show that it has a comparable performance with SVM. Keywords — Linear discriminant analysis (LDA), kernel LDA (KLDA), conjugate gradient algorithm, nonlinear discriminant clas-sifier. I. I NTRODUCTION S INCE the support vector machine (SVM) introduced a general kernel method, which can transform the input space to a higher dimensional feature space via an implicit nonlinear mapping function, many linear methods can be generalized intotheir nonlinear forms by the kernel trick, such as kernelprinciple component analysis (KPCA) [1], kernel linear discri-minant analysis (KLDA) [2], [3], kernel fisher discriminant(KFD) [4], etc.The traditional LDA can find the optimal projection topreserve the cluster structure in linearly separable data, whileKLDA can overcome the limitation due to non-linearly separa-ble data. The optimal solution fo r KLDA is obtained by solvinga general eigenvalue problem, but the within-class scattermatrix is often singular. The authors in [3] recommendedsolving this difficulty by generalized singular value decom-position. The authors in [4] added a multiple of the identitymatrix to the within-class scatter matrix, and made it becomepositive definite.In this paper, a fast and stable iterative algorithm for KLDAin the two-class case is proposed to avoid the eigenvaluedecomposition in singular case. The iteration procedure basedon the conjugate gradient algorithm converges very fast andstably. The proposed KLDA is used as a two-class classifierlike SVM. And it is compared with SVM classifier in theexperiments.

Domain described support vector classifier for multi-classification problems

In this paper, a novel classifier for multi-classification problems is proposed. The proposed classifier, based on the Bayesian optimal decision theory, tries to model the decision boundaries via the posterior probability distributions constructed from support vector domain description rather than to model them via the optimal hyperplanes constructed from two-class support vector machines. Experimental results show that the proposed method is more accurate and efficient for multi-classification problems.

Using one-class and two-class SVMs for multiclass image annotation

We propose using one-class, two-class, and multiclass SVMs to annotate images for supporting keyword retrieval of images. Providing automatic annotation requires an accurate mapping of images' low-level perceptual features (e.g., color and texture) to some high-level semantic labels (e.g., landscape, architecture, and animals). Much work has been performed in this area; however, there is a lack of ability to assess the quality of annotation. In this paper, we propose a confidence-based dynamic ensemble (CDE), which employs a three-level classification scheme. At the base-level, CDE uses one-class support vector machines (SVMs) to characterize a confidence factor for ascertaining the correctness of an annotation (or a class prediction) made by a binary SVM classifier. The confidence factor is then propagated to the multiclass classifiers at subsequent levels. CDE uses the confidence factor to make dynamic adjustments to its member classifiers so as to improve class-prediction accuracy, to accommodate new semantics, and to assist in the discovery of useful low-level features. Our empirical studies on a large real-world data set demonstrate CDE to be very effective.

A<tex>$nu$</tex>-Insensitive SVM Approach for Compliance Monitoring of the Conservation Reserve Program

We study an automatic compliance monitoring approach for U.S. Department of Agriculture's (USDA) Conservation Reserve Program (CRP). CRP compliance monitoring checks each CRP tract regarding its contract stipulations, and is formulated as an unsupervised classification of Landsat imageries given the CRP reference data. Assuming the majority of a CRP tract is compliant, we want to locate the non-CRP outliers. A one-class support vector machine (OCSVM) is used to separate minor outliers (non-CRP) from the majority (CRP). /spl nu/ is an important OCSVM parameter that controls the percentage of outliers and is unknown here. Usually, /spl nu/ estimation may be complicated or computationally expensive. We propose a /spl nu/-insensitive approach by incorporating both the OCSVM and two-class support vector machine (TCSVM) sequentially. Specifically, support vector machine scores obtained from the OCSVM, which indicate the distances between data samples and the classification hyperplane in a feature space, are used to select sufficient and reliable training samples for the TCSVM. Simulation results show the effectiveness and robustness of the proposed method.

Recognizing objects on cluttered backgrounds

This paper deals with recognition of known 3-D objects in different orientations on cluttered backgrounds. As a recognition technique we apply support vector machines (SVMs). To cope with the cluttered background a tree structure of masks is introduced for each object. SVMs are then computed by masking the training sets with the appropriate masks. One- and two-class SVMs are combined in the recognition process. One-class SVMs, used at the first stage, allow us to avoid the “non-object” class generation usually required to classify unknown objects or other parts of a scene. Two-class SVMs are further applied to resolve the recognition process when necessary. The proposed method is compared with two other approaches and as demonstrated by experimental results it is robust to cluttered backgrounds. The advantage of the method is its ability to classify the pattern as unknown which has a valuable effect on false positive rate.