Semi-supervised Support Vector Machines Research Articles

Help-Training for semi-supervised support vector machines

In this paper, we propose to reinforce the Self-Training strategy in semi-supervised mode by using a generative classifier that may help to train the main discriminative classifier to label the unlabeled data. We call this semi-supervised strategy Help-Training and apply it to training kernel machine classifiers as support vector machines (SVMs) and as least squares support vector machines. In addition, we propose a model selection strategy for semi-supervised training. Experimental results on both artificial and real problems demonstrate that Help-Training outperforms significantly the standard Self-Training. Moreover, compared to other semi-supervised methods developed for SVMs, our Help-Training strategy often gives the lowest error rate.

Near-duplicate keyframe retrieval by semi-supervised learning and nonrigid image matching

Near-duplicate keyframe (NDK) retrieval techniques are critical to many real-world multimedia applications. Over the last few years, we have witnessed a surge of attention on studying near-duplicate image/keyframe retrieval in the multimedia community. To facilitate an effective approach to NDK retrieval on large-scale data, we suggest an effective Multi-Level Ranking (MLR) scheme that effectively retrieves NDKs in a coarse-to-fine manner. One key stage of the MLR ranking scheme is how to learn an effective ranking function with extremely small training examples in a near-duplicate detection task. To attack this challenge, we employ a semi-supervised learning method, semi-supervised support vector machines, which is able to significantly improve the retrieval performance by exploiting unlabeled data. Another key stage of the MLR scheme is to perform a fine matching among a subset of keyframe candidates retrieved from the previous coarse ranking stage. In contrast to previous approaches based on either simple heuristics or rigid matching models, we propose a novel Nonrigid Image Matching (NIM) approach to tackle near-duplicate keyframe retrieval from real-world video corpora in order to conduct an effective fine matching. Compared with the conventional methods, the proposed NIM approach can recover explicit mapping between two near-duplicate images with a few deformation parameters and find out the correct correspondences from noisy data simultaneously. To evaluate the effectiveness of our proposed approach, we performed extensive experiments on two benchmark testbeds extracted from the TRECVID2003 and TRECVID2004 corpora. The promising results indicate that our proposed method is more effective than other state-of-the-art approaches for near-duplicate keyframe retrieval.

Semi-supervised pattern classification of medical images: Application to mild cognitive impairment (MCI)

Many progressive disorders are characterized by unclear or transient diagnoses for specific subgroups of patients. Commonly used supervised pattern recognition methodology may not be the most suitable approach to deriving image-based biomarkers in such cases, as it relies on the availability of categorically labeled data (e.g., patients and controls). In this paper, we explore the potential of semi-supervised pattern classification to provide image-based biomarkers in the absence of precise diagnostic information for some individuals. We employ semi-supervised support vector machines (SVM) and apply them to the problem of classifying MR brain images of patients with uncertain diagnoses. We examine patterns in serial scans of ADNI participants with mild cognitive impairment (MCI), and propose that in the absence of sufficient follow-up evaluations of individuals with MCI, semi-supervised strategy is potentially more appropriate than the fully-supervised paradigm employed up to date.

A self-trained ensemble with semisupervised SVM: An application to pixel classification of remote sensing imagery

In this article, we present a semisupervised support vector machine that uses self-training approach. We then construct an ensemble of semisupervised SVM classifiers to address the problem of pixel classification of remote sensing images. Semisupervised support vector machines (S^3VMs) are based on applying the margin maximization principle to both labeled and unlabeled samples. The ensemble of SVM classifiers recognizes the conceptual similarity between component classifiers from the same data source. The effectiveness of the proposed technique is first demonstrated for two numeric remote sensing data described in terms of feature vectors and then identifying different land cover regions in remote sensing imagery. Experimental results on these datasets show that employing this learning scheme can increase the accuracy level. The performance of the ensemble is compared with one of its component classifier and conventional SVM in terms of accuracy and quantitative cluster validity indices.

Stochastic Optimization Algorithms for Support Vector Machines Classification

In this paper, we consider the problem of semi-supervised binary classification by Support Vector Machines (SVM). This problem is explored as an unconstrained and non-smooth optimization task when part of the available data is unlabelled. We apply non-smooth optimization techniques to classification where the objective function considered is non-convex and non-differentiable and so difficult to minimize. We explore and compare the properties of Simulated Annealing and of Simultaneous Perturbation Stochastic Approximation (SPSA) algorithms (SPSA with the Lipschitz Perturbation Operator, SPSA with the Uniform Perturbation Operator, Standard Finite Difference Approximation) for semi-supervised SVM classification. Numerical results are given, obtained by running the proposed methods on several standard test problems drawn from the binary classification literature. The performance of the classifiers were evaluated by analyzing Receiver Operating Characteristics (ROC).

Stochastic approximation algorithms for support vector machines semi-supervised binary classification

In this paper,we consider the problem of semi-supervisedbinary classificationby Support Vector Machines (SVM). This problem is explored as an unconstrained and non-smooth optimization task when part of the available data is unlabelled. We apply non-smooth optimization techniques to classification problems where the objective function considered is non-convex and non-differentiable and so difficult to minimize. We explore and compare the properties of Stochastic Approximation algorithms (Simultaneous Perturbation Stochastic Approximation (SPSA) with the Lipschitz Perturbation Operator, SPSA with the Uniform Perturbation Operator, and Standard Finite Difference Approximation) for semi-supervised SVM classification. We present some numerical results obtained by running the proposed methods on several standard test problems drawn from the binary classification literature.

Optimization Techniques for Semi-Supervised Support Vector Machines

Due to its wide applicability, the problem of semi-supervised classification is attracting increasing attention in machine learning. Semi-Supervised Support Vector Machines (S3VMs) are based on applying the margin maximization principle to both labeled and unlabeled examples. Unlike SVMs, their formulation leads to a non-convex optimization problem. A suite of algorithms have recently been proposed for solving S3VMs. This paper reviews key ideas in this literature. The performance and behavior of various S3VMs algorithms is studied together, under a common experimental setting.

Semisupervised Classification of Hyperspectral Images by SVMs Optimized in the Primal

This paper addresses classification of hyperspectral remote sensing images with kernel-based methods defined in the framework of semisupervised support vector machines (S3VMs). In particular, we analyzed the critical problem of the nonconvexity of the cost function associated with the learning phase of S3VMs by considering different (S3VMs) techniques that solve optimization directly in the primal formulation of the objective function. As the nonconvex cost function can be characterized by many local minima, different optimization techniques may lead to different classification results. Here, we present two implementations, which are based on different rationales and optimization methods. The presented techniques are compared with S3VMs implemented in the dual formulation in the context of classification of real hyperspectral remote sensing images. Experimental results point out the effectiveness of the techniques based on the optimization of the primal formulation, which provided higher accuracy and better generalization ability than the S3VMs optimized in the dual formulation

A Semisupervised Support Vector Machines Algorithm for BCI Systems

As an emerging technology, brain-computer interfaces (BCIs) bring us new communication interfaces which translate brain activities into control signals for devices like computers, robots, and so forth. In this study, we propose a semisupervised support vector machine (SVM) algorithm for brain-computer interface (BCI) systems, aiming at reducing the time-consuming training process. In this algorithm, we apply a semisupervised SVM for translating the features extracted from the electrical recordings of brain into control signals. This SVM classifier is built from a small labeled data set and a large unlabeled data set. Meanwhile, to reduce the time for training semisupervised SVM, we propose a batch-mode incremental learning method, which can also be easily applied to the online BCI systems. Additionally, it is suggested in many studies that common spatial pattern (CSP) is very effective in discriminating two different brain states. However, CSP needs a sufficient labeled data set. In order to overcome the drawback of CSP, we suggest a two-stage feature extraction method for the semisupervised learning algorithm. We apply our algorithm to two BCI experimental data sets. The offline data analysis results demonstrate the effectiveness of our algorithm.

TEXTURE SEGMENTATION USING SEMI-SUPERVISED SUPPORT VECTOR MACHINES

Segmentation of natural textures has been investigated by developing a novel semi-supervised support vector machines (S3VM) algorithm with multiple constraints. Unlike conventional segmentation algorithms the proposed method does not classify the textures but classifies the uniform-texture regions and the regions of boundaries. Also the overall algorithm does not use any training set as used by all other learning algorithms such as conventional SVMs. During the process, the images are restored from high spatial frequency noise. Then various-order statistics of the textures within a sliding two-dimensional window are measured. K-mean algorithm is used to initialise the clustering procedure by labelling part of the class members and the classifier parameters. Therefore at this stage we have both the training and the working sets. A non-linear S3VM is then developed to exploit both sets to classify all the regions. The convex algorithm maximises a defined cost function by incorporating a number of constraints. The algorithm has been applied to combinations of a number of natural textures. It is demonstrated that the algorithm is robust, with negligible misclassification error. However, for complex textures there may be a minor misplacement of the edges.