Support Vector Machine Active Learning Research Articles

Efficient Reliability Analysis of Structures Using Symbiotic Organisms Search-Based Active Learning Support Vector Machine

Reliability-based design optimization considers the uncertainties that lie in the designing process of resilient buildings and structures. To model uncertainty, the major challenge is to lower the high computational expense incurred by the double-loop approach, where the design optimization (outer loop) repeatedly calls the reliability analysis of each structural design (inner loop). An alternative is to convert the reliability constraints to deterministic constraints by using optimality conditions. Yet, the approximated results are often inaccurate when constraint functions are highly non-linear, non-continuous, or non-differentiable. To achieve better accuracy while attaining sufficient flexibility, the present study proposes a new framework to classify the structural designs into feasible/infeasible designs. The proposed framework is called SOS-ASVM by integrating the symbiotic organism search (SOS) and the active-learning support vector machine (ASVM). ASVM is adopted as the surrogate model, while SOS is used to seek more representative samples to improve the classification accuracy of ASVM. The SOS-ASVM was validated by comparisons with popular classification tools: conventional support vector machine, artificial neural network, and Kriging model. Three practical engineering cases are used to demonstrate the performance of the SOS-ASVM: a cantilever beam, a bracket structure, and a 25-bar space truss. The comparison results confirm the superiority of the proposed framework to other tools.

A Detection Method for Collapsed Buildings Combining Post-Earthquake High-Resolution Optical and Synthetic Aperture Radar Images

The detection of collapsed buildings based on post-earthquake remote sensing images is conducive to eliminating the dependence on pre-earthquake data, which is of great significance to carry out emergency response in time. The difficulties in obtaining or lack of elevation information, as strong evidence to determine whether buildings collapse or not, is the main challenge in the practical application of this method. On the one hand, the introduction of double bounce features in synthetic aperture radar (SAR) images are helpful to judge whether buildings collapse or not. On the other hand, because SAR images are limited by imaging mechanisms, it is necessary to introduce spatial details in optical images as supplements in the detection of collapsed buildings. Therefore, a detection method for collapsed buildings combining post-earthquake high-resolution optical and SAR images was proposed by mining complementary information between traditional visual features and double bounce features from multi-source data. In this method, a strategy of optical and SAR object set extraction based on an inscribed center (OpticalandSAR-ObjectsExtraction) was first put forward to extract a unified optical-SAR object set. Based on this, a quantitative representation of collapse semantic knowledge in double bounce (DoubleBounceCollapseSemantic) was designed to bridge a semantic gap between double bounce and collapse features of buildings. Ultimately, the final detection results were obtained based on the improved active learning support vector machines (SVMs). The multi-group experimental results of post-earthquake multi-source images show that the overall accuracy (OA) and the detection accuracy for collapsed buildings (Pcb) of the proposed method can reach more than 82.39% and 75.47%. Therefore, the proposed method is significantly superior to many advanced methods for comparison.

Binary classification method for efficient and accurate system reliability analyses of layered soil slopes

ABSTRACT To enhance the computational efficiency in slope reliability analyses, a binary classification method (BCM) that takes advantage of a judgement-based strength reduction method (SRM) and an active-learning support vector machine (SVM) was developed to conduct system reliability analyses of layered soil slopes. The SVM was naturally employed to establish a binary classifier via the judgment technique to approximate the true limit state function because the stability state (e.g. stable or unstable) of a slope can be determined without calculating its exact factor of safety according to the SRM. An active-learning technique was developed to iteratively search training samples in the vicinity of the border between safe and failure domains, to update the SVM classifier with the aid of a modified initial sampling rule. Then, Latin hypercube sampling was employed, together with the obtained SVM classifier, to compute the slope system probability of failure. Three representative examples taken from the literature were employed to evaluate the performance of the proposed method. The proposed BCM shows great computational efficiency compared with existing methods. It reduced the computational cost to several minutes for simple slopes and to approximately 30 minutes for a complex real case, while maintaining a good computational accuracy.

Multiclass SVM active learning algorithm based on decision directed acyclic graph and one versus one

The classical training algorithms of support vector machines (SVM) are supervised learning algorithms which based on large-scale labeled samples, while these labeled samples are not easy to be acquired or labeled costly and class–unbalanced dataset, meanwhile these SVM algorithms are originally designed for the solution of two-class problems. To solve these problems of SVM, An Active learning algorithm based on decision directed acyclic graph (DDAG) for SVM is proposed in the paper, which train the multiclass SVMs using as few labeled instances as possible while maintaining the same SVM performance, or achieving the generalization performance of SVM classification as good as possible. The experimental results on the UCI data show that the proposed approach can achieve higher clasification accuracy, but using less labeled samples, while improving generalization performance and ruducing the marking costs of SVM training.

Support vector machine active learning by Hessian regularization

It is time-consuming and expensive to gather and label the growing multimedia data that is easily accessible with the prodigious development of Internet technology and digital sensors. Hence, it is essential to develop a technique that can efficiently be utilized for the large-scale multimedia data especially when labeled data is rare. Active learning is showing to be one useful approach that greedily chooses queries from unlabeled data to be labeled for further learning and then minimizes the estimated expected learning error. However, most active learning methods only take into account the labeled data in the training of the classifier. In this paper, we introduce a semi-supervised algorithm to learn the classifier and then perform active learning scheme on top of the semi-supervised scheme. Particularly, we employ Hessian regularization into support vector machine to boost the classifier. Hessian regularization exploits the potential geometry structure of data space (including labeled and unlabeled data) and then significantly leverages the performance in each round. To evaluate the proposed algorithm, we carefully conduct extensive experiments including image segmentation and human activity recognition on popular datasets respectively. The experimental results demonstrate that our method can achieve a better performance than the traditional active learning methods.

Consumer Preference Elicitation of Complex Products Using Fuzzy Support Vector Machine Active Learning

As technology advances, new products (e.g., digital cameras, computer tablets, etc.) have become increasingly more complex. Researchers often face considerable challenges in understanding consumers’ preferences for such products. This paper proposes an adaptive decompositional framework to elicit consumers’ preferences for complex products. The proposed method starts with collaborative-filtered initial part-worths, followed by an adaptive question selection process that uses a fuzzy support vector machine active learning algorithm to adaptively refine the individual-specific preference estimate after each question. Our empirical and synthetic studies suggest that the proposed method performs well for product categories equipped with as many as 70 to 100 attribute levels, which is typically considered prohibitive for decompositional preference elicitation methods. In addition, we demonstrate that the proposed method provides a natural remedy for a long-standing challenge in adaptive question design by gauging the possibility of response errors on the fly and incorporating the results into the survey design. This research also explores in a live setting how responses from previous respondents may be used to facilitate active learning of the focal respondent’s product preferences. Overall, the proposed approach offers new capabilities that complement existing preference elicitation methods, particularly in the context of complex products. Data, as supplemental material, are available at http://dx.doi.org/10.1287/mksc.2015.0946 .

Strategy of active learning support vector machine for image retrieval

This study proposes a new method for content-based image retrieval by finding an optimal classifier. The optimal classifier is achieved by a new active learning support vector machine (SVM) which combines the model selection with the active learning. The unlabelled samples close to the boundary of the SVM classifier are selected based on the feature similarity for the active learning, and the adaptive regularisation is used to select the optimal model. The combination of model selection with active learning accelerates the convergence of the classifier. The new method can improve the image retrieval accuracy and reduce the time consumption. The experimental results show that the proposed method has a better performance with fewer samples and less time consumption for image retrieval.

Challenges in Content-Based Image Indexing of Cultural Heritage Collections: Support vector machine active learning with applications to text classification

Cultural heritage collections are being digitized and made available through online tools. Due to the large volume of documents being digitized, not enough manpower is available to provide useful annotations. Is this article, we discuss the use of automatic tools to both automatically index the documents (i.e., provide labels) and search through the collections. We detail the challenges specific to these collections as well as research directions that must be followed to answer the questions raised by these new data.

A new SVM-based active feedback scheme for image retrieval

Relevance feedback has emerged as a powerful tool to boost the retrieval performance in content-based image retrieval (CBIR). Support vector machine (SVM) active learning is one popular and successful technique for relevance feedback in CBIR. Despite the success, for conventional SVM active learning, the users are usually not so patience to label a large number of training instances in the relevance feedback round. To overcome this limitation, a new SVM-based active feedback using ensemble multiple classifiers is proposed in this paper. Firstly, we select the most informative images by using active learning method for user to label, and quickly learn a boundary that separates the images that satisfy the user׳s query concept from the rest of the dataset. Then, a set of moderate accurate one-class SVM classifiers are trained separately by using different sub-features vectors. Finally, we compute the weight vector of component SVM classifiers dynamically by using the parameters for positive and negative samples, and combine the results of the component classifiers to form an output code as a hypothesized solution to the overall image retrieval problem. Experiments on large databases show that the proposed algorithms are significantly more effective than the state-of-the-art approaches.

Active learning SVM with regularization path for image classification

In classification problems, many different active learning techniques are often adopted to find the most informative samples for labeling in order to save human labors. Among them, active learning support vector machine (SVM) is one of the most representative approaches, in which model parameter is usually set as a fixed default value during the whole learning process. Note that model parameter is closely related to the training set. Hence dynamic parameter is desirable to make a satisfactory learning performance. To target this issue, we proposed a novel algorithm, called active learning SVM with regularization path, which can fit the entire solution path of SVM for every value of model parameters. In this algorithm, we first traced the entire solution path of the current classifier to find a series of candidate model parameters, and then used unlabeled samples to select the best model parameter. Besides, in the initial phase of training, we constructed a training sample sets by using an improved K-medoids cluster algorithm. Experimental results conducted from real-world data sets showed the effectiveness of the proposed algorithm for image classification problems.

Geometric Optimum Experimental Design for Collaborative Image Retrieval

Relevance feedback (RF) schemes have been widely designed to improve the performance of content-based image retrieval. Despite the success, it is not appropriate to require the user to label a large number of samples in RF. Collaborative image retrieval (CIR) aims to reduce the labeling efforts of the user by resorting to the auxiliary information. Support vector machine (SVM) active learning can select ambiguous samples as the most informative ones for the user to label with the help of the optimal hyperplane of SVM, and thus alleviate the labeling efforts of conventional RF. However, the optimal hyperplane of SVM is usually unstable and inaccurate with small-sized training data, and this is always the case in image retrieval since the user would not like to label a large number of feedback samples and cannot label each sample accurately all the time. In this paper, we propose a novel active learning method, i.e., geometric optimum experimental design (GOED), to select multiple representative samples in the database as the most informative ones for the user to label. Especially, GOED can alleviate the small-sized training data problem by leveraging the geometric structure of unlabeled samples in the reproducing kernel Hilbert space and thus further enhance the performance of image retrieval. Different from the conventional manifold regularization framework, the new method can effectively select the most informative samples for the user to label in image retrieval. By minimizing the expected average prediction variance on the test data, GOED has a clear geometric interpretation to select a set of the most representative samples in the database iteratively with the global optimum. Compared with the popular SVM active learning, our method is label-independent and can effectively avoid various potential problems caused by insufficient and inexactly labeled samples in RF, and is more appropriate and useful for image retrieval. Extensive experiments on both synthetic datasets and a real-world image database have been conducted to show the advantages of the proposed GOED for CIR.

Improving Relevance Feedback in Image Retrieval by Incorporating Unlabelled Images

In content-base image retrieval, relevance feedback (RF) schemes based on support vector machine (SVM) have been widely used to narrow the semantic gap between low-level visual features and high-level human perception. However, the performance of image retrieval with SVM active learning is known to be poor when the training data is insufficient. In this paper, the problem is solved by incorporating the unlabelled images into the learning process. We proposed a semi-supervised active learning algorithm which uses not only labeled training samples but also unlabeled ones to build better models. In relevance feedback, active learning algorithm is often used to reduce the cost of labeling by selecting only the most informative data. In addition, we introduced a semi-supervised approach which employed Nearest-Neighbor technique to label the unlabeled sample with a certain degree of uncertainty in its class information. Using these samples, Fuzzy support vector machine (FSVM) which takes into account the fuzzy nature of some training samples during its training is trained. We compared our method with standard active SVM on a database of 10,000 images, the experiment results show that the efficiency of SVM active learning can be improved by incorporating unlabelled images, and thus improve the overall retrieval performance. DOI: http://dx.doi.org/10.11591/telkomnika.v11i7.2807

An Improved Support Vector Machine Active Learning for Audio Event Detection

An Improved Support Vector Machine Active Learning for Audio Event Detection

Active learning with adaptive regularization

In classification problems, active learning is often adopted to alleviate the laborious human labeling efforts, by finding the most informative samples to query the labels. One of the most popular query strategy is selecting the most uncertain samples for the current classifier. The performance of such an active learning process heavily relies on the learned classifier before each query. Thus, stepwise classifier model/parameter selection is quite critical, which is, however, rarely studied in the literature. In this paper, we propose a novel active learning support vector machine algorithm with adaptive model selection. In this algorithm, before each new query, we trace the full solution path of the base classifier, and then perform efficient model selection using the unlabeled samples. This strategy significantly improves the active learning efficiency with comparatively inexpensive computational cost. Empirical results on both artificial and real world benchmark data sets show the encouraging gains brought by the proposed algorithm in terms of both classification accuracy and computational cost. ► Model selection is very important for active learning. ► An adaptive model is more suitable for active learning than a fixed model. ► We propose a novel active learning SVM algorithm with adaptive model selection. ► We use unlabeled data to validate the model parameter in our algorithm. ► We investigate how to properly use the data in active learning scenario.

Asymmetric Semi-Supervised Boosting Scheme for Interactive Image Retrieval

Support vector machine (SVM) active learning plays a key role in interactive content-based image retrieval (CBIR) community. However, regular SVM active learning is challenged by what we call the small example problem and the asymmetric distribution problem. This paper attempts to integrate merits of semi-supervised learning, ensemble learning, and active learning into interactive CBIR. Concretely, unlabeled images are exploited to facilitate boosting by helping augment diversity among base SVM classifiers, and then learned ensemble model is used to identify most informative images for active learning. In particular, a bias-weighting mechanism is developed to guide ensemble model to pay more attention on positive images than negative images. Experiments on 5000 Corel images show that proposed method yields better retrieval performance by an amount of 0.16 in mean average precision compared to regular SVM active learning, which is more effective than some existing improved variants of SVM active learning.

Medical Image Retrieval Based on Semi-Supervised Learning

Among various content-based image retrieval (CBIR) methods based on active learning, support vector machine(SVM) active learning is popular for its application to relevance feedback in CBIR. However, the regular SVM active learning has two main drawbacks when used for relevance feedback. Furthermore, it’s difficult to collect vast amounts of labeled data and easy for unlabeled data to image examples. Therefore, it is necessary to define conditions to utilize the unlabeled examples enough. This paper presented a method of medical images retrieval about semi-supervised learning based on SVM for relevance feedback in CBIR. This paper also introduced an algorithm about defining two learners, both learners are re-trained after every relevance feedback round, and then each of them gives every image in a rank. Experiments show that using semi-supervised learning idea in CBIR is beneficial, and the proposed method achieves better performance than some existing methods.

Semisupervised SVM batch mode active learning with applications to image retrieval

Support vector machine (SVM) active learning is one popular and successful technique for relevance feedback in content-based image retrieval (CBIR). Despite the success, conventional SVM active learning has two main drawbacks. First, the performance of SVM is usually limited by the number of labeled examples. It often suffers a poor performance for the small-sized labeled examples, which is the case in relevance feedback. Second, conventional approaches do not take into account the redundancy among examples, and could select multiple examples that are similar (or even identical). In this work, we propose a novel scheme for explicitly addressing the drawbacks. It first learns a kernel function from a mixture of labeled and unlabeled data, and therefore alleviates the problem of small-sized training data. The kernel will then be used for a batch mode active learning method to identify the most informative and diverse examples via a min-max framework. Two novel algorithms are proposed to solve the related combinatorial optimization: the first approach approximates the problem into a quadratic program, and the second solves the combinatorial optimization approximately by a greedy algorithm that exploits the merits of submodular functions. Extensive experiments with image retrieval using both natural photo images and medical images show that the proposed algorithms are significantly more effective than the state-of-the-art approaches. A demo is available at http://msm.cais.ntu.edu.sg/LSCBIR/.

Automatic skin segmentation and tracking in sign language recognition

Skin segmentation and tracking play an important role in sign language recognition. A framework for segmenting and tracking skin objects from signing videos is described. It mainly consists of two parts: a skin colour model and a skin object tracking system. The skin colour model is first built based on the combination of support vector machine active learning and region segmentation. Then, the obtained skin colour model is integrated with the motion and position information to perform segmentation and tracking. The tracking system is able to predict occlusions among any of the skin objects using a Kalman filter (KF). Moreover, the skin colour model can be updated with the help of tracking to handle illumination variation. Experimental evaluations using real-world gesture videos and comparison with other existing algorithms demonstrate the effectiveness of the proposed work.

SVM-based active feedback in image retrieval using clustering and unlabeled data

In content-based image retrieval, relevance feedback is studied extensively to narrow the gap between low-level image feature and high-level semantic concept. However, most methods are challenged by small sample size problem since users are usually not so patient to label a large number of training instances in the relevance feedback round. In this paper, this problem is solved by two strategies: (1) designing a new active selection criterion to select images for user's feedback. It takes both the informative and the representative measures into consideration, thus the diversities between these images are increased while their informative powers are kept. With this new criterion, more information gain can be obtained from the feedback images; and (2) incorporating unlabeled images within the co-training framework. Unlabeled data partially alleviates the training data scarcity problem, thus improves the efficiency of support vector machine (SVM) active learning. Systematic experimental results verify the superiority of our method over existing active learning methods.

Second-Order SMO Improves SVM Online and Active Learning

Iterative learning algorithms that approximate the solution of support vector machines (SVMs) have two potential advantages. First, they allow online and active learning. Second, for large data sets, computing the exact SVM solution may be too time-consuming, and an efficient approximation can be preferable. The powerful LASVM iteratively approaches the exact SVM solution using sequential minimal optimization (SMO). It allows efficient online and active learning. Here, this algorithm is considerably improved in speed and accuracy by replacing the working set selection in the SMO steps. A second-order working set selection strategy, which greedily aims at maximizing the progress in each single step, is incorporated.