Semi-labeled Samples Research Articles

Semi-supervised learning with constrained virtual support vector machines for classification of remote sensing image data

We introduce two semi-supervised models for the classification of remote sensing image data. The models are built upon the framework of Virtual Support Vector Machines (VSVM). Generally, VSVM follow a two-step learning procedure: A Support Vector Machines (SVM) model is learned to determine and extract labeled samples that constitute the decision boundary with the maximum margin between thematic classes, i.e., the Support Vectors (SVs). The SVs govern the creation of so-called virtual samples. This is done by modifying, i.e., perturbing, the image features to which a decision boundary needs to be invariant. Subsequently, the classification model is learned for a second time by using the newly created virtual samples in addition to the SVs to eventually find a new optimal decision boundary. Here, we extend this concept by (i) integrating a constrained set of semi-labeled samples when establishing the final model. Thereby, the model constrainment, i.e., the selection mechanism for including solely informative semi-labeled samples, is built upon a self-learning procedure composed of two active learning heuristics. Additionally, (ii) we consecutively deploy semi-labeled samples for the creation of semi-labeled virtual samples by modifying the image features of semi-labeled samples that have become semi-labeled SVs after an initial model run. We present experimental results from classifying two multispectral data sets with a sub-meter geometric resolution. The proposed semi-supervised VSVM models exhibit the most favorable performance compared to related SVM and VSVM-based approaches, as well as (semi-)supervised CNNs, in situations with a very limited amount of available prior knowledge, i.e., labeled samples.

Combining active learning and transductive support vector machines for sea ice detection

Sea ice can cause some of the most prominent marine disasters in polar and high latitude regions, and remote sensing technology provides an important means to detect such hazards. The accuracy of sea ice detection depends on the number and quality of labeled samples, but because of environmental conditions in sea ice regions, acquisition of labeled samples can be time-consuming and labor-intensive. To solve this problem, we propose a combined active learning (AL) and semisupervised learning (SSL) classification framework for sea ice detection. At first, we acquire most informative and representative samples by AL; then labeled samples acquired by AL are used as the initial labeled samples for SSL, in this framework, we not only choose the most valuable samples but also use the large number of unlabeled samples to enhance the classification accuracy. In AL phase, we use two different sampling strategies: uncertainty and diversity. In the SSL phase, we utilize a sampling function integrating AL to acquire semilabeled samples, and we use a transductive support vector machine as a classification model. We analyze three remote sensing images (hyperspectral and multispectral) and conduct detailed comparative analyses between the proposed method and others. Our proposed method achieves the highest classification accuracies (89.9734%, 97.4919%, and 89.7166%) in both experiments. These results show that the proposed method exhibits better overall performance than other methods and can be effectively applied to sea ice detection using remote sensing.

Feature extraction of hyperspectral images using boundary semi-labeled samples and hybrid criterion

Feature extraction is a very important preprocessing step for classification of hyperspectral images. The linear discriminant analysis (LDA) method fails to work in small sample size situations. Moreover, LDA has poor efficiency for non-Gaussian data. LDA is optimized by a global criterion. Thus, it is not sufficiently flexible to cope with the multi-modal distributed data. We propose a new feature extraction method in this paper, which uses the boundary semi-labeled samples for solving small sample size problem. The proposed method, which called hybrid feature extraction based on boundary semi-labeled samples (HFE-BSL), uses a hybrid criterion that integrates both the local and global criteria for feature extraction. Thus, it is robust and flexible. The experimental results with three real hyperspectral images show the good efficiency of HFE-BSL compared to some popular and state-of-the-art feature extraction methods.

A Novel Classification Algorithm Based on Incremental Semi-Supervised Support Vector Machine.

For current computational intelligence techniques, a major challenge is how to learn new concepts in changing environment. Traditional learning schemes could not adequately address this problem due to a lack of dynamic data selection mechanism. In this paper, inspired by human learning process, a novel classification algorithm based on incremental semi-supervised support vector machine (SVM) is proposed. Through the analysis of prediction confidence of samples and data distribution in a changing environment, a “soft-start” approach, a data selection mechanism and a data cleaning mechanism are designed, which complete the construction of our incremental semi-supervised learning system. Noticeably, with the ingenious design procedure of our proposed algorithm, the computation complexity is reduced effectively. In addition, for the possible appearance of some new labeled samples in the learning process, a detailed analysis is also carried out. The results show that our algorithm does not rely on the model of sample distribution, has an extremely low rate of introducing wrong semi-labeled samples and can effectively make use of the unlabeled samples to enrich the knowledge system of classifier and improve the accuracy rate. Moreover, our method also has outstanding generalization performance and the ability to overcome the concept drift in a changing environment.

A new approach to estimate a priori probabilities in remote sensing digital image classification

An adaptive classifier designed for high-dimensional image data classification has recently been proposed. The main feature in this proposed classifier consists of a novel method to acquire semilabeled samples in an iterative way during the classification process itself. The semilabeled samples are then added to the set of available labeled samples, thus increasing the total number of training samples and consequently the reliability of the estimated values for the parameters in a parametric classifier. In this study we propose the insertion of an additional step into the iterative procedure implemented in the adaptive classifier to produce more realistic estimates of the a priori probability of the classes. The proposed procedure to estimate the a priori probability of the classes makes use of the same weighted approach used by the adaptive classifier, is computationally efficient, and can be easily implemented. Tests using hyperspectral image data have shown that the proposed approach can contribute to increased accuracies in the classification process.

An ensemble-driven k-NN approach to ill-posed classification problems

This paper addresses the supervised classification of remote-sensing images in problems characterized by relatively small-size training sets with respect to the input feature space and the number of classifier parameters (ill-posed classification problems). An ensemble-driven approach based on the k-nearest neighbor ( k-NN) classification technique is proposed. This approach effectively exploits semilabeled samples (i.e., original unlabeled samples labeled by the classification process) to increase the accuracy of the classification process. Experimental results obtained on ill-posed classification problems confirm the effectiveness of the proposed approach, which significantly increases both the accuracy and the reliability of classification maps.

A Semilabeled-Sample-Driven Bagging Technique for Ill-Posed Classification Problems

In this letter, a semilabeled-sample-driven bootstrap aggregating (bagging) technique based on a co-inference (inductive and transductive) framework is proposed for addressing ill-posed classification problems. The novelties of the proposed technique lie in: 1) the definition of a general classification strategy for ill-posed problems by the joint use of training and semilabeled samples (i.e., original unlabeled samples labeled by the classification process); and 2) the design of an effective bagging method (driven by semilabeled samples) for a proper exploitation of different classifiers based on bootstrapped hybrid training sets. Although the proposed technique is general and can be applied to any classification algorithm, in this letter multilayer perceptron neural networks (MLPs) are used to develop the basic classifier of the proposed architecture. In this context, a novel cost function for the training of MLPs is defined, which properly considers the contribution of semilabeled samples in the learning of each member of the ensemble. The experimental results, which are obtained on different ill-posed classification problems, confirm the effectiveness of the proposed technique.

An adaptive method for combined covariance estimation and classification

In this paper, a family of adaptive covariance estimators is proposed to mitigate the problem of limited training samples for application to hyperspectral data analysis in quadratic maximum likelihood classification. These estimators are the combination of adaptive classification procedures and regularized covariance estimators. In these proposed estimators, the semi-labeled samples (whose labels are determined by a decision rule) are incorporated in the process of determining the optimal regularized parameters and estimating those supportive covariance matrices that formulate final regularized covariance estimators. In all experiments with simulated and real remote sensing data, these proposed combined covariance estimators achieved significant improvement on statistics estimation and classification accuracy over conventional regularized covariance estimators and an adaptive maximum likelihood classifier (MLC). The degree of improvement increases with dimensions, especially for ill-posed or very ill-posed problems where the total number of training samples is smaller than the number of dimensions.

An adaptive classifier design for high-dimensional data analysis with a limited training data set

Proposes a self-learning and self-improving adaptive classifier to mitigate the problem of small training sample size that can severely affect the recognition accuracy of classifiers when the dimensionality of the multispectral data is high. This proposed adaptive classifier utilizes classified samples (referred to as semilabeled samples) in addition to original training samples iteratively. In order to control the influence of semilabeled samples, the proposed method gives full weight to the training samples and reduced weight to semilabeled samples. The authors show that by using additional semilabeled samples that are available without extra cost, the additional class label information may be extracted and utilized to enhance statistics estimation and hence improve the classifier performance, and therefore the Hughes phenomenon (peak phenomenon) may be mitigated. Experimental results show this proposed adaptive classifier can improve the classification accuracy as well as representation of estimated statistics significantly.