Unlabeled Ones Research Articles

ASCENT: Active Supervision for Semi-Supervised Learning

Active learning algorithms attempt to overcome the labeling bottleneck by asking queries from large collection of unlabeled examples. Existing batch mode active learning algorithms sufferfrom three limitations: (1) The methods that are based on similarityfunction or optimizing certain diversity measurement, in which may lead to suboptimal performance and produce the selected set with redundant examples. (2) The models with assumption on data are hard in finding images that are both informative and representative. (3) The problem of noise labels has been an obstacle for algorithms. In this paper, we propose a novel active learning method that makes embeddings of labeled examples to those of unlabeled ones and back via deep neural networks. The active scheme makes correct association cycles that end up at the same class from that the association was started, which considers both the informativeness and representativeness of examples, as well as being robust to the noise labels. We apply our active learning method to semi-supervised classification and clustering. The submodular function is designed to reduce the redundancy of the selected examples. Specifically, we incorporate our batch mode active scheme into the classification approaches, in which the generalization ability is improved. For semi-supervised clustering, we try to use our active scheme for constraints to make fast convergence and perform better than unsupervised clustering. Finally, we apply our active learning method to data filtering. To validate the effectiveness of the proposed algorithms, extensive experiments are conducted on diversity benchmark datasets for different tasks, i.e., classification, clustering, and data filtering, and the experimental results demonstrate consistent and substantial improvements over the state-of-the-art approaches.

Semisupervised Change Detection Using Graph Convolutional Network

Most change detection (CD) methods are unsupervised as collecting substantial multitemporal training data is challenging. Unsupervised CD methods are driven by heuristics and lack the capability to learn from data. However, in many real-world applications, it is possible to collect a small amount of labeled data scattered across the analyzed scene. Such a few scattered labeled samples in the pool of unlabeled samples can be effectively handled by graph convolutional network (GCN) that has recently shown good performance in semisupervised single-date analysis, to improve change detection performance. Based on this, we propose a semisupervised CD method that encodes multitemporal images as a graph via multiscale parcel segmentation that effectively captures the spatial and spectral aspects of the multitemporal images. The graph is further processed through GCN to learn a multitemporal model. Information from the labeled parcels is propagated to the unlabeled ones over training iterations. By exploiting the homogeneity of the parcels, the model is used to infer the label at a pixel level. To show the effectiveness of the proposed method, we tested it on a multitemporal Very High spatial Resolution (VHR) data set acquired by Pleiades sensor over Trento, Italy.

SSAH: Semi-Supervised Adversarial Deep Hashing with Self-Paced Hard Sample Generation

Deep hashing methods have been proved to be effective and efficient for large-scale Web media search. The success of these data-driven methods largely depends on collecting sufficient labeled data, which is usually a crucial limitation in practical cases. The current solutions to this issue utilize Generative Adversarial Network (GAN) to augment data in semi-supervised learning. However, existing GAN-based methods treat image generations and hashing learning as two isolated processes, leading to generation ineffectiveness. Besides, most works fail to exploit the semantic information in unlabeled data. In this paper, we propose a novel Semi-supervised Self-pace Adversarial Hashing method, named SSAH to solve the above problems in a unified framework. The SSAH method consists of an adversarial network (A-Net) and a hashing network (H-Net). To improve the quality of generative images, first, the A-Net learns hard samples with multi-scale occlusions and multi-angle rotated deformations which compete against the learning of accurate hashing codes. Second, we design a novel self-paced hard generation policy to gradually increase the hashing difficulty of generated samples. To make use of the semantic information in unlabeled ones, we propose a semi-supervised consistent loss. The experimental results show that our method can significantly improve state-of-the-art models on both the widely-used hashing datasets and fine-grained datasets.

Discriminative Adversarial Domain Adaptation

Given labeled instances on a source domain and unlabeled ones on a target domain, unsupervised domain adaptation aims to learn a task classifier that can well classify target instances. Recent advances rely on domain-adversarial training of deep networks to learn domain-invariant features. However, due to an issue of mode collapse induced by the separate design of task and domain classifiers, these methods are limited in aligning the joint distributions of feature and category across domains. To overcome it, we propose a novel adversarial learning method termed Discriminative Adversarial Domain Adaptation (DADA). Based on an integrated category and domain classifier, DADA has a novel adversarial objective that encourages a mutually inhibitory relation between category and domain predictions for any input instance. We show that under practical conditions, it defines a minimax game that can promote the joint distribution alignment. Except for the traditional closed set domain adaptation, we also extend DADA for extremely challenging problem settings of partial and open set domain adaptation. Experiments show the efficacy of our proposed methods and we achieve the new state of the art for all the three settings on benchmark datasets.

A Soft-Voting Ensemble Based Co-Training Scheme Using Static Selection for Binary Classification Problems

In recent years, a forward-looking subfield of machine learning has emerged with important applications in a variety of scientific fields. Semi-supervised learning is increasingly being recognized as a burgeoning area embracing a plethora of efficient methods and algorithms seeking to exploit a small pool of labeled examples together with a large pool of unlabeled ones in the most efficient way. Co-training is a representative semi-supervised classification algorithm originally based on the assumption that each example can be described by two distinct feature sets, usually referred to as views. Since such an assumption can hardly be met in real world problems, several variants of the co-training algorithm have been proposed dealing with the absence or existence of a naturally two-view feature split. In this context, a Static Selection Ensemble-based co-training scheme operating under a random feature split strategy is outlined regarding binary classification problems, where the type of the base ensemble learner is a soft-Voting one composed of two participants. Ensemble methods are commonly used to boost the predictive performance of learning models by using a set of different classifiers, while the Static Ensemble Selection approach seeks to find the most suitable structure of ensemble classifier based on a specific criterion through a pool of candidate classifiers. The efficacy of the proposed scheme is verified through several experiments on a plethora of benchmark datasets as statistically confirmed by the Friedman Aligned Ranks non-parametric test over the behavior of classification accuracy, F1-score, and Area Under Curve metrics.

Multi-Cue Semi-Supervised Color Constancy With Limited Training Samples

Color constancy is one of the fundamental tasks in computer vision. Many supervised methods, including recently proposed Convolutional Neural Networks (CNN)-based methods, have been proved to work well on this problem, but they often require a sufficient number of labeled data. However, it is expensive and time-consuming to collect a large number of labeled training images with accurately measured illumination. In order to reduce the dependence on labeled images and leverage unlabeled ones without measured illumination, we propose a novel semi-supervised framework with limited training samples for illumination estimation. Our key insight is that the images with similar features from different cues will share similar lighting conditions. Consequently, three graphs based on three visual cues, low-level RGB color distribution, mid-level initial illuminant estimates and high-level scene content, are constructed to represent the relationship among different images. Then a multi-cue semi-supervised color constancy method (MSCC) is proposed after integrating these three graphs into a unified model. Extensive experiments on benchmark datasets demonstrate that our proposed MSCC method outperforms nearly all the existing supervised methods with limited labeled samples. Even with no unlabeled samples, MSCC still obtains better performance and stableness than most supervised methods.

Label propagation algorithm based on Roll-back detection and credibility assessment.

The traditional label propagation algorithm (LPA) iteratively propagates labels from a small number of labeled samples to many unlabeled ones based on the sample similarities. However, due to the randomness of label propagations, and LPA's weak ability to deal with uncertain points, the label error may be continuously expanded during the propagation process. In this paper, the algorithm label propagation based on roll-back detection and credibility assessment (LPRC) is proposed. A credit evaluation of the unlabeled samples is carried out before the selection of samples in each round of label propagation, which makes sure that the samples with more certainty can be labeled first. Furthermore, a roll-back detection mechanism is introduced in the iterative process to improve the label propagation accuracy. At last, our method is compared with 9 algorithms based on UCI datasets, and the results demonstrated that our method can achieve better classification performance, especially when the number of labeled samples is small. When the labeled samples only account for 1% of the total sample number of each synthetic dataset, the classification accuracy of LPRC improved by at least 26.31% in dataset circles, and more than 13.99%, 15.22% than most of the algorithms compared in dataset moons and varied, respectively. When the labeled samples account for 2% of the total sample number of each dataset in UCI datasets, the accuracy (take the average value of 50 experiments) of LPRC improved in an average value of 23.20% in dataset wine, 20.82% in dataset iris, 4.25% in dataset australian, and 6.75% in dataset breast. And the accuracy increases with the number of labeled samples.

Semi-supervised Seizure Prediction with Generative Adversarial Networks.

Many outstanding studies have reported promising results in seizure prediction that is considered one of the most challenging predictive data analysis. This is mainly because electroencephalogram (EEG) bio-signal intensity is very small, in μV range, and there are significant sensing difficulties given physiological and non-physiological artifacts. In this article, we propose an approach that can make use of not only labeled EEG signals but also the unlabeled ones which are more accessible. We also suggest the use of data fusion to further improve the seizure prediction accuracy. Data fusion in our vision includes EEG signals, cardiogram signals, body temperature and time. We use the short-time Fourier transform on 28-s EEG windows as a pre-processing step. A generative adversarial network (GAN) is trained in an unsupervised manner where information of seizure onset is disregarded. The trained Discriminator of the GAN is then used as feature extractor. Features generated by the feature extractor are classified by two fully-connected layers (can be replaced by any classifier) for the labeled EEG signals. This semi-supervised seizure prediction method achieves area under the operating characteristic curve (AUC) of 77.68% and 75.47% for the CHBMIT scalp EEG dataset and the Freiburg Hospital intracranial EEG dataset, respectively. Unsupervised training without the need of labeling is important because not only it can be performed in real-time during EEG signal recording, but also it does not require feature engineering effort for each patient.

Safe semi-supervised learning: a brief introduction

Semi-supervised learning constructs the predictive model by learning from a few labeled training examples and a large pool of unlabeled ones. It has a wide range of application scenarios and has attracted much attention in the past decades. However, it is noteworthy that although the learning performance is expected to be improved by exploiting unlabeled data, some empirical studies show that there are situations where the use of unlabeled data may degenerate the performance. Thus, it is advisable to be able to exploit unlabeled data safely. This article reviews some research progress of safe semi-supervised learning, focusing on three types of safeness issue: data quality, where the training data is risky or of low-quality; model uncertainty, where the learning algorithm fails to handle the uncertainty during training; measure diversity, where the safe performance could be adapted to diverse measures.

Robust Deep Softmax Regression Against Label Noise for Unsupervised Domain Adaptation

Domain adaptation aims to generalize the classification model from a source domain to a different but related target domain. Recent studies have revealed the benefit of deep convolutional features trained on a large dataset (e.g. ImageNet) in alleviating domain discrepancy. However, literatures show that the transferability of features decreases as (i) the difference between the source and target domains increases, or (ii) the layers are toward the top layers. Therefore, even with deep features, domain adaptation remains necessary. In this paper, we propose a novel unsupervised domain adaptation (UDA) model for deep neural networks, which is learned with the labeled source samples and the unlabeled target ones simultaneously. For target samples without labels, pseudo labels are assigned to them according to their maximum classification scores during training of the UDA model. However, due to the domain discrepancy, label noise generally is inevitable, which degrades the performance of the domain adaptation model. Thus, to effectively utilize the target samples, three specific robust deep softmax regression (RDSR) functions are performed for them with high, medium and low classification confidence respectively. Extensive experiments show that our method yields the state-of-the-art results, demonstrating the effectiveness of the robust deep softmax regression classifier in UDA.

A Semi-Supervised Regression Algorithm for Grade Prediction of Students in Distance Learning Courses

Applying data mining methods in the educational field has gained a lot of attention among researchers in recent years. Educational Data Mining has turned into an effective tool for uncovering hidden relationships in educational data and predicting students’ learning outcomes. Several supervised methods have been successfully applied with the purpose of identifying students at risk of failing or of predicting their academic performance. Recently, the implementation of Semi-Supervised Learning (SSL) methods in the educational process indicated their superiority over the supervised ones. SSL is an emerging subfield of machine learning seeking to effectively exploit a small pool of labeled examples together with a large pool of unlabeled ones. On this basis, a small number of students’ data from previous years may be used as the training set of a learning model to predict future outcomes of current students. A number of rewarding studies deal with the implementation of classification methods in the educational field in contrast to regression, which is deemed to be a slightly touched task. In this paper, a novel semi-supervised regression (SSR) algorithm is presented for predicting the final grade of undergraduate students in a distance online course. To the best of our knowledge there is no study dealing with the implementation of SSR methods in the educational field. A plethora of attributes related to students’ characteristics, academic performance and interaction within the course online platform form the training set, while several experiments were carried out confirming the superiority of the proposed algorithm over familiar regression methods. The experiment results show that the predictive performance of the proposed algorithm is increasing significantly over time, achieving a MAE value of less than 1.2358 before the middle of the academic year, which provides the advantage of early warnings and interventions.

Crowd Counting With Limited Labeling Through Submodular Frame Selection

Automated crowd counting is valuable for intelligent transportation systems, as it can help to improve the emergency planning and prevent congestion in transit hubs such as train stations and airports. Semi-supervised crowd counting aims to estimate the number of pedestrians in an ongoing scene using a combination of a small number of labeled frames and a large number of unlabeled ones. However, existing methods do not incorporate ways to effectively select informative frames as labeled training samples, resulting in low accuracy on unseen crowd scenes. We propose a submodular method to select the most informative frames from the image sequences of crowds. Specifically, the method selects the most representative images to guarantee the information coverage, by maximizing the similarities between the group of selected images and the image sequence. In addition, these frames are chosen to avoid redundancies and preserve diversity. Finally, our semi-supervised method incorporates graph Laplacian regularization and spatiotemporal constraints. Extensive experiments on three benchmark data sets demonstrate that our proposed approach achieves higher accuracy compared with the state-of-the-art regression methods and competitive performance with deep convolutional models, especially when the number of labeled data is exceptionally small.

Semi-supervised One-Pass Multi-view Learning with Variable Features and Views

Traditional supervised multi-view learning machines aim to process multi-view data sets which consist of labeled instances from multiple views. While they cannot deal with semi-supervised data sets whose training instances consist of both labeled and unlabeled ones. Moreover, with the limitation of storage and process ability, some learning machines cannot process large-scale data sets. Furthermore, some instances maybe have missing features or views and traditional multi-view learning machines have no ability to process the data sets with variable features and views. Thus, this paper develops a semi-supervised one-pass multi-view learning with variable features and views (SOMVFV) so as to process the large-scale semi-supervised data sets with variable features and views. Related experiments on some supervised, semi-supervised, large-scale, and small-scale data sets validate the effectiveness of our proposed SOMVFV and we can get the following conclusions, (1) SOMVFV can process multiple kinds of special data sets; (2) compared with most learning machines used in our experiments, the better performance of SOMVFV is significant; (3) compared with missing views, missing features has a greater influence on the classification accuracy.

Opinion Spam Detection based on Annotation Extension and Neural Networks

Online reviews play an increasingly important role in the purchase decisions of potential customers. Incidentally, driven by the desire to gain profit or publicity, spammers may be hired to write fake reviews and promote or demote the reputation of products or services. Correspondingly, opinion spam detection has attracted attention from both business and research communities in recent years. However, unlike other tasks such as news classification or blog classification, the existing review spam datasets are typically limited due to the expensiveness of human annotation, which may further affect detection performance even if excellent classifiers have been developed. We propose a novel approach in this paper to boost opinion spam detection performance by fully utilizing the existing labelled small-size dataset. We first design an annotation extension scheme that uses extra tree classifiers to train multiple estimators and then iteratively generate reliable labelled samples from unlabeled ones. Subsequently, we examine neural network scenarios on a newly extended dataset to learn the distributed representation. Experimental results suggest that the proposed approach has better generalization capability and improved performance than state-of-the-art methods.

Ensembles of density estimators for positive-unlabeled learning

Positive-Unlabeled (PU) learning works by considering a set of positive samples, and a (usually larger) set of unlabeled ones. This challenging setting requires algorithms to cleverly exploit dependencies hidden in the unlabeled data in order to build models able to accurately discriminate between positive and negative samples. We propose to exploit probabilistic generative models to characterize the distribution of the positive samples, and to label as reliable negative samples those that are in the lowest density regions with respect to the positive ones. The overall framework is flexible enough to be applied to many domains by leveraging tools provided by years of research from the probabilistic generative model community. In addition, we show how to create mixtures of generative models by adopting a well-known bagging method from the discriminative framework as an effective and cheap alternative to the classical Expectation Maximization. Results on several benchmark datasets show the performance and flexibility of the proposed approach.

Development of the covalent antibody-DNA conjugates technology for detection of IgE and IgM antibodies by immuno-PCR.

Immuno-PCR (iPCR) is one of the methods used for the detection of a wide range of analytes and features the high sensitivity of the polymerase chain reaction (PCR) method. iPCR uses antibodies coupled to DNA, followed by the amplification of the attached DNA using RT-PCR. Two major types of antibody-DNA conjugates are currently used, which are obtained as a result of non-covalent (biotin-streptavidin) or covalent interactions. Using a strain-promoted azide-alkyne cycloaddition (SPAAC), we synthesized covalent DNA-antibody conjugates, optimized the reaction conditions, and developed an efficient protocol for the purification of conjugates, with which all unreacted antibodies and oligonucleotides are separated. Covalent DNA-antibody conjugates were tested with iPCR assays that were previously developed for the detection of IgE and IgM antibodies with the use of the supramolecular complex of 5'- and 3'-biotinylated DNA and streptavidin. The results show that the modification of antibodies with amino groups did not allow us to obtain monolabeled antibodies or antibodies with a strictly defined number of DNA-labels. The degree of labeling determined by the dyes introduced through the azido group reflects the actual labeling degree statistically. If the average labeling degree for azido groups is 1.1, the conjugates contain 25% mono-labeled antibodies, 50% double-labeled antibodies, and 25% unlabeled ones. The specificity of the monoclonal antibody to human IgE (BE5) changed after conjugation with the oligonucleotide. The sensitivity of iPCR in the detection of IgM antibodies produced against the LeC disaccharide using a covalent conjugate was similar to that of a supramolecular complex of 5'- and 3'-biotinylated DNA and streptavidin, but the new procedure is two steps shorter.

Epileptic Seizure Forecasting With Generative Adversarial Networks

Many outstanding studies have reported promising results in seizure forecasting, one of the most challenging predictive data analysis problems. This is mainly because electroencephalogram (EEG) bio-signal intensity is very small, in $\mu \text{V}$ range, and there are significant sensing difficulties given physiological and non-physiological artifacts. Today the process of accurate epileptic seizure identification and data labeling is done by neurologists. The current unpredictability of epileptic seizure activities together with the lack of reliable treatment for patients living with drug resistant forms of epilepsy creates an urgency for research into accurate, sensitive and patient-specific seizure forecasting. Most seizure forecasting algorithms use only labeled data for training purposes. As the seizure data is labeled manually by neurologists, preparing the labeled data is expensive and time consuming, making the best use of the data critical. In this article, we propose an approach that can make use of not only labeled EEG signals but also the unlabeled ones which are more accessible. We use the short-time Fourier transform on 28-s EEG windows as a pre-processing step. A generative adversarial network (GAN) is trained in an unsupervised manner where information of seizure onset is disregarded. The trained Discriminator of the GAN is then used as a feature extractor. Features generated by the feature extractor are classified by two fully-connected layers (can be replaced by any classifier) for the labeled EEG signals. This semi-supervised patient-specific seizure forecasting method achieves an out-of-sample testing area under the operating characteristic curve (AUC) of 77.68%, 75.47% and 65.05% for the CHB-MIT scalp EEG dataset, the Freiburg Hospital intracranial EEG dataset and the EPILEPSIAE dataset, respectively. Unsupervised training without the need for labeling is important because not only it can be performed in real-time during EEG signal recording, but also it does not require feature engineering effort for each patient. To the best of our knowledge, this is the first application of GAN to seizure forecasting.

Semi-Supervised Non-Negative Matrix Factorization With Dissimilarity and Similarity Regularization

In this article, we propose a semi-supervised non-negative matrix factorization (NMF) model by means of elegantly modeling the label information. The proposed model is capable of generating discriminable low-dimensional representations to improve clustering performance. Specifically, a pair of complementary regularizers, i.e., similarity and dissimilarity regularizers, is incorporated into the conventional NMF to guide the factorization. And, they impose restrictions on both the similarity and dissimilarity of the low-dimensional representations of data samples with labels as well as a small number of unlabeled ones. The proposed model is formulated as a well-posed constrained optimization problem and further solved with an efficient alternating iterative algorithm. Moreover, we theoretically prove that the proposed algorithm can converge to a limiting point that meets the Karush-Kuhn-Tucker conditions. Extensive experiments as well as comprehensive analysis demonstrate that the proposed model outperforms the state-of-the-art NMF methods to a large extent over five benchmark data sets, i.e., the clustering accuracy increases to 82.2% from 57.0%.

Model Matching: A Novel Framework to use Clustering Strategy to Solve the Classification Problem

It is a common practice to handle labeled data with classifiers and unlabeled ones with clusterings. The traditional Bayesian network classifiers (BNC$^{\mathcal {T}}\text{s}$ ) learned from labeled training set $\mathcal {T}$ directly map the unlabeled test instance into the network structure to calculate the conditional probability for the classification, which neglects the information hidden in the unlabeled data and will result in classification bias. To address this issue, we propose a novel learning framework, called model matching, that uses the “clustering” strategy to solve the classification problem. The labeled data is divided into several clusters according to the different class label to learn a set of BNC$^{\mathcal {T}}\text{s}$ and a corresponding set of BNC$^{p}\text{s}$ is built for each unlabeled test instance. To make a classification, the cross entropy method is applied to compare the structural similarity between BNC$^{\mathcal {T}}$ and BNC p . The extensive experimental results on 46 datasets from the University of California at Irvine (UCI) machine learning repository demonstrate that for BNCs model matching helps improve the generalization performance and outperforms the several state-of-the-art classifiers like tree-augmented naive Bayes and Random forest.

Remote Sensing Imagine Change Detection Using Semi-Supervised Neural Networks

By comparing and contrasting photographs of the same location taken at various periods using remote sensing technology, we may determine whether or not there have been any changes to the land cover there. Identifying shifts in a timely manner will facilitate the identification of natural disasters and climate change, the evaluation of damage, and the rehabilitation of affected areas. Both supervised and unsupervised methods have been utilised historically for change detection. In practise, specialists can only gather a small number of tagged patterns to use in solving the change detection issue. In this case, supervised techniques are ineffective, and if unsupervised methods are employed instead, some scarce but useful labels are wasted. In this dissertation, we offer a novel methodology to enhance change detection in the face of data scarcity by combining a small number of labelled patterns with a large number of unlabeled ones. In this study, many neural network architectures are employed to develop active and semi-supervised learning change detection methods. For better change detection when only a small number of labelled patterns are available, we redesign two unsupervised learning based neural networks, a modified self-organizing feature map and a Hopfield type neural network, in a semi-supervised framework.