Semi-supervised Learning Scenarios Research Articles

Constantly optimized mean teacher for semi-supervised 3D MRI image segmentation.

The mean teacher model and its variants, as important methods in semi-supervised learning, have demonstrated promising performance in magnetic resonance imaging (MRI) data segmentation. However, the superior performance of teacher model through exponential moving average (EMA) is limited by the unreliability of unlabeled image, resulting in potentially unreliable predictions. In this paper, we propose a framework to optimized the teacher model with reliable expert-annotated data while preserving the advantages of EMA. To avoid the tight coupling that results from EMA, we leverage data augmentations to provide two distinct perspectives for the teacher and student models. The teacher model adopts weak data augmentation to provide supervision for the student model and optimizes itself with real annotations, while the student uses strong data augmentation to avoid overfitting on noise information. In addition, double softmax helps the model resist noise and continue learning meaningful information from the images, which is a key component in the proposed model. Extensive experiments show that the proposed method exhibits competitive performance on the Left Atrium segmentation MRI dataset (LA) and the Brain Tumor Segmentation MRI dataset (BraTS2019). For the LA dataset, we achieved a dice of 91.02% using only 20% labeled data, which is close to the dice of 91.14% obtained by the supervised approach using 100% labeled data. For the BraTs2019 dataset, the proposed method achieved 1.02% and 1.92% improvement on 5% and 10% labeled data, respectively, compared to the best baseline method on this dataset. This study demonstrates that the proposed model can be a potential candidate for medical image segmentation in semi-supervised learning scenario.

Robust hyperspectral image classification using generative adversarial networks

This paper introduces Sill-Rgan, a novel Generative Adversarial Network (GAN) designed to improve hyperspectral image (HSI) classification under varying lighting conditions. Sill-Rgan uniquely maps different light condition domains, enhancing sample classification robustness and generating new virtual samples. Addressing challenges like high spectral dimensionality and noise in HSI classification, our approach utilizes a deep proxy-based learning framework. It integrates and improves advanced GAN models and multitask networks for optimal training stability and loss function optimization. The model's mapping network is adept at generating domain-specific latent codes, enabling the transformation of original hyperspectral data into enhanced versions. Extensive experiments conducted on hyperspectral datasets of agricultural products under diverse indoor and outdoor lighting conditions confirm the effectiveness of Sill-Rgan. The results highlight the model's adaptability in both supervised and semi-supervised learning scenarios, yielding exceptional classification accuracy and enhanced data quality. The versatile potential of Sill-Rgan extends its applicability to a broad range of spectral data classifications, underlining its significant contribution to hyperspectral imaging. This advancement opens new avenues in machine vision systems, particularly in scenarios with dynamic lighting challenges.

An Interpretable and Accurate Deep-Learning Diagnosis Framework Modeled With Fully and Semi-Supervised Reciprocal Learning.

The deployment of automated deep-learning classifiers in clinical practice has the potential to streamline the diagnosis process and improve the diagnosis accuracy, but the acceptance of those classifiers relies on both their accuracy and interpretability. In general, accurate deep-learning classifiers provide little model interpretability, while interpretable models do not have competitive classification accuracy. In this paper, we introduce a new deep-learning diagnosis framework, called InterNRL, that is designed to be highly accurate and interpretable. InterNRL consists of a student-teacher framework, where the student model is an interpretable prototype-based classifier (ProtoPNet) and the teacher is an accurate global image classifier (GlobalNet). The two classifiers are mutually optimised with a novel reciprocal learning paradigm in which the student ProtoPNet learns from optimal pseudo labels produced by the teacher GlobalNet, while GlobalNet learns from ProtoPNet's classification performance and pseudo labels. This reciprocal learning paradigm enables InterNRL to be flexibly optimised under both fully- and semi-supervised learning scenarios, reaching state-of-the-art classification performance in both scenarios for the tasks of breast cancer and retinal disease diagnosis. Moreover, relying on weakly-labelled training images, InterNRL also achieves superior breast cancer localisation and brain tumour segmentation results than other competing methods.

Weakly Supervised AUC Optimization: A Unified Partial AUC Approach.

Since acquiring perfect supervision is usually difficult, real-world machine learning tasks often confront inaccurate, incomplete, or inexact supervision, collectively referred to as weak supervision. In this work, we present WSAUC, a unified framework for weakly supervised AUC optimization problems, which covers noisy label learning, positive-unlabeled learning, multi-instance learning, and semi-supervised learning scenarios. Within the WSAUC framework, we first frame the AUC optimization problems in various weakly supervised scenarios as a common formulation of minimizing the AUC risk on contaminated sets, and demonstrate that the empirical risk minimization problems are consistent with the true AUC. Then, we introduce a new type of partial AUC, specifically, the reversed partial AUC (rpAUC), which serves as a robust training objective for AUC maximization in the presence of contaminated labels. WSAUC offers a universal solution for AUC optimization in various weakly supervised scenarios by maximizing the empirical rpAUC. Theoretical and experimental results under multiple settings support the effectiveness of WSAUC on a range of weakly supervised AUC optimization tasks.

Neighbor-aware deep multi-view clustering via graph convolutional network

Multi-view clustering (MVC) enhances the clustering performance of data by combining correlation information from different views. However, most existing MVC approaches process each sample independently and ignore the correlation amongst samples, resulting in reduced clustering performance. Although graph convolution network (GCN) can naturally capture correlation amongst samples by integrating the neighbors and structural information into representation learning, it is used in the semi-supervised learning scenario. In this paper, we propose a neighbor-aware deep MVC framework based on GCN (NMvC-GCN) for clustering multi-view samples and training GCN in a fully unsupervised manner. In addition, we design a consensus regularization to learn the common representations and introduce a clustering embedding layer to jointly optimize the clustering task and representation learning, so that the correlation amongst samples and that between the clustering task and representation learning can be fully explored. Extensive experiments on 10 datasets illustrate that NMvC-GCN significantly outperforms the state-of-the-art MVC methods. Our code will be released at https://github.com/dugzzuli/NMvC-GCN.

Two-stage natural scene image classification with noise discovering and label-correlation mining

In natural scene classification, it is common that one natural scene image may belong to multiple categories concurrently; as a result, multi-label learning has become a research hotspot. Despite recent rapid developments in multi-label learning, the increasing amount of high-dimensional data poses great challenges—such as redundant features and high computational costs—to conventional multi-label learning models. Most contemporary strategies for dealing with this issue depend on forcing feature learning into multi-label models. Notably, however, these approaches seldom pay attention to the label correlation and propagation in the feature subspace. To address this issue, we introduce an alternative multi-label feature learning solution that incorporates both labeled and unlabeled information. Unlike existing multi-label learning models , which rely on clean and trustworthy training datasets, we argue that in semi-supervised learning scenarios, the unlabeled data can be easily corrupted by noise or outliers, which causes the model performance to degrade. We next extract the label correlation and propagate the label information to discover the noise or outliers. Subsequently, our model adaptively searches the optimal feature subspace to reduce the influence of redundant features for high-dimensional data. The effectiveness of the proposed model is demonstrated by experimental observations on artificial and real-world datasets.

A Novel Approach Based on Generative Adversarial Network for Interference Detection in Wireless Communications

With the rapid growth of wireless devices, the communication environment gets complex. The detection of interference or unauthorized signals can improve spectrum efficiency, which is a key technology for limited spectrum resources. Traditional detection methods analyze the parameter characteristics of the received signal. But it is difficult to detect interference with the same time and frequency as the original signal by those feature engineering. As a classical problem in deep learning, anomaly detection is usually solved by supervised learning. But a more challenging situation is to detect unknown or invisible anomalies. It means that the number of abnormal samples is insufficient and the data is highly biased toward the normal samples. In this paper, a wireless communication interference detection algorithm based on generative adversarial network (GAN) is proposed. In the semi-supervised learning scenario, the algorithm detects the time-frequency overlapped interference by the reconstruction strategy. The generator adopts the encoder-decoder-encoder architecture. In the training process, the model jointly learns the data distribution of normal samples by minimizing the distance in both the signal space and the latent space. In the inference phase, a large distance metric implies an abnormal sample. Experiments on simulated communication datasets show the superiority of the proposed algorithm.

Label-Noise Robust Deep Generative Model for Semi-Supervised Learning

Deep generative models have demonstrated an excellent ability to generate data by learning their distribution. Despite their unsupervised nature, these models can be implemented in semi-supervised learning scenarios by treating the class labels as additional latent variables. In this article, we propose a deep generative model for semi-supervised learning that offsets label noise, which is a ubiquitous feature in large-scale datasets owing to the high cost of annotation. We assume that noisy labels are generated from true labels and employ a noise transition matrix to describe the transition. We estimate this matrix by adjusting its entries to minimize its difference with the true transition matrix and use the estimated matrix to formulate the objective function for inference, which consists of an evidence lower bound and a classification risk. However, because directly minimizing the latter with noisy labels may result in an inaccurate classifier, we propose a statistically consistent estimator for computing the classification risk solely with noisy data. Empirical results on benchmark datasets demonstrate that the proposed model improves the classification performance over that of the baseline algorithms. We also present a case study on semiconductor manufacturing. Additionally, we empirically show that the proposed model, as a generative model, is capable of reconstructing data even with noisy labels.

Low-rank supervised and semi-supervised multi-metric learning for classification

Multi-metric learning is an important technique for improving classification performance since learning a single metric is usually insufficient for complex data. Most of the existing multi-metric learning approaches have high computational complexity. In this work, two multi-metric learning frameworks proposed to perform supervised and semi-supervised classifications respectively. Based on the frameworks, we first design a low-rank multi-metric learning model (LSMML) for supervised classification, in which multiple local class metrics as well as one global metric are jointly trained. A joint regularization scheme, composed of LogDet divergence and low-rank term, is also well-designed to incorporate prior knowledge for improving generalization. By learning appropriate metrics, LSMML not only captures the local nonlinear discriminate information of each class to reduce the probability of misclassification but also enhances the stability, alleviates the computational burden, and avoids the risk of overfitting. Then, we extend LSMML to a semi-supervised learning scenario and propose a low-rank semi-supervised multi-metric learning approach (LSeMML) to process data with scarce labels. Alternating iterative algorithms are designed to optimize both LSMML and LSeMML. At each iteration, we only require to perform geodesic convex optimizations, with closed-form solutions and low computational cost. In supervised and semi-supervised settings respectively, numerical simulations are carried out on different databases, which shows that the proposed LSMML and LSeMML have a simple form, fast training speed, and good classification performance.

Learning Kernel for Conditional Moment-Matching Discrepancy-Based Image Classification.

Conditional maximum mean discrepancy (CMMD) can capture the discrepancy between conditional distributions by drawing support from nonlinear kernel functions; thus, it has been successfully used for pattern classification. However, CMMD does not work well on complex distributions, especially when the kernel function fails to correctly characterize the difference between intraclass similarity and interclass similarity. In this paper, a new kernel learning method is proposed to improve the discrimination performance of CMMD. It can be operated with deep network features iteratively and thus denoted as KLN for abbreviation. The CMMD loss and an autoencoder (AE) are used to learn an injective function. By considering the compound kernel, that is, the injective function with a characteristic kernel, the effectiveness of CMMD for data category description is enhanced. KLN can simultaneously learn a more expressive kernel and label prediction distribution; thus, it can be used to improve the classification performance in both supervised and semisupervised learning scenarios. In particular, the kernel-based similarities are iteratively learned on the deep network features, and the algorithm can be implemented in an end-to-end manner. Extensive experiments are conducted on four benchmark datasets, including MNIST, SVHN, CIFAR-10, and CIFAR-100. The results indicate that KLN achieves the state-of-the-art classification performance.

Diagnostic Sparse Connectivity Networks with Regularization Template

Dynamic systems are often monitored with multivariate time series where each dimension represents a local component measured through a (virtual) sensor. Performing accurate diagnostic for dynamic systems while simultaneously taking into account their similarities/distinctions, is a non-trivial task. To this end, we develop an adaptive regularization approach to learning sparse connectivity structures in complex dynamic systems. The learned connectivity networks shed lights on the structural compositions of the system and hence can serve as highly informative inputs for various machine learning tasks such as classification. In particular, we focus on high-dimensional and semi-supervised learning scenarios and present a joint learning approach to recover system-wise connectivity patterns by adaptively constructing a shared, sparsity-inducing regularization template across all systems. The shared template can be physically interpreted and used as a modeling template for analyzing new systems. Moreover, our approach has the flexibility to incorporate supervising information such as must-links and cannot-links for constructing regularization templates. Overall, our approach, named sparse adaptive regularization (SAR), can extract structure-related connectivity features efficiently and effectively, and result in significant improvements for machine learning tasks in dynamic systems. We benchmark our approach against the state-of-the-art methods with real-world data. Our results demonstrate the superiority of our approach.

On the Sample Complexity of HGR Maximal Correlation Functions for Large Datasets

The Hirschfeld-Gebelein-Rényi (HGR) maximal correlation and the corresponding functions have been shown useful in many machine learning scenarios. In this paper, we study the sample complexity of estimating the HGR maximal correlation functions by the alternating conditional expectation (ACE) algorithm using training samples from large datasets. Specifically, we develop a mathematical framework to characterize the learning errors between the maximal correlation functions computed from the true distribution, and the functions estimated from the ACE algorithm. For both supervised and semi-supervised learning scenarios, we establish the analytical expressions for the error exponents of the learning errors. Furthermore, we demonstrate that for large datasets, the upper bounds for the sample complexity of learning the HGR maximal correlation functions by the ACE algorithm can be expressed using the established error exponents. Moreover, with our theoretical results, we investigate the sampling strategy for different types of samples in semi-supervised learning with a total sampling budget constraint, and an optimal sampling strategy is developed to maximize the error exponent of the learning error. Finally, the numerical simulations are presented to support our theoretical results.

Rank-based self-training for graph convolutional networks

Graph Convolutional Networks (GCNs) have been established as a fundamental approach for representation learning on graphs, based on convolution operations on non-Euclidean domain, defined by graph-structured data. GCNs and variants have achieved state-of-the-art results on classification tasks, especially in semi-supervised learning scenarios. A central challenge in semi-supervised classification consists in how to exploit the maximum of useful information encoded in the unlabeled data. In this paper, we address this issue through a novel self-training approach for improving the accuracy of GCNs on semi-supervised classification tasks. A margin score is used through a rank-based model to identify the most confident sample predictions. Such predictions are exploited as an expanded labeled set in a second-stage training step. Our model is suitable for different GCN models. Moreover, we also propose a rank aggregation of labeled sets obtained by different GCN models. The experimental evaluation considers four GCN variations and traditional benchmarks extensively used in the literature. Significant accuracy gains were achieved for all evaluated models, reaching results comparable or superior to the state-of-the-art. The best results were achieved for rank aggregation self-training on combinations of the four GCN models.

Learning Better Representations for Audio-Visual Emotion Recognition with Common Information

Audio-visual emotion recognition aims to distinguish human emotional states by integrating the audio and visual data acquired in the expression of emotions. It is crucial for facilitating the affect-related human-machine interaction system by enabling machines to intelligently respond to human emotions. One challenge of this problem is how to efficiently extract feature representations from audio and visual modalities. Although progresses have been made by previous works, most of them ignore common information between audio and visual data during the feature learning process, which may limit the performance since these two modalities are highly correlated in terms of their emotional information. To address this issue, we propose a deep learning approach in order to efficiently utilize common information for audio-visual emotion recognition by correlation analysis. Specifically, we design an audio network and a visual network to extract the feature representations from audio and visual data respectively, and then employ a fusion network to combine the extracted features for emotion prediction. These neural networks are trained by a joint loss, combining: (i) the correlation loss based on Hirschfeld-Gebelein-Rényi (HGR) maximal correlation, which extracts common information between audio data, visual data, and the corresponding emotion labels, and (ii) the classification loss, which extracts discriminative information from each modality for emotion prediction. We further generalize our architecture to the semi-supervised learning scenario. The experimental results on the eNTERFACE’05 dataset, BAUM-1s dataset, and RAVDESS dataset show that common information can significantly enhance the stability of features learned from different modalities, and improve the emotion recognition performance.

Exploring Domain Knowledge for Facial Expression-Assisted Action Unit Activation Recognition

Current works on facial action unit (AU) activation recognition typically include supervised training using AU-annotated training images. Compared to facial expression labeling, AU annotation is a time-consuming, expensive, and error-prone process. Domain knowledge refers to the strong probabilistic dependencies between facial expressions and AUs, as well as dependencies among AUs. To take advantage of this, we avoid the time-consuming process of AU annotation and introduce a new AU activation recognition method that learns AU classifiers from domain knowledge, and requires only expression-annotated facial images. Specifically, we first generate pseudo AU labels according to the probabilistic dependencies between expressions and AUs as well as correlations among AUs summarized from domain knowledge. Then, we propose to use a Restricted Boltzmann Machine to model AU label prior distribution from the generated pseudo AU data. After that, we train AU classifiers from expression-annotated facial images and the learned prior model by maximizing the log likelihood of AU classifiers with regard to the learned AU label prior. The proposed AU activation recognition can also be extended to semi-supervised learning scenarios with partially AU-annotated facial images. Experimental results on four benchmark databases demonstrate the effectiveness of the proposed approach in learning AU classifiers from domain knowledge.

SCOs: Semi-Supervised Co-Selection by a Similarity Preserving Approach

In this paper, we focus on co-selection of instances and features in the semi-supervised learning scenario. In this context, co-selection becomes a more challenging problem as data contain labeled and unlabeled examples sampled from the same population. To carry out such semi-supervised co-selection, we propose a unified framework, called sCOs, which efficiently integrates labeled and unlabeled parts into the co-selection process. The framework is based on introducing both a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">sparse regularization term</i> and a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">similarity preserving approach</i> . It evaluates the usefulness of features and instances in order to select the most relevant ones, simultaneously. We propose two efficient algorithms that work for both convex and nonconvex functions. To the best of our knowledge, this paper offers, for the first time ever, a study utilizing nonconvex penalties for the co-selection of semi-supervised learning tasks. Experimental results on some known benchmark datasets are provided for validating sCOs and comparing it with some representative methods in the state-of-the art.

Robust Semisupervised Graph Classifier Learning With Negative Edge Weights

In a semisupervised learning scenario, (possibly noisy) partially observed labels are used as input to train a classifier in order to assign labels to unclassified samples. In this paper, we construct a complete graph-based binary classifier given only samples’ feature vectors and partial labels. Specifically, we first build appropriate similarity graphs with positive and negative edge weights connecting all samples based on internode feature distances. By viewing a binary classifier as a piecewise constant graph signal, we cast classifier learning as a signal restoration problem via a classical maximum a posteriori (MAP) formulation. One unfortunate consequence of negative edge weights is that the graph Laplacian matrix $\mathbf {L}$ can be indefinite, and previously proposed graph-signal smoothness prior $\mathbf {x}^T \mathbf {L}\mathbf {x}$ for candidate signal $\mathbf {x}$ can lead to pathological solutions. In response, we derive a minimum-norm perturbation matrix $\boldsymbol{\Delta }$ that preserves $\mathbf {L}$ 's eigen-structure—based on a fast lower-bound computation of $\mathbf {L}$ 's smallest negative eigenvalue via a novel application of the Haynsworth inertia additivity formula—so that $\mathbf {L}+ \boldsymbol{\Delta }$ is positive semidefinite, resulting in a stable signal prior. Further, instead of forcing a hard binary decision for each sample, we define the notion of generalized smoothness on graphs that promotes ambiguity in the classifier signal. Finally, we propose an algorithm based on iterative reweighted least squares that solves the posed MAP problem efficiently. Extensive simulation results show that our proposed algorithm outperforms both SVM variants and previous graph-based classifiers using positive-edge graphs noticeably.

Weakly Supervised Facial Action Unit Recognition With Domain Knowledge.

Current facial action unit (AU) recognition typically includes supervised training, where the fully AU annotated training images are required. Due to the nuances of facial appearance and individual differences, AU annotation is a time-consuming, expensive, and error-prone process. Facial expression is relatively simple to label, since facial expressions describe facial behavior globally and the number of expressions appearing on a face is much less than that of AUs. Furthermore, there exist strong dependencies between AUs and expressions, referred to as domain knowledge. Such domain knowledge is inherent in facial anatomy and facial behavior. Therefore, in this paper, we propose a novel weakly supervised AU recognition method to jointly learn multiple AU classifiers with expression annotations but without any AU annotations by leveraging domain knowledge. Specifically, we first summarize the expression-dependent AU ranking from the domain knowledge of conditional probabilities of AUs given expressions. Then, we formulate the weakly supervised AU recognition as a multilabel ranking problem and propose an efficient learning algorithm to solve it. Furthermore, we extend the proposed weakly supervised AU recognition method to a semi-supervised learning scenario when partial AU labeled samples are available. Experimental results on three benchmark databases demonstrate that the proposed method can successfully exploit domain knowledge for multiple AU recognition and, thus, outperforms both state-of-the-art weakly supervised AU recognition method and the semi-supervised AU recognition method.

RGCLI: Robust Graph that Considers Labeled Instances for Semi-Supervised Learning

Graph-based semi-supervised learning (SSL) provides a powerful framework for the modeling of manifold structures in high-dimensional spaces. Additionally, graph representation is effective for the propagation of the few initial labels existing in training data. Graph-based SSL requires robust graphs as input for an accurate data mining task, such as classification. In contrast to most graph construction methods, which ignore the labeled instances available in SSL scenarios, a previous study proposed a graph-construction method, named GBILI, to exploit the informativeness conveyed by such instances available in a semi-supervised classification domain. Here, we have improved the method proposing an optimized algorithm referred to as Robust Graph that Considers Labeled Instances (RGCLI) for the generation of more robust graphs. The contributions of this paper are threefold: i) reduction of GBILI time complexity from quadratic to O(nklogn). This enhancement allows addressing large datasets; ii) demonstration of RGCLI mathematical properties, proving the constructed graph is an optimal graph to model the smoothness assumption of SSL; and iii) evaluation of the efficacy of the proposed approach in a comprehensive semi-supervised classification scenario with several datasets, including an image segmentation task, which needs a large graph to represent the image. Such experiments show the use of labeled vertices in the graph construction process improves the graph topology, hence, the learning task in which it will be employed.

PClass+: A Novel Evolving Semi-Supervised Classifier

A novel evolving semi-supervised classifier, namely Parsimonious Classifier+ (pClass+), is proposed in this paper. pClass+ enhances a recently developed classifier, namely pClass, for a semi-supervised learning scenario. As with its predecessor, pClass+ is capable of initiating its learning process from scratch with an empty rule base and adopts an open network structure, where fuzzy rules are evolved, pruned, and recalled automatically on demands. The novelty of pClass+ lies in an online active learning technique, which decreases operator’s annotation efforts and expedites its training process. pClass+ is also equipped with a new parameter identification strategy to cope with the class overlapping situation. The efficacy of pClass+ has been experimentally validated with numerous synthetic and real-world study cases, confirmed by thorough statistical tests and comparisons against state-of-the art classifiers, where pClass+ outperforms its counterparts in achieving the best trade-off between accuracy and complexity.