Semi-supervised Tasks Research Articles

Cross-view discrepancy-dependency network for volumetric medical image segmentation

The limited data poses a crucial challenge for deep learning-based volumetric medical image segmentation, and many methods have tried to represent the volume by its subvolumes (i.e., multi-view slices) for alleviating this issue. However, such methods generally sacrifice inter-slice spatial continuity. Currently, a promising avenue involves incorporating multi-view information into the network to enhance volume representation learning, but most existing studies tend to overlook the discrepancy and dependency across different views, ultimately limiting the potential of multi-view representations. To this end, we propose a cross-view discrepancy-dependency network (CvDd-Net) to task with volumetric medical image segmentation, which exploits multi-view slice prior to assist volume representation learning and explore view discrepancy and view dependency for performance improvement. Specifically, we develop a discrepancy-aware morphology reinforcement (DaMR) module to effectively learn view-specific representation by mining morphological information (i.e., boundary and position of object). Besides, we design a dependency-aware information aggregation (DaIA) module to adequately harness the multi-view slice prior, enhancing individual view representations of the volume and integrating them based on cross-view dependency. Extensive experiments on four medical image datasets (i.e., Thyroid, Cervix, Pancreas, and Glioma) demonstrate the efficacy of the proposed method on both fully-supervised and semi-supervised tasks.

Attention-based stackable graph convolutional network for multi-view learning

In multi-view learning, graph-based methods like Graph Convolutional Network (GCN) are extensively researched due to effective graph processing capabilities. However, most GCN-based methods often require complex preliminary operations such as sparsification, which may bring additional computation costs and training difficulties. Additionally, as the number of stacking layers increases in most GCN, over-smoothing problem arises, resulting in ineffective utilization of GCN capabilities. In this paper, we propose an attention-based stackable graph convolutional network that captures consistency across views and combines attention mechanism to exploit the powerful aggregation capability of GCN to effectively mitigate over-smoothing. Specifically, we introduce node self-attention to establish dynamic connections between nodes and generate view-specific representations. To maintain cross-view consistency, a data-driven approach is devised to assign attention weights to views, forming a common representation. Finally, based on residual connectivity, we apply an attention mechanism to the original projection features to generate layer-specific complementarity, which compensates for the information loss during graph convolution. Comprehensive experimental results demonstrate that the proposed method outperforms other state-of-the-art methods in multi-view semi-supervised tasks.

Nucleus-Aware Self-Supervised Pretraining Using Unpaired Image-to-Image Translation for Histopathology Images.

Self-supervised pretraining attempts to enhance model performance by obtaining effective features from unlabeled data, and has demonstrated its effectiveness in the field of histopathology images. Despite its success, few works concentrate on the extraction of nucleus-level information, which is essential for pathologic analysis. In this work, we propose a novel nucleus-aware self-supervised pretraining framework for histopathology images. The framework aims to capture the nuclear morphology and distribution information through unpaired image-to-image translation between histopathology images and pseudo mask images. The generation process is modulated by both conditional and stochastic style representations, ensuring the reality and diversity of the generated histopathology images for pretraining. Further, an instance segmentation guided strategy is employed to capture instance-level information. The experiments on 7 datasets show that the proposed pretraining method outperforms supervised ones on Kather classification, multiple instance learning, and 5 dense-prediction tasks with the transfer learning protocol, and yields superior results than other self-supervised approaches on 8 semi-supervised tasks. Our project is publicly available at https://github.com/zhiyuns/UNITPathSSL.

Structure-Aware DropEdge Toward Deep Graph Convolutional Networks.

It has been discovered that graph convolutional networks (GCNs) encounter a remarkable drop in performance when multiple layers are piled up. The main factor that accounts for why deep GCNs fail lies in oversmoothing, which isolates the network output from the input with the increase of network depth, weakening expressivity and trainability. In this article, we start by investigating refined measures upon DropEdge-an existing simple yet effective technique to relieve oversmoothing. We term our method as DropEdge ++ for its two structure-aware samplers in contrast to DropEdge: layer-dependent (LD) sampler and feature-dependent (FD) sampler. Regarding the LD sampler, we interestingly find that increasingly sampling edges from the bottom layer yields superior performance than the decreasing counterpart as well as DropEdge. We theoretically reveal this phenomenon with mean-edge-number (MEN), a metric closely related to oversmoothing. For the FD sampler, we associate the edge sampling probability with the feature similarity of node pairs and prove that it further correlates the convergence subspace of the output layer with the input features. Extensive experiments on several node classification benchmarks, including both full-and semi-supervised tasks, illustrate the efficacy of DropEdge ++ and its compatibility with a variety of backbones by achieving generally better performance over DropEdge and the no-drop version.

Semi-supervised learning with missing values imputation

Incomplete instances with various missing attributes in many real-world applications have brought challenges to the classification tasks. Unsupervised imputation is often employed to replace the missing values with substitute values before supervised classification. However, this process often separates the imputation and classification, which may lead to inferior performance since the separated two tasks ignore the data distribution and label information contained in each other. Besides, traditional methods may rely on improper assumptions to initialize the missing values, whereas the unreliability of such initialization might degrade the performance. To address these problems, a novel semi-supervised conditional normalizing flow (SSCFlow) is proposed in this paper. SSCFlow combines unsupervised imputation and supervised classification as a joint semi-supervised task, which estimates the conditional distribution of incomplete instances to facilitate the imputation and classification simultaneously. Moreover, SSCFlow treats the initialized missing values as corrupted initial imputations and iteratively reconstructs their latent representations to approximate their true conditional distribution. Experiments on real-world datasets demonstrate the robustness and effectiveness of the proposed algorithm.

ATOM: Self-supervised human action recognition using atomic motion representation learning

Self-supervised learning (SSL) is a promising method for gaining perception and common sense from unlabelled data. Existing approaches to analyzing human body skeletons address the problem similar to SSL models for image and video understanding, but pixel data is far more challenging than coordinates. This paper presents ATOM, an SSL model designed for skeleton-based data analysis. Unlike video-based SSL approaches, ATOM leverages atomic movements within skeleton actions to achieve a more fine-grained representation. The proposed architecture predicts the action order at the frame level, leading to improved perceptions and representations of each action. ATOM outperforms state-of-the-art approaches in two well-known datasets (NTU RGB + D and NTU-120 RGB + D), and its weight transferability enables performance improvements on supervised and semi-supervised tasks, up to 4.4% (3.3% p.p.) and 14.1% (6.3% p.p.), respectively, in Top-1 Accuracy.

MTCSNet: Mean Teachers Cross-Supervision Network for Semi-Supervised Cloud Detection

Cloud detection methods based on deep learning depend on large and reliable training datasets to achieve high detection accuracy. There will be a significant impact on their performance, however when the training data are insufficient or when the label quality is low. Thus, to alleviate this problem, a semi-supervised cloud detection method, named the mean teacher cross-supervision cloud detection network (MTCSNet) is proposed. This method enforces both consistency and accuracy on two cloud detection student network branches, which are perturbed with different initializations, for the same input image. For each of the two student branches, the respective teacher branches, used to generate high-quality pseudo labels, are constructed using an exponential moving average method (EMA). A pseudo one-hot label, produced by one teacher network branch, supervises the other student network branch with the standard cross-entropy loss, and vice versa. To incorporate additional prior information into the model, the presented method uses near-infrared bands instead of red bands as model inputs and injects strong data augmentations on unlabeled images fed into the student model. This induces the model to learn richer representations and ensure consistency constraints on the predictions of the same unlabeled image across different batches. To attain a more refined equilibrium between the supervised and semi-supervised loss in the training process, the proposed cloud detection network learns the optimal weights based on homoscedastic uncertainty, thus effectively exploiting the advantages of semi-supervised tasks and elevating the overall performance. Experiments on the SPARCS and GF1-WHU public cloud detection datasets show that the proposed method outperforms several state-of-the-art semi-supervised algorithms when only a limited number of labeled samples are available.

Ada-MIP: Adaptive Self-supervised Graph Representation Learning via Mutual Information and Proximity Optimization

Self-supervised graph-level representation learning has recently received considerable attention. Given varied input distributions, jointly learning graphs’ unique and common features is vital to downstream tasks. Inspired by graph contrastive learning (GCL), which targets maximizing the agreement between graph representations from different views, we propose an Ada ptive self-supervised framework, Ada-MIP, considering both M utual I nformation between views (unique features) and inter-graph P roximity (common features). Specifically, Ada-MIP learns graphs’ unique information through a learnable and probably injective augmenter, which can acquire more adaptive views compared to the augmentation strategies applied by existing GCL methods; to learn graphs’ common information, we employ graph kernels to calculate graphs’ proximity and learn graph representations among which the precomputed proximity is preserved. By sharing a global encoder, graphs’ unique and common information can be well integrated into the graph representations learned by Ada-MIP. Ada-MIP is also extendable to semi-supervised scenarios, with our experiments confirming its superior performance in both unsupervised and semi-supervised tasks.

Dual Teacher: A Semisupervised Cotraining Framework for Cross-Domain Ship Detection

Cross-domain ship detection tries to identify Synthetic Aperture Radar (SAR) ship by adapting knowledge from labeled optical images, without labor-intensive annotations. In practical applications, a few ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e.g</i> ., one or three samples) labeled SAR samples are available, which provides an additional supervision for SAR ships. However, the existing cross-domain methods ignore the SAR supervision (a few labeled and unlabeled SAR images), which limits their performances in a practical and under-investigated task: semi-supervised cross-domain ship detection. In this paper, a Dual Teacher framework is proposed to address the mutual interference between the optical supervision and the SAR supervision. First, both optical and SAR supervision are decomposed into two sub-tasks: cross-domain task and semi-supervised task. Then, both cross-domain task and semi-supervised task can be learned interactively in two individual teacher-student models. The teacher-student models generate pseudo-labels on unlabeled SAR images by a teacher network and fine-tune the student network. Finally, the Dual Teacher framework retrains two teacher-student models in co-training strategies. Both cross-domain dataset and semi-supervised dataset are exploited to jointly improve the pseudo-label quality. The effectiveness of the Dual Teacher framework has been fully experimentally demonstrated. The code is available at https://github.com/XiangtaoZheng/DualTeacher.

Incorporating Linear Regression Problems Into an Adaptive Framework With Feasible Optimizations

Accompanied with the increasing popularity of linear regression approaches, most of the existing minimization problems are related with several convex measurements, e.g., <formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex>$\ell_1$</tex></formula> / <formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex>$\ell_2$</tex></formula> / <formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex>$\ell_{2,1}$</tex></formula> -norm of a vector and <formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex>$L_1$</tex></formula> / <formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex>$L_{2,1}$</tex></formula> /Frobenius/nuclear norm of a matrix, where the regularized function and the loss function are usually studied for two objective terms case by case, respectively. To address this issue, this work combines these linear regression problems into a unified expression framework by employing an adaptive and flexible function, in which we need to choose different variable elements and adjust an inner parameter, properly. Besides this, they are equipped with some corresponding relationships and their interesting properties. Intuitively speaking, the proposed framework can generalize several traditional linear regression formulations and even more complex ones into an extended representation. For further optimizations, an iteratively re-weighted penalty solution (IRwPS) is devised without any inner loops, making the iteration programming easy to perform. Meanwhile, the theoretical results are provided for guaranteeing that the mathematical convergence analysis is solid and meaningful. Finally, by performing real-world applications in supervised, unsupervised, and semi-supervised tasks, numerical experiments are conducted to validate the theoretical properties and the superiority over some of the state-of-the-art.

Consistency Regularization for Deep Face Anti-Spoofing

Face anti-spoofing (FAS) plays a crucial role in securing face recognition systems. Empirically, given an image, a model with more consistent output on different views (i.e., augmentations) of this image usually performs better. Motivated by this exciting observation, we conjecture that encouraging feature consistency of different views may be a promising way to boost FAS models. In this paper, we explore this way thoroughly by enhancing both Embedding-level and Prediction-level Consistency Regularization (EPCR) in FAS. Specifically, at the embedding level, we design a dense similarity loss to maximize the similarities between all positions of two intermediate feature maps in a self-supervised fashion; while at the prediction level, we optimize the mean square error between the predictions of two views. Notably, our EPCR is free of annotations and can directly integrate into semi-supervised learning schemes. Considering different application scenarios, we further design five diverse semi-supervised protocols to measure semi-supervised FAS techniques. We conduct extensive experiments to show that EPCR can significantly improve the performance of several supervised and semi-supervised tasks on benchmark datasets. The codes and protocols are available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/clks-wzz/EPCR</uri> .

Multi-point attention-based semi-supervised learning for diabetic retinopathy classification

In recent years, the severity classification of some well-known diseases has gradually become a focus of researchers, especially diabetic retinopathy (DR) recognition caused by diabetes. Existing diagnostic methods usually require many annotated fundus images for training. However, in practical situations, the high time and economic cost of annotating images make it unaffordable for many researchers. In this paper, we design a semi-supervised learning framework to explore the associations between unlabeled data, with the help of a small number of labeled samples, for accurate fundus image classification of DR. Through the analysis of semi-supervised tasks, it can be observed that: (1) an attention mechanism that fits the data can improve the network’s potential to extract critical features; (2) unlabeled data is as potentially valuable as labeled ones, and the association between these unknown samples can further improve the model’s performance. Therefore, we propose a Multi-point Attention-based Semi-supervised Learning (MASL) framework that efficiently utilizes the massive unlabeled data for accurate DR classification. Specifically, we propose a multi-point attention mechanism that enables the model to extract subtle features from multiple perspectives of fundus images and discard invalid regions. In addition, we design a new self-supervision mechanism to force the model to perform mandatory similarity mining on unknown samples, maximizing the potential of squeezing unlabeled data based on the confidence of the selected unannotated samples. Sufficient experimental results demonstrate that our MASL method significantly outperforms other methods on two public datasets.

Deep semi-supervised clustering for multi-variate time-series

Huge amount of data are nowadays produced by a large and disparate family of sensors, which typically measure multiple variables over time. Such rich information can be profitably organized as multivariate time-series. Collect enough labelled samples to set up supervised analysis for such kind of data is challenging while a reasonable assumption is to dispose of a limited background knowledge that can be injected in the analysis process. In this context, semi-supervised clustering methods represent a well suited tool to get the most out of such reduced amount of knowledge. With the aim to deal with multivariate time-series analysis under a limited background knowledge setting, we propose a semi-supervised (constrained) deep embedding time-series clustering framework that exploits knowledge supervision modeled as Must- and Cannot-link constraints. More in detail, our proposal, named conDetSEC (constrained Deep embedding time SEries Clustering), is based on Gated Recurrent Units (GRUs) with the aim to explicitly manage the temporal dimension associated to multi-variate time series data. conDetSEC implements a procedure in which an embedding generation step is combined with a clustering refinement step. Both steps exploit the small amount of available knowledge provided by Must- and Cannot-link constraints. More specifically, during the data embedding generation the constraints are used by jointly optimizing the network parameters via both unsupervised and semi-supervised tasks, while at the refinement step they are used in conjunction with the goal to stretch the embedding manifold towards the clustering centroids to recover a more clear cluster structure. Experimental evaluation on real-world benchmarks coming from diverse domains has highlighted the effectiveness of our proposal in comparison with state-of-the-art unsupervised and semi-supervised time-series clustering methods.

Investigating Topic Modeling Techniques to Extract Meaningful Insights in Italian Long COVID Narration.

Through an adequate survey of the history of the disease, Narrative Medicine (NM) aims to allow the definition and implementation of an effective, appropriate, and shared treatment path. In the present study different topic modeling techniques are compared, as Latent Dirichlet Allocation (LDA) and topic modeling based on BERT transformer, to extract meaningful insights in the Italian narration of COVID-19 pandemic. In particular, the main focus was the characterization of Post-acute Sequelae of COVID-19, (i.e., PASC) writings as opposed to writings by health professionals and general reflections on COVID-19, (i.e., non-PASC) writings, modeled as a semi-supervised task. The results show that the BERTopic-based approach outperforms the LDA-base approach by grouping in the same cluster the 97.26% of analyzed documents, and reaching an overall accuracy of 91.97%.

CoCoS: Enhancing Semi-supervised Learning on Graphs with Unlabeled Data via Contrastive Context Sharing

Graph Neural Networks (GNNs) have recently become a popular framework for semi-supervised learning on graph-structured data. However, typical GNN models heavily rely on labeled data in the learning process, while ignoring or paying little attention to the data that are unlabeled but available. To make full use of available data, we propose a generic framework, Contrastive Context Sharing (CoCoS), to enhance the learning capacity of GNNs for semi-supervised tasks. By sharing the contextual information among nodes estimated to be in the same class, different nodes can be correlated even if they are unlabeled and remote from each other in the graph. Models can therefore learn different combinations of contextual patterns, which improves the robustness of node representations. Additionally, motivated by recent advances in self-supervised learning, we augment the context sharing strategy by integrating with contrastive learning, which naturally correlates intra-class and inter-class data. Such operations utilize all available data for training and effectively improve a model's learning capacity. CoCoS can be easily extended to a wide range of GNN-based models with little computational overheads. Extensive experiments show that CoCoS considerably enhances typical GNN models, especially when labeled data are sparse in a graph, and achieves state-of-the-art or competitive results in real-world public datasets. The code of CoCoS is available online.

Lithology identification by adaptive feature aggregation under scarce labels

Lithology identification plays an important role in petroleum exploration. However, due to the high cost of labeling by cores and cuttings, the application of semi-supervised method in lithology identification should be considered. There is a main problem in the past graph-based semi-supervised methods, that is, the connection strengths (i.e., edges) among samples are calculated before training and therefore fixed. However, these pre-handcrafted connections strengths determine the graph structure and therefore affect the performance of the trained model. Obviously, there inevitably exist inappropriate connection settings, which motivates us to consider the dynamic adjustment of these connection strengths via training. First, the graph attention network is used to complete the semi-supervised task in the process of lithology identification. This is the first time to our knowledge that the graph attention network is applied to lithology identification. This method can learn edge weights and mitigate interference due to the error of initial edge weights. Second, two methods are used in graph neural network to establish the connection between logging data and transforming them into graph structure data. Third, through a lot of experiments, the graph attention network using depth similarity (GAT-D) outperforms other algorithms on two metrics (macro-avg-f1 score of 73.73% and macro-avg-recall of 80.54% in dataset A). Hence, converting logging data into graph data and then using graph neural network to realize semi-supervised classification could improve the performance of lithology identification.

ESW Edge Weights: Ensemble Stochastic Watershed Edge Weights for Hyperspectral Image Classification

Hyperspectral image (HSI) classification is a topic of active research. One of the main challenges of HSI classification is the lack of reliable labelled samples. Various semi-supervised and unsupervised classification methods are proposed to handle the low number of labelled samples. Chief among them are graph convolution networks (GCN) and their variants. These approaches exploit the graph structure for semi-supervised and unsupervised classification. While several of these methods implicitly construct edge-weights, to our knowledge, not much work has been done to estimate the edge-weights explicitly. In this article, we estimate the edge-weights explicitly and use them for the downstream classification tasks - both semi-supervised and unsupervised. The proposed edge-weights are based on two key insights - (a) Ensembles reduce the variance and (b) Classes in HSI datasets and feature similarity have only one-sided implications. That is, while same classes would have similar features, similar features do not necessarily imply the same classes. Exploiting these, we estimate the edge-weights using an aggregate of ensembles of watersheds over subsamples of features. These edge weights are evaluated for both semi-supervised and unsupervised classification tasks. The evaluation for semi-supervised tasks uses Random-Walk based approach. For the unsupervised case, we use a simple filter using a graph convolution network (GCN). In both these cases, the proposed edge weights outperform the traditional approaches to compute edge-weights - Euclidean distances and cosine similarities. Fascinatingly, with the proposed edge-weights, the simplest GCN obtained results comparable to the recent state-of-the-art.

Learning adaptive criteria weights for active semi-supervised learning

Batch mode active learning (BMAL) is devoted to training trustful learning models with scarce labeled samples by efficiently asking the ground truth annotations of the most beneficial unlabeled points for supervision with the feedback of an expert. Particularly, BMAL algorithms always sample points based on the decent-designed criteria, such as (un)certainty and representativeness, etc. However, present BMAL approaches consistently are afflicted with one limitation: They simply integrate the sampling criteria with fixed weights to select instances for supervised training, which may yield suboptimal batch acquisition since the criteria values of the plentiful candidate unlabeled samples would fluctuate after retraining the classifier with the newly augmented training set. Instead, the weights of sampling criteria should be allocated appropriately. To overcome this problem, this work proposes a novel Adaptive Criteria Weights batch selection algorithm, abbreviated ACW, which dynamically adjusts the importance of (un)certainty and representativeness to choose critical instances for semi-supervised learning. A submodular function is employed to recognize a diverse mini-batch from the selected batch of samples. We apply our proposed ACW batch sampling algorithm to two types of essential semi-supervised tasks, i.e., semi-supervised classification and semi-supervised clustering. To the best of our knowledge, this work is the first devoted attempt to explore adaptive mechanism of criteria weights in the context of active learning. The superiority and effectiveness of ACW against the present state-of-the-art BMAL approaches have also been demonstrated by the encouraging experimental results.

Every node counts: Self-ensembling graph convolutional networks for semi-supervised learning

Graph convolutional network (GCN) provides a powerful means for graph-based semi-supervised tasks. However, as a localized first-order approximation of spectral graph convolution, the classic GCN can not take full advantage of unlabeled data, especially when the unlabeled node is far from labeled ones. To capitalize on the information from unlabeled nodes to boost the training for GCN, we propose a novel framework named Self-Ensembling GCN (SEGCN), which marries GCN with Mean Teacher – a powerful self-ensemble learning mechanism for semi-supervised task. SEGCN contains a student model and a teacher model. As a student, it not only learns to correctly classify the labeled nodes, but also tries to be consistent with the teacher on unlabeled nodes in more challenging situations, such as a high dropout rate and graph corrosion. As a teacher, it averages the student model weights and generates more accurate predictions to lead the student. In such a mutual-promoting process, both labeled and unlabeled samples can be fully utilized for backpropagating effective gradients to train GCN. In a variety of semi-supervised classification benchmarks, i.e. Citeseer, Cora, Pubmed and NELL, we validate that the proposed method matches the state of the arts in the classification accuracy. The code is publicly available at https://github.com/RoyalVane/SEGCN.

Going Deep: Graph Convolutional Ladder-Shape Networks

Neighborhood aggregation algorithms like spectral graph convolutional networks (GCNs) formulate graph convolutions as a symmetric Laplacian smoothing operation to aggregate the feature information of one node with that of its neighbors. While they have achieved great success in semi-supervised node classification on graphs, current approaches suffer from the over-smoothing problem when the depth of the neural networks increases, which always leads to a noticeable degradation of performance. To solve this problem, we present graph convolutional ladder-shape networks (GCLN), a novel graph neural network architecture that transmits messages from shallow layers to deeper layers to overcome the over-smoothing problem and dramatically extend the scale of the neural networks with improved performance. We have validated the effectiveness of proposed GCLN at a node-wise level with a semi-supervised task (node classification) and an unsupervised task (node clustering), and at a graph-wise level with graph classification by applying a differentiable pooling operation. The proposed GCLN outperforms original GCNs, deep GCNs and other state-of-the-art GCN-based models for all three tasks, which were designed from various perspectives on six real-world benchmark data sets.