Semi-supervised Learning Method Research Articles

Knowledge Guided Semi-supervised Learning for Quality Assessment of User Generated Videos

Perceptual quality assessment of user generated content (UGC) videos is challenging due to the requirement of large scale human annotated videos for training. In this work, we address this challenge by first designing a self-supervised Spatio-Temporal Visual Quality Representation Learning (ST-VQRL) framework to generate robust quality aware features for videos. Then, we propose a dual-model based Semi Supervised Learning (SSL) method specifically designed for the Video Quality Assessment (SSL-VQA) task, through a novel knowledge transfer of quality predictions between the two models. Our SSL-VQA method uses the ST-VQRL backbone to produce robust performances across various VQA datasets including cross-database settings, despite being learned with limited human annotated videos. Our model improves the state-of-the-art performance when trained only with limited data by around 10%, and by around 15% when unlabelled data is also used in SSL. Source codes and checkpoints are available at https://github.com/Shankhanil006/SSL-VQA.

From Toxic to Trustworthy: Using Self-Distillation and Semi-supervised Methods to Refine Neural Networks

Despite the tremendous success of deep neural networks (DNNs) across various fields, their susceptibility to potential backdoor attacks seriously threatens their application security, particularly in safety-critical or security-sensitive ones. Given this growing threat, there is a pressing need for research into purging backdoors from DNNs. However, prior efforts on erasing backdoor triggers not only failed to withstand increasingly powerful attacks but also resulted in reduced model performance. In this paper, we propose From Toxic to Trustworthy (FTT), an innovative approach to eliminate backdoor triggers while simultaneously enhancing model accuracy. Following the stringent and practical assumption of limited availability of clean data, we introduce a self-attention distillation (SAD) method to remove the backdoor by aligning the shallow and deep parts of the network. Furthermore, we first devise a semi-supervised learning (SSL) method that leverages ubiquitous and available poisoned data to further purify backdoors and improve accuracy. Extensive experiments on various attacks and models have shown that our FTT can reduce the attack success rate from 97% to 1% and improve the accuracy of 4% on average, demonstrating its effectiveness in mitigating backdoor attacks and improving model performance. Compared to state-of-the-art (SOTA) methods, our FTT can reduce the attack success rate by 2 times and improve the accuracy by 5%, shedding light on backdoor cleansing.

ANEDL: Adaptive Negative Evidential Deep Learning for Open-Set Semi-supervised Learning

Semi-supervised learning (SSL) methods assume that labeled data, unlabeled data and test data are from the same distribution. Open-set semi-supervised learning (Open-set SSL) con- siders a more practical scenario, where unlabeled data and test data contain new categories (outliers) not observed in labeled data (inliers). Most previous works focused on out- lier detection via binary classifiers, which suffer from insufficient scalability and inability to distinguish different types of uncertainty. In this paper, we propose a novel framework, Adaptive Negative Evidential Deep Learning (ANEDL) to tackle these limitations. Concretely, we first introduce evidential deep learning (EDL) as an outlier detector to quantify different types of uncertainty, and design different uncertainty metrics for self-training and inference. Furthermore, we propose a novel adaptive negative optimization strategy, making EDL more tailored to the unlabeled dataset containing both inliers and outliers. As demonstrated empirically, our proposed method outperforms existing state-of-the-art methods across four datasets.

A General Model for Aggregating Annotations AcrossSimple, Complex, and Multi-object Annotation Tasks (Abstract Reprint)

Human annotations are vital to supervised learning, yet annotators often disagree on the correct label, especially as annotation tasks increase in complexity. A common strategy to improve label quality is to ask multiple annotators to label the same item and then aggregate their labels. To date, many aggregation models have been proposed for simple categorical or numerical annotation tasks, but far less work has considered more complex annotation tasks, such as those involving open-ended, multivariate, or structured responses. Similarly, while a variety of bespoke models have been proposed for specific tasks, our work is the first we are aware of to introduce aggregation methods that generalize across many, diverse complex tasks, including sequence labeling, translation, syntactic parsing, ranking, bounding boxes, and keypoints. This generality is achieved by applying readily available task-specific distance functions, then devising a task-agnostic method to model these distances between labels, rather than the labels themselves. This article presents a unified treatment of our prior work on complex annotation modeling and extends that work with investigation of three new research questions. First, how do complex annotation task and dataset properties impact aggregation accuracy? Second, how should a task owner navigate the many modeling choices in order to maximize aggregation accuracy? Finally, what tests and diagnoses can verify that aggregation models are specified correctly for the given data? To understand how various factors impact accuracy and to inform model selection, we conduct large-scale simulation studies and broad experiments on real, complex datasets. Regarding testing, we introduce the concept of unit tests for aggregation models and present a suite of such tests to ensure that a given model is not mis-specified and exhibits expected behavior. Beyond investigating these research questions above, we discuss the foundational concept and nature of annotation complexity, present a new aggregation model as a conceptual bridge between traditional models and our own, and contribute a new general semisupervised learning method for complex label aggregation that outperforms prior work.

Unknown-Aware Graph Regularization for Robust Semi-supervised Learning from Uncurated Data

Recent advances in semi-supervised learning (SSL) have relied on the optimistic assumption that labeled and unlabeled data share the same class distribution. However, this assumption is often violated in real-world scenarios, where unlabeled data may contain out-of-class samples. SSL with such uncurated unlabeled data leads training models to be corrupted. In this paper, we propose a robust SSL method for learning from uncurated real-world data within the context of open-set semi-supervised learning (OSSL). Unlike previous works that rely on feature similarity distance, our method exploits uncertainty in logits. By leveraging task-dependent predictions of logits, our method is capable of robust learning even in the presence of highly correlated outliers. Our key contribution is to present an unknown-aware graph regularization (UAG), a novel technique that enhances the performance of uncertainty-based OSSL frameworks. The technique addresses not only the conflict between training objectives for inliers and outliers but also the limitation of applying the same training rule for all outlier classes, which are existed on previous uncertainty-based approaches. Extensive experiments demonstrate that UAG surpasses state-of-the-art OSSL methods by a large margin across various protocols. Codes are available at https://github.com/heejokong/UAGreg.

Linear semantic transformation for semi-supervised medical image segmentation

Medical image segmentation is a focus research and foundation in developing intelligent medical systems. Recently, deep learning for medical image segmentation has become a standard process and succeeded significantly, promoting the development of reconstruction, and surgical planning of disease diagnosis. However, semantic learning is often inefficient owing to the lack of supervision of feature maps, resulting in that high-quality segmentation models always rely on numerous and accurate data annotations. Learning robust semantic representation in latent spaces remains a challenge. In this paper, we propose a novel semi-supervised learning framework to learn vital attributes in medical images, which constructs generalized representation from diverse semantics to realize medical image segmentation. We first build a self-supervised learning part that achieves context recovery by reconstructing space and intensity of medical images, which conduct semantic representation for feature maps. Subsequently, we combine semantic-rich feature maps and utilize simple linear semantic transformation to convert them into image segmentation. The proposed framework was tested using five medical segmentation datasets. Quantitative assessments indicate the highest scores of our method on IXI (73.78%), ScaF (47.50%), COVID-19-Seg (50.72%), PC-Seg (65.06%), and Brain-MR (72.63%) datasets. Finally, we compared our method with the latest semi-supervised learning methods and obtained 77.15% and 75.22% DSC values, respectively, ranking first on two representative datasets. The experimental results not only proved that the proposed linear semantic transformation was effectively applied to medical image segmentation, but also presented its simplicity and ease-of-use to pursue robust segmentation in semi-supervised learning. Our code is now open at: https://github.com/QingYunA/Linear-Semantic-Transformation-for-Semi-Supervised-Medical-Image-Segmentation.

Neural collapse inspired semi-supervised learning with fixed classifier

Pseudo-labeling-based approaches are gaining prominence in Semi-Supervised Learning (SSL). Recent studies have identified that the key bottleneck in this methodology is addressing insufficient, incorrect, and imbalanced pseudo-labels. In this paper, we argue that the intrinsic problem behind this bottleneck is classifier bias, i.e., the classifier's prototypes suffer from poor uniformity. Further, inspired by neural collapse that reveals an optimal structure under supervised training scenarios, we address this classifier bias by utilizing an offline simplex Equiangular Tight Frame (ETF) classifier with maximally and equally separated prototypes. During the training phase, we maintain the prototypes of the classifier as fixed and concentrate on refining the feature encoder. Specifically, we integrate a straightforward clustering-based pseudo-labeling strategy with information maximization for feature learning. In practice, the fixed ETF classifier prevents the model from falling into a detrimental cycle, where a biased classifier exacerbates misaligned features, further perpetuating this bias. Furthermore, the clustering-based pseudo-labeling strategy reduces the dependency on complex threshold-adjusting mechanisms and effectively navigates the quantity-quality trade-off that plagues existing SSL methods. Leveraging these methodologies, we develop a simple yet powerful approach, termed ETF-SSL. Extensive experiments across Image, Text, and Audio datasets demonstrate that ETF-SSL can achieve competitive or superior performance compared to existing approaches. This success highlights the benefits of using a fixed ETF classifier in SSL and points to promising directions for future research in this area. The code is available at: https://github.com/yichenwang231/ETFSSL.

Semi-Supervised Machine Learning Method for Predicting Observed Individual Risk Preference Using Gallup Data

Risk and uncertainty play a vital role in almost every significant economic decision, and an individual’s propensity to make riskier decisions also depends on various circumstances. This article aims to investigate the effects of social and economic covariates on an individual’s willingness to take general risks and extends the scope of existing works by using quantitative measures of risk-taking from the GPS and Gallup datasets (in addition to the qualitative measures used in the literature). Based on the available observed risk-taking data for one year, this article proposes a semi-supervised machine learning-based approach that can efficiently predict the observed risk index for those countries/individuals for years when the observed risk-taking index was not collected. We find that linear models are insufficient to capture certain patterns among risk-taking factors, and non-linear models, such as random forest regression, can obtain better root mean squared values than those reported in past literature. In addition to finding factors that agree with past studies, we also find that subjective well-being influences risk-taking behavior.

Semi-supervised nonnegative matrix factorization with label propagation and constraint propagation

Semi-supervised nonnegative matrix factorization (SNMF) improves the clustering effect of nonnegative matrix factorization (NMF) by integrating label information. However, as one of the most commonly used semi-supervised learning methods, label propagation algorithm (LP) has some limitations due to its heavy dependence on the predefined similarity matrix. To address this deficiency and effectively use limited supervisory information, a structural NMF algorithm with label propagation and constraint propagation (PNLP-SCNMF) is proposed in this paper, which simultaneously performs data representation and classification in a joint manner. Both positive label information and negative label information are introduced into the proposed method, and the factor matrix is constrained by label information to construct the tri-decomposition form. Constraint propagation algorithm (CPA) is used to correct the constructed similarity matrix, so that the low-dimensional subspace obtained by matrix decomposition can retain the geometric structure characteristics of the original sample space. In addition, an efficient iterative update optimization scheme is proposed to solve this objective function, and its convergence is proved theoretically. A large number of comparative experiments on nine real datasets show that the proposed method can consistently achieve higher clustering accuracy in image clustering tasks.

A survey on semi-supervised graph clustering

Semi-Supervised Graph Clustering (SSGC) has emerged as a pivotal field at the intersection of graph clustering and semi-supervised learning (SSL), offering innovative solutions to intricate data analysis problems. However, despite its significance and wide-ranging applications, there exists a notable void in the literature—a comprehensive survey specifically dedicated to SSGC techniques and their diverse applications remains conspicuously absent. Addressing this gap, this paper presents a systematic and comprehensive review of SSGC methodologies, spanning from well-established approaches to cutting-edge developments. Through a meticulous categorization, critical examination, and insightful discussion of these techniques, this survey not only illuminates the current landscape of SSGC but also identifies unexplored avenues for exploration and innovation. In this paper we present a comprehensive survey of conventional graph construction, graph clustering, SSL methods, evaluation metrics and their primary development process. We then provide a taxonomy of SSGC techniques based on their structures and principles and thoroughly analyze and discuss the related models. Furthermore, we review the applications of SSGC in various fields. Lastly, we summarize the limitations of current SSGC algorithms and discuss the future directions of the field.

Online semi-supervised active learning ensemble classification for evolving imbalanced data streams

Concept drift is a core challenge in classification tasks of data streams. Although many drift adaptation methods have been presented, most of them assume that labels of all data are available, which is impractical in many real-world applications. Additionally, the absence of label makes the imbalance ratio of an imbalanced data stream difficultly being obtained in time, providing the inaccurate guidance for resampling and causing poor generalization. To tackle the joint challenges, an online semi-supervised active learning method is proposed to classifier imbalanced data streams with concept drift. A newly-arrived data is first added to the sliding window, and then assigned a pseudo label in terms of its nearest cluster. Meanwhile, semi-supervised clustering algorithm offers its predicted label. Based on the above two predictive labels, cluster-based query strategy provides the criteria for the evaluation and selection of representative instances. More especially, the uncertainty and importance of instances are defined to synthetically evaluate its representativeness. After obtaining true labels of typical ones, ensemble classifier is updated by all instances in current sliding window. Experimental results on 13 synthetic and real data streams indicate that the proposed method outperforms six comparative methods on both G-mean and Recall under various labeling budgets.

Semi-supervised adversarial deep learning for capacity estimation of battery energy storage systems

Battery energy storage systems (BESS) play a pivotal role in energy management, and the precise estimation of battery capacity is crucial for optimizing their performance and ensuring reliable power supply. Deep learning methodologies applied to battery capacity estimation have exhibited exemplary performance. However, deep learning methods necessitate supervised training with a significant volume of labeled data, presenting challenges for data collection in industrial scenarios. Moreover, a diverse range of battery types in industrial settings makes it difficult to develop capacity estimation models for different types of batteries from scratch. To address these issues, a semi-supervised adversarial deep learning (SADL) method is proposed for lithium-ion battery capacity estimation. Initially, a subset of labeled lithium-ion battery data, coupled with a subset of unlabeled data, is collected. Voltage and current data are then transformed into capacity increment features. Subsequently, an adversarial training strategy is employed, subjecting labeled and unlabeled data to adversarial training to enhance the performance of SADL. Finally, the effectiveness of the SADL method in estimating the capacity of other lithium-ion batteries is analysed. Experimental results demonstrate that the SADL method accurately estimates the capacity of various battery types, showcasing an RMSE error of approximately 2%, surpassing the performance of other methods. The proposed SADL method emerges as a promising solution for the precise estimation of lithium-ion battery capacity in BESS.

Semi-supervised deep learning for molecular clump verification

Context. A reliable molecular clump detection algorithm is essential for studying these clumps. Existing detection algorithms for molecular clumps still require that detected candidates be verified manually, which is impractical for large-scale data. Semi-supervised learning methods, especially those based on deep features, have the potential to accomplish the task of molecular clump verification thanks to the powerful feature extraction capability of deep networks. Aims. Our main objective is to develop an automated method for the verification of molecular clump candidates. This method utilises a 3D convolutional neural network (3D CNN) to extract features of molecular clumps and employs semi-supervised learning to train the model, with the aim being to improve its generalisation ability and data utilisation. It addresses the issue of insufficient labelled samples in traditional supervised learning and enables the model to better adapt to new, unlabelled samples, achieving high accuracy in the verification of molecular clumps. Methods. We propose SS-3D-Clump, a semi-supervised deep clustering method that jointly learns the parameters of a 3D CNN and the cluster assignments of the generated features for automatic verification of molecular clumps. SS-3D-Clump iteratively classifies the features with the Constrained-KMeans and uses these class labels as supervision to update the weights of the entire network. Results. We used CO data from the Milky Way Imaging Scroll Painting project covering 350 square degrees in the Milky Way’s first, second, and third quadrants. The ClumpFind algorithm was applied to extract molecular clump candidates in these regions, which were subsequently verified using SS-3D-Clump. The SS-3D-Clump model, trained on a dataset comprising three different density regions, achieved an accuracy of 0.933, a recall rate of 0.955, a precision rate of 0.945, and an F1 score of 0.950 on the corresponding test dataset. These results closely align with those obtained through manual verification. Conclusions. Our experiments demonstrate that the SS-3D-Clump model achieves high accuracy in the automated verification of molecular clumps. It effectively captures the essential features of the molecular clumps and overcomes the challenge of limited labelled samples in supervised learning by using unlabelled samples through semi-supervised learning. This enhancement significantly improves the generalisation capability of the SS-3D-Clump model, allowing it to adapt effectively to new and unlabelled samples. Consequently, SS-3D-Clump can be integrated with any detection algorithm to create a comprehensive framework for the automated detection and verification of molecular clumps.

Text Attribute Control via Closed-Loop Disentanglement

Abstract Changing an attribute of a text without changing the content usually requires first disentangling the text into irrelevant attributes and content representations. After that, in the inference phase, the representation of one attribute is tuned to a different value, expecting that the corresponding attribute of the text can also be changed accordingly. The usual way of disentanglement is to add some constraints on the latent space of an encoder-decoder architecture, including adversarial-based constraints and mutual-information-based constraints. However, previous semi-supervised processes of attribute change are usually not enough to guarantee the success of attribute change and content preservation. In this paper, we propose a novel approach to achieve a robust control of attributes while enhancing content preservation. In this approach, we use a semi-supervised contrastive learning method to encourage the disentanglement of attributes in latent spaces. Differently from previous works, we re-disentangle the reconstructed sentence and compare the re-disentangled latent space with the original latent space, which makes a closed-loop disentanglement process. This also helps content preservation. In addition, the contrastive learning method is also able to replace the role of minimizing mutual information and adversarial training in the disentanglement process, which alleviates the computation cost. We conducted experiments on three text datasets, including the Yelp Service review dataset, the Amazon Product review dataset, and the GoEmotions dataset. The experimental results show the effectiveness of our model.

Gearbox fault diagnosis method based on improved semi-supervised MTDL and GAF

Aiming at the problem that it takes a long time and high cost to obtain complete labeled data under intelligent fault diagnosis and unlabeled data is not used. This paper proposes an improved semi-supervised mean teacher deep learning (MTDL) and Gramian angle field (GAF) fusion diagnostic method. This method fully utilizes a small number of labeled samples and a large number of unlabeled samples to deeply mine invisible fault features and potential physical correlations. At the same time, it solves the problem of losing the inter-data correlation structure when one-dimensional time series signals are used as inputs for neural networks. The GAF-MTDL method uses consistency regularization and modifies the network structure in the mean teacher algorithm into a semi-supervised deep learning model enhanced by WideResNet. The experimental results show that the proposed GAF-MTDL method saves a lot of manual labeling costs, improves the recognition accuracy and generalization ability, and can achieve excellent prediction accuracy with very little labeled data. In the end, the accuracy of planetary gear fault identification reached 98.22% under the labeling rate of 20%, and the accuracy of fault identification reached 99.98% through the verification of the bearing data set of Case Western Reserve University. The value of this research is to bring an efficient and low-cost technology to the field of industrial intelligent fault diagnosis, which can significantly improve the accuracy of fault identification.

FiFrauD: Unsupervised Financial Fraud Detection in Dynamic Graph Streams

Given a stream of financial transactions between traders in an e-market, how can we accurately detect fraudulent traders and suspicious behaviors in real time? Despite the efforts made in detecting these fraudsters, this field still faces serious challenges, including the ineffectiveness of existing methods for the complex and streaming environment of e-markets. As a result, it is still difficult to quickly and accurately detect suspected traders and behavior patterns in real-time transactions, and it is still considered an open problem. To solve this problem and alleviate the existing challenges, in this article, we propose FiFrauD, which is an unsupervised, scalable approach that depicts the behavior of manipulators in a transaction stream. In this approach, real-time transactions between traders are converted into a stream of graphs and, instead of using supervised and semi-supervised learning methods, fraudulent traders are detected precisely by exploiting density signals in graphs. Specifically, we reveal the traits of fraudulent traders in the market and propose a novel metric from this perspective, i.e., graph topology, time, and behavior. Then, we search for suspicious blocks by greedily optimizing the proposed metric. Theoretical analysis demonstrates upper bounds for FiFrauD's effectiveness in catching suspicious trades. Extensive experiments on five real-world datasets with both actual and synthetic labels demonstrate that FiFrauD achieves significant accuracy improvements compared with state-of-the-art fraud detection methods. Also, it can find various suspicious behavior patterns in a linear runtime and provide interpretable results. Furthermore, FiFrauD is resistant to the camouflage tactics used by fraudulent traders.

A semi-supervised framework for computational fluid dynamics prediction

Data-driven deep learning approach heavily relies on the diversity and quantity of data. Acquiring data in the computational fluid dynamics (CFD) domain is a time and computationally intensive process. This paper proposes a semi-supervised learning method called discriminative regression fitters (DRF) for aerodynamic prediction of airfoils. DRF utilizes neural networks’ memory property to dynamically divide pseudo-labeled data into easy and difficult subsets using a model of Gaussian distribution. The method classifies unlabeled data based on loss and updates the pseudo-labeled data, improving the model’s generalization capability. Experiments on airfoil regression task datasets show that DRF achieves similar or better prediction accuracy than fully supervised approaches. It reduces data acquisition time by 70%. Ablation studies and qualitative results verify the effectiveness of DRF. The surrogate model obtained from DRF is extended to airfoil optimization, demonstrating its practicality. DRF provides a promising direction for improving the regression task while reducing the reliance on large amounts of CFD data.

Semi-supervised medical image classification via distance correlation minimization and graph attention regularization

We propose a novel semi-supervised learning method to leverage unlabeled data alongside minimal annotated data and improve medical imaging classification performance in realistic scenarios with limited labeling budgets to afford data annotations. Our method introduces distance correlation to minimize correlations between feature representations from different views of the same image encoded with non-coupled deep neural networks architectures. In addition, it incorporates a data-driven graph-attention based regularization strategy to model affinities among images within the unlabeled data by exploiting their inherent relational information in the feature space. We conduct extensive experiments on four medical imaging benchmark data sets involving X-ray, dermoscopic, magnetic resonance, and computer tomography imaging on single and multi-label medical imaging classification scenarios. Our experiments demonstrate the effectiveness of our method in achieving very competitive performance and outperforming several state-of-the-art semi-supervised learning methods. Furthermore, they confirm the suitability of distance correlation as a versatile dependence measure and the benefits of the proposed graph-attention based regularization for semi-supervised learning in medical imaging analysis.

CycleGAN-Based Data Augmentation for Subgrade Disease Detection in GPR Images with YOLOv5

Vehicle-mounted ground-penetrating radar (GPR) technology is an effective means of detecting railway subgrade diseases. However, existing methods of GPR data interpretation largely rely on manual identification, which is not only inefficient but also highly subjective. This paper proposes a semi-supervised deep learning method to identify railway subgrade diseases. This method addresses the sample imbalance problem in the defect dataset by utilizing a data augmentation method based on a generative adversarial network model. An initial network model for disease identification is obtained by training the YOLOv5 network with a small number of existing samples. The intelligently extended samples are then labeled to achieve a balance in the disease samples. The network is trained to improve the recognition accuracy of the intelligent model using a more complete dataset. The experimental results show that the accuracy of the proposed method can reach up to 94.53%, which is 23.85% higher than that of the supervised learning model without an extended dataset. This has strong industrial application value for railway subgrade disease detection as the potential learning ability of the model can be explored to a greater extent, thereby improving the recognition accuracy of subgrade diseases.

Multi-Augmentation-Based Contrastive Learning for Semi-Supervised Learning

Semi-supervised learning has been proven to be effective in utilizing unlabeled samples to mitigate the problem of limited labeled data. Traditional semi-supervised learning methods generate pseudo-labels for unlabeled samples and train the classifier using both labeled and pseudo-labeled samples. However, in data-scarce scenarios, reliance on labeled samples for initial classifier generation can degrade performance. Methods based on consistency regularization have shown promising results by encouraging consistent outputs for different semantic variations of the same sample obtained through diverse augmentation techniques. However, existing methods typically utilize only weak and strong augmentation variants, limiting information extraction. Therefore, a multi-augmentation contrastive semi-supervised learning method (MAC-SSL) is proposed. MAC-SSL introduces moderate augmentation, combining outputs from moderately and weakly augmented unlabeled images to generate pseudo-labels. Cross-entropy loss ensures consistency between strongly augmented image outputs and pseudo-labels. Furthermore, the MixUP is adopted to blend outputs from labeled and unlabeled images, enhancing consistency between re-augmented outputs and new pseudo-labels. The proposed method achieves a state-of-the-art performance (accuracy) through extensive experiments conducted on multiple datasets with varying numbers of labeled samples. Ablation studies further investigate each component’s significance.