Semi-supervised Learning Method Research Articles

Semi-Supervised Medical Image Segmentation Based on Deep Consistent Collaborative Learning

In the realm of medical image analysis, the cost associated with acquiring accurately labeled data is prohibitively high. To address the issue of label scarcity, semi-supervised learning methods are employed, utilizing unlabeled data alongside a limited set of labeled data. This paper presents a novel semi-supervised medical segmentation framework, DCCLNet (deep consistency collaborative learning UNet), grounded in deep consistent co-learning. The framework synergistically integrates consistency learning from feature and input perturbations, coupled with collaborative training between CNN (convolutional neural networks) and ViT (vision transformer), to capitalize on the learning advantages offered by these two distinct paradigms. Feature perturbation involves the application of auxiliary decoders with varied feature disturbances to the main CNN backbone, enhancing the robustness of the CNN backbone through consistency constraints generated by the auxiliary and main decoders. Input perturbation employs an MT (mean teacher) architecture wherein the main network serves as the student model guided by a teacher model subjected to input perturbations. Collaborative training aims to improve the accuracy of the main networks by encouraging mutual learning between the CNN and ViT. Experiments conducted on publicly available datasets for ACDC (automated cardiac diagnosis challenge) and Prostate datasets yielded Dice coefficients of 0.890 and 0.812, respectively. Additionally, comprehensive ablation studies were performed to demonstrate the effectiveness of each methodological contribution in this study.

Semi-supervised PolSAR Image Change Detection using Similarity Matching

Abstract. The lack of precisely labeled data limits the development of supervised polarimetric synthetic aperture radar (PolSAR) image change detection. Therefore, semi-supervised deep learning methods have recently demonstrated their significant capability for PolSAR image change detection. Similarity Matching (SimMatch) improves the performance of semi-supervised learning tasks across different benchmark datasets and different settings. Introducing SimMatch into the field of PolSAR image change detection can improve the performance of semi-supervised PolSAR image change detection under limited labeled data conditions. Usually, semi-supervision solves the problem of insufficient labeled data by generating pseudo-labels. However, when the pseudo-label method is simply applied, the model will fit on the confident but wrong pseudo-labels, resulting in poor performance. SimMatch offers a solution by requiring the strongly augmented view to share the same semantic similarity (i.e. label prediction) and instance characteristics (i.e. similarity between instances) with a weak augmented view for more intrinsic feature matching. Besides, by using a labeled memory buffer, the two similarities can be isomorphically transformed with each other by introducing the aggregating and unfolding techniques. Therefore, the semantic and instance pseudo-labels can be mutually propagated, and then, the detection performance of the PolSAR image change detection is improved. Experimental results on real PolSAR datasets demonstrated that SimMatch is an effective semi-supervised PolSAR change detection method and its performance surpasses some well-known change detection methods. Compared to the fully-supervised algorithm CWNN, the semi-supervised SimMatch algorithm can improve accuracy by up to 14.4%.

Boundary sample-based class-weighted semi-supervised learning for malignant tumor classification of medical imaging.

Medical image classification plays a pivotal role within the field of medicine. Existing models predominantly rely on supervised learning methods, which necessitate large volumes of labeled data for effective training. However, acquiring and annotating medical image data is both an expensive and time-consuming endeavor. In contrast, semi-supervised learning methods offer a promising approach by harnessing limited labeled data alongside abundant unlabeled data to enhance the performance of medical image classification. Nonetheless, current methods often encounter confirmation bias due to noise inherent in self-generated pseudo-labels and the presence of boundary samples from different classes. To overcome these challenges, this study introduces a novel framework known as boundary sample-based class-weighted semi-supervised learning (BSCSSL) for medical image classification. Our method aims to alleviate the impact of intra- and inter-class boundary samples derived from unlabeled data. Specifically, we address reliable confidential data and inter-class boundary samples separately through the utilization of an inter-class boundary sample mining module. Additionally, we implement an intra-class boundary sample weighting mechanism to extract class-aware features specific to intra-class boundary samples. Rather than discarding such intra-class boundary samples outright, our approach acknowledges their intrinsic value despite the difficulty associated with accurate classification, as they contribute significantly to model prediction. Experimental results on widely recognized medical image datasets demonstrate the superiority of our proposed BSCSSL method over existing semi-supervised learning approaches. By enhancing the accuracy and robustness of medical image classification, our BSCSSL approach yields considerable implications for advancing medical diagnosis and future research endeavors.

A review of machine learning methods used for educational data

AbstractIntegrating machine learning (ML) methods in educational research has the potential to greatly impact upon research, teaching, learning and assessment by enabling personalised learning, adaptive assessment and providing insights into student performance, progress and learning patterns. To reveal more about this notion, we investigated ML approaches used for educational data analysis in the last decade and provided recommendations for further research. Using a systematic literature review (SLR), we examined 77 publications from two large and high-impact databases for educational research using bibliometric mapping and evaluative review analysis. Our results suggest that the top five most frequently used keywords were similar in both databases. The majority of the publications (88%) utilised supervised ML approaches for predicting students’ performances and finding learning patterns. These methods include decision trees, support vector machines, random forests, and logistic regression. Semi-supervised learning methods were less frequently used, but also demonstrated promising results in predicting students’ performance. Finally, we discuss the implications of these results for statisticians, researchers, and policymakers in education.

Reservoir fluid identification based on multi-head attention with UMAP

Gas logging curves obtained during drilling, along with conventional well logs, typically serve as foundational data for identifying reservoir fluids. Traditional methods for identifying layers containing oil or gas often suffer from limitations such as inadequate feature extraction, underutilization of differences in data points, and limited applicability. These limitations lead to reduced accuracy in predicting reservoir fluid segmentation. To address the challenge of spatiotemporal feature extraction in logging curves, this paper presents a fluid identification method based on deep learning networks and multivariate sequence classification. The method takes into account both the sequential variations in geological strata and the relationships among different curves for fluid prediction. Our proposed method utilizes a Transformer multi-head attention mechanism and follows a series of steps. First, an improved Symbolic Aggregate approXimation (SAX) is employed to assess the response characteristics of the logging curves to the target fluid and identify sensitive curves based on information similarity. Subsequently, the semi-supervised learning method UMAP is applied to identify layers with high interpretability. Hierarchical training is then employed to mitigate the impact of noise and high-dimensional nonlinearity on identification results. Finally, the processed vector curves are input into the attention module layer, where features of each layer are extracted in an encoding-decoding manner. This method has been tested and validated using gas logging data from 12 wells in a gas field offshore China. The overall prediction accuracy can exceed 94%, and the prediction accuracy for a single gas layer is over 88%. When compared to other machine learning approaches, such as K-means, decision tree, and XGBoost, this method exhibits superior identification accuracy and generalization ability, especially for unbalanced samples.

Semi-supervised learning in diagnosis of infant hip dysplasia towards multisource ultrasound images.

Automated diagnosis of infant hip dysplasia is heavily affected by the individual differences among infants and ultrasound machines. Hip sonographic images of 493 infants from various ultrasound machines were collected in the Department of Orthopedics in Yangzhou Maternal and Child Health Care Service Centre. Herein, we propose a semi-supervised learning method based on a feature pyramid network (FPN) and a contrastive learning scheme based on a Siamese architecture. A large amount of unlabeled data of ultrasound images was used via the Siamese network in the pre-training step, and then a small amount of annotated data for anatomical structures was adopted to train the model for landmark identification and standard plane recognition. The method was evaluated on our collected dataset. The method achieved a mean Dice similarity coefficient (DSC) of 0.7873 and a mean Hausdorff distance (HD) of 5.0102 in landmark identification, compared to the model without contrastive learning, which had a mean DSC of 0.7734 and a mean HD of 6.1586. The accuracy, precision, and recall of standard plane recognition were 95.4%, 91.64%, and 94.86%, respectively. The corresponding area under the curve (AUC) was 0.982. This study proposes a semi-supervised deep learning method following Graf's principle, which can better utilize a large volume of ultrasound images from various devices and infants. This method can identify the landmarks of infant hips more accurately than manual operators, thereby improving the efficiency of diagnosis of infant hip dysplasia.

An enhanced semi-supervised learning method with self-supervised and adaptive threshold for fault detection and classification in urban power grids

An enhanced semi-supervised learning method with self-supervised and adaptive threshold for fault detection and classification in urban power grids

Forest fire susceptibility assessment under small sample scenario: A semi-supervised learning approach using transductive support vector machine

Forest fires threaten global ecosystems, socio-economic structures, and public safety. Accurately assessing forest fire susceptibility is critical for effective environmental management. Supervised learning methods dominate this assessment, relying on a substantial dataset of forest fire occurrences for model training. However, obtaining precise forest fire location data remains challenging. To address this issue, semi-supervised learning emerges as a viable solution, leveraging both a limited set of collected samples and unlabeled data containing environmental factors for training. Our study employed the transductive support vector machine (TSVM), a key semi-supervised learning method, to assess forest fire susceptibility in scenarios with limited samples. We conducted a comparative analysis, evaluating its performance against widely used supervised learning methods. The assessment area for forest fire susceptibility lies in Dayu County, Jiangxi Province, China, renowned for its vast forest cover and frequent fire incidents. We analyzed and generated maps depicting forest fire susceptibility, evaluating prediction accuracies for both supervised and semi-supervised learning methods across various small sample scenarios (e.g., 4, 8, 12, 16, 20, 24, 28, and 32 samples). Our findings indicate that TSVM exhibits superior prediction accuracy compared to supervised learning with limited samples, yielding more plausible forest fire susceptibility maps. For instance, at sample sizes of 4, 16, and 28, TSVM achieves prediction accuracies of approximately 0.8037, 0.9257, and 0.9583, respectively. In contrast, random forests, the top performers in supervised learning, demonstrate accuracies of approximately 0.7424, 0.8916, and 0.9431, respectively, for the same small sample sizes. Additionally, we discussed three key aspects: TSVM parameter configuration, the impact of unlabeled sample size, and performance within typical sample sizes. Our findings support semi-supervised learning as a promising approach compared to supervised learning for forest fire susceptibility assessment and mapping, particularly in scenarios with small sample sizes.

A novel semisupervised learning method with textual information for financial distress prediction

AbstractFinancial distress prediction (FDP) has attracted high attention from many financial institutions. Utilizing supervised learning‐based methods in FDP, however, is time consuming and labor intensive. Therefore, in this paper, we exploit active‐pSVM method, which combines potential data distribution information and existing expert experience to solve FDP problem. Moreover, with the increasingly popular textual information, we construct several features on our protocol that are based on the Management Discussion and Analysis (MD&A) text information. Using datasets that are collected in different time windows from the listed Chinese companies, we conducted an extensive experiment and were able to confirm a better efficiency of our active‐pSVM, when compared with some common supervised learning‐based methods. Our study also covers the application of MD&A text information on weakly supervised learning model in FDP.

TCP-RBA: Semi-supervised learning for traditional chinese painting classification with random brushwork augment

Traditional Chinese painting (TCP), culturally significant, reflects China’s rich history and aesthetics. In recent years, TCP classification has shown impressive performance, but obtaining accurate annotations for these tasks is time-consuming and expensive, involving professional art experts. To address this challenge, we present a semi-supervised learning (SSL) method for traditional painting classification, achieving exceptional results even with a limited number of labels. To improve global representation learning, we employ the self-attention-based MobileVit model as the backbone network. Furthermore, We present a data augmentation strategy, Random Brushwork Augment (RBA), which integrates brushwork to enhance the performance. Comparative experiments confirm the effectiveness of TCP-RBA in Chinese painting classification, demonstrating outstanding accuracy of 88.27% on the test dataset, even with only 10 labels, each representing a single class.

Self-training with dual uncertainty for semi-supervised MRI image segmentation

The goal of semi-supervised MRI image segmentation is to train high-accuracy segmentation models using a small amount of labeled data and a large amount of unlabeled data, thereby alleviating the burden of manual annotation by physicians. Self-training is a traditional semi-supervised learning method aimed at acquiring additional image features by generating pseudo-labels for unlabeled data. However, this method may lead to an unstable training process due to differences between samples. Additionally, the predictions of model may contain erroneous noise, leading to low-confidence pseudo-labels. To address these issues, we proposed a dual uncertainty-guided self-training framework (DUST) for semi-supervised MRI image segmentation. Specifically, the two uncertainties consist of sample-level uncertainty and pixel-level uncertainty. The sample-level uncertainty is intended to achieve a stable and smooth training process. The pixel-level uncertainty is intended to rectify pseudo-labels. We conducted a series of experiments on two public MRI image datasets, ACDC2017 and Prostate2018. Compared to the baseline, we improved the Dice scores by 5.0% and 4.0% over the two datasets, respectively. Furthermore, the experimental results indicate that the proposed method has certain advantages compared to the comparative methods. This validates the feasibility and applicability of our method.

Semi-supervised pathological image segmentation via cross distillation of multiple attentions and Seg-CAM consistency

Segmentation of pathological images is a crucial step for accurate cancer diagnosis. However, acquiring dense annotations of such images for training is labor-intensive and time-consuming. To address this issue, Semi-Supervised Learning (SSL) has the potential for reducing the annotation cost, but it is challenged by a large number of unlabeled training images. In this paper, we propose a novel SSL method based on Cross Distillation of Multiple Attentions and Seg-CAM Consistency (CDMA+) to effectively leverage unlabeled images. First, we propose a Multi-attention Tri-decoder Network (MTNet) that consists of a shared encoder and three decoders, with each decoder using a different attention mechanism that calibrates features in different aspects to generate diverse outputs. Second, we introduce Cross Decoder Knowledge Distillation (CDKD) between the three decoders, allowing them to learn from each other’s soft labels to mitigate the negative impact of incorrect pseudo labels during training. Subsequently, motivated by the observation that the Class Activation Maps (CAMs) derived from the classification task could provide a rough segmentation, we employ an auxiliary classification head and introduce a consistency constraint between the CAM and segmentation results, i.e. Seg-CAM consistency. Additionally, uncertainty minimization is applied to the average prediction of the three decoders, which further regularizes predictions on unlabeled images and encourages inter-decoder consistency. Our proposed CDMA+ was compared with eight state-of-the-art SSL methods on two public pathological image datasets, and the experimental results showed that our method outperforms the other approaches under different annotation ratios. The code is available at https://github.com/HiLab-git/CDMA.

A semi-supervised learning method based on pseudo-label iterative purification for intelligent fault diagnosis of rolling bearing

Bearing fault diagnosis based on semi-supervised learning shows good results in solving the scarcity of data in industrial scenarios. Pseudo-label samples can be used to expand labeled datasets, thereby to solve the shortage of training samples. However, when pseudo-labeling is employed to improve the performance of the model, the low confidence level of pseudo-labels will have a negative impact on the model. To solve this problem, the pseudo-label iterative purification method is proposed. Firstly, the graph neural network is employed to aggregate samples in order to increase the degree of dissimilarity among target domain data belonging to distinct categories. Subsequently, the pseudo-labeled samples are purified, while multiple positive-negative discriminators are trained on particular data from the source domain to accurately identify the target domain samples containing the pseudo-labels. Finally, the backbone network is fine-tuned by the purified pseudo-labeled samples. Results of experiments on the self-built bearing dataset and the Paderborn University bearing dataset validate that the method proposed in this article exhibits outstanding performance.

SC-SSL: Self-Correcting Collaborative and Contrastive Co-Training Model for Semi-Supervised Medical Image Segmentation.

Image segmentation achieves significant improvements with deep neural networks at the premise of a large scale of labeled training data, which is laborious to assure in medical image tasks. Recently, semi-supervised learning (SSL) has shown great potential in medical image segmentation. However, the influence of the learning target quality for unlabeled data is usually neglected in these SSL methods. Therefore, this study proposes a novel self-correcting co-training scheme to learn a better target that is more similar to ground-truth labels from collaborative network outputs. Our work has three-fold highlights. First, we advance the learning target generation as a learning task, improving the learning confidence for unannotated data with a self-correcting module. Second, we impose a structure constraint to encourage the shape similarity further between the improved learning target and the collaborative network outputs. Finally, we propose an innovative pixel-wise contrastive learning loss to boost the representation capacity under the guidance of an improved learning target, thus exploring unlabeled data more efficiently with the awareness of semantic context. We have extensively evaluated our method with the state-of-the-art semi-supervised approaches on four public-available datasets, including the ACDC dataset, M&Ms dataset, Pancreas-CT dataset, and Task_07 CT dataset. The experimental results with different labeled-data ratios show our proposed method's superiority over other existing methods, demonstrating its effectiveness in semi-supervised medical image segmentation.

Interval Type-2 Fuzzy Neural Network Based on Active Semi-Supervised Learning for Non-Stationary Industrial Processes

Accurate models ensure the efficient and safe operations of industrial processes. However, modeling of industrial processes with non-stationary characteristic is challenging. In this study, an interval type-2 fuzzy neural network (IT2FNN) based on active semi-supervised learning (ASSL-IT2FNN) is proposed for such industrial processes. First, an IT2FNN with adaptive fuzzy membership function (FMF) and hierarchical learning method is utilized to achieve considerable modeling accuracy and efficiency. Second, to better tackle with the non-stationary characteristic, an unsupervised distribution detection method is proposed to identify the occurrence moments of concept drift actively from the perspective of probability and spatial projection. Then, by an active semi-supervised learning method, the concept drift samples with partial labels are used to build a subnetwork, guaranteeing the performance of the constructed IT2FNN models with incomplete data in non-stationary environments. Finally, the proposed ASSL-IT2FNN is verified by benchmark simulations and real industrial data, and the experimental results demonstrate its outperformance. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —Concept drift is an important indicator of a non-stationary environment. It manifests as changes in the distribution of samples over time, which may reduce the accuracy of models constructed based on historical samples. This study aims to alleviate the impact of concept drift on model performance in non-stationary industrial processes while reduce the cost of sample labeling. By using the adaptive FMF and hierarchical learning method, the designed IT2FNN can effectively deal with the inherent nonlinearity of complicated industrial processes. The samples representing concept drift are actively identified by the Kullback-Leibler (KL) divergence and maximum mean difference (MMD), enabling timely detections of concept drift moments in unsupervised situations. The active semi-supervised learning is used to adapt the constructed model to handle the concept drift, wherein a subnetwork is designed based on concept drift samples and similar historical samples.

MACHINE LEARNING FOR ENHANCED CLASSROOM HOMOGENEITY IN PRIMARY EDUCATION

A homogeneous distribution of students in a class is accepted as a key factor for overall success in primary education. A class of students with similar attributes normally increases academic success. It is also a fact that general academic success might be lower in some classes where students have different intelligence and academic levels. In this study, a class distribution model is proposed by using some data science algorithms over a small number of students’ dataset. With unsupervised and semi supervised learning methods in machine learning and data mining, a group of students is equally distributed to classes, taking into account some criteria. This model divides a group of students into clusters by the considering students’ different qualitative and quantitative characteristics. A draft study is carried out by predicting the effectiveness and efficiency of the presented approaches. In addition, some process elements such as quantitative and qualitative characteristics of a student, data acquisition style, digitalization of attributes, and creating a future prediction are also included in this study. Satisfactory and promising experimental results are received using a set of algorithms over collected datasets for classroom scenarios. As expected, a clear and concrete evaluation between balanced and unbalanced class distributions cannot be performed since these two scenarios for the class distributions cannot be applicable at the same time.

ConvNeXt-MHC: improving MHC-peptide affinity prediction by structure-derived degenerate coding and the ConvNeXt model.

Peptide binding to major histocompatibility complex (MHC) proteins plays a critical role in T-cell recognition and the specificity of the immune response. Experimental validation such peptides is extremely resource-intensive. As a result, accurate computational prediction of binding peptides is highly important, particularly in the context of cancer immunotherapy applications, such as the identification of neoantigens. In recent years, there is a significant need to continually improve the existing prediction methods to meet the demands of this field. We developed ConvNeXt-MHC, a method for predicting MHC-I-peptide binding affinity. It introduces a degenerate encoding approach to enhance well-established panspecific methods and integrates transfer learning and semi-supervised learning methods into the cutting-edge deep learning framework ConvNeXt. Comprehensive benchmark results demonstrate that ConvNeXt-MHC outperforms state-of-the-art methods in terms of accuracy. We expect that ConvNeXt-MHC will help us foster new discoveries in the field of immunoinformatics in the distant future. We constructed a user-friendly website at http://www.combio-lezhang.online/predict/, where users can access our data and application.

Effects of non-landslide sampling strategies on machine learning models in landslide susceptibility mapping

This study aims to explore the effects of different non-landslide sampling strategies on machine learning models in landslide susceptibility mapping. Non-landslide samples are inherently uncertain, and the selection of non-landslide samples may suffer from issues such as noisy or insufficient regional representations, which can affect the accuracy of the results. In this study, a positive-unlabeled (PU) bagging semi-supervised learning method was introduced for non-landslide sample selection. In addition, buffer control sampling (BCS) and K-means (KM) clustering were applied for comparative analysis. Based on landslide data from Qiaojia County, Yunnan Province, China, collected in 2014, three machine learning models, namely, random forest, support vector machine, and CatBoost, were used for landslide susceptibility mapping. The results show that the quality of samples selected using different non-landslide sampling strategies varies significantly. Overall, the quality of non-landslide samples selected using the PU bagging method is superior, and this method performs best when combined with CatBoost for predicting (AUC = 0.897) landslides in very high and high susceptibility zones (82.14%). Additionally, the KM results indicated overfitting, displaying high accuracy for validation but poor statistical outcomes for zoning. The BCS results were the worst.

FedDUS: Lung tumor segmentation on CT images through federated semi-supervised with dynamic update strategy

Background and ObjectiveLung tumor annotation is a key upstream task for further diagnosis and prognosis. Although deep learning techniques have promoted automation of lung tumor segmentation, there remain challenges impeding its application in clinical practice, such as a lack of prior annotation for model training and data-sharing among centers. MethodsIn this paper, we use data from six centers to design a novel federated semi-supervised learning (FSSL) framework with dynamic model aggregation and improve segmentation performance for lung tumors. To be specific, we propose a dynamically updated algorithm to deal with model parameter aggregation in FSSL, which takes advantage of both the quality and quantity of client data. Moreover, to increase the accessibility of data in the federated learning (FL) network, we explore the FAIR data principle while the previous federated methods never involve. ResultThe experimental results show that the segmentation performance of our model in six centers is 0.9348, 0.8436, 0.8328, 0.7776, 0.8870 and 0.8460 respectively, which is superior to traditional deep learning methods and recent federated semi-supervised learning methods. ConclusionThe experimental results demonstrate that our method is superior to the existing FSSL methods. In addition, our proposed dynamic update strategy effectively utilizes the quality and quantity information of client data and shows efficiency in lung tumor segmentation. The source code is released on (https://github.com/GDPHMediaLab/FedDUS).

RoPDA: Robust Prompt-Based Data Augmentation for Low-Resource Named Entity Recognition

Data augmentation has been widely used in low-resource NER tasks to tackle the problem of data sparsity. However, previous data augmentation methods have the disadvantages of disrupted syntactic structures, token-label mismatch, and requirement for external knowledge or manual effort. To address these issues, we propose Robust Prompt-based Data Augmentation (RoPDA) for low-resource NER. Based on pre-trained language models (PLMs) with continuous prompt, RoPDA performs entity augmentation and context augmentation through five fundamental augmentation operations to generate label-flipping and label-preserving examples. To optimize the utilization of the augmented samples, we present two techniques: self-consistency filtering and mixup. The former effectively eliminates low-quality samples with a bidirectional mask, while the latter prevents performance degradation arising from the direct utilization of labelflipping samples. Extensive experiments on three popular benchmarks from different domains demonstrate that RoPDA significantly improves upon strong baselines, and also outperforms state-of-the-art semi-supervised learning methods when unlabeled data is included.