Performance Of Semi-supervised Learning Research Articles

Semi-supervised learning methods for weed detection in turf.

Accurate weed detection is a prerequisite for precise automatic precision herbicide application. Previous research has adopted the laborious and time-consuming approach of manually labeling and processing large image data sets to develop deep neural networks for weed detection. This research introduces a novel semi-supervised learning (SSL) approach for detecting weeds in turf. The performance of SSL was compared with that of ResNet50, a fully supervised learning (FSL) method, in detecting and differentiating sub-images containing weeds from those containing only turfgrass. Compared with ResNet50, the evaluated SSL methods, Π-model, Mean Teacher, and FixMatch, increased the classification accuracy by 2.8%, 0.7%, and 3.9%, respectively, when only 100 labeled images per class were utilized. FixMatch was the most efficient and reliable model, as it exhibited higher accuracy (≥0.9530) and F1 scores (≥0.951) with fewer labeled data (50 per class) in the validation and testing data sets than the other neural networks evaluated. These results reveal that the SSL deep neural networks are capable of being highly accurate while requiring fewer labeled training images, thus being more time- and labor-efficient than the FSL method. © 2024 Society of Chemical Industry.

Dynamic Weighted Adversarial Learning for Semi-Supervised Classification under Intersectional Class Mismatch

Nowadays, class-mismatch problem has drawn intensive attention in Semi-Supervised Learning (SSL), where the classes of labeled data are assumed to be only a subset of the classes of unlabeled data. However, in a more realistic scenario, the labeled data and unlabeled data often share some common classes while they also have their individual classes, which leads to an “intersectional class-mismatch” problem. As a result, existing SSL methods are often confused by these individual classes and suffer from performance degradation. To address this problem, we propose a novel Dynamic Weighted Adversarial Learning (DWAL) framework to properly utilize unlabeled data for boosting the SSL performance. Specifically, to handle the influence of the individual classes in unlabeled data (i.e., Out-Of-Distribution classes), we propose an enhanced adversarial domain adaptation to dynamically assign weight for each unlabeled example from the perspectives of domain adaptation and a class-wise weighting mechanism, which consists of transferability score and prediction confidence value. Besides, to handle the influence of the individual classes in labeled data (i.e., private classes), we propose a dissimilarity maximization strategy to suppress the inaccurate correlations caused by the examples of individual classes within labeled data. Therefore, our DWAL can properly make use of unlabeled data to acquire an accurate SSL classifier under intersectional class-mismatch setting, and extensive experimental results on five public datasets demonstrate the effectiveness of the proposed model over other state-of-the-art SSL methods.

Generalization of vision pre-trained models for histopathology

Out-of-distribution (OOD) generalization, especially for medical setups, is a key challenge in modern machine learning which has only recently received much attention. We investigate how different convolutional pre-trained models perform on OOD test data—that is data from domains that have not been seen during training—on histopathology repositories attributed to different trial sites. Different trial site repositories, pre-trained models, and image transformations are examined as specific aspects of pre-trained models. A comparison is also performed among models trained entirely from scratch (i.e., without pre-training) and models already pre-trained. The OOD performance of pre-trained models on natural images, i.e., (1) vanilla pre-trained ImageNet, (2) semi-supervised learning (SSL), and (3) semi-weakly-supervised learning (SWSL) models pre-trained on IG-1B-Targeted are examined in this study. In addition, the performance of a histopathology model (i.e., KimiaNet) trained on the most comprehensive histopathology dataset, i.e., TCGA, has also been studied. Although the performance of SSL and SWSL pre-trained models are conducive to better OOD performance in comparison to the vanilla ImageNet pre-trained model, the histopathology pre-trained model is still the best in overall. In terms of top-1 accuracy, we demonstrate that diversifying the images in the training using reasonable image transformations is effective to avoid learning shortcuts when the distribution shift is significant. In addition, XAI techniques—which aim to achieve high-quality human-understandable explanations of AI decisions—are leveraged for further investigations.

Graph-Based Self-Training for Semi-Supervised Deep Similarity Learning.

Semi-supervised learning is a learning pattern that can utilize labeled data and unlabeled data to train deep neural networks. In semi-supervised learning methods, self-training-based methods do not depend on a data augmentation strategy and have better generalization ability. However, their performance is limited by the accuracy of predicted pseudo-labels. In this paper, we propose to reduce the noise in the pseudo-labels from two aspects: the accuracy of predictions and the confidence of the predictions. For the first aspect, we propose a similarity graph structure learning (SGSL) model that considers the correlation between unlabeled and labeled samples, which facilitates the learning of more discriminative features and, thus, obtains more accurate predictions. For the second aspect, we propose an uncertainty-based graph convolutional network (UGCN), which can aggregate similar features based on the learned graph structure in the training phase, making the features more discriminative. It can also output the uncertainty of predictions in the pseudo-label generation phase, generating pseudo-labels only for unlabeled samples with low uncertainty; thus, reducing the noise in the pseudo-labels. Further, a positive and negative self-training framework is proposed, which combines the proposed SGSL model and UGCN into the self-training framework for end-to-end training. In addition, in order to introduce more supervised signals in the self-training process, negative pseudo-labels are generated for unlabeled samples with low prediction confidence, and then the positive and negative pseudo-labeled samples are trained together with a small number of labeled samples to improve the performance of semi-supervised learning. The code is available upon request.

A self-training subspace clustering algorithm based on adaptive confidence for gene expression data.

Gene clustering is one of the important techniques to identify co-expressed gene groups from gene expression data, which provides a powerful tool for investigating functional relationships of genes in biological process. Self-training is a kind of important semi-supervised learning method and has exhibited good performance on gene clustering problem. However, the self-training process inevitably suffers from mislabeling, the accumulation of which will lead to the degradation of semi-supervised learning performance of gene expression data. To solve the problem, this paper proposes a self-training subspace clustering algorithm based on adaptive confidence for gene expression data (SSCAC), which combines the low-rank representation of gene expression data and adaptive adjustment of label confidence to better guide the partition of unlabeled data. The superiority of the proposed SSCAC algorithm is mainly reflected in the following aspects. 1) In order to improve the discriminative property of gene expression data, the low-rank representation with distance penalty is used to mine the potential subspace structure of data. 2) Considering the problem of mislabeling in self-training, a semi-supervised clustering objective function with label confidence is proposed, and a self-training subspace clustering framework is constructed on this basis. 3) In order to mitigate the negative impact of mislabeled data, an adaptive adjustment strategy based on gravitational search algorithm is proposed for label confidence. Compared with a variety of state-of-the-art unsupervised and semi-supervised learning algorithms, the SSCAC algorithm has demonstrated its superiority through extensive experiments on two benchmark gene expression datasets.

The High Separation Probability Assumption for Semi-Supervised Learning

We propose a novel semi-supervised learning (SSL) assumption—the high separation probability (HSP) assumption—that is complementary to the common low density separation (LDS) assumption, and a highly accurate and efficient SSL algorithm, the transductive minimax probability machine (TMPM). Under the HSP assumption, a preferred classifier should ensure that with high probability the samples are far away from the decision boundary. Similarly, the LDS assumption states that the decision boundary should lie in a low-density region. A remarkable property of HSP is that it provides an elegant theoretical framework for analyzing the generalization performance of SSL, while it remains difficult for the LDS assumption to have a quantitative criterion on how it fits the data. Moreover, it has the same paradigm as the minimax probability machine (MPM) and yields an intractable mixed-integer optimization for solving the transductive label assignment problem. Consequently, we propose TMPM with a label-switching strategy and iterative manner to formalize our approximate solution. Theoretically, we analyze the computational complexity, convergence property, and generalization error bound of TMPM. Empirically, we show that TMPM achieves competitive performance on a wide range of datasets, while being almost one order of magnitude faster than existing SSL approaches.

Semi-Supervised Clustering via Cannot Link Relationship for Multiview Data

Due to the diversity of data modalities, the research interest of multi-view clustering is gradually increasing, in the field of large-data analytics, particularly in clustering. However, the greater part of current multi-view clustering methods is mainly in view of unsupervised learning, which leads to unpredictable results and algorithmic instability. Besides, they ignore the diversity of graphs, which is not desirable in practical applications, because the characteristic properties of each view are different. To solve these problems, inspired by the outstanding performance of semi-supervised learning in machine learning, we propose a valid semi-supervised multi-view spectral clustering algorithm. We use the pre-set labels as prior knowledge to obtain the overall distribution of the remaining unlabeled data. Tensor minimization Schatten <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$p$ </tex-math></inline-formula> -norm is utilized to mine the mutual information hidden in multiple views. Meanwhile, we also use the cannot-link as another semi-supervised constraint to update the graph. Our proposed algorithm is generally 5%-10% better than the comparison algorithms in view of the experimental results on five datasets, and our algorithm is relatively fast with the computational complexity of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathcal {O}({T({n^{2}}\log (n) + {n^{2}} + {u^{2}}l + ulc + uc\log (c))})$ </tex-math></inline-formula> , where <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$T$ </tex-math></inline-formula> denotes the number of iterations and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$n$ </tex-math></inline-formula> , <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$l$ </tex-math></inline-formula> , <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$u$ </tex-math></inline-formula> represent the number of samples, the number of labeled and unlabeled samples, respectively, which shows that our proposed method has broad application prospects.

Semi-Supervised Skin Lesion Segmentation via Iterative Mask Optimization

Deep learning-based skin lesion segmentation methods have achieved promising results in the community. However, they are usually based on fully supervised learning and require many high-quality ground truths. Labeling the ground truths takes a lot of labor, material, and financial resources. We propose a novel semi-supervised skin lesion segmentation method to solve this problem. First, a hierarchical image segmentation algorithm is used to generate optimal segmentation maps. Then, fully supervised training is performed on a small part of the images with ground truths. The resulting pseudo masks are generated to train the rest of the images. The optimal segmentation maps are utilized in this process to refine the pseudo masks. Experiments show that the proposed method can improve the performance of semi-supervised learning for skin lesion segmentation by reducing the gap with fully supervised learning methods. Moreover, it can reduce the workload of labeling the ground truths. Extensive experiments are conducted on the open dataset to validate the efficiency of the proposed method. The results show that our method is competitive in improving the quality of semi-supervised segmentation.

Supervised and semi-supervised 3D organ localisation in CT images combining reinforcement learning with imitation learning

Computer aided diagnostics often requires analysis of a region of interest (ROI) within a radiology scan, and the ROI may be an organ or a suborgan. Although deep learning algorithms have the ability to outperform other methods, they rely on the availability of a large amount of annotated data. Motivated by the need to address this limitation, an approach to localisation and detection of multiple organs based on supervised and semi-supervised learning is presented here. It draws upon previous work by the authors on localising the thoracic and lumbar spine region in CT images. The method generates six bounding boxes of organs of interest, which are then fused to a single bounding box. The results of experiments on localisation of the Spleen, Left and Right Kidneys in CT Images using supervised and semi supervised learning (SSL) demonstrate the ability to address data limitations with a much smaller data set and fewer annotations, compared to other state-of-the-art methods. The SSL performance was evaluated using three different mixes of labelled and unlabelled data (i.e. 30:70,35:65,40:60) for each of lumbar spine, spleen left and right kidneys respectively. The results indicate that SSL provides a workable alternative especially in medical imaging where it is difficult to obtain annotated data.

A safe semi-supervised graph convolution network.

In the semi-supervised learning field, Graph Convolution Network (GCN), as a variant model of GNN, has achieved promising results for non-Euclidean data by introducing convolution into GNN. However, GCN and its variant models fail to safely use the information of risk unlabeled data, which will degrade the performance of semi-supervised learning. Therefore, we propose a Safe GCN framework (Safe-GCN) to improve the learning performance. In the Safe-GCN, we design an iterative process to label the unlabeled data. In each iteration, a GCN and its supervised version (S-GCN) are learned to find the unlabeled data with high confidence. The high-confidence unlabeled data and their pseudo labels are then added to the label set. Finally, both added unlabeled data and labeled ones are used to train a S-GCN which can achieve the safe exploration of the risk unlabeled data and enable safe use of large numbers of unlabeled data. The performance of Safe-GCN is evaluated on three well-known citation network datasets and the obtained results demonstrate the effectiveness of the proposed framework over several graph-based semi-supervised learning methods.

Transfer of semi-supervised broad learning system in electroencephalography signal classification

Electroencephalography (EEG) signal classification is a crucial part in motor imagery brain–computer interface (BCI) system. Traditional supervised learning methods have performed well pleasing in EEG classification. Unfortunately, the unlabeled samples are easier to collect than labeled samples. In addition, recent studies have shown that it may degenerate performance of semi-supervised learning by exploiting unlabeled samples without selection. To address these issues, a novel semi-supervised broad learning system with transfer learning (TSS-BLS) is proposed in this paper. First, the pseudo-labels of unlabeled samples are obtained using the joint distribution adaptation algorithm. TSS-BLS is then constructed by an improved manifold regularization framework containing both labeled and pseudo-label information. Finally, the effectiveness of the proposed TSS-BLS is evaluated on three BCI competition datasets and four benchmark datasets from UCI repository and compared with seven state-of-the-art algorithms, including ELM, SS-ELM, HELM, SVM, LapSVM, BLS and GSS-BLS. Experimental results show that the performance of TSS-BLS is superior to BLS and GSS-BLS on average. It is thereby shown that TSS-BLS is safe and efficient for EEG classification.

Semi-supervised GAN-based Radiomics Model for Data Augmentation in Breast Ultrasound Mass Classification

Background and ObjectiveThe capability of deep learning radiomics (DLR) to extract high-level medical imaging features has promoted the use of computer-aided diagnosis of breast mass detected on ultrasound. Recently, generative adversarial network (GAN) has aided in tackling a general issue in DLR, i.e., obtaining a sufficient number of medical images. However, GAN methods require a pair of input and labeled images, which require an exhaustive human annotation process that is very time-consuming. The aim of this paper is to develop a radiomics model based on a semi-supervised GAN method to perform data augmentation in breast ultrasound images. MethodsA total of 1447 ultrasound images, including 767 benign masses and 680 malignant masses were acquired from a tertiary hospital. A semi-supervised GAN model was developed to augment the breast ultrasound images. The synthesized images were subsequently used to classify breast masses using a convolutional neural network (CNN). The model was validated using a 5-fold cross-validation method. ResultsThe proposed GAN architecture generated high-quality breast ultrasound images, verified by two experienced radiologists. The improved performance of semi-supervised learning increased the quality of the synthetic data produced in comparison to the baseline method. We achieved more accurate breast mass classification results (accuracy 90.41%, sensitivity 87.94%, specificity 85.86%) with our synthetic data augmentation compared to other state-of-the-art methods. ConclusionThe proposed radiomics model has demonstrated a promising potential to synthesize and classify breast masses on ultrasound in a semi-supervised manner.

Classifying Medical Document in Bahasa Indonesia using Semi-Supervised Learning

The medical domain has always been an all-time important domain since healthiness is everyone’s purpose. People find medical document resources in the sea of data and information, such as the web. To support information retrieval and knowledge dissemination through the web, we analyze the use of semi-supervised learning to classify medical-related documents. The semi-supervised learning technique is chosen to show the possibilities of creating good classifiers with limited human supervision. In this research, we use the Naïve Bayes and Pseudo Labeling technique. We analyze different labeled:unlabeled data ratios of the training dataset in the experiment, starting from 4:3, 3:4, 2:5, and 1:6, to see the semi-supervised learning performance with different levels of human supervision. We get a relatively similar result in terms of classification average accuracy (81%-83%). Interestingly, in one experiment, the highest accuracy of the 1:6 ratio (85%) outperforms the 2:5 ratio (82%) and has the same accuracy as the 4:3 (85%). However, the standard deviation of the accuracy in the 1:6 ratio is the highest, amongst others (4.183). Finally, semi-supervised learning can be used to create a great classifier model of the medical domain in Bahasa Indonesia with less human supervision.

Multi-class motor imagery EEG classification using collaborative representation-based semi-supervised extreme learning machine

Both labeled and unlabeled data have been widely used in electroencephalographic (EEG)-based brain-computer interface (BCI). However, labeled EEG samples are generally scarce and expensive to collect, while unlabeled samples are considered to be abundant in real applications. Although the semi-supervised learning (SSL) allows us to utilize both labeled and unlabeled data to improve the classification performance as against supervised algorithms, it has been reported that unlabeled data occasionally undermine the performance of SSL in some cases. To overcome this challenge, we propose a collaborative representation-based semi-supervised extreme learning machine (CR-SSELM) algorithm to evaluate the risk of unlabeled samples by a new safety-control mechanism. Specifically, the ELM model is firstly used to predict unlabeled samples and then the collaborative representation (CR) approach is employed to reconstruct the unlabeled samples according to the obtained prediction results, from which the risk degree of unlabeled sample is defined. A risk-based regularization term is then constructed accordingly and embedded into the objective function of the SS-ELM. Experiments conducted on benchmark and EEG datasets demonstrate that the proposed method outperforms the ELM and SS-ELM algorithm. Moreover, the proposed CR-SSELM even offers the best performance while SS-ELM yields worse performance compared with its supervised counterpart (ELM). Graphical abstract This paper proposes a collaborative representation-based semi-supervised extreme learning machine (CR-SSELM) algorithm to evaluate the risk of unlabeled samples by a new safety-control mechanism. It is aim to solve the safety problem of SS-ELM method that SS-ELM yields worse performance than ELM. With the help of safety mechanism, the performance of our method is still better than supervised ELM method.

Snowball: Iterative Model Evolution and Confident Sample Discovery for Semi-Supervised Learning on Very Small Labeled Datasets

In this work, we develop a joint sample discovery and iterative model evolution method for semi-supervised learning on very small labeled training sets. We propose a master-teacher-student model framework to provide multi-layer guidance during the model evolution process with multiple iterations and generations. The teacher model is constructed by performing an exponential moving average of the student models obtained from past training steps. The master network combines the knowledge of the student and teacher models with additional access to newly discovered samples. The master and teacher models are then used to guide the training of the student network by enforcing the consistence between their predictions of unlabeled samples and evolve all models when more and more samples are discovered. Our extensive experiments demonstrate that the discovering confident samples from the unlabeled dataset, once coupled with the above master-teacher-student network evolution, can significantly improve the overall semi-supervised learning performance. For example, on the CIFAR-10 dataset, with a very small set of 250 labeled samples, our method achieves an error rate of 11.81 %, more than 38 % lower than the state-of-the-art method Mean-Teacher (49.91 %).

Collaborative Graph Convolutional Networks: Unsupervised Learning Meets Semi-Supervised Learning

Graph convolutional networks (GCN) have achieved promising performance in attributed graph clustering and semi-supervised node classification because it is capable of modeling complex graphical structure, and jointly learning both features and relations of nodes. Inspired by the success of unsupervised learning in the training of deep models, we wonder whether graph-based unsupervised learning can collaboratively boost the performance of semi-supervised learning. In this paper, we propose a multi-task graph learning model, called collaborative graph convolutional networks (CGCN). CGCN is composed of an attributed graph clustering network and a semi-supervised node classification network. As Gaussian mixture models can effectively discover the inherent complex data distributions, a new end to end attributed graph clustering network is designed by combining variational graph auto-encoder with Gaussian mixture models (GMM-VGAE) rather than the classic k-means. If the pseudo-label of an unlabeled sample assigned by GMM-VGAE is consistent with the prediction of the semi-supervised GCN, it is selected to further boost the performance of semi-supervised learning with the help of the pseudo-labels. Extensive experiments on benchmark graph datasets validate the superiority of our proposed GMM-VGAE compared with the state-of-the-art attributed graph clustering networks. The performance of node classification is greatly improved by our proposed CGCN, which verifies graph-based unsupervised learning can be well exploited to enhance the performance of semi-supervised learning.

A Safe Semi-supervised Classification Algorithm Using Multiple Classifiers Ensemble

In order to improve the performance of semi-supervised learning, a safe semi-supervised classification algorithm using multiple classifiers ensemble (S3C-MC) is proposed. First, unlabeled samples are filtered and unlabeled samples with small ambiguity are selected for semi-supervised learning. Next, the labeled training set is sampled to multiple subsets and they generate multiple classifiers to predict the filtered unlabeled sample respectively. The predicted label is formed by multiple classifiers with weighted voting mechanism, and the weight of classifier is changing constantly according to the correctness of the prediction for unlabeled samples by classifier. Then, security verification is carried out to ensure that the classifier evolves in the direction of error reduction when the new sample is added. Only the label making classifiers error lower and having the same predictive value with the three classifiers in security verification is added into the labeled set to expand the number of labeled sets. Finally, the algorithm iterates until the unlabeled sample set is empty. The experiment is carried out on the UCI data set and the result shows that the proposed S3C-MC has good safety and has a higher classification rate.

Impact of Fuzziness Measures on the Performance of Semi-supervised Learning

Usage of fuzziness in the study of semi-supervised learning is relatively new. In this study, the divide-and-conquer strategy is used to investigate the performance of semi-supervised learning. To this end, testing dataset is divided into three categories, namely low, medium and high-fuzzy samples based on the magnitude of fuzziness of each sample. It is experimentally confirmed that if the low-fuzzy samples are added from the testing dataset to the original training dataset and the model is retrained, then the accuracy can be improved. To measure the amount of fuzziness of each sample, four different fuzziness measuring models are used in this study. Experimental results support that improvement of accuracy is dependent on which fuzziness measuring model is used to measure the fuzziness of each sample. Wilcoxon signed-rank test shows that choosing a specific fuzziness measuring model is significant or not. Finally, from the Wilcoxon signed-rank test, the best model is chosen, which can be used along with semi-supervised learning to improve its performance.

Adaptive Safe Semi-Supervised Extreme Machine Learning

Semi-supervised learning (SSL) based on manifold regularization (MR) is an excellent learning framework. However, the performance of SSL heavily depends on the construction of manifold graph and the safety degrees of unlabeled samples. Due to the construction of manifold graph and safety degrees of unlabeled samples are usually pre-construct before classification and fixed during the classification learning process, which results independent with the subsequent classification. Aiming at the above problems, we propose a unified adaptive safe semi-supervised learning (Adap-SaSSL) framework. This framework adaptively constructs a manifold graph while adaptively calculating the safety degrees of unlabeled samples. Specifically, the weights of manifold graph and its parameters, as well as the safety degrees of unlabeled samples will be optimized during the learning process rather than being calculated in advance. Finally, we then develop and implement a adaptive safe classification method based on the Adap-SaSSL framework, which is called adaptive safe semi-supervised extreme learning machine (AdSafe-SSELM). Experimental results on artificial, benchmark and image datasets show that the performance of AdSafe-SSELM is effective and reliable compared to other algorithms.

GAN-Based Semi-Supervised Learning Approach for Clinical Decision Support in Health-IoT Platform

With the development of the Internet of Things (IoT) technology, its application in the medical field becomes more and more extensive. However, with a dramatic increase in medical data obtained from the IoT-based health service system, labeling a large number of medical data requires high cost and relevant domain knowledge. Therefore, how to use a small number of labeled medical data reasonably to build an efficient and high-quality clinical decision support model in the IoT-based platform has been an urgent research topic. In this paper, we propose a novel semi-supervised learning approach in association with generative adversarial networks (GANs) for supporting clinical decision making in the IoT-based health service system. In our approach, GAN is adopted to not only increase the number of labeled data but also to compensate the imbalanced labeled classes with additional artificial data in order to improve the semi-supervised learning performance. Extensive evaluations on a collection of benchmarks and real-world medical datasets show that the proposed technique outperforms the others and provides a potential solution for practical applications.