Semi-supervised Learning Model Research Articles

Evidential Pseudo-Label Ensemble for semi-supervised classification

Semi-supervised learning (SSL) focuses on effectively exploiting both a limited number of labeled data and relatively rich unlabeled data. Recent SSL researches have achieved great progress with pseudo-labeling. However, incorrect pseudo-label may cause severe error propagation, which has been widely recognized and termed as confirmation bias. Furthermore, vanilla ensemble strategies aiming at alleviating this issue will lead to serious conservative labeling due to the averaging property. Accordingly, they could not sufficiently explore the unlabeled data. In this paper, we proposed a novel SSL method termed Evidential Pseudo-Label Ensemble (EPLE) which aims to generate more accurate pseudo-labels with evidence support. Specifically, multiple networks with different augmented strategies are introduced to generate complementary evidence for unlabeled samples, and induce confident predictions with Dempster–Shafer theory (DST), which effectively addresses error propagation and confirmation bias. To enhance the consistency of predictions for the same samples with different noises, we employ multiple augmentation strategies for both weak and strong augmentations. We conduct experiments on standard semi-supervised learning datasets, including CIFAR-10/100, SVHN and STL-10. Extensive experiments show the superior performance of our proposed EPLE compared to state-of-the-art SSL models.

A data extension framework of seismic-induced gravelly soil liquefaction based on semi-supervised methods

In studies of gravelly soil liquefaction discrimination, the number of training samples significantly affects model performance. However, historical earthquakes provide limited instances of gravelly soil liquefaction, resulting in no guarantee of the generalization performance of existing models. Additionally, valuable data from liquefiable critical layer points and unlabeled points outside the critical layer are often overlooked in stratigraphic exploration, despite their high value. Therefore, in this paper, a data extension method for semi-supervised learning based on the Markov chain Monte Carlo–Bayesian network (MCMC-BN) method is proposed and a semi-MCMC-BN gravelly soil liquefaction discrimination model is constructed. By effectively mining the available site liquefaction information, the proposed method increases the sample size of the gravelly soil liquefaction dataset by approximately 4 times, expanding 120 initial liquefaction samples to 476. The results demonstrate that the extended liquefaction database improves the distribution of liquefaction data for gravelly soils. The semi-supervised learning model outperforms the supervised learning model, highlighting the feasibility of the semi-supervised approach for gravelly soil liquefaction discrimination. In addition, debris flow data for the Wenchuan area, China, are used to validate the effectiveness of the proposed method. Finally, the impacts of the confidence level magnitude, critical soil layer selection, and different machine learning methods on the data extension outcomes are discussed. The expanded database can provide data support to studying gravelly soil liquefaction, and the proposed data expansion framework can be used for related research in other fields.

Prediction of heavy-oil combustion emissions with a semi-supervised learning model considering variable operation conditions

Accurate and reliable prediction of combustion emissions is essential for combustion optimization adjustment. Existing data-driven approaches are limited by insufficient labeled data and low robustness, resulting in low prediction accuracy. To address these limitations, a semi-supervised learning model consisting of adversarial denoising autoencoder and Gaussian process regression is proposed for combustion emissions prediction. The unsupervised adversarial denoising autoencoder is applied for feature extraction of the flame image, and the supervised Gaussian process regression is utilized for feature recognition to estimate the CO2 and NOx emissions concentrations. Especially, a structural similarity-based loss function is developed to improve the adversarial denoising autoencoder training efficiency. During the experiments, the heavy-oil flame images under variable operation conditions are captured to verify the performance of the semi-supervised learning model. Results indicate that the model provides accurate emissions prediction with a prediction time of 61.38 ms/f (milliseconds per frame), where the prediction accuracy for the CO2 and NOx emissions are R2=0.97 and R2=0.98, respectively. The confidence intervals generated by the model cover the actual observations and confirm the reliability of the predictions.

A Semi-Supervised Learning Model for Predicting User Attributes Based on Ladder Network

A Semi-Supervised Learning Model for Predicting User Attributes Based on Ladder Network

Social relation-driven consensus reaching in large-scale group decision-making using semi-supervised classification

The fundamental goal of group decision making (GDM) is to improve consensus amongst experts and reduce individual conflicts of interest in the process of alternative selection. By analysing the social network relationships of decision-makers (DMs), such as trust and preference similarity relationships, more effective consensus-reaching mechanisms can be designed. Previous studies developed supervised classification algorithms for DM groups of more than 1000 participants on social networks. However, in large-scale social networks, the social connection relations of DMs cannot be completely investigated as human resources and costs during the decision-making process impose limitations. These DMs cannot be effectively trained to build a classification model, which is referred to as unlabelled DMs and results in a large amount of information being ignored during the model training process, leading to an unfair process that can deepen the conflict amongst experts. To address the information loss problem, this study proposes a semi-supervised learning model for social networks with incomplete trust relations, aiming to effectively absorb the preference information of unlabelled DMs into the classification model, thereby improving the effectiveness of classification management. Specifically, the cost-sensitive semi-supervised support vector machine (SVM) introduced in the inner product space is subsequently used by the DMs' classification model to accurately divide unlabelled DMs. A minimum cost adjustment consensus model is then constructed based on the DM subgroups. Numerical experiments on urban renewal were conducted to verify the effectiveness of the proposed method. The empirical and simulation results demonstrated that the proposed method can decrease total consensus costs for social-network GDM with missing trust-relationship information.

Development and validation of a semi-supervised deep learning model for automatic retinopathy of prematurity staging

Retinopathy of prematurity (ROP) is currently one of the leading causes of infant blindness worldwide. Recently significant progress has been made in deep learning-based computer-aided diagnostic methods. However, deep learning often requires a large amount of annotated data for model optimization, but this requires long hours of effort by experienced doctors in clinical scenarios. In contrast, a large number of unlabeled images are relatively easy to obtain. In this paper, we propose a new semi-supervised learning framework to reduce annotation costs for automatic ROP staging. We design two consistency regularization strategies, prediction consistency loss and semantic structure consistency loss, which can help the model mine useful discriminative information from unlabeled data, thus improving the generalization performance of the classification model. Extensive experiments on a real clinical dataset show that the proposed method promises to greatly reduce the labeling requirements in clinical scenarios while achieving good classification performance.

Urban surface classification using semi-supervised domain adaptive deep learning models and its application in urban environment studies.

High-resolution urban surface information, e.g., the fraction of impervious/pervious surface, is pivotal in studies of local thermal/wind environments and air pollution. In this study, we introduced and validated a domain adaptive land cover classification model, to automatically classify Google Earth images into pixel-based land cover maps. By combining domain adaptation (DA) and semi-supervised learning (SSL) techniques, our model demonstrates its effectiveness even when trained with a limited dataset derived from Gaofen2 (GF2) satellite images. The model's overall accuracy on the translated GF2 dataset improved significantly from 19.5% to 75.2%, and on the Google Earth image dataset from 23.1% to 61.5%. The overall accuracy is 2.9% and 3.4% higher than when using only DA. Furthermore, with this model, we derived land cover maps and investigated the impact of land surface composition on the local meteorological parameters and air pollutant concentrations in the three most developed urban agglomerations in China, i.e., Beijing, Shanghai and the Great Bay Area (GBA). Our correlation analysis reveals that air temperature exhibits a strong positive correlation with neighboring artificial impervious surfaces, with Pearson correlation coefficients higher than 0.6 in all areas except during the spring in the GBA. However, the correlation between air pollutants and land surface composition is notably weaker and more variable. The primary contribution of this paper is to provide an efficient method for urban land cover extraction which will be of great value for assessing the urban surface composition, quantifying the impact of land use/land cover, and facilitating the development of informed policies.

Computational screening of 2D anode materials with robust thermal and electrical properties for lithium-ion batteries

Two-dimensional materials AB2-type and AB-type materials not only have robust electrical properties, but also have robust thermal stability, which can reduce the risk of thermal runaway of the battery. While experimental and theoretical calculations to explore the entire phase space are both very time-consuming, using machine learning combining with first principle calculations, a new material TiF that has high electronic conductivity (bandgap = 0 eV), low energy barrier (0.28 eV), high capacity (400.82 mA h/g), high thermal stability and moderate open circuit voltage (2.21 V) is screened out from 1913 materials. TiF possesses great electronic properties to be used as anode materials and its stability at high temperatures offers the feasibility of reducing the risk of thermal runaway. Besides TiF, the other 12 2D materials obtained from the screening have been used in experiments as electrodes. In addition, the screening method combining first principle calculations with new Positive and Negative Semi-supervised (PNS) machine learning model can greatly reduce the screening time and under limited samples, classification performance increases on iteration, which benefits us to complete the phase space exploration of two-dimensional AB2-type and AB-type materials.

MTAN: A semi-supervised learning model for kidney tumor segmentation.

Medical image segmentation is crucial in disease diagnosis and treatment planning. Deep learning (DL) techniques have shown promise. However, optimizing DL models requires setting numerous parameters, and demands substantial labeled datasets, which are labor-intensive to create. This study proposes a semi-supervised model that can utilize labeled and unlabeled data to accurately segment kidneys, tumors, and cysts on CT images, even with limited labeled samples. An end-to-end semi-supervised learning model named MTAN (Mean Teacher Attention N-Net) is designed to segment kidneys, tumors, and cysts on CT images. The MTAN model is built on the foundation of the AN-Net architecture, functioning dually as teachers and students. In its student role, AN-Net learns conventionally. In its teacher role, it generates objects and instructs the student model on their utilization to enhance learning quality. The semi-supervised nature of MTAN allows it to effectively utilize unlabeled data for training, thus improving performance and reducing overfitting. We evaluate the proposed model using two CT image datasets (KiTS19 and KiTS21). In the KiTS19 dataset, MTAN achieved segmentation results with an average Dice score of 0.975 for kidneys and 0.869 for tumors, respectively. Moreover, on the KiTS21 dataset, MTAN demonstrates its robustness, yielding average Dice scores of 0.977 for kidneys, 0.886 for masses, 0.861 for tumors, and 0.759 for cysts, respectively. The proposed MTAN model presents a compelling solution for accurate medical image segmentation, particularly in scenarios where the labeled data is scarce. By effectively utilizing the unlabeled data through a semi-supervised learning approach, MTAN mitigates overfitting concerns and achieves high-quality segmentation results. The consistent performance across two distinct datasets, KiTS19 and KiTS21, underscores model's reliability and potential for clinical reference.

Collaborative Wheat Lodging Segmentation Semi-Supervised Learning Model Based on RSE-BiSeNet Using UAV Imagery

Lodging is a common natural disaster during wheat growth. The accurate identification of wheat lodging is of great significance for early warnings and post-disaster assessment. With the widespread use of unmanned aerial vehicles (UAVs), large-scale wheat lodging monitoring has become very convenient. In particular, semantic segmentation is widely used in the recognition of high-resolution field scene images from UAVs, providing a new technical path for the accurate identification of wheat lodging. However, there are still problems, such as insufficient wheat lodging data, blurred image edge information, and the poor accuracy of small target feature extraction, which limit the recognition of wheat lodging. To this end, the collaborative wheat lodging segmentation semi-supervised learning model based on RSE-BiseNet is proposed in this study. Firstly, ResNet-18 was used in the context path of BiSeNet to replace the original backbone network and introduce squeeze-and-excitation (SE) attention, aiming to enhance the expression ability of wheat lodging characteristics. Secondly, the segmentation effects of the collaborative semi-supervised and fully supervised learning model based on RSE-BiSeNet were compared using the self-built wheat lodging dataset. Finally, the test results of the proposed RSE-BiSeNet model were compared with classic network models such as U-Net, BiseNet, and DeepLabv3+. The experimental results showed that the wheat lodging segmentation model based on RSE-BiSeNet collaborative semi-supervised learning has a good performance. The method proposed in this study can also provide references for remote sensing UAVs, other field crop disaster evaluations, and production assistance.

Research on habitat quality assessment and decision-making based on Semi-supervised Ensemble Learning method—Daxia River Basin, China

Habitat quality is an indicator of the ecological evolution of a region, and evaluating habitat quality utilizing Machine Learning approaches can reflect the ecological status of a region more objectively. Gridded statistical ecological factors in the study region were utilized to build the Ecological Information State Layer (EISL) in the grid space. Calculate the Environmental Quality Index (EQI) of the sampled grid to evaluate the performance index of the models on the sampled grid. The model with the optimal performance index is selected for the task of habitat quality classification in the research region, and then especially decision-making advice is offered by inverting the degree of influence of ecological elements on habitat quality. The results show that: (1) The Semi-supervised Ensemble Learning model (Tri-training) Accuracy, Kappa coefficient, and F1-score are 0.93, 0.89, and 0.92, respectively, as the optimal model for the habitat quality classification task. (2) Based on the results of the ecological categories calculation, the current ecosystem quality of the southwestern section of the Daxia River Basin is maintained as a positive indicator. the ecosystem quality inside the living space centered on the cities of Linxia and Hezuo shows negative indicator changes. (3) The calculation results of the inverse importance of ecological factors show that vegetation index and human population density are the key factors impacting the habitat quality of the Daxia River Basin. In the ecological management of Daxia River Basin, the degree of spatial aggregation of people's settlements should be reduced while preserving and expanding vegetation cover. Using Tri-training's comprehensive analysis of multiple ecological factors, regional habitat quality can be accurately assessed.

Consistency regularisation in varying contexts and feature perturbations for semi-supervised semantic segmentation of histology images

Semantic segmentation of various tissue and nuclei types in histology images is fundamental to many downstream tasks in the area of computational pathology (CPath). In recent years, Deep Learning (DL) methods have been shown to perform well on segmentation tasks but DL methods generally require a large amount of pixel-wise annotated data. Pixel-wise annotation sometimes requires expert’s knowledge and time which is laborious and costly to obtain. In this paper, we present a consistency based semi-supervised learning (SSL) approach that can help mitigate this challenge by exploiting a large amount of unlabelled data for model training thus alleviating the need for a large annotated dataset. However, SSL models might also be susceptible to changing context and features perturbations exhibiting poor generalisation due to the limited training data. We propose an SSL method that learns robust features from both labelled and unlabelled images by enforcing consistency against varying contexts and feature perturbations. The proposed method incorporates context-aware consistency by contrasting pairs of overlapping images in a pixel-wise manner from changing contexts resulting in robust and context invariant features. We show that cross-consistency training makes the encoder features invariant to different perturbations and improves the prediction confidence. Finally, entropy minimisation is employed to further boost the confidence of the final prediction maps from unlabelled data. We conduct an extensive set of experiments on two publicly available large datasets (BCSS and MoNuSeg) and show superior performance compared to the state-of-the-art methods.

RECOGNITION OF ATRIAL FIBRILLATION BASED ON CNN-LSTM AND LAPLACIAN SUPPORT VECTOR MACHINE

Atrial fibrillation (AF) is a common arrhythmia associated with cardiac death and stroke. Therefore, the early detection of AF is of critical importance and the wearable long-term ECG monitoring system is one promising way. In order to assist the cardiologists in identifying potential AF in a tremendous amount of long-term ECG data, this study proposed an automatic detector combining deep learning and semi-supervised learning in view of the difficulty of obtaining a large number of labeled data in clinical practice. Three R-R interval features and two nonlinear features extracted from ECG samples, combined with 16-dimension deep learning features extracted by CNN-LSTM, are put into the semi-supervised machine learning model Laplacian Support Vector Machine (LapSVM) for AF detection. The proposed method showed very promising performance, with an accuracy of 99.63%, a sensitivity of 99.70%, a specificity of 99.57% and an F_score of 99.59% on the AFDB dataset. It still achieved an accuracy of 98% when the proportion of the training set was reduced, and achieved an accuracy of 96% on the SPHD collected clinically. The results show that the proposed method can classify AF and non-AF with a higher accuracy, and has excellent generalization performance in different categories of subjects, which is in line with clinical scenarios. The proposed method is also conducive to solving the clinical cases with little labeled data.

Federated Semi-Supervised Learning for Medical Image Segmentation via Pseudo-Label Denoising.

Distributed big data and digital healthcare technologies have great potential to promote medical services, but challenges arise when it comes to learning predictive model from diverse and complex e-health datasets. Federated Learning (FL), as a collaborative machine learning technique, aims to address the challenges by learning a joint predictive model across multi-site clients, especially for distributed medical institutions or hospitals. However, most existing FL methods assume that clients possess fully labeled data for training, which is often not the case in e-health datasets due to high labeling costs or expertise requirement. Therefore, this work proposes a novel and feasible approach to learn a Federated Semi-Supervised Learning (FSSL) model from distributed medical image domains, where a federated pseudo-labeling strategy for unlabeled clients is developed based on the embedded knowledge learned from labeled clients. This greatly mitigates the annotation deficiency at unlabeled clients and leads to a cost-effective and efficient medical image analysis tool. We demonstrated the effectiveness of our method by achieving significant improvements compared to the state-of-the-art in both fundus image and prostate MRI segmentation tasks, resulting in the highest Dice scores of 89.23% and 91.95% respectively even with only a few labeled clients participating in model training. This reveals the superiority of our method for practical deployment, ultimately facilitating the wider use of FL in healthcare and leading to better patient outcomes.

The use of Wasserstein Generative Adversarial Networks in searches for new resonances at the LHC.

In the search for physics beyond the standard model, machine learning classifiers provide methods for extracting signals from background processes in data produced at the LHC. Semi-supervised machine learning models are trained on a labeled background and unlabelled signal. When using semi-supervised techniques in the training of machine learning models, over-training can lead to background events incorrectly being labeled as signal events. The extent of false signals generated must therefore be quantified before semi-supervised techniques can be used in resonance searches. In this study, a frequentest methodology is presented to quantify the extent of fake signals generated in the training of semi supervised DNN classifiers when confronting side-bands and the signal regions. The use of a WGAN is explored as a machine learning based data generator.

Broiler Mobility Assessment via a Semi-Supervised Deep Learning Model and Neo-Deep Sort Algorithm.

Mobility is a vital welfare indicator that may influence broilers' daily activities. Classical broiler mobility assessment methods are laborious and cannot provide timely insights into their conditions. Here, we proposed a semi-supervised Deep Learning (DL) model, YOLOv5 (You Only Look Once version 5), combined with a deep sort algorithm conjoined with our newly proposed algorithm, neo-deep sort, for individual broiler mobility tracking. Initially, 1650 labeled images from five days were employed to train the YOLOv5 model. Through semi-supervised learning (SSL), this narrowly trained model was then used for pseudo-labeling 2160 images, of which 2153 were successfully labeled. Thereafter, the YOLOv5 model was fine-tuned on the newly labeled images. Lastly, the trained YOLOv5 and the neo-deep sort algorithm were applied to detect and track 28 broilers in two pens and categorize them in terms of hourly and daily travel distances and speeds. SSL helped in increasing the YOLOv5 model's mean average precision (mAP) in detecting birds from 81% to 98%. Compared with the manually measured covered distances of broilers, the combined model provided individual broilers' hourly moved distances with a validation accuracy of about 80%. Eventually, individual and flock-level mobilities were quantified while overcoming the occlusion, false, and miss-detection issues.

Quantum Machine Learning for Security Assessment in the Internet of Medical Things (IoMT)

Internet of Medical Things (IoMT) is an ecosystem composed of connected electronic items such as small sensors/actuators and other cyber-physical devices (CPDs) in medical services. When these devices are linked together, they can support patients through medical monitoring, analysis, and reporting in more autonomous and intelligent ways. The IoMT devices; however, often do not have sufficient computing resources onboard for service and security assurance while the medical services handle large quantities of sensitive and private health-related data. This leads to several research problems on how to improve security in IoMT systems. This paper focuses on quantum machine learning to assess security vulnerabilities in IoMT systems. This paper provides a comprehensive review of both traditional and quantum machine learning techniques in IoMT vulnerability assessment. This paper also proposes an innovative fused semi-supervised learning model, which is compared to the state-of-the-art traditional and quantum machine learning in an extensive experiment. The experiment shows the competitive performance of the proposed model against the state-of-the-art models and also highlights the usefulness of quantum machine learning in IoMT security assessments and its future applications.

A Contrastive Model with Local Factor Clustering for Semi-Supervised Few-Shot Learning

Learning novel classes with a few samples per class is a very challenging task in deep learning. To mitigate this issue, previous studies have utilized an additional dataset with extensively labeled samples to realize transfer learning. Alternatively, many studies have used unlabeled samples that originated from the novel dataset to achieve few-shot learning, i.e., semi-supervised few-shot learning. In this paper, an easy but efficient semi-supervised few-shot learning model is proposed to address the embeddings mismatch problem that results from inconsistent data distributions between the novel and base datasets, where samples with the same label approach each other while samples with different labels separate from each other in the feature space. This model emphasizes pseudo-labeling guided contrastive learning. We also develop a novel local factor clustering module to improve the ability to obtain pseudo-labels from unlabeled samples, and this module fuses the local feature information of labeled and unlabeled samples. We report our experimental results on the mini-ImageNet and tiered-ImageNet datasets for both five-way one-shot and five-way five-shot settings and achieve better performance than previous models. In particular, the classification accuracy of our model is improved by approximately 11.53% and 14.87% compared to the most advanced semi-supervised few-shot learning model we know in the five-way one-shot scenario. Moreover, ablation experiments in this paper show that our proposed clustering strategy demonstrates accuracy improvements of about 4.00% in the five-way one-shot and five-way five-shot scenarios compared to two popular clustering methods.

Rank Based Two Stage Semi-Supervised Deep Learning Model for X-Ray Images Classification

Rank Based Two Stage Semi-Supervised Deep Learning Model for X-Ray Images Classification

A Cross-Domain Semi-Supervised Zero-Shot Learning Model for the Classification of Hyperspectral Images

A Cross-Domain Semi-Supervised Zero-Shot Learning Model for the Classification of Hyperspectral Images