Semi-supervised Learning Approach Research Articles

Seismic arrival-time picking on distributed acoustic sensing data using semi-supervised learning

Distributed Acoustic Sensing (DAS) is an emerging technology for earthquake monitoring and subsurface imaging. However, its distinct characteristics, such as unknown ground coupling and high noise level, pose challenges to signal processing. Existing machine learning models optimized for conventional seismic data struggle with DAS data due to its ultra-dense spatial sampling and limited manual labels. We introduce a semi-supervised learning approach to address the phase-picking task of DAS data. We use the pre-trained PhaseNet model to generate noisy labels of P/S arrivals in DAS data and apply the Gaussian mixture model phase association (GaMMA) method to refine these noisy labels and build training datasets. We develop PhaseNet-DAS, a deep learning model designed to process 2D spatio-temporal DAS data to achieve accurate phase picking and efficient earthquake detection. Our study demonstrates a method to develop deep learning models for DAS data, unlocking the potential of integrating DAS in enhancing earthquake monitoring.

Classification of Hemilabile Ligands Using Machine Learning.

Hemilabile ligands have the capacity to partially disengage from a metal center, providing a strategy to balance stability and reactivity in catalysis, but they are not straightforward to identify. We identify ligands in the Cambridge Structural Database that have been crystallized with distinct denticities and are thus identifiable as hemilabile ligands. We implement a semi-supervised learning approach using a label-spreading algorithm to augment a small negative set that is supported by heuristic rules of ligand and metal co-occurrence. We show that a heuristic based on coordinating atom identity alone is not sufficient to identify whether a ligand is hemilabile, and our trained machine-learning classification models are instead needed to predict whether a bi-, tri-, or tetradentate ligand is hemilabile with high accuracy and precision. Feature importance analysis of our models shows that the second, third, and fourth coordination spheres all play important roles in ligand hemilability.

ILF: A Quantum Semi-Supervised Learning Approach for Binary Classification

ILF: A Quantum Semi-Supervised Learning Approach for Binary Classification

A Comparative Study of Landslide Susceptibility Mapping Using Bagging PU Learning in Class-Prior Probability Shift Datasets

Landslide susceptibility mapping is typically based on binary prediction probabilities. However, non-landslide samples in modeling datasets are often unlabeled data, and the phenomenon of class-priori shift, that is, the proportion of landslide samples frequently deviates from real-world scenarios and is spatially heterogeneous. By comparing the classification performance and predicted probability distributions across multiple unbalanced datasets with known and unknown sample proportions, this study assesses the landslide susceptibility model’s generalization ability in the context of class-prior shifts. The study investigates the potential of Bagging PU Learning, a semi-supervised learning approach, in improving the generalization performance of landslide susceptibility models and proposes the Bagging PU-GDBT algorithm. Our findings highlight the effectiveness of Bagging PU Learning in enhancing the recall of landslides and the generalization capabilities of models on unbalanced datasets. This method reduces prediction uncertainties, especially in high and very high susceptibility zones. Furthermore, results emphasize the superiority of models trained on balanced datasets with 1:1 sample ratio for landslide susceptibility mapping over those trained on unbalanced datasets.

Boosting semi-supervised learning with Contrastive Complementary Labeling

Semi-supervised learning (SSL) approaches have achieved great success in leveraging a large amount of unlabeled data to learn deep models. Among them, one popular approach is pseudo-labeling which generates pseudo labels only for those unlabeled data with high-confidence predictions. As for the low-confidence ones, existing methods often simply discard them because these unreliable pseudo labels may mislead the model. Unlike existing methods, we highlight that these low-confidence data can be still beneficial to the training process. Specifically, although we cannot determine which class a low-confidence sample belongs to, we can assume that this sample should be very unlikely to belong to those classes with the lowest probabilities (often called complementary classes/labels). Inspired by this, we propose a novel Contrastive Complementary Labeling (CCL) method that constructs a large number of reliable negative pairs based on the complementary labels and adopts contrastive learning to make use of all the unlabeled data. Extensive experiments demonstrate that CCL significantly improves the performance on top of existing advanced methods and is particularly effective under the label-scarce settings. For example, CCL yields an improvement of 2.43% over FixMatch on CIFAR-10 only with 40 labeled data.

Semi-Supervised Machine Learning Approaches for DDOS Attack Detection

Network infrastructures are the target of several attacks. These include intrusions into the confidentiality, integrity, and availability of the network. The network's availability is impacted by a persistent attack known as a distributed denial-of-service (DDoS) attack. Such an assault is carried out using a command and control (C & C) technique. To detect these assaults, numerous researchers have put forth various machine learning-based solutions. In this paper, we are going to detect different DDoS attacks by various methods and evaluate their performance. This experiment made use of the KD99 dataset. The normal and assault samples were classified using the random forest technique. The classification of 99.76% of the samples was accurate. By strategically selecting clusters and incorporating the insights gained from the small labelled dataset, a portion of the unlabelled clusters can be assigned labels, effectively converting raw data into useful training examples. This enriched dataset is then used to train an improved classifier that can better generalize and adapt to the dynamic nature of DDoS attacks.

An efficient segmentation method based on semi-supervised learning for seafloor monitoring in Pujada Bay, Philippines

The semantic segmentation of marine images makes it easier to describe seafloor scenes and track marine organisms. However, creating human-annotated datasets for image segmentation requires significant time and effort. Therefore, this study proposes a semisupervised learning approach that combines the mean-teacher model and U-Net architecture for segmenting seafloor images obtained from the Philippines. The proposed method performs segmentation for categories, including corals, sea urchins, sea stars, and seagrass. Traditional manual labelling methods are used for coral, sea urchins, and sea stars. For the seagrass category, which is challenging to label manually, we used the K-means clustering algorithm to obtain corresponding labelled datasets based on the characteristics of such images and evaluated the feasibility. Compared with the U-Net-based supervised method, the semi-supervised method used in this study achieved good results and accuracy values, even with fewer labelled images.

MTAN: A semi-supervised learning model for kidney tumor segmentation.

Medical image segmentation is crucial in disease diagnosis and treatment planning. Deep learning (DL) techniques have shown promise. However, optimizing DL models requires setting numerous parameters, and demands substantial labeled datasets, which are labor-intensive to create. This study proposes a semi-supervised model that can utilize labeled and unlabeled data to accurately segment kidneys, tumors, and cysts on CT images, even with limited labeled samples. An end-to-end semi-supervised learning model named MTAN (Mean Teacher Attention N-Net) is designed to segment kidneys, tumors, and cysts on CT images. The MTAN model is built on the foundation of the AN-Net architecture, functioning dually as teachers and students. In its student role, AN-Net learns conventionally. In its teacher role, it generates objects and instructs the student model on their utilization to enhance learning quality. The semi-supervised nature of MTAN allows it to effectively utilize unlabeled data for training, thus improving performance and reducing overfitting. We evaluate the proposed model using two CT image datasets (KiTS19 and KiTS21). In the KiTS19 dataset, MTAN achieved segmentation results with an average Dice score of 0.975 for kidneys and 0.869 for tumors, respectively. Moreover, on the KiTS21 dataset, MTAN demonstrates its robustness, yielding average Dice scores of 0.977 for kidneys, 0.886 for masses, 0.861 for tumors, and 0.759 for cysts, respectively. The proposed MTAN model presents a compelling solution for accurate medical image segmentation, particularly in scenarios where the labeled data is scarce. By effectively utilizing the unlabeled data through a semi-supervised learning approach, MTAN mitigates overfitting concerns and achieves high-quality segmentation results. The consistent performance across two distinct datasets, KiTS19 and KiTS21, underscores model's reliability and potential for clinical reference.

SPLAL: Similarity-based pseudo-labeling with alignment loss for semi-supervised medical image classification

Medical image classification presents significant challenges due to limited labeled samples and class imbalance resulting from varying disease prevalence. In many real-world scenarios, a scarcity of labeled images pertaining to specific ailments coexists with an abundance of unlabeled ones. The process of annotating these unlabeled medical images by domain experts is both time-intensive and financially burdensome. Consequently, medical image classification remains an active and evolving research area. Semi-supervised learning (SSL) techniques hold promise in addressing these challenges by effectively leveraging both labeled and unlabeled data. However, the application of SSL to medical image classification necessitates overcoming two primary obstacles: (1) the reliable estimation of pseudo-labels for unlabeled images and (2) the mitigation of biases stemming from class imbalance. In this work, we introduce a novel semi-supervised learning approach, denoted as SPLAL, which adeptly addresses these challenges. SPLAL leverages the concept of class prototypes and employs a weighted combination of classifiers to predict dependable pseudo-labels for a subset of unlabeled medical images. Additionally, we introduce an alignment loss term designed to alleviate model biases, particularly those arising from the overrepresentation of majority classes. This alignment loss mechanism ensures the model’s consistency in delivering the same classification output for diverse augmented images of the same disease. To evaluate the effectiveness of our proposed approach, we conducted comprehensive experiments on two publicly accessible benchmark datasets for medical image classification. Our empirical findings unequivocally demonstrate that SPLAL outperforms several state-of-the-art semi-supervised learning methods across various evaluation metrics.

AnGeL: Fully-Automated Analog Circuit Generator Using a Neural Network Assisted Semi-Supervised Learning Approach

AnGeL: Fully-Automated Analog Circuit Generator Using a Neural Network Assisted Semi-Supervised Learning Approach

Semi-Supervised Domain Alignment Learning for Single Image Dehazing.

Convolutional neural networks (CNNs) have attracted much research attention and achieved great improvements in single-image dehazing. However, previous learning-based dehazing methods are mainly trained on synthetic data, which greatly degrades their generalization capability on natural hazy images. To address this issue, this article proposes a semi-supervised learning approach for single-image dehazing, where both synthetic and realistic images are leveraged during training. Considering the situation that it is hard to obtain the realistic pairs of hazy and haze-free images, how to utilize the realistic data is not a trivial work. In this article, a domain alignment module is introduced to narrow the distribution distance between synthetic data and realistic hazy images in a latent feature space. Meanwhile, a haze-aware attention module is designed to describe haze densities of different regions in the image, thus adaptively responds for different hazy areas. Furthermore, the dark channel prior is introduced to the framework to improve the quality of the unsupervised learning results by considering the statistical characters of haze-free images. Such a semi-supervised design can significantly address the domain shift issue between the synthetic and realistic data, and improve generalization performance in the real world. Experiments indicate that the proposed method obtains state-of-the-art performance on both public synthetic and realistic hazy images with better visual results.

Semi-supervised imbalanced classification of wafer bin map defects using a Dual-Head CNN

Wafer Bin Map (WBM) defect patterns are a critical aspect of identifying the root cause of manufacturing defects in the semiconductor industry. Semi-supervised learning (SSL) approaches have gained popularity for this purpose, as they can leverage both labeled and unlabeled data to improve model performance. However, SSL of WBM defect patterns is challenging due to class imbalance, where some defect classes have many more examples than others. Most of the existing SSL approaches assume a balanced dataset and often fail to provide satisfactory results when applied to imbalanced class problems. To address this issue, this work proposes a novel Dual-Head Convolutional Neural Network (CNN) architecture that contains two classifier heads. One classifier head maximizes overall classification scores, while the other aims to maximize per-class classification scores, providing equal attention to both majority and minority classes. The proposed CNN architecture uses pseudo-labels selected based on the outputs of these two classifiers to expand the labeled training set, which is then used to retrain the CNN. In this way, highly confident pseudo-labels are selected even from the minority classes, leading to better model training. Experiments show that the proposed approach is effective in handling class-imbalanced classification of WBM defect patterns, reporting state-of-the-art classification with an F1 score of 0.918, accuracy of 98.2% and a mean per-class accuracy of 91.7% using a lightweight ResNet-10 model as the backbone on the real-world public WBM dataset, WM-811K. The proposed approach’s success suggests that it could be a valuable tool for improving the accuracy and reliability of WBM defect pattern classification in semiconductor manufacturing. The code is available at https://github.com/M-Siyamalan/SSL-DHCNN.

A Machine Learning Approach to Predicting Physical Activity Levels in Adolescents

The ongoing evolution of technology has had both positive and negative effects on modern society. On the positive side, it has significantly improved the ease with which various activities can be performed. However, it has also had a negative impact by reducing physical activity. This reduction in physical activity, in turn, increases the risk of chronic diseases that contribute to global mortality rates. This research aims to assess the effectiveness of machine learning in predicting the physical activity levels of adolescents. The study utilizes data from accelerometers, specifically the ActiGraph GT3X. The research methodology employs a semi-supervised machine learning approach, using the support vector machine and decision tree algorithms to make these predictions. The sample comprises 61 adolescents (males = 17, female = 44), including high school students and university students aged 18-21, from the West Java region. The results from the machine learning model using the decision tree algorithm indicated a model accuracy of 97.50% in predicting physical activity levels. In contrast, the accuracy obtained from the performance analysis using the confusion matrix for the support vector machine model was 92.5%. Based on these accuracy levels, the decision tree algorithm outperforms the support vector machine algorithm's accuracy. Further analyses involving different models are necessary to determine which algorithm offers the highest level of accuracy.

Energy Consumption Auditing Anomaly Detection System of Robotic Manipulators based on a Generative Adversarial Network

Unexpected anomalies pose significant risks to the health and security of intelligent manufacturing systems. This paper proposes a generative adversarial network (GAN)-based anomaly detection framework specifically for monitoring robotic manipulator operation using a side-channel energy auditing mechanism. To tackle the limitation arising from the lack of labeled data, the GAN model is trained by a semi-supervised learning approach that identifies anomalies during online operations as outliers. The overfitting is purposely utilized during the model training to enlarge the difference between normal energy consumption patterns used for training and anomalous profiles in real-time testing. In addition, the GAN model is modified to use multiple discriminators to analyze the individual energy profile associated with each joint or motor. The anomaly is detected by evaluating the mean and standard deviation values of anomaly scores' distribution, and both values are continuously updated by Welford's algorithm in real time to take into account the effect of environmental variations during operations. The detection performance on our custom dataset demonstrates the feasibility of the proposed pipeline. Specifically, for physical attacks, the framework can achieve an accuracy of approximately 0.93 for instant-wise detection and 0.84 for event-wise detection.

Inter- and intra-uncertainty based feature aggregation model for semi-supervised histopathology image segmentation

Acquiring pixel-level annotations is often limited in applications such as histology studies that require domain expertise. Various semi-supervised learning approaches have been developed to work with limited ground truth annotations, such as the popular teacher-student models. However, hierarchical prediction uncertainty within the student model (intra-uncertainty) and image prediction uncertainty (inter-uncertainty) have not been fully utilized by existing methods. To address these issues, we first propose a novel inter- and intra-uncertainty regularization method to measure and constrain both inter- and intra-inconsistencies in the teacher-student architecture. We also propose a new two-stage network with pseudo-mask guided feature aggregation (PG-FANet) as the segmentation model. The two-stage structure complements with the uncertainty regularization strategy to avoid introducing extra modules in solving uncertainties and the aggregation mechanisms enable multi-scale and multi-stage feature integration. Comprehensive experimental results over the MoNuSeg and CRAG datasets show that our PG-FANet outperforms other state-of-the-art methods and our semi-supervised learning framework yields competitive performance with a limited amount of labeled data.

Consistency regularisation in varying contexts and feature perturbations for semi-supervised semantic segmentation of histology images

Semantic segmentation of various tissue and nuclei types in histology images is fundamental to many downstream tasks in the area of computational pathology (CPath). In recent years, Deep Learning (DL) methods have been shown to perform well on segmentation tasks but DL methods generally require a large amount of pixel-wise annotated data. Pixel-wise annotation sometimes requires expert’s knowledge and time which is laborious and costly to obtain. In this paper, we present a consistency based semi-supervised learning (SSL) approach that can help mitigate this challenge by exploiting a large amount of unlabelled data for model training thus alleviating the need for a large annotated dataset. However, SSL models might also be susceptible to changing context and features perturbations exhibiting poor generalisation due to the limited training data. We propose an SSL method that learns robust features from both labelled and unlabelled images by enforcing consistency against varying contexts and feature perturbations. The proposed method incorporates context-aware consistency by contrasting pairs of overlapping images in a pixel-wise manner from changing contexts resulting in robust and context invariant features. We show that cross-consistency training makes the encoder features invariant to different perturbations and improves the prediction confidence. Finally, entropy minimisation is employed to further boost the confidence of the final prediction maps from unlabelled data. We conduct an extensive set of experiments on two publicly available large datasets (BCSS and MoNuSeg) and show superior performance compared to the state-of-the-art methods.

Semi-Supervised Learning Method for the Augmentation of an Incomplete Image-Based Inventory of Earthquake-Induced Soil Liquefaction Surface Effects

Soil liquefaction often occurs as a secondary hazard during earthquakes and can lead to significant structural and infrastructure damage. Liquefaction is most often documented through field reconnaissance and recorded as point locations. Complete liquefaction inventories across the impacted area are rare but valuable for developing empirical liquefaction prediction models. Remote sensing analysis can be used to rapidly produce the full spatial extent of liquefaction ejecta after an event to inform and supplement field investigations. Visually labeling liquefaction ejecta from remotely sensed imagery is time-consuming and prone to human error and inconsistency. This study uses a partially labeled liquefaction inventory created from visual annotations by experts and proposes a pixel-based approach to detecting unlabeled liquefaction using advanced machine learning and image processing techniques, and to generating an augmented inventory of liquefaction ejecta with high spatial completeness. The proposed methodology is applied to aerial imagery taken from the 2011 Christchurch earthquake and considers the available partial liquefaction labels as high-certainty liquefaction features. This study consists of two specific comparative analyses. (1) To tackle the limited availability of labeled data and their spatial incompleteness, a semi-supervised self-training classification via Linear Discriminant Analysis is presented, and the performance of the semi-supervised learning approach is compared with supervised learning classification. (2) A post-event aerial image with RGB (red-green-blue) channels is used to extract color transformation bands, statistical indices, texture components, and dimensionality reduction outputs, and performances of the classification model with different combinations of selected features from these four groups are compared. Building footprints are also used as the only non-imagery geospatial information to improve classification accuracy by masking out building roofs from the classification process. To prepare the multi-class labeled data, regions of interest (ROIs) were drawn to collect samples of seven land cover and land use classes. The labeled samples of liquefaction were also clustered into two groups (dark and light) using the Fuzzy C-Means clustering algorithm to split the liquefaction pixels into two classes. A comparison of the generated maps with fully and manually labeled liquefaction data showed that the proposed semi-supervised method performs best when selected high-ranked features of the two groups of statistical indices (gradient weight and sum of the band squares) and dimensionality reduction outputs (first and second principal components) are used. It also outperforms supervised learning and can better augment the liquefaction labels across the image in terms of spatial completeness.

Malicious Traffic Classification via Edge Intelligence in IIoT

The proliferation of smart devices in the 5G era of industrial IoT (IIoT) produces significant traffic data, some of which is encrypted malicious traffic, creating a significant problem for malicious traffic detection. Malicious traffic classification is one of the most efficient techniques for detecting malicious traffic. Although it is a labor-intensive and time-consuming process to gather large labeled datasets, the majority of prior studies on the classification of malicious traffic use supervised learning approaches and provide decent classification results when a substantial quantity of labeled data is available. This paper proposes a semi-supervised learning approach for classifying malicious IIoT traffic. The approach utilizes the encoder–decoder model framework to classify the traffic, even with a limited amount of labeled data available. We sample and normalize the data during the data-processing stage. In the semi-supervised model-building stage, we first pre-train a model on a large unlabeled dataset. Subsequently, we transfer the learned weights to a new model, which is then retrained using a small labeled dataset. We also offer an edge intelligence model that considers aspects such as computation latency, transmission latency, and privacy protection to improve the model’s performance. To achieve the lowest total latency and to reduce the risk of privacy leakage, we first create latency and privacy-protection models for each local, edge, and cloud. Then, we optimize the total latency and overall privacy level. In the study of IIoT malicious traffic classification, experimental results demonstrate that our method reduces the model training and classification time with 97.55% accuracy; moreover, our approach boosts the privacy-protection factor.

A machine learning-based score for precise echocardiographic assessment of cardiac remodelling in hypertensive young adults.

Accurate staging of hypertension-related cardiac changes, before the development of significant left ventricular hypertrophy, could help guide early prevention advice. We evaluated whether a novel semi-supervised machine learning approach could generate a clinically meaningful summary score of cardiac remodelling in hypertension. A contrastive trajectories inference approach was applied to data collected from three UK studies of young adults. Low-dimensional variance was identified in 66 echocardiography variables from participants with hypertension (systolic ≥160 mmHg) relative to a normotensive group (systolic < 120 mmHg) using a contrasted principal component analysis. A minimum spanning tree was constructed to derive a normalized score for each individual reflecting extent of cardiac remodelling between zero (health) and one (disease). Model stability and clinical interpretability were evaluated as well as modifiability in response to a 16-week exercise intervention. A total of 411 young adults (29 ± 6 years) were included in the analysis, and, after contrastive dimensionality reduction, 21 variables characterized >80% of data variance. Repeated scores for an individual in cross-validation were stable (root mean squared deviation = 0.1 ± 0.002) with good differentiation of normotensive and hypertensive individuals (area under the receiver operating characteristics 0.98). The derived score followed expected hypertension-related patterns in individual cardiac parameters at baseline and reduced after exercise, proportional to intervention compliance (P = 0.04) and improvement in ventilatory threshold (P = 0.01). A quantitative score that summarizes hypertension-related cardiac remodelling in young adults can be generated from a computational model. This score might allow more personalized early prevention advice, but further evaluation of clinical applicability is required.

Neural Data-to-Text Generation Based on Small Datasets: Comparing the Added Value of Two Semi-Supervised Learning Approaches on Top of a Large Language Model

AbstractThis study discusses the effect of semi-supervised learning in combination with pretrained language models for data-to-text generation. It is not known whether semi-supervised learning is still helpful when a large-scale language model is also supplemented. This study aims to answer this question by comparing a data-to-text system only supplemented with a language model, to two data-to-text systems that are additionally enriched by a data augmentation or a pseudo-labeling semi-supervised learning approach.Results show that semi-supervised learning results in higher scores on diversity metrics. In terms of output quality, extending the training set of a data-to-text system with a language model using the pseudo-labeling approach did increase text quality scores, but the data augmentation approach yielded similar scores to the system without training set extension. These results indicate that semi-supervised learning approaches can bolster output quality and diversity, even when a language model is also present.