Semi-supervised Deep Learning Research Articles

SMS Scam Detection Application Based on Optical Character Recognition for Image Data Using Unsupervised and Deep Semi-Supervised Learning

SMS Scam Detection Application Based on Optical Character Recognition for Image Data Using Unsupervised and Deep Semi-Supervised Learning

Detecting floating litter in freshwater bodies with semi-supervised deep learning

Researchers and practitioners have extensively utilized supervised Deep Learning methods to quantify floating litter in rivers and canals. These methods require the availability of large amount of labeled data for training. The labeling work is expensive and laborious, resulting in small open datasets available in the field compared to the comprehensive datasets for computer vision, e.g., ImageNet. Fine-tuning models pre-trained on these larger datasets helps improve litter detection performances and reduces data requirements. Yet, the effectiveness of using features learned from generic datasets is limited in large-scale monitoring, where automated detection must adapt across different locations, environmental conditions, and sensor settings. To address this issue, we propose a two-stage semi-supervised learning method to detect floating litter based on the Swapping Assignments between multiple Views of the same image (SwAV). SwAV is a self-supervised learning approach that learns the underlying feature representation from unlabeled data. In the first stage, we used SwAV to pre-train a ResNet50 backbone architecture on about 100k unlabeled images. In the second stage, we added new layers to the pre-trained ResNet50 to create a Faster R-CNN architecture, and fine-tuned it with a limited number of labeled images (≈1.8k images with 2.6k annotated litter items). We developed and validated our semi-supervised floating litter detection methodology for images collected in canals and waterways of Delft (the Netherlands) and Jakarta (Indonesia). We tested for out-of-domain generalization performances in a zero-shot fashion using additional data from Ho Chi Minh City (Vietnam), Amsterdam and Groningen (the Netherlands). We benchmarked our results against the same Faster R-CNN architecture trained via supervised learning alone by fine-tuning ImageNet pre-trained weights. The findings indicate that the semi-supervised learning method matches or surpasses the supervised learning benchmark when tested on new images from the same training locations. We measured better performances when little data (≈200 images with about 300 annotated litter items) is available for fine-tuning and with respect to reducing false positive predictions. More importantly, the proposed approach demonstrates clear superiority for generalization on the unseen locations, with improvements in average precision of up to 12.7%. We attribute this superior performance to the more effective high-level feature extraction from SwAV pre-training from relevant unlabeled images. Our findings highlight a promising direction to leverage semi-supervised learning for developing foundational models, which have revolutionized artificial intelligence applications in most fields. By scaling our proposed approach with more data and compute, we can make significant strides in monitoring to address the global challenge of litter pollution in water bodies.

Dirty road extraction from GF-2 images by semi-supervised deep learning method for arid and semiarid regions of southern Mongolia

ABSTRACT The uncontrolled proliferation of natural roads in arid regions has exacerbated regional land degradation and desertification, presenting substantial challenges to their accurate mapping owing to their dynamic and obscure features. Moreover, the high cost of data annotation restricts the availability of comprehensively labelled datasets, which are essential for advanced remote sensing processing and natural road detection. This study dedicated to implement a semi-supervised deep learning method for dirty road extraction in southern Mongolia. A new thematic semantic segmentation dataset of natural roads was established firstly to address scarcity of annotation datasets this region. A semi-supervised UniMatch structure was designed consequently. Operating with high-resolution GaoFen-2 images, this approach minimises the need for extensive manual annotation, achieving an IOU of 73.51% and MIOU of 86.37%. This method significantly reduces labour and time costs associated with manual and fully supervised methods. These observations provide a valuable data source and methodology for addressing natural road expansion in arid regions. They can aid governments in evaluating transportation infrastructure in remote areas, and analysing dirty road traffic impact on environment.

Detection of diffusely abnormal white matter in multiple sclerosis on multiparametric brain MRI using semi-supervised deep learning

In addition to focal lesions, diffusely abnormal white matter (DAWM) is seen on brain MRI of multiple sclerosis (MS) patients and may represent early or distinct disease processes. The role of MRI-observed DAWM is understudied due to a lack of automated assessment methods. Supervised deep learning (DL) methods are highly capable in this domain, but require large sets of labeled data. To overcome this challenge, a DL-based network (DAWM-Net) was trained using semi-supervised learning on a limited set of labeled data for segmentation of DAWM, focal lesions, and normal-appearing brain tissues on multiparametric MRI. DAWM-Net segmentation performance was compared to a previous intensity thresholding-based method on an independent test set from expert consensus (N = 25). Segmentation overlap by Dice Similarity Coefficient (DSC) and Spearman correlation of DAWM volumes were assessed. DAWM-Net showed DSC > 0.93 for normal-appearing brain tissues and DSC > 0.81 for focal lesions. For DAWM-Net, the DAWM DSC was 0.49 ± 0.12 with a moderate volume correlation (ρ = 0.52, p < 0.01). The previous method showed lower DAWM DSC of 0.26 ± 0.08 and lacked a significant volume correlation (ρ = 0.23, p = 0.27). These results demonstrate the feasibility of DL-based DAWM auto-segmentation with semi-supervised learning. This tool may facilitate future investigation of the role of DAWM in MS.

Semi-supervised Double Deep Learning Temporal Risk Prediction (SeDDLeR) with Electronic Health Records

Background:Risk prediction plays a crucial role in planning for prevention, monitoring, and treatment. Electronic Health Records (EHRs) offer an expansive repository of temporal medical data encompassing both risk factors and outcome indicators essential for effective risk prediction. However, challenges emerge due to the lack of readily available gold-standard outcomes and the complex effects of various risk factors. Compounding these challenges are the false positives in diagnosis codes, and formidable task of pinpointing the onset timing in annotations. Objective:We develop a Semi-supervised Double Deep Learning Temporal Risk Prediction (SeDDLeR) algorithm based on extensive unlabeled longitudinal Electronic Health Records (EHR) data augmented by a limited set of gold standard labels on the binary status information indicating whether the clinical event of interest occurred during the follow-up period. Methods:The SeDDLeR algorithm calculates an individualized risk of developing future clinical events over time using each patient’s baseline EHR features via the following steps: (1) construction of an initial EHR-derived surrogate as a proxy for the onset status; (2) deep learning calibration of the surrogate along gold-standard onset status; and (3) semi-supervised deep learning for risk prediction combining calibrated surrogates and gold-standard onset status. To account for missing onset time and heterogeneous follow-up, we introduce temporal kernel weighting. We devise a Gated Recurrent Units (GRUs) module to capture temporal characteristics. We subsequently assess our proposed SeDDLeR method in simulation studies and apply the method to the Massachusetts General Brigham (MGB) Biobank to predict type 2 diabetes (T2D) risk. Results:SeDDLeR outperforms benchmark risk prediction methods, including Semi-parametric Transformation Model (STM) and DeepHit, with consistently best accuracy across experiments. SeDDLeR achieved the best C-statistics ( 0.815, SE 0.023; vs STM +.084, SE 0.030, P-value .004; vs DeepHit +.055, SE 0.027, P-value .024) and best average time-specific AUC (0.778, SE 0.022; vs STM + 0.059, SE 0.039, P-value .067; vs DeepHit + 0.168, SE 0.032, P-value <0.001) in the MGB T2D study. Conclusion:SeDDLeR can train robust risk prediction models in both real-world EHR and synthetic datasets with minimal requirements of labeling event times. It holds the potential to be incorporated for future clinical trial recruitment or clinical decision-making.

Wind turbine blade defect detection with a semi-supervised deep learning framework

To increase the economic efficiency of utilizing wind turbines, an enhanced object detection model based on the small version of the fifth generation of the You Look Only Once algorithm (YOLOv5s) is proposed in this paper, which can effectively detect cracks on the surface of wind turbine blades using images captured by unmanned aerial vehicles. To improve the extraction of light-colored and low-definition images, Omni-Dimensional Dynamic Convolution (ODConv) and Dynamic Head Module components (DyHead) are introduced in the proposed method, while a lightweight Group Shuffle convolution (GSConv) module is utilized to accelerate the model inference speed without sacrificing detection performance. Furthermore, a semi-supervised learning strategy is developed to reduce human labors in annotating images. Extensive experiments demonstrate that the proposed model outperforms the original YOLOv5s in terms of both detection accuracy and inference speed. Besides, the proposed model has good performance against state-of-the-art methods. Furthermore, the experiments validate the efficacy of the proposed semi-supervised learning strategy.

Automatic fault detection in grid-connected photovoltaic systems via variational autoencoder-based monitoring

Anomaly detection is indispensable for ensuring the reliable operation of grid-connected photovoltaic (PV) systems. This study introduces a semi-supervised deep learning approach for fault detection in such systems. The method leverages a variational autoencoder (VAE) to extract features and identify anomalies. By training the VAE on normal operation data, a compact latent space representation is created. Abnormal observations, indicating faults, exhibit distinct feature vectors in this latent space. Multiple anomaly detection algorithms, including Isolation Forest, Epileptic Envelope, Local Outlier Factor, and One-Class SVM, are employed to discern normal and abnormal observations. This semi-supervised approach only requires fault-free data for training, without labeled faults, making it attractive in practice. A publicly available dataset, the Grid-connected PV System Faults (GPVS-Faults) dataset, which includes data from a PV plant operating in both maximum power point tracking (MPPT) and intermediate power point tracking (IPPT) switching modes, is used for evaluation. The proposed approach is assessed across various fault scenarios, such as partial shading, inverter faults, and MPPT/IPPT controller faults in boost converters. The outcomes underscore the effectiveness of VAE-based techniques in accurately identifying these faults, with accuracy rates reaching up to 92.90% for MPPT mode and 92.99% for IPPT mode, thus contributing to the robustness of fault detection in grid-connected PV systems.

A novel semi-supervised deep learning method for enhancing discriminability and diversity in EEG-based emotion recognition task

Numerous deep learning models have been introduced for EEG-based Emotion recognition tasks. Nevertheless, the majority of these models are fully supervised, demanding substantial amounts of labeled EEG signals. The labeling process of EEG signals is both time-intensive and costly, involving numerous trials and meticulous analysis by experts. Recently, some advanced semi-supervised algorithms that can achieve a competitive performance with fully-supervised methods by using only a small set of labeled data have been presented. However, these algorithms are primarily developed for the image data type, and naïve adaptation of them for EEG applications results in unsatisfactory performance. To address this issue, we present a robust semi-supervised EEG-based method that exploits the best techniques from advanced semi-supervised algorithms in the computer vision domain enriched with novel regularization terms for unlabeled signals. The proposed regularization terms improve both the discriminability and diversity of the model’s predictions and effectively leverage prior knowledge about the class distributions, thereby achieving a superior performance compared to the distribution alignment techniques in state-of-the-art methods. We evaluate our method on the DEAP dataset for cross-subject valence/arousal emotion recognition tasks, and on the SEED in a cross-session setting. The results indicate that the proposed method consistently surpasses the peer methods at different numbers of labeled data by a large margin.

Multi-Dimensional Manifolds Consistency Regularization for semi-supervised remote sensing semantic segmentation

Semi-supervised semantic segmentation in remote sensing is critical for urban planning, environmental monitoring and disaster response. The high cost and time required for high-quality data annotation limits its wider application. Traditional semi-supervised deep learning methods, which operate in a single dimension, limit model robustness and generalization. Our study addresses this issue by proposing an effective semi-supervised learning method. This method improves model robustness and generalization in remote sensing semantic segmentation. We introduce the Multi-Dimensional Manifolds Consistency Regularization (MDMCR) approach. It applies multi-dimensional perturbations to input images and features, expanding the sample library and improving learning efficiency. Our method has been rigorously tested on various datasets. With only 1/8 of the data labeled, it achieved mean Intersection over Union (mIoU) scores of 74.48% on ISPRS Vaihingen and 78.80% on Potsdam. With only 5% labeled data, it reached 49.93% mIoU on DeepGlobe Roads and 57.90% on Massachusetts Roads. These results show the superiority of our method over existing techniques.

Deep semi-supervised learning for recovering traceability links between issues and commits

Traceability links between issues and commits record valuable information about the evolutionary history of software projects. Unfortunately, these links are often missing. While deep learning stands as the current state-of-the-art (SOTA) in automated traceability links recovery (TLR), its effectiveness is faced with the practical problem of limited labeled data during training. To overcome this challenge, in this paper, we propose DSSLink, a novel method based on deep semi-supervised learning, enhancing deep-learning-based link recovery tasks. DSSLink first learns knowledge from labeled data through pre-trained model and then leverages deep semi-supervised learning to infer pseudo-labels on unlabeled data. The extended dataset of pseudo-labeled and labeled data re-trains the deep learning model in an iterative process. Our extensive evaluations are conducted on two SOTA traceability methods (T-BERT and BTLink) across four GitHub projects and 11 Apache projects. Specifically, the maximum F1-score improvements for GitHub and Apache projects reached 22.9% and 43.5%, respectively. Evaluation results show that DSSLink is effective in enhancing TLR performance and outperforms TraceFUN, a recent approach that utilizes unlabeled data for TLR. The source code of DSSLink is available at https://github.com/DSSLink.Editor’s note: Open Science material was validated by the Journal of Systems and Software Open Science Board.

Semi-Supervised Training for (Pre-Stack) Seismic Data Analysis

A lack of labeled examples is a problem in different domains, such as text and image processing, medicine, and static reservoir characterization, because supervised learning relies on vast volumes of these data to perform successfully, but this is quite expensive. However, large amounts of unlabeled data exist in these domains. The deep semi-supervised learning (DSSL) approach leverages unlabeled data to improve supervised learning performance using deep neural networks. This approach has succeeded in image recognition, text classification, and speech recognition. Nevertheless, there have been few works on pre-stack seismic reservoir characterization, in which knowledge of rock and fluid properties is fundamental for oil exploration. This paper proposes a methodology to estimate acoustic impedance using pre-stack seismic data and DSSL with a recurrent neural network. The few labeled datasets for training were pre-processed from raw seismic and acoustic impedance data from five borehole logs. The results showed that the acoustic impedance estimation at the well location and outside it was better predicted by the DSSL compared to the supervised version of the same neural network. Therefore, employing a large amount of unlabeled data can be helpful in the development of seismic data interpretation systems.

Semi-supervised PolSAR Image Change Detection using Similarity Matching

Abstract. The lack of precisely labeled data limits the development of supervised polarimetric synthetic aperture radar (PolSAR) image change detection. Therefore, semi-supervised deep learning methods have recently demonstrated their significant capability for PolSAR image change detection. Similarity Matching (SimMatch) improves the performance of semi-supervised learning tasks across different benchmark datasets and different settings. Introducing SimMatch into the field of PolSAR image change detection can improve the performance of semi-supervised PolSAR image change detection under limited labeled data conditions. Usually, semi-supervision solves the problem of insufficient labeled data by generating pseudo-labels. However, when the pseudo-label method is simply applied, the model will fit on the confident but wrong pseudo-labels, resulting in poor performance. SimMatch offers a solution by requiring the strongly augmented view to share the same semantic similarity (i.e. label prediction) and instance characteristics (i.e. similarity between instances) with a weak augmented view for more intrinsic feature matching. Besides, by using a labeled memory buffer, the two similarities can be isomorphically transformed with each other by introducing the aggregating and unfolding techniques. Therefore, the semantic and instance pseudo-labels can be mutually propagated, and then, the detection performance of the PolSAR image change detection is improved. Experimental results on real PolSAR datasets demonstrated that SimMatch is an effective semi-supervised PolSAR change detection method and its performance surpasses some well-known change detection methods. Compared to the fully-supervised algorithm CWNN, the semi-supervised SimMatch algorithm can improve accuracy by up to 14.4%.

Real-time coronary artery segmentation in CAG images: A semi-supervised deep learning strategy

BackgroundWhen treating patients with coronary artery disease and concurrent renal concerns, we often encounter a conundrum: how to achieve a clearer view of vascular details while minimizing the contrast and radiation doses during percutaneous coronary intervention (PCI). Our goal is to use deep learning (DL) to create a real-time roadmap for guiding PCI. To this end, segmentation, a critical first step, paves the way for detailed vascular analysis. Unlike traditional supervised learning, which demands extensive labeling time and manpower, our strategy leans toward semi-supervised learning. This method not only economizes on labeling efforts but also aims at reducing contrast and radiation exposure. Methods and resultsCAG data sourced from eight tertiary centers in Taiwan, comprising 500 labeled and 8952 unlabeled images. Employing 400 labels for training and reserving 100 for validation, we built a U-Net based network within a teacher-student architecture. The initial teacher model was updated with 8952 unlabeled images inputted, employing a quality control strategy involving consistency regularization and RandAugment. The optimized teacher model produced pseudo-labels for label expansion, which were then utilized to train the final student model. We attained an average dice similarity coefficient of 0.9003 for segmentation, outperforming supervised learning methods with the same label count. Even with only 5 % labels for semi-supervised training, the results surpassed a supervised method with 100 % labels inputted. This semi-supervised approach's advantage extends beyond single-frame prediction, yielding consistently superior results in continuous angiography films. ConclusionsHigh labeling cost hinders DL training. Semi-supervised learning, quality control, and pseudo-label expansion can overcome this. DL-assisted segmentation potentially provides a real-time PCI roadmap and further diminishes radiation and contrast doses.

Semi-supervised learning in diagnosis of infant hip dysplasia towards multisource ultrasound images.

Automated diagnosis of infant hip dysplasia is heavily affected by the individual differences among infants and ultrasound machines. Hip sonographic images of 493 infants from various ultrasound machines were collected in the Department of Orthopedics in Yangzhou Maternal and Child Health Care Service Centre. Herein, we propose a semi-supervised learning method based on a feature pyramid network (FPN) and a contrastive learning scheme based on a Siamese architecture. A large amount of unlabeled data of ultrasound images was used via the Siamese network in the pre-training step, and then a small amount of annotated data for anatomical structures was adopted to train the model for landmark identification and standard plane recognition. The method was evaluated on our collected dataset. The method achieved a mean Dice similarity coefficient (DSC) of 0.7873 and a mean Hausdorff distance (HD) of 5.0102 in landmark identification, compared to the model without contrastive learning, which had a mean DSC of 0.7734 and a mean HD of 6.1586. The accuracy, precision, and recall of standard plane recognition were 95.4%, 91.64%, and 94.86%, respectively. The corresponding area under the curve (AUC) was 0.982. This study proposes a semi-supervised deep learning method following Graf's principle, which can better utilize a large volume of ultrasound images from various devices and infants. This method can identify the landmarks of infant hips more accurately than manual operators, thereby improving the efficiency of diagnosis of infant hip dysplasia.

Graph deep learning recognition of port ship behavior patterns from a network approach

The detection and recognition of ship mobile behavior patterns based on automatic identification system (AIS) data are crucial for ship navigation and maritime supervision. However, present methods, such as rule-based constraints and density clustering extraction, still suffer from restrictions imposed by artificial settings of parameters and thresholds. The lack of neighborhood context information in unstructured and serialized ship trajectory vector data makes it challenging to incorporate advanced methods such as deep learning for pattern learning. This study builds multiple semi-supervised deep learning graph neural network (GNN) models based on the message passing paradigm for recognizing ship mobile behavior patterns by constructing an AIS trajectory network and computing multi-order node features through Delaunay triangulation. It can get rid of the dependence on parameters of traditional rule-based and unsupervised learning methods. It also reduces the dependence on auxiliary information in trajectory qualitative by introducing multi-order node features. We conducted the experiments on cargo ship data from New York and Singapore to test our method. The model's classification accuracy is more than 99% for the New York dataset and more than 95% for the Singapore dataset. This method can be extended to other types of ships and traffic trajectory vector data with the excellent application potential of GNN in pattern recognition of vector mobile data. The code and experimental data are available at: https://github.com/destiny1103/DT-GNN.

A robust semi-supervised deep learning approach for emotion recognition using EEG signals

A robust semi-supervised deep learning approach for emotion recognition using EEG signals

A Critical Analysis of Deep Semi-Supervised Learning Approaches for Enhanced Medical Image Classification

Deep semi-supervised learning (DSSL) is a machine learning paradigm that blends supervised and unsupervised learning techniques to improve the performance of various models in computer vision tasks. Medical image classification plays a crucial role in disease diagnosis, treatment planning, and patient care. However, obtaining labeled medical image data is often expensive and time-consuming for medical practitioners, leading to limited labeled datasets. DSSL techniques aim to address this challenge, particularly in various medical image tasks, to improve model generalization and performance. DSSL models leverage both the labeled information, which provides explicit supervision, and the unlabeled data, which can provide additional information about the underlying data distribution. That offers a practical solution to resource-intensive demands of data annotation, and enhances the model’s ability to generalize across diverse and previously unseen data landscapes. The present study provides a critical review of various DSSL approaches and their effectiveness and challenges in enhancing medical image classification tasks. The study categorized DSSL techniques into six classes: consistency regularization method, deep adversarial method, pseudo-learning method, graph-based method, multi-label method, and hybrid method. Further, a comparative analysis of performance for six considered methods is conducted using existing studies. The referenced studies have employed metrics such as accuracy, sensitivity, specificity, AUC-ROC, and F1 score to evaluate the performance of DSSL methods on different medical image datasets. Additionally, challenges of the datasets, such as heterogeneity, limited labeled data, and model interpretability, were discussed and highlighted in the context of DSSL for medical image classification. The current review provides future directions and considerations to researchers to further address the challenges and take full advantage of these methods in clinical practices.

Triple-task mutual consistency for semi-supervised 3D medical image segmentation

Semi-supervised deep learning algorithm is an effective means of medical image segmentation. Among these methods, multi-task learning with consistency regularization has achieved outstanding results. However, most of the existing methods usually simply embed the Signed Distance Map (SDM) task into the network, which underestimates the potential ability of SDM in edge awareness and leads to excessive dependence between tasks. In this work, we propose a novel triple-task mutual consistency (TTMC) framework to enhance shape and edge awareness capabilities, and overcome the task dependence problem underestimated in previous work. Specifically, we innovatively construct the Signed Attention Map (SAM), a novel fusion image with attention mechanism, and use it as an auxiliary task for segmentation to enhance the edge awareness ability. Then we implement a triple-task deep network, which jointly predicts the voxel-wise classification map, the Signed Distance Map and the Signed Attention Map. In our proposed framework, an optimized differentiable transformation layer associates SDM with voxel-wise classification map and SAM prediction, while task-level consistency regularization utilizes unlabeled data in an unsupervised manner. Evaluated on the public Left Atrium dataset and NIH Pancreas dataset, our proposed framework achieves significant performance gains by effectively utilizing unlabeled data, outperforming recent state-of-the-art semi-supervised segmentation methods. Code is available at https://github.com/Saocent/TTMC.

EEG power spectra parameterization and adaptive channel selection towards semi-supervised seizure prediction

Background:The seizure prediction algorithms have demonstrated their potential in mitigating epilepsy risks by detecting the pre-ictal state using ongoing electroencephalogram (EEG) signals. However, most of them require high-density EEG, which is burdensome to the patients for daily monitoring. Moreover, prevailing seizure models require extensive training with significant labeled data which is very time-consuming and demanding for the epileptologists. Method:To address these challenges, here we propose an adaptive channel selection strategy and a semi-supervised deep learning model respectively to reduce the number of EEG channels and to limit the amount of labeled data required for accurate seizure prediction. Our channel selection module is centered on features from EEG power spectra parameterization that precisely characterize the epileptic activities to identify the seizure-associated channels for each patient. The semi-supervised model integrates generative adversarial networks and bidirectional long short-term memory networks to enhance seizure prediction. Results:Our approach is evaluated on the CHB-MIT and Siena epilepsy datasets. With utilizing only 4 channels, the method demonstrates outstanding performance with an AUC of 93.15% on the CHB-MIT dataset and an AUC of 88.98% on the Siena dataset. Experimental results also demonstrate that our selection approach reduces the model parameters and training time. Conclusions:Adaptive channel selection coupled with semi-supervised learning can offer the possible bases for a light weight and computationally efficient seizure prediction system, making the daily monitoring practical to improve patients’ quality of life.

Semi-supervised deep learning based on label propagation algorithm for debris flow susceptibility assessment in few-label scenarios

Semi-supervised deep learning based on label propagation algorithm for debris flow susceptibility assessment in few-label scenarios