Semi-supervised Learning Approach Research Articles

A fatigue crack prediction method based on inductive semi-supervised learning and Lamb-wave monitoring for orthotropic steel bridge deck

Orthotropic steel bridge deck suffers from fatigue cracking due to its orthotropic configuration and cyclic vehicular load. Hence, fatigue crack monitoring and prediction are crucial for addressing this issue. The Lamb-wave technique has potential in monitoring the growth of fatigue cracks, whereas most of the existing signal processing methods show poor performance in the crack prediction. To improve the accuracy and reliability of fatigue crack prediction, an inductive semi-supervised learning (ISL) approach is proposed in conjunction with Lamb-wave monitoring. The presented method is first verified by conducting a fatigue test of orthotropic steel bridge deck loading by three load conditions which would lead to different types of cracks, where lightweight Lamb-wave monitoring devices were adopted. The approach was further applied in field monitoring to validate its effectiveness in real engineering structures. The results demonstrate that the proposed method can recognize the initiation of fatigue cracks by giving a predicted time interval in which the real initiation moment locates. Similar increasing trends are found between the predicted and real crack lengths during the growth of cracks, while a relative fluctuate is existing in the predicted results, and the prediction accuracy of crack lengths can be regarded as 2 mm. For bridge field monitoring, the real fatigue crack length was also well predicted by the proposed method, exhibiting an average error of 0.82 mm in comparison with the real values. In general, the research findings emphasize the adaptability of the proposed method for fatigue crack prediction in practical engineering.

A scoping review of machine learning applied to peripheral nerve interfaces.

Peripheral nerve interfaces (PNIs) can enable communication with the peripheral nervous system and have a broad range of applications including in bioelectronic medicine and neuroprostheses. They can modulate neural activity through stimulation or monitor conditions by recording from the peripheral nerves. The recent growth of Machine Learning (ML) has led to the application of a wide variety of ML techniques to PNIs, especially in circumstances where the goal is classification or regression. However, the extent to which ML has been applied to PNIs or the range of suitable ML techniques has not been documented. Therefore, a scoping review was conducted to determine and understand the state of ML in the PNI field. The review searched five databases and included 63 studies after full-text review. Most studies incorporated a supervised learning approach to classify activity, with the most common algorithms being some form of neural network (artificial neural network, convolutional neural network or recurrent neural network). Unsupervised, semi-supervised and reinforcement learning approaches are currently underutilized and could be better leveraged to improve performance in this domain.

Translation Consistent Semi-supervised Segmentation for 3D Medical Images.

3D medical image segmentation methods have been successful, but their dependence on large amounts of voxel-level annotated data is a disadvantage that needs to be addressed given the high cost to obtain such annotation. Semi-supervised learning (SSL) solves this issue by training models with a large unlabelled and a small labelled dataset. The most successful SSL approaches are based on consistency learning that minimises the distance between model responses obtained from perturbed views of the unlabelled data. These perturbations usually keep the spatial input context between views fairly consistent, which may cause the model to learn segmentation patterns from the spatial input contexts instead of the foreground objects. In this paper, we introduce the Translation Consistent Co-training (TraCoCo) which is a consistency learning SSL method that perturbs the input data views by varying their spatial input context, allowing the model to learn segmentation patterns from foreground objects. Furthermore, we propose a new Confident Regional Cross entropy (CRC) loss, which improves training convergence and keeps the robustness to co-training pseudo-labelling mistakes. Our method yields state-of-the-art (SOTA) results for several 3D data benchmarks, such as the Left Atrium (LA), Pancreas-CT (Pancreas), and Brain Tumor Segmentation (BraTS19). Our method also attains best results on a 2D-slice benchmark, namely the Automated Cardiac Diagnosis Challenge (ACDC), further demonstrating its effectiveness. Our code, training logs and checkpoints are available at https://github.com/yyliu01/ TraCoCo.

Application of the Semi-Supervised Learning Approach for Pavement Defect Detection.

Road surface quality is essential for driver comfort and safety, making it crucial to monitor pavement conditions and detect defects in real time. However, the diversity of defects and the complexity of ambient conditions make it challenging to develop an effective and robust classification and detection algorithm. In this study, we adopted a semi-supervised learning approach to train ResNet-18 for image feature retrieval and then classification and detection of pavement defects. The resulting feature embedding vectors from image patches were retrieved, concatenated, and randomly sampled to model a multivariate normal distribution based on the only one-class training pavement image dataset. The calibration pavement image dataset was used to determine the defect score threshold based on the receiver operating characteristic curve, with the Mahalanobis distance employed as a metric to evaluate differences between normal and defect pavement images. Finally, a heatmap derived from the defect score map for the testing dataset was overlaid on the original pavement images to provide insight into the network's decisions and guide measures to improve its performance. The results demonstrate that the model's classification accuracy improved from 0.868 to 0.887 using the expanded and augmented pavement image data based on the analysis of heatmaps.

Deep learning for surgical workflow analysis: a survey of progresses, limitations, and trends

Automatic surgical workflow analysis, which aims to recognize the ongoing surgical events in videos, is fundamental for developing context-aware computer-assisted systems. This paper reviews representative surgical workflow recognition algorithms based on deep learning, outlining their merits, limitations, and future research directions. The literature survey was performed on three large bibliographic databases, covering 67 lary sources, which were comparatively analyzed in terms of spatial feature modeling, spatio-temporal feature modeling, input pre-processing, regularization and post-processing algorithms, as well as learning strategies. Then, common public datasets and evaluation metrics for surgical workflow recognition are also described in detail. Finally, we discuss all literature from different perspectives, and point out the challenges, possible solutions and future trends. The need for more diverse and larger datasets, the potential of unsupervised and semi-supervised learning approaches, comprehensive and equitable metrics, establishing complete regulatory and data standards, and interoperability will be key challenges in translating models to clinical operating rooms. And we propose that surgical activity anticipation and employing large language model as training assistant are interesting research directions in surgical workflow analysis.

Detecting Retinopathy From Optical Coherence Tomography Images Using a Novel Augmentation-Based Semi-Supervised Learning Approach

Detecting Retinopathy From Optical Coherence Tomography Images Using a Novel Augmentation-Based Semi-Supervised Learning Approach

Semi-supervised machine learning method for predicting homogeneous ancestry groups to assess Hardy-Weinberg equilibrium in diverse whole-genome sequencing studies

Large-scale, multi-ethnic whole-genome sequencing (WGS) studies, such as the National Human Genome Research Institute Genome Sequencing Program's Centers for Common Disease Genomics (CCDG), play an important role in increasing diversity for genetic research. Before performing association analyses, assessing Hardy-Weinberg equilibrium (HWE) is a crucial step in quality control procedures to remove low quality variants and ensure valid downstream analyses. Diverse WGS studies contain ancestrally heterogeneous samples; however, commonly used HWE methods assume that the samples are homogeneous. Therefore, directly applying these to the whole dataset can yield statistically invalid results. To account for this heterogeneity, HWE can be tested on subsets of samples that have genetically homogeneous ancestries and the results aggregated at each variant. To facilitate valid HWE subset testing, we developed a semi-supervised learning approach that predicts homogeneous ancestries based on the genotype. This method provides a convenient tool for estimating HWE in the presence of population structure and missing self-reported race and ethnicities in diverse WGS studies. In addition, assessing HWE within the homogeneous ancestries provides reliable HWE estimates that will directly benefit downstream analyses, including association analyses in WGS studies. We applied our proposed method on the CCDG dataset, predicting homogeneous genetic ancestry groups for 60,545 multi-ethnic WGS samples to assess HWE within each group.

Automated PIM Detection in Wireless Networks via Smart Data-Driven Techniques

As the number of base stations keeps increasing due to ever-growing wireless services and pervasive connectivity, the operational costs for network service providers rises proportionally. In addition, with the evolution of wireless technologies and the development of next generation networks such as 5G and 6G, network operators are inevitably expected to encounter more complex challenges. One such network issue is the Passive Intermodulation (PIM) problem that is observed in both 4G and 5G networks. It degrades the user experience and radio resource efficiency while leading to an operational overhead for detecting and mitigating it on the operator side. Although there is a significant body of work regarding PIM detection and cancellation methods, the majority of such studies depends on hardware solutions and manual investigation by network engineers, which is costly in terms of time and labor. In this paper, we propose two methods using unsupervised and semi-supervised Machine Learning (ML) approaches, namely a time-series-based anomaly detection technique and an autoencoder-based one, for identifying PIM problems in network sites. The proposed solutions utilize a set of Key Performance Indicator (KPI) data of base stations obtained from network management systems for a significantly long time interval and detect possible PIM problems without the need for a human in the loop. We measure and analyze the performance of our solutions, with the guidance of experienced network engineers, on our collected dataset.

Recent advances in complementary label learning

Complementary Label Learning (CLL), a crucial aspect of weakly supervised learning, has seen significant theoretical and practical advancements. However, a comprehensive review of the field has been lacking. This survey provides the first exhaustive compilation and synthesis of state-of-the-art CLL approaches, filling a critical gap in the literature and serving as a foundational resource for the community. Key contributions of this survey include an extensive categorization of CLL methodologies, clarifying diverse algorithms based on their principles and applications. This classification scheme enhances understanding of the CLL landscape, highlighting its versatility across varied settings. Additionally, the survey examines the evolution of CLL, showcasing its adaptability and potential in addressing complex applications. It also explores experimental frameworks, including processes for generating complementary labels and datasets and numerical evaluation of existing state-of-the-art. Moreover, the survey delves into how CLL integrates with and enhances other weakly supervised and semi-supervised learning approaches, deepening understanding of its role in the broader machine learning ecosystem. Overall, this survey not only compiles CLL research but also guides future explorations, steering the field towards new horizons in weakly supervised learning.

Enhancing Multi-Object Detection in Ultrasound Images Through Semi-Supervised Learning, Focal Loss and Relation of Frame

ObjectiveTo identify musculoskeletal anatomical structures in real time by using deep learning techniques. MethodsAn automated annotation system based on deep learning neural networks was designed to aid in the real-time identification of anatomical structures. Additionally, novel algorithms aimed at diminishing model training duration while enhancing accuracy were introduced. In this study, we proposed a semi-supervised learning (SSL) approach that substantially reduced annotation time. We also adopted the focal loss (FL) method to enhance the accuracy of challenging structures. Additionally, during the inference stage, we harnessed the temporal continuity of video frames, which involved leveraging information from preceding frames to facilitate recognition of structures in the current image. Training the model through a combination of SSL and FL yielded superior performance compared with supervised learning, while also substantially mitigating any expense linked to annotations. During inference, the incorporation of frame continuity helped to avoid discontinuity and bolster accuracy. ResultsForearm tissue detection was demonstrated by properly configuring the SSL approach, including FL and the filtering threshold. Comparable performance with supervised learning was achieved while only using 30% of the training data. The real-time experimental results also demonstrated that implementing relation of frame reduced the number of missing frames during inference and successfully increased the confidence scores of detected objects. ConclusionThis proposed system has the potential to aid medical professionals in efficiently and effectively diagnosing musculoskeletal disorders, ultimately leading to enhanced patient outcomes.

CoFix: Advancing Semi-Supervised Learning with Noisy Label Mitigation Through Sample Selection and Consistent Regularization

Deep Neural Networks (DNNs) have revolutionized various fields through their ability to model complex patterns, yet their performance critically hinges on the availability of large-scale, accurately labeled datasets. The costly and time-consuming process of creating such datasets has motivated research into Learning with Noisy Labels (LNL), which aims to reduce reliance on perfect labels. This paper introduces CoFix, an innovative LNL framework that integrates sample selection with semi-supervised learning techniques to address the challenge of noisy labels in training. CoFix employs a Gaussian Mixture Model (GMM) to dynamically segment the training data into clean and noisy subsets, leveraging semi-supervised learning approaches on both. Inspired by the FixMatch algorithm, CoFix refines consistent regularization and pseudo-labeling strategies, enhancing augmentation strategies and temperature sharpening techniques. Additionally, CoFix explores label smoothing to augment the loss function, further refining model performance. Our experiments demonstrate CoFix’s superiority over state-of-the-art methods, achieving significant improvements in fewer training epochs, particularly in lower noise scenarios. The robustness and versatility of CoFix are evident through its consistent performance across various benchmark datasets and noise levels. The contributions of this paper include a novel LNL method with enhanced generalization capability, an investigation into the impact of label smoothing on the loss function, and extensive testing that confirms CoFix’s efficiency and adaptability to different noise levels.

Using Semi-supervised Domain Adaptation to Enhance EEG-Based Cross-Task Mental Workload Classification Performance.

Mental workload (MWL) assessment is critical for accident prevention and operator safety. However, achieving cross-task generalization of MWL classification models is a significant challenge for real-world applications. Classifiers trained on labeled samples from one task often experience a notable performance drop when directly applied to samples from other tasks, limiting its use cases. To address this issue, we propose a semi-supervised cross-task domain adaptation (SCDA) method using power spectral density (PSD) features for MWL recognition across tasks (MATB-II and n-back). Our results demonstrated that the SCDA method achieved the best cross-task classification performance on our data and COG-BCI public dataset, with accuracies of 90.98% ± 9.36% and 96.61% ± 4.35%, respectively. Furthermore, in the cross-task classification of cross-subject scenarios, SCDA showed the highest average accuracy (75.39% ± 9.56% on our data, 90.98% ± 9.36% on the COG-BCI public dataset). The findings indicate that the semi-supervised transfer learning approach using PSD features is feasible and effective for cross-task MWL assessment.

Automatic Road Crack Detection Using Convolutional Neural Network Based on Semi-Supervised Learning

Crack detection in pavements is a critical task for infrastructure maintenance, but it often requires extensive manual labeling of training samples, which is both time-consuming and labor-intensive. To address this challenge, this paper proposes a semi-supervised learning approach based on a DenseNet classification model to detect pavement cracks more efficiently. The primary objective is to leverage a small set of labeled samples to improve the model's performance by incorporating a large number of unlabeled samples through semi-supervised learning. This method enhances the DenseNet model's ability to generalize by iteratively learning from new unlabeled datasets. As a result, the proposed approach not only reduces the need for extensive manual labeling but also mitigates issues related to label inconsistency and errors in the original labels. The experimental results demonstrate that the semi-supervised DenseNet model achieves a prediction precision of 96.77% and a recall of 94.17%, with an F1 score of 95.45% and an Intersectidn over Union (IoU) of 91.30%. These metrics highlight the model's high accuracy and effectiveness in crack detection. The proposed method not only improves label quality and model performance but also offers practical value for engineering applications in the field of pavement maintenance, making it a valuable tool for infrastructure management.

Semi-supervised lung adenocarcinoma histopathology image classification based on multi-teacher knowledge distillation.

In this study, we propose a semi-supervised learning scheme using a patch-based deep learning framework to tackle the challenge of high-precision classification of seven lung tumor growth patterns, despite having a small amount of labeled data in whole slide images (WSIs). This scheme aims to enhance generalization ability with limited data and reduce dependence on large amounts of labeled data. It effectively addresses the common challenge of high demand for labeled data in medical image analysis.

Approach. To address these challenges, the study employs a semi-supervised learning approach enhanced by a dynamic confidence threshold mechanism. This mechanism adjusts based on the quantity and quality of pseudo labels generated. This dynamic thresholding mechanism helps avoid the imbalance of pseudo-label categories and the low number of pseudo-labels that may result from a higher fixed threshold. Furthermore, the research introduces a multi-teacher knowledge distillation technique. This technique adaptively weights predictions from multiple teacher models to transfer reliable knowledge and safeguard student models from low-quality teacher predictions.

Main results. The framework underwent rigorous training and evaluation using a dataset of 150 WSIs, each representing one of the seven growth patterns. The experimental results demonstrate that the framework is highly accurate in classifying lung tumor growth patterns in histopathology images. Notably, the performance of the framework is comparable to that of fully supervised models and human pathologists. In addition, the framework's evaluation metrics on a publicly available dataset are higher than those of previous studies, indicating good generalizability.

Significance. This research demonstrates that a semi-supervised learning approach can achieve results comparable to fully supervised models and expert pathologists, thus opening new possibilities for efficient and cost-effective medical images analysis. The implementation of dynamic confidence thresholding and multi-teacher knowledge distillation techniques represents a significant advancement in applying deep learning to complex medical image analysis tasks. This advancement could lead to faster and more accurate diagnoses, ultimately improving patient outcomes and fostering the overall progress of healthcare technology.&#xD.

Semi-supervised abdominal multi-organ segmentation by object-redrawing.

Multi-organ segmentation is a critical task in medical imaging, with wide-ranging applications in both clinical practice and research. Accurate delineation of organs from high-resolution 3D medical images, such as CT scans, is essential for radiation therapy planning, enhancing treatment outcomes, and minimizing radiation toxicity risks. Additionally, it plays a pivotal role in quantitative image analysis, supporting various medical research studies. Despite its significance, manual segmentation of multiple organs from 3D images is labor-intensive and prone to low reproducibility due to high interoperator variability. Recent advancements in deep learning have led to several automated segmentation methods, yet many rely heavily on labeled data and human anatomy expertise. In this study, our primary objective is to address the limitations of existing semi-supervised learning (SSL) methods for abdominal multi-organ segmentation. We aim to introduce a novel SSL approach that leverages unlabeled data to enhance the performance of deep neural networks in segmenting abdominal organs. Specifically, we propose a method that incorporates a redrawing network into the segmentation process to correct errors and improve accuracy. Our proposed method comprises three interconnected neural networks: a segmentation network for image segmentation, a teacher network for consistency regularization, and a redrawing network for object redrawing. During training, the segmentation network undergoes two rounds of optimization: basic training and readjustment. We adopt the Mean-Teacher model as our baseline SSL approach, utilizing labeled and unlabeled data. However, recognizing significant errors in abdominal multi-organ segmentation using this method alone, we introduce the redrawing network to generate redrawn images based on CT scans, preserving original anatomical information. Our approach is grounded in the generative process hypothesis, encompassing segmentation, drawing, and assembling stages. Correct segmentation is crucial for generating accurate images. In the basic training phase, the segmentation network is trained using both labeled and unlabeled data, incorporating consistency learning to ensure consistent predictions before and after perturbations. The readjustment phase focuses on reducing segmentation errors by optimizing the segmentation network parameters based on the differences between redrawn and original CT images. We evaluated our method using two publicly available datasets: the beyond the cranial vault (BTCV) segmentation dataset (training: 44, validation: 6) and the abdominal multi-organ segmentation (AMOS) challenge 2022 dataset (training:138, validation:16). Our results were compared with state-of-the-art SSL methods, including MT and dual-task consistency (DTC), using the Dice similarity coefficient (DSC) as an accuracy metric. On both datasets, our proposed SSL method consistently outperformed other methods, including supervised learning, achieving superior segmentation performance for various abdominal organs. These findings demonstrate the effectiveness of our approach, even with a limited number of labeled data. Our novel semi-supervised learning approach for abdominal multi-organ segmentation addresses the challenges associated with this task. By integrating a redrawing network and leveraging unlabeled data, we achieve remarkable improvements in accuracy. Our method demonstrates superior performance compared to existing SSL and supervised learning methods. This approach holds great promise in enhancing the precision and efficiency of multi-organ segmentation in medical imaging applications.

Artificial neural networks with better analysis reliability in data mining

If there are relatively few cases, semi-supervised learning approaches make advantage of a large amount of unlabeled data to assist develop a better classifier. To expand the labeled training set and update the classifier, a fundamental method is to select and label the unlabeled instances for which the current classifier has higher classification confidence. This approach is primarily used in two distinct semi-supervised learning paradigms: co-training and self-training. However, compared to self-labeled examples that would be tagged by a classifier, the real labeled instances will be more trustworthy. Incorrect label assignment to unlabeled occurrences might potentially compromise the classifier's accuracy in classification. This research presents a novel instance selection method based on actual labeled data. This will take into account the classifier's current performance on unlabeled data in addition to its performance on actual labeled data alone. This uses the accuracy changes in the newly trained classifier over the original labeled data as a criterion in each iteration to determine whether or not the selected most confident unlabeled examples would be accepted by a subsequent iteration. Naïve Bayes (NB) will be used as the basic classifier in the co-training and self-training studies. The findings indicate that the accuracy and categorization of self-training and co-training will be greatly enhanced by SIS. As compared to semi-supervised classification methods, it will enhance accuracy, precision, recall, and F1 score, according to the findings.

Semi-supervised Triply Robust Inductive Transfer Learning

In this work, we propose a Semi-supervised Triply Robust Inductive transFer LEarning (STRIFLE) approach, which integrates heterogeneous data from a label-rich source population and a label-scarce target population and utilizes a large amount of unlabeled data simultaneously to improve the learning accuracy in the target population. Specifically, we consider a high dimensional covariate shift setting and employ two nuisance models, a density ratio model and an imputation model, to combine transfer learning and surrogate-assisted semi-supervised learning strategies organically and achieve triple robustness. While the STRIFLE approach assumes the target and source populations to share the same conditional distribution of outcome Y given both the surrogate features S and predictors X , it allows the true underlying model of Y � X to differ between the two populations due to the potential covariate shift in S and X . Different from double robustness, even if both nuisance models are misspecified or the distribution of Y � S , X is not the same between the two populations when the shifted source population and the target population share enough similarities, the triply robust STRIFLE estimator can still partially utilize the source population, and it is guaranteed to be no worse than the target-only surrogate-assisted semi-supervised estimator with an additional error term from transferability detection. These desirable properties of our estimator are established theoretically and verified in finite samples via extensive simulation studies. We utilize the STRIFLE estimator to train a Type II diabetes polygenic risk prediction model for the African American target population by transferring knowledge from electronic health records linked genomic data observed in a larger European source population.

Finding core labels for maximizing generalization of graph neural networks

Graph neural networks (GNNs) have become a popular approach for semi-supervised graph representation learning. GNNs research has generally focused on improving methodological details, whereas less attention has been paid to exploring the importance of labeling the data. However, for semi-supervised learning, the quality of training data is vital. In this paper, we first introduce and elaborate on the problem of training data selection for GNNs. More specifically, focusing on node classification, we aim to select representative nodes from a graph used to train GNNs to achieve the best performance. To solve this problem, we are inspired by the popular lottery ticket hypothesis, typically used for sparse architectures, and we propose the following subset hypothesis for graph data: “There exists a core subset when selecting a fixed-size dataset from the dense training dataset, that can represent the properties of the dataset, and GNNs trained on this core subset can achieve a better graph representation”. Equipped with this subset hypothesis, we present an efficient algorithm to identify the core data in the graph for GNNs. Extensive experiments demonstrate that the selected data (as a training set) can obtain performance improvements across various datasets and GNNs architectures.

A topological approach for semi-supervised learning

Nowadays, Machine Learning and Deep Learning methods have become the state-of-the-art approach to solve data classification tasks. In order to use those methods, it is necessary to acquire and label a considerable amount of data; however, this is not straightforward in some fields, since data annotation is time consuming and might require expert knowledge. This challenge can be tackled by means of semi-supervised learning methods that take advantage of both labelled and unlabelled data. In this work, we present new semi-supervised learning methods based on techniques from Topological Data Analysis (TDA). In particular, we have created two semi-supervised learning methods following two topological approaches. In the former, we have used a homological approach that consists in studying the persistence diagrams associated with the data using the bottleneck and Wasserstein distances. In the latter, we have considered the connectivity of the data. In addition, we have carried out a thorough analysis of the developed methods using 9 tabular datasets with low and high dimensionality. The results show that the developed semi-supervised methods outperform the results obtained with models trained with only manually labelled data, and are an alternative to other classical semi-supervised learning algorithms.

Optimization of semi-supervised generative adversarial network models: a survey

PurposeWith the development of intelligent technology, deep learning has made significant progress and has been widely used in various fields. Deep learning is data-driven, and its training process requires a large amount of data to improve model performance. However, labeled data is expensive and not readily available.Design/methodology/approachTo address the above problem, researchers have integrated semi-supervised and deep learning, using a limited number of labeled data and many unlabeled data to train models. In this paper, Generative Adversarial Networks (GANs) are analyzed as an entry point. Firstly, we discuss the current research on GANs in image super-resolution applications, including supervised, unsupervised, and semi-supervised learning approaches. Secondly, based on semi-supervised learning, different optimization methods are introduced as an example of image classification. Eventually, experimental comparisons and analyses of existing semi-supervised optimization methods based on GANs will be performed.FindingsFollowing the analysis of the selected studies, we summarize the problems that existed during the research process and propose future research directions.Originality/valueThis paper reviews and analyzes research on generative adversarial networks for image super-resolution and classification from various learning approaches. The comparative analysis of experimental results on current semi-supervised GAN optimizations is performed to provide a reference for further research.