Semi-supervised Learning Method Research Articles

Semi-supervised machine learning for motor eccentricity fault diagnosis

Eccentricity is one major indicator of mechanical faults in electric machines and needs to be detected early to avoid machine failures. Data-driven techniques based on machine learning and deep learning algorithms have been proposed in recent years for motor fault detection. However, majority of these methods use supervised learning algorithms and require large, labelled datasets, which can be challenging to obtain. In this paper, we propose a semi-supervised learning method based on a deep generative model using variational auto-encoder for eccentricity fault quantification. Good prediction accuracy can be achieved when only a small subset of training data has labels.

Semi-supervised cross-lingual speech emotion recognition

Performance in Speech Emotion Recognition (SER) on a single language has increased greatly in the last few years thanks to the use of deep learning techniques. However, cross-lingual SER remains a challenge in real-world applications due to two main factors: the first is the big gap among the source and the target domain distributions; the second factor is the major availability of unlabeled utterances in contrast to the labeled ones for the new language. Taking into account previous aspects, we propose a Semi-Supervised Learning (SSL) method for cross-lingual emotion recognition when only few labeled examples in the target domain (i.e. the new language) are available. Our method is based on a Transformer and it adapts to the new domain by exploiting a pseudo-labeling strategy on the unlabeled utterances. In particular, the use of a hard and soft pseudo-labels approach is investigated. We thoroughly evaluate the performance of the proposed method in a speaker-independent setup on both the source and the new language and show its robustness across five languages belonging to different linguistic strains. The experimental findings indicate that the unweighted accuracy is increased by an average of 40% compared to state-of-the-art methods.

An uncertainty-aware self-training framework with consistency regularization for the multilabel classification of common computed tomography signs in lung nodules.

Computed tomography (CT) signs of lung nodules play an important role in indicating lung nodules' malignancy, and accurate automatic classification of these signs can help doctors improve their diagnostic efficiency. However, few relevant studies targeting multilabel classification (MLC) of nodule signs have been conducted. Moreover, difficulty in obtaining labeled data also restricts this avenue of research to a large extent. To address these problems, a multilabel automatic classification system for nodule signs is proposed, which consists of a 3-dimensional (3D) convolutional neural network (CNN) and an efficient new semi-supervised learning (SSL) framework. Two datasets were used in our experiments: Lung Nodule Analysis 16 (LUNA16), a public dataset for lung nodule classification, and a private dataset of nodule signs. The private dataset contains 641 nodules, 454 of which were annotated with 6 important signs by radiologists. Our classification system consists of 2 main parts: a 3D CNN model and an SSL method called uncertainty-aware self-training framework with consistency regularization (USC). In the system, supervised training is performed with labeled data, and simultaneously, an uncertainty-and-confidence-based strategy is used to select pseudo-labeled samples for unsupervised training, thus jointly realizing the optimization of the model. For the MLC of nodule signs, our proposed 3D CNN achieved satisfactory results with a mean average precision (mAP) of 0.870 and a mean area under the curve (AUC) of 0.782. In semi-supervised experiments, compared with supervised learning, our proposed SSL method could increase the mAP by 7.6% (from 0.730 to 0.806) and the mean AUC by 8.1% (from 0.631 to 0.712); it thus efficiently utilized the unlabeled data and achieved superior performance improvement compared to the recently advanced methods. We realized the optimal MLC of lung nodule signs with our proposed 3D CNN. Our proposed SSL method can also offer an efficient solution for the insufficiency of labeled data that may exist in the MLC tasks of 3D medical images.

A Survey on Deep Semi-Supervised Learning

Deep semi-supervised learning is a fast-growing field with a range of practical applications. This paper provides a comprehensive survey on both fundamentals and recent advances in deep semi-supervised learning methods from perspectives of model design and unsupervised loss functions. We first present a taxonomy for deep semi-supervised learning that categorizes existing methods, including deep generative methods, consistency regularization methods, graph-based methods, pseudo-labeling methods, and hybrid methods. Then we provide a comprehensive review of 60 representative methods and offer a detailed comparison of these methods in terms of the type of losses, architecture differences, and test performance results. In addition to the progress in the past few years, we further discuss some shortcomings of existing methods and provide some tentative heuristic solutions for solving these open problems.

ProMatch: Semi-Supervised Learning with Prototype Consistency

Recent state-of-the-art semi-supervised learning (SSL) methods have made significant advancements by combining consistency-regularization and pseudo-labeling in a joint learning paradigm. The core concept of these methods is to identify consistency targets (pseudo-labels) by selecting predicted distributions with high confidence from weakly augmented unlabeled samples. However, they often face the problem of erroneous high confident pseudo-labels, which can lead to noisy training. This issue arises due to two main reasons: (1) when the model is poorly calibrated, the prediction of a single sample may be overconfident and incorrect, and (2) propagating pseudo-labels from unlabeled samples can result in error accumulation due to the margin between the pseudo-label and the ground-truth label. To address this problem, we propose a novel consistency criterion called Prototype Consistency (PC) to improve the reliability of pseudo-labeling by leveraging the prototype similarities between labeled and unlabeled samples. First, we instantiate semantic-prototypes (centers of embeddings) and prediction-prototypes (centers of predictions) for each category using memory buffers that store the features of labeled examples. Second, for a given unlabeled sample, we determine the most similar semantic-prototype and prediction-prototype by assessing the similarities between the features of the unlabeled sample and the prototypes of the labeled samples. Finally, instead of using the prediction of the unlabeled sample as the pseudo-label, we select the most similar prediction-prototype as the consistency target, as long as the predicted category of the most similar prediction-prototype, the ground-truth category of the most similar semantic-prototype, and the ground-truth category of the most similar prediction-prototype are equivalent. By combining the PC approach with the techniques developed by the MixMatch family, our proposed ProMatch framework demonstrates significant performance improvements compared to previous algorithms on datasets such as CIFAR-10, CIFAR-100, SVHN, and Mini-ImageNet.

Uncertainty-aware and dynamically-mixed pseudo-labels for semi-supervised defect segmentation

Deep learning-based defect segmentation is one of the important tasks of machine vision in automated inspection. Supervised learning methods have recently achieved remarkable performance on this task. However, the effectiveness of the supervised methods is limited by the scarcity and high cost of pixel-level annotation of training data. Semi-supervised learning methods have been proposed for training deep learning networks using a limited amount of labeled data along with additional unlabeled data for image segmentation. Most of these methods are based on consistency regularization and pseudo labeling, where the predictions on unlabeled samples often come with noise and are unreliable, resulting in poor segmentation performance. To alleviate this problem, we propose uncertainty-aware pseudo labels, which are generated from dynamically mixed predictions of multiple decoders that leverage a shared encoder network. The estimated uncertainty guides the pseudo-label-based supervision and regularizes the training when using the unlabeled samples. In our experiments on four public datasets for defect segmentation, the proposed method outperformed the fully supervised baseline and six state-of-the-art semi-supervised segmentation methods. We also conducted an extensive ablation study to demonstrate the effectiveness of our approach in various settings. The implementation code for this work is available at https://github.com/djene-mengistu/UAPS.

Can Glaucoma Suspect Data Help to Improve the Performance of Glaucoma Diagnosis?

The presence of imbalanced datasets in medical applications can negatively affect deep learning methods. This study aims to investigate how the performance of convolutional neural networks (CNNs) for glaucoma diagnosis can be improved by addressing imbalanced learning issues through utilizing glaucoma suspect samples, which are often excluded from studies because they are a mixture of healthy and preperimetric glaucomatous eyes, in a semi-supervised learning approach. A baseline 3D CNN was developed and trained on a real-world glaucoma dataset, which is naturally imbalanced (like many other real-world medical datasets). Then, three methods, including reweighting samples, data resampling to form balanced batches, and semi-supervised learning on glaucoma suspect data were applied to practically assess their impacts on the performances of the trained methods. The proposed method achieved a mean accuracy of 95.24%, an F1 score of 97.42%, and an area under the curve of receiver operating characteristic (AUC ROC) of 95.64%, whereas the corresponding results for the traditional supervised training using weighted cross-entropy loss were 92.88%, 96.12%, and 92.72%, respectively. The obtained results show statistically significant improvements in all metrics. Exploiting glaucoma suspect eyes in a semi-supervised learning method coupled with resampling can improve glaucoma diagnosis performance by mitigating imbalanced learning issues. Clinical imbalanced datasets may negatively affect medical applications of deep learning. Utilizing data with uncertain diagnosis, such as glaucoma suspects, through a combination of semi-supervised learning and class-imbalanced learning strategies can partially address the problems of having limited data and learning on imbalanced datasets.

A semi-supervised algorithm for improving the consistency of crowdsourced datasets: The COVID-19 case study on respiratory disorder classification

Background and Objective: Cough audio signal classification is a potentially useful tool in screening for respiratory disorders, such as COVID-19. Since it is dangerous to collect data from patients with contagious diseases, many research teams have turned to crowdsourcing to quickly gather cough sound data. The COUGHVID dataset enlisted expert physicians to annotate and diagnose the underlying diseases present in a limited number of recordings. However, this approach suffers from potential cough mislabeling, as well as disagreement between experts.Methods: In this work, we use a semi-supervised learning (SSL) approach – based on audio signal processing tools and interpretable machine learning models – to improve the labeling consistency of the COUGHVID dataset for 1) COVID-19 versus healthy cough sound classification 2) distinguishing wet from dry coughs, and 3) assessing cough severity. First, we leverage SSL expert knowledge aggregation techniques to overcome the labeling inconsistencies and label sparsity in the dataset. Next, our SSL approach is used to identify a subsample of re-labeled COUGHVID audio samples that can be used to train or augment future cough classifiers.Results: The consistency of the re-labeled COVID-19 and healthy data is demonstrated in that it exhibits a high degree of inter-class feature separability: 3x higher than that of the user-labeled data. Similarly, the SSL method increases this separability by 11.3x for cough type and 5.1x for severity classifications. Furthermore, the spectral differences in the user-labeled audio segments are amplified in the re-labeled data, resulting in significantly different power spectral densities between healthy and COVID-19 coughs in the 1-1.5 kHz range (p=1.2×10−64), which demonstrates both the increased consistency of the new dataset and its explainability from an acoustic perspective. Finally, we demonstrate how the re-labeled dataset can be used to train a COVID-19 classifier, achieving an AUC of 0.797.Conclusions: We propose a SSL expert knowledge aggregation technique for the field of cough sound classification for the first time, and demonstrate how it can be used to combine the medical knowledge of multiple experts in an explainable fashion, thus providing abundant, consistent data for cough classification tasks.

A semi-supervised learning-based quality evaluation system for digital chest radiographs.

Digital radiography is the most commonly utilized medical imaging technique worldwide, and the quality of radiographs plays a crucial role in accurate disease diagnosis. Therefore, evaluating the quality of radiographs is an essential step in medical examinations. However, manual evaluation can be time-consuming, labor-intensive, and prone to interobserver differences, making it less reliable. To alleviate the workload of radiographic technologists and enhance the efficiency of radiograph quality evaluation, it is crucial to develop rapid and reliable quality evaluation methods and establish a set of quantitative evaluation standards. To address this, we have proposed a quality evaluation system for digital radiographs that utilizes deep learning techniques to achieve fast and precise evaluation. The evaluation of frontal chest radiograph quality involves assessing patient positioning through semantic segmentation and foreign body detection. For lung, scapula, and clavicle segmentation in digital chest radiographs, a residual connection-based convolutional neural network π-ResUNet, was proposed. Criteria for patient positioning evaluation were established based on the segmentation and manual evaluation results. A convolutional neural network, FasterRCNN, was utilized to detect and localize foreign bodies in digital chest radiographs. To enhance the performance of both neural networks, a semi-supervised learning (SSL) strategy was implemented by incorporating a consistency loss that leverages a large number of unlabeled digital radiographs. We also trained the network using the fully supervised learning (FSL) strategy and compared their performance on the test set. The ChestXRay-14 and object-CXR datasets were used throughout the process. By comparing with the manual annotation, the proposed network, trained using the SSL method, achieved a high Dice similarity coefficient (DSC) of 0.96, 0.88, and 0.88 for lung, scapula, and clavicle segmentation, respectively, outperforming the network trained with the FSL method. In addition, for foreign body detection, the proposed SSL method was superior to the FSL method, achieving an AUC (Area under receiver operating characteristic curve, Area under ROC curve) of 0.90 and an FROC (Free-response ROC) of 0.77 on the test dataset. The experimental results show that our proposed system is well-suited for radiograph quality evaluation, with the semi-supervised learning method further improving the network's performance. The proposed method can evaluate the quality of a chest radiograph from two aspects-patient positioning and foreign body detection-within 1 s, offering a promising tool in radiograph quality evaluation.

Efficient Information-Theoretic Large-Scale Semi-Supervised Metric Learning via Proxies

Semi-supervised metric learning intends to learn a distance function from the limited labeled data as well as a large amount of unlabeled data to better gauge the similarities of any two instances than using a general distance function. However, most existing semi-supervised metric learning methods rely on the manifold assumptions to mine the rich discriminant information of the unlabeled data, which breaks the intrinsic connection between the manifold regularizer-building process and the subsequent metric learning. Moreover, these methods usually encounter high computational or memory overhead. To solve these issues, we develop a novel method entitled Information-Theoretic Large-Scale Semi-Supervised Metric Learning via Proxies (ISMLP). ISMLP aims to simultaneously learn multiple proxy vectors as well as a Mahalanobis matrix and forms the semi-supervised metric learning as the probability distribution optimization parameterized by the Mahalanobis distance between the instance and each proxy vector. ISMLP maximizes the entropy of the labeled data and minimizes that of the unlabeled data to follow the entropy regularization, in this way, the labeled part and unlabeled part can be integrated in a meaningful way. Furthermore, the time complexity of the proposed method has a linear dependency concerning the number of instances, thereby, can be extended to the large-scale dataset without incurring too much time. Experiments on multiple datasets demonstrate the superiority of the proposed method over the compared methods used in the experiments.

Deep Bregman divergence for self-supervised representations learning

Neural Bregman divergence measures the divergence of data points using convex neural networks, which is beyond Euclidean distance and capable of capturing divergence over distributions. The non-Euclidean geometry is not well explored in deep representation learning and remains a challenging endeavor for self-supervised representation learning. In this paper, we propose deep Bregman divergences for self-supervised pretext task learning, where we aim to enhance self-supervised embedding representation by training additional networks based on functional Bregman divergences. Our framework can capture the divergence of embedding distributions and improve the quality of learned representation using an arbitrary Bregman divergence over data embedding. Specifically, we develop a novel self-supervised architecture and a new divergence loss that measures the asymmetric distance of arbitrary Bergman divergences of neural networks. We show that the combination of self-supervised contrastive learning and our proposed method outperforms the baseline as well as most established methods for self-supervised and semi-supervised learning on multiple classifications and object detection tasks and datasets. Moreover, the learned representations generalize well when transferred to other datasets and tasks.

High-order features of a single linear corneal laceration image are valuable biomarkers in an intelligent multimodal analytic strategy for corneal laceration reconstruction

BackgroundIn corneal lacerations, the absence of high-order image features as biomarkers to guide surgical strategy is a limiting factor. The absence of multimodal data restricts the development of automated reconstruction designs for corneal laceration. The present study is aimed at training and optimizing the model based on high-order features from corneal laceration images and real suture samples and completing the intelligent promotion of whole corneal laceration suture auxiliary decision-making with the two-step method of automatic wound identification and stitch position prediction. MethodsBased on the images of isolated corneal wound samples, a fully supervised U-Net learning method and consistent regular semisupervised learning method based on the mean-teacher model were used to identify the wounds. The DDice coefficient was used to evaluate the segmentation and recognition effect. Traditional image processing technology was used to predict the needle entry and exit points of wound sutures based on medical suture principles. The prediction effect was evaluated by viewpoint similarity. ResultsAfter training the wound recognition model based on 2400 corneal images and corresponding incision labels, the DDice coefficients of supervised U-Net with or without postprocessing results were 0.902 and 0.817, respectively. The Dice coefficients of the semisupervisedmean-teacher model with or without postprocessing were 0.921 and 0.843, respectively. The key point similarity of wound stitch position prediction was 0.872 ± 0.021. ConclusionThis new automated method for corneal laceration identification and stitch position generation based on novel biomarkers and multimodal data is expected to assist doctors treating corneal lacerations to quickly formulate a primary suturing strategy.

Coarse-Refined Consistency Learning Using Pixel-Level Features for Semi-Supervised Medical Image Segmentation.

Pixel-level annotations are extremely expensive for medical image segmentation tasks as both expertise and time are needed to generate accurate annotations. Semi-supervised learning (SSL) for medical image segmentation has recently attracted growing attention because it can alleviate the exhausting manual annotations for clinicians by leveraging unlabeled data. However, most of the existing SSL methods do not take pixel-level information (e.g., pixel-level features) of labeled data into account, i.e., the labeled data are underutilized. Hence, in this work, an innovative Coarse-Refined Network with pixel-wise Intra-patch ranked loss and patch-wise Inter-patch ranked loss (CRII-Net) is proposed. It provides three advantages: i) it can produce stable targets for unlabeled data, as a simple yet effective coarse-refined consistency constraint is designed; ii) it is very effective for the extreme case where very scarce labeled data are available, as the pixel-level and patch-level features are extracted by our CRII-Net; and iii) it can output fine-grained segmentation results for hard regions (e.g., blurred object boundaries and low-contrast lesions), as the proposed Intra-Patch Ranked Loss (Intra-PRL) focuses on object boundaries and Inter-Patch Ranked loss (Inter-PRL) mitigates the adverse impact of low-contrast lesions. Experimental results on two common SSL tasks for medical image segmentation demonstrate the superiority of our CRII-Net. Specifically, when there are only 4% labeled data, our CRII-Net improves the Dice similarity coefficient (DSC) score by at least 7.49% when compared to five classical or state-of-the-art (SOTA) SSL methods. For hard samples/regions, our CRII-Net also significantly outperforms other compared methods in both quantitative and visualization results.

SS-TBN: A Semi-Supervised Tri-Branch Network for COVID-19 Screening and Lesion Segmentation.

Insufficient annotated data and minor lung lesions pose big challenges for computed tomography (CT)-aided automatic COVID-19 diagnosis at an early outbreak stage. To address this issue, we propose a Semi-Supervised Tri-Branch Network (SS-TBN). First, we develop a joint TBN model for dual-task application scenarios of image segmentation and classification such as CT-based COVID-19 diagnosis, in which pixel-level lesion segmentation and slice-level infection classification branches are simultaneously trained via lesion attention, and individual-level diagnosis branch aggregates slice-level outputs for COVID-19 screening. Second, we propose a novel hybrid semi-supervised learning method to make full use of unlabeled data, combining a new double-threshold pseudo labeling method specifically designed to the joint model and a new inter-slice consistency regularization method specifically tailored to CT images. Besides two publicly available external datasets, we collect internal and our own external datasets including 210,395 images (1,420 cases vs. 498 controls) from ten hospitals. Experimental results show that the proposed method achieves state-of-the-art performance in COVID-19 classification with limited annotated data even if lesions are subtle, and that segmentation results promote interpretability for diagnosis, suggesting the potential of the SS-TBN in early screening in insufficient labeled data situations at the early stage of a pandemic outbreak like COVID-19.

A Universal Feature Extractor Based on Self-Supervised Pre-Training for Fault Diagnosis of Rotating Machinery under Limited Data

To address the limited data problem in real-world fault diagnosis, previous studies have primarily focused on semi-supervised learning and transfer learning methods. However, these approaches often struggle to obtain the necessary data, failing to fully leverage the potential of easily obtainable unlabeled data from other devices. In light of this, this paper proposes a novel network architecture, named Signal Bootstrap Your Own Latent (SBYOL), which utilizes unlabeled vibration signals to address the challenging issues of variable working conditions, strong noise, and limited data in rotating machinery fault diagnosis. The architecture consists of a self-supervised pre-training-based fault feature recognition network and a diagnosis network based on knowledge transfer. The fault feature recognition network uses ResNet-18 as the backbone network for self-supervised pre-training and transfers the trained fault feature extractor to the target diagnostic object. Additionally, a unique vibration signal data augmentation technique, time–frequency signal transformation (TFST), is proposed specifically for rotating machinery fault diagnosis, which addresses the key task of contrastive learning and achieves high-precision fault diagnosis with very few labeled samples. Experimental results demonstrate that the proposed diagnostic model outperforms other methods in both extremely limited sample and strong noise scenarios and can transfer unlabeled data utilization between similar and even different device types.

Detection of Intermittent Oscillation in Process Control Loops with Semi-Supervised Learning

Oscillations in the control loops indicate the poor performance of the control loops. The occurrence of oscillations in the process control loop is quite high in the industry, so it needs to be reduced so that the control loop can work properly. The first step for oscillation reduction is oscillation detection. One type of oscillation that is difficult to detect is intermittent oscillation. The smart factory concept encourages the development of the intermittent oscillation detection system using machine learning by being implemented online. Therefore, in this study an online intermittent oscillation detection program is built using K-nearest neighbor (KNN)-based Semi-supervised learning (SSL) method. The SSL method applied is self-training. The training data was obtained by a simulation of the Tennessee Eastman Process. The data is segmented based on window size and extracted time series features. The extracted data is used to build a model to detect oscillations caused by stiction, tuning errors, and external disturbances in the reactor. The model is implemented online with sliding windows and MQTT. The best accuracy and F1-score of the model obtained are 96.15% and 95.15%. In online detection, the model detects the type of oscillation with an average time of 305 seconds.

Semisupervised Learning with Report-guided Pseudo Labels for Deep Learning-based Prostate Cancer Detection Using Biparametric MRI.

To evaluate a novel method of semisupervised learning (SSL) guided by automated sparse information from diagnostic reports to leverage additional data for deep learning-based malignancy detection in patients with clinically significant prostate cancer. This retrospective study included 7756 prostate MRI examinations (6380 patients) performed between January 2014 and December 2020 for model development. An SSL method, report-guided SSL (RG-SSL), was developed for detection of clinically significant prostate cancer using biparametric MRI. RG-SSL, supervised learning (SL), and state-of-the-art SSL methods were trained using 100, 300, 1000, or 3050 manually annotated examinations. Performance on detection of clinically significant prostate cancer by RG-SSL, SL, and SSL was compared on 300 unseen examinations from an external center with a histopathologically confirmed reference standard. Performance was evaluated using receiver operating characteristic (ROC) and free-response ROC analysis. P values for performance differences were generated with a permutation test. At 100 manually annotated examinations, mean examination-based diagnostic area under the ROC curve (AUC) values for RG-SSL, SL, and the best SSL were 0.86 ± 0.01 (SD), 0.78 ± 0.03, and 0.81 ± 0.02, respectively. Lesion-based detection partial AUCs were 0.62 ± 0.02, 0.44 ± 0.04, and 0.48 ± 0.09, respectively. Examination-based performance of SL with 3050 examinations was matched by RG-SSL with 169 manually annotated examinations, thus requiring 14 times fewer annotations. Lesion-based performance was matched with 431 manually annotated examinations, requiring six times fewer annotations. RG-SSL outperformed SSL in clinically significant prostate cancer detection and achieved performance similar to SL even at very low annotation budgets.Keywords: Annotation Efficiency, Computer-aided Detection and Diagnosis, MRI, Prostate Cancer, Semisupervised Deep Learning Supplemental material is available for this article. Published under a CC BY 4.0 license.

Semi-supervised transfer learning with hierarchical self-regularization

Both semi-supervised learning and transfer learning aim at lowering the annotation burden for training models. However, such two tasks are usually studied separately, i.e. most semi-supervised learning algorithms train models from scratch while transfer learning assumes pre-trained models as the initialization. In this work, we focus on a previously-less-concerned setting that further reduces the annotation efforts through incorporating both semi-supervised and transfer learning, where specifically a pre-trained source model is used as the initialization of semi-supervised learning. As those powerful pre-trained models are ubiquitously available nowadays and can considerably benefit various down-streaming tasks, such a setting is relevant to real-world applications yet challenging to design effective algorithms.Aiming at enabling transfer learning under semi-supervised settings, we propose a hierarchical self-regularization mechanism to exploit unlabeled samples more effectively, where a novel self-regularizer has been introduced to incorporate both individual-level and population-level regularization terms. The former term employs self-distillation to regularize learned deep features for each individual sample, and the latter one enforces self-consistency on feature distributions between labeled and unlabeled samples. Samples involved in both regularizers are weighted by an adaptive strategy, where self-regularization effects of both terms are adaptively controlled by the confidence of every sample. To validate our algorithm, exhaustive experiments have been conducted on diverse datasets such as CIFAR-10 for general object recognition, CUB-200-2011/MIT-indoor-67 for fine-grained classification and MURA for medical image classification. Compared with state-of-the-art semi-supervised learning methods including Pseudo Label, Mean Teacher, MixMatch and FixMatch, our algorithm demonstrates two advantages: first of all, the proposed approach adopts a new point of view to tackle problems caused by inadequate supervision and achieves very competitive results; then, it is complementary to these state-of-the-art methods and thus can be combined with them to get additional improvements. Furthermore, our method can also be applied to fully supervised transfer learning and self-supervised learning. We have published our code at https://github.com/SHI-Labs/Semi-Supervised-Transfer-Learning.

Predicting pathological complete response based on weakly and semi-supervised joint learning from breast cancer MRI.

Neoadjuvant chemotherapy (NAC) is the standard treatment for breast cancer patients. Patients achieving complete pathological response (pCR) after NAC usually have a good prognosis. However, automatic pCR prediction has been a challenging problem due to lacking well annotations in 3D MRI. Thus far, unifying different annotation information to predict the tumor's early response to NAC has not been systematically addressed. This paper proposes a weakly and semi-supervised joint learning method that integrates attentional features from multi-parametric MRI with radiomic features for predicting pCR to NAC in breast cancer patients. The attention-based multi-instance learning (MIL) is first developed to generate informative MRI bag-level features and mine key instances. The mean-teacher framework is then employed to segment tumor regions in a semi-supervised setting for extracting radiomic features. We perform experiments on 442 patients' data and show that our method achieves an AUC value of 0.85 in pCR prediction, which is superior to comparative methods. It is also shown that learning from multi-parametric MRI outperforms that of single-parameter MRI in pCR prediction.

Safe co-training for semi-supervised regression

Co-training is a popular semi-supervised learning method. The learners exchange pseudo-labels obtained from different views to reduce the accumulation of errors. One of the key issues is how to ensure the quality of pseudo-labels. However, the pseudo-labels obtained during the co-training process may be inaccurate. In this paper, we propose a safe co-training (SaCo) algorithm for regression with two new characteristics. First, the safe labeling technique obtains pseudo-labels that are certified by both views to ensure their reliability. It differs from popular techniques of using two views to assign pseudo-labels to each other. Second, the label dynamic adjustment strategy updates the previous pseudo-labels to keep them up-to-date. These pseudo-labels are predicted using the augmented training data. Experiments are conducted on twelve datasets commonly used for regression testing. Results show that SaCo is superior to other co-training style regression algorithms and state-of-the-art semi-supervised regression algorithms.