Semi-supervised Learning Method Research Articles

Uncertainty-guided cross learning via CNN and transformer for semi-supervised honeycomb lung lesion segmentation

Objective. Deep learning networks such as convolutional neural networks (CNN) and Transformer have shown excellent performance on the task of medical image segmentation, however, the usual problem with medical images is the lack of large-scale, high-quality pixel-level annotations, which is a very time-consuming and laborious task, and its further leads to compromised the performance of medical image segmentation under limited annotation conditions. Approach. In this paper, we propose a new semi-supervised learning method, uncertainty-guided cross learning, which uses a limited number of annotated samples along with a large number of unlabeled images to train the network. Specifically, we use two networks with different learning paradigms, CNN and Transformer, for cross learning, and use the prediction of one of them as a pseudo label to supervise the other, so that they can learn from each other, fully extract the local and global features of the images, and combine explicit and implicit consistency regularization constraints with pseudo label methods. On the other hand, we use epistemic uncertainty as a guiding message to encourage the model to learn high-certainty pixel information in high-confidence regions, and minimize the impact of erroneous pseudo labels on the overall learning process to improve the performance of semi-supervised segmentation methods. Main results. We conducted honeycomb lung lesion segmentation experiments using a honeycomb lung CT image dataset, and designed several sets of comparison experiments and ablation experiments to validate the effectiveness of our method. The final experimental results show that the Dice coefficient of our proposed method reaches 88.49% on the test set, and our method achieves state-of-the-art performance in honeycomb lung lesion segmentation compared to other semi-supervised learning methods. Significance. Our proposed method can effectively improve the accuracy of segmentation of honeycomb lung lesions, which provides an important reference for physicians in the diagnosis and treatment of this disease.

A General Model for Aggregating Annotations Across Simple, Complex, and Multi-Object Annotation Tasks

Human annotations are vital to supervised learning, yet annotators often disagree on the correct label, especially as annotation tasks increase in complexity. A common strategy to improve label quality is to ask multiple annotators to label the same item and then aggregate their labels. To date, many aggregation models have been proposed for simple categorical or numerical annotation tasks, but far less work has considered more complex annotation tasks, such as those involving open-ended, multivariate, or structured responses. Similarly, while a variety of bespoke models have been proposed for specific tasks, our work is the first we are aware of to introduce aggregation methods that generalize across many, diverse complex tasks, including sequence labeling, translation, syntactic parsing, ranking, bounding boxes, and keypoints. This generality is achieved by applying readily available task-specific distance functions, then devising a task-agnostic method to model these distances between labels, rather than the labels themselves. This article presents a unified treatment of our prior work on complex annotation modeling and extends that work with investigation of three new research questions. First, how do complex annotation task and dataset properties impact aggregation accuracy? Second, how should a task owner navigate the many modeling choices in order to maximize aggregation accuracy? Finally, what tests and diagnoses can verify that aggregation models are specified correctly for the given data? To understand how various factors impact accuracy and to inform model selection, we conduct large-scale simulation studies and broad experiments on real, complex datasets. Regarding testing, we introduce the concept of unit tests for aggregation models and present a suite of such tests to ensure that a given model is not mis-specified and exhibits expected behavior. Beyond investigating these research questions above, we discuss the foundational concept and nature of annotation complexity, present a new aggregation model as a conceptual bridge between traditional models and our own, and contribute a new general semisupervised learning method for complex label aggregation that outperforms prior work.

CDGAN-BERT: Adversarial constraint and diversity discriminator for semi-supervised text classification

Semi-supervised generative adversarial learning (SS-GAN) significantly improves the model’s performance with limited labeled data, especially on text classification. However, these SS-GAN-like works pay more attention to the more real properties of the generated samples and ignore their adversarial nature, which may prevent the model from learning more higher-quality data representations. And the existing single discriminator may produce discrimination ambiguity when discriminating against tasks with different discriminative properties (simultaneously identifying real or fake and classifying categories), which may harm the model’s classification performance. In this paper, we propose a novel CDGAN-BERT, an SS-GAN-based architecture with the adversarial constraint and a diversity discriminator. Specifically, CDGAN-BERT first focuses on adversarial constraint, which calculates the model’s intrinsic state representation with the reciprocal of Mean Squared Error (MSE), further keeping adversarial of the generated samples relative to the real data distribution. Then, a diversity discriminator is designed to improve the model’s classification performance by alleviating discriminative ambiguity, which refines tasks with different discriminative attributes by adding an additional authenticity discriminator. We validate our model on 6 text datasets and achieve significant improvements, especially with limited supervision. The experimental results show that our method outperforms or gets comparable results to other state-of-the-art semi-supervised learning methods on several datasets. Especially for the datasets of QC-Fine and QC-Coarse with limited labeled data, our CDGAN-BERT achieves the best average Micro-F1 scores of 72.5% and 92.037%, respectively.

A domain knowledge powered hybrid regularization strategy for semi-supervised breast cancer diagnosis

Semi-supervised learning has attracted much attention in medical image analysis as annotating medical images is substantially difficult. Existing semi-supervised learning methods mainly utilize some regularization strategies to encourage a network to make similar predictions for each sample under various perturbations (e.g. rotating and flipping). However, as these strategies are originally designed for natural images, they do not fully utilize the specific attributes in the medical domain. In this paper, we propose DK-HRS, a domain knowledge-powered hybrid regularization strategy for semi-supervised breast cancer diagnosis. More specifically, DK-HRS generates four types of perturbed samples, namely, domain knowledge-augmented samples, weakly and strongly-augmented samples, and virtual adversarial samples. These perturbed samples represent transformations of medical images from different aspects. DK-HRS utilizes the FixMatch, a popular consistency regularization method as the backbone, and integrates two additional modules: a virtual adversarial training module, and a domain knowledge-based contrastive learning module. Experimental results on four breast ultrasound datasets demonstrate that, by incorporating medical domain knowledge, DK-HRS achieves superior performance and outperforms some state-of-the-art semi-supervised methods by a large margin.

A data extension framework of seismic-induced gravelly soil liquefaction based on semi-supervised methods

In studies of gravelly soil liquefaction discrimination, the number of training samples significantly affects model performance. However, historical earthquakes provide limited instances of gravelly soil liquefaction, resulting in no guarantee of the generalization performance of existing models. Additionally, valuable data from liquefiable critical layer points and unlabeled points outside the critical layer are often overlooked in stratigraphic exploration, despite their high value. Therefore, in this paper, a data extension method for semi-supervised learning based on the Markov chain Monte Carlo–Bayesian network (MCMC-BN) method is proposed and a semi-MCMC-BN gravelly soil liquefaction discrimination model is constructed. By effectively mining the available site liquefaction information, the proposed method increases the sample size of the gravelly soil liquefaction dataset by approximately 4 times, expanding 120 initial liquefaction samples to 476. The results demonstrate that the extended liquefaction database improves the distribution of liquefaction data for gravelly soils. The semi-supervised learning model outperforms the supervised learning model, highlighting the feasibility of the semi-supervised approach for gravelly soil liquefaction discrimination. In addition, debris flow data for the Wenchuan area, China, are used to validate the effectiveness of the proposed method. Finally, the impacts of the confidence level magnitude, critical soil layer selection, and different machine learning methods on the data extension outcomes are discussed. The expanded database can provide data support to studying gravelly soil liquefaction, and the proposed data expansion framework can be used for related research in other fields.

Predicting invasive species distributions using incremental ensemble-based pseudo-labeling

Control strategies for preventing the spread of invasive species require their accurate geographical distribution. Species distribution models (SDMs) that can predict potential habitats of invasive species and thereby derive habitat suitability maps have become a valuable tool for supporting regulatory strategies. To date, machine learning (ML)-based approaches have outperformed profile and statistical-based approaches in terms of species distribution prediction accuracy. However, ML-based approaches often suffer from poor predictive performance when there is insufficient labeled data. Recently, pseudo-labeling (PL), a semi-supervised learning method, has been proposed to alleviate this problem, but pseudo-labels generated using a single-teacher model with very few labeled data points are generally biased and not suitable for training SDMs. In this paper, we propose a novel prediction scheme for invasive species distributions using incremental ensemble-based PL (SDP-EPL). We first build an ensemble-based teacher using multiple conventional SDMs and then incrementally construct a training set, starting with very few labeled data points, by repeating the following process: (i) generating pseudo-labels for unlabeled data using the teacher model, (ii) appending the pseudo-labeled data representing high or low habitat suitability to the training set, and (iii) training the teacher model using the updated training set. We then train a student SDM using the training set. Based on extensive experiments using citizen science datasets for three species, we show that the proposed scheme outperforms other commonly used SDMs in terms of diverse evaluation metrics and achieves performance improvements of up to 14.61% and 5.45% compared to the baseline and state-of-the-art models, respectively. We also demonstrate the effectiveness of the ensemble teacher model and incremental labeling in terms of predictive accuracy.

Booster-Driven Consistency Training Framework for Semi-Supervised Learning

The paper proposes an inductive semi-supervised learning method, called Booster-Driven Consistency Training (BDCT). In our work, we extend consistency training by designing a “booster” module for aggregating multi-view information, preventing the networks from collapsing into each other, and constructing better targets in parallel with the existing consistency training method. By booster, we mean that the one network is treated as the booster to help the training of the other two co-trained and superior networks, “spacecraft”. Furthermore, BDCT integrates smoothness enforcing into the designed fast feedback loop to ensure the effectiveness and the robustness of training. Meanwhile, adversarial examples are exploited to maintain the diversity among the networks that are learned to be smooth on the low dimensional manifold. BDCT demonstrates satisfactory performance in comparison with state-of-the-art semi-supervised deep learning methods and extensive experiments validate the effectiveness of the “booster” module.

Optical Flow Estimation in Dense Foggy Scenes With Domain-Adaptive Networks

Estimating optical flow in dense foggy scenes is a challenging task. The basic assumptions for computing flow such as brightness and gradient constancy become invalid. To address the problem, we introduce a semi-supervised deep learning method that can learn from real fog images without requiring the corresponding optical flow ground-truths. Our method is a multi-task network, integrating the domain transformation and optical flow networks in one framework. The domain transformation is performed between foggy and clean domains. Under our semi-supervised training strategy, we train our method in a supervised manner with a pair of synthetic fog images, their corresponding clean images and optical flow ground-truths. Subsequently, given a pair of real fog images and a pair of clean images that are not corresponding to each other (unpaired), in the next training batch, we train our network in an unsupervised manner. The supervised and unsupervised training processes are alternated iteratively. Since our method relies on unsupervised learning for real data, we show that it can be used for test-time training. We show in our experiments that performing test-time training improves the results further on our test data. Extensive experiments show that our method outperforms the state-of-the-art methods in estimating optical flow in dense foggy scenes.

Abstract A112: A semi-supervised approach to classify atypical BRAF mutations to identify effective targeted therapies in colorectal cancer

Abstract Due to a minority of atypical BRAF alleles being classified as either activating and RAS-independent (class 2) or kinase-impaired and RAS-dependent (class 3), few pre-clinical models of these atypical BRAF mutations exist, limiting efforts to discover effective targeted therapies for these tumors. Here we evaluated the activity of 100 different mutant BRAF alleles (6 Class 2, 10 Class 3, 84 unclassified) using Reverse Phase Protein Array (RPPA) profiles and then used principal component analysis (PCA) and semi-supervised clustering learning methods to classify the 84 unclassified mutations. To identify the protein features that best defined each class of BRAF mutation (Class 1,2,3 and non-activating) pairwise t-tests were performed on the RPPA profiles identifying a total of 43 proteomic features. PCA with K-means clustering was performed; known Class 2 and Class 3 samples separated from each other and non-activating controls. Within the same cluster, previously unclassified mutations were assigned to the same class as mutations with known classification; 8 mutations were classified as Class 1, 42 Class 2, 28 Class 3, and 6 non-activating.Analyzing RPPA profiles across each BRAF class with several markers utilized by Yao and colleagues showed similar patterns in protein expression levels between the putative and Yao classification systems (MAPK_pT202_Y204: p = 0.57, n = 13 for Class 2 and p = 0.62, n = 21 for Class 3; Cyclin-D1: p = 0.83, n = 8 for Class 2 and p = 0.26, n = 17 for Class 3). Cell viability analysis of BRAF-mutant Ba/F3 cells using Student’s t-test yielded no significant differences in mean AUC values between new and old classification systems (class 2: p = 0.53, n = 63, Cohen’s d = 0.22; class 3: p = 0.93, n = 40, Cohen’s d = 0.03), further validating that the newly established classification system is consistent with the Yao classes.Network analysis was also performed to uncover essential oncogenic interactions for each BRAF class. KRAS, NRAS, and PIK3CA were found to have higher CERES scores for class-2 and -3 cells, while atypical BRAF yielded lower CERES scores. This suggests that in cells carrying atypical BRAF mutations, BRAF is less independently important for cell survival and it must partner with other oncogenes to drive tumor growth. Several key genes, including PIK3CA, EGFR, and MEK, were identified as potential drug targets for CRC cell lines with class-3 atypical BRAF mutations. In the cell lines, NCIH508 (Yao class 3) and HT55 (putative class 3), Trametinib (MEK inhibitor) and Afatinib (EGFR inhibitor) yielded IC 50 values below 3 µM (0.1 µM for HT55 with Afatinib and for NCIH508 with Trametinib), while Dactolisib (PIK3CA inhibitor) yielded an IC50 value of 0.1 µM.In conclusion, we extended the previous BRAF-mutant Yao classification system to establish the Yao Classification System Plus, which includes 84 more atypical BRAF mutations. This allows for the development of greater therapeutic options to target tumors carrying these newly characterized mutations and expand care options for many CRC patients. Citation Format: Abhinav B Madduri, Yang Liu, Bela G Nelson, Darren J Wang, Kaitlin E Sanders, Naadir H Jamal, Nidhi Sahni, John Paul Shen. A semi-supervised approach to classify atypical BRAF mutations to identify effective targeted therapies in colorectal cancer [abstract]. In: Proceedings of the AACR-NCI-EORTC Virtual International Conference on Molecular Targets and Cancer Therapeutics; 2023 Oct 11-15; Boston, MA. Philadelphia (PA): AACR; Mol Cancer Ther 2023;22(12 Suppl):Abstract nr A112.

Rewarded Semi-Supervised Re-Identification on Identities Rarely Crossing Camera Views.

Semi-supervised person re-identification (Re-ID) is an important approach for alleviating annotation costs when learning to match person images across camera views. Most existing works assume that training data contains abundant identities crossing camera views. However, this assumption is not true in many real-world applications, especially when images are captured in nonadjacent scenes for Re-ID in wider areas, where the identities rarely cross camera views. In this work, we operate semi-supervised Re-ID under a relaxed assumption of identities rarely crossing camera views, which is still largely ignored in existing methods. Since the identities rarely cross camera views, the underlying sample relations across camera views become much more uncertain, and deteriorate the noise accumulation problem in many advanced Re-ID methods that apply pseudo labeling for associating visually similar samples. To quantify such uncertainty, we parameterize the probabilistic relations between samples in a relation discovery objective for pseudo label training. Then, we introduce reward quantified by identification performance on a few labeled data to guide learning dynamic relations between samples for reducing uncertainty. Our strategy is called the Rewarded Relation Discovery (R 2 D), of which the rewarded learning paradigm is under-explored in existing pseudo labeling methods. To further reduce the uncertainty in sample relations, we perform multiple relation discovery objectives learning to discover probabilistic relations based on different prior knowledge of intra-camera affinity and cross-camera style variation, and fuse the complementary knowledge of different probabilistic relations by similarity distillation. To better evaluate semi-supervised Re-ID on identities rarely crossing camera views, we collect a new real-world dataset called REID-CBD, and perform simulation on benchmark datasets. Experiment results show that our method outperforms a wide range of semi-supervised and unsupervised learning methods.

Semi-Supervised Representation Learning for Segmentation on Medical Volumes and Sequences.

Benefiting from the massive labeled samples, deep learning-based segmentation methods have achieved great success for two dimensional natural images. However, it is still a challenging task to segment high dimensional medical volumes and sequences, due to the considerable efforts for clinical expertise to make large scale annotations. Self/semi-supervised learning methods have been shown to improve the performance by exploiting unlabeled data. However, they are still lack of mining local semantic discrimination and exploitation of volume/sequence structures. In this work, we propose a semi-supervised representation learning method with two novel modules to enhance the features in the encoder and decoder, respectively. For the encoder, based on the continuity between slices/frames and the common spatial layout of organs across subjects, we propose an asymmetric network with an attention-guided predictor to enable prediction between feature maps of different slices of unlabeled data. For the decoder, based on the semantic consistency between labeled data and unlabeled data, we introduce a novel semantic contrastive learning to regularize the feature maps in the decoder. The two parts are trained jointly with both labeled and unlabeled volumes/sequences in a semi-supervised manner. When evaluated on three benchmark datasets of medical volumes and sequences, our model outperforms existing methods with a large margin of 7.3% DSC on ACDC, 6.5% on Prostate, and 3.2% on CAMUS when only a few labeled data is available. Further, results on the M&M dataset show that the proposed method yields improvement without using any domain adaption techniques for data from unknown domain. Intensive evaluations reveal the effectiveness of representation mining, and superiority on performance of our method. The code is available at https://github.com/CcchenzJ/BootstrapRepresentation.

An active semi-supervised transfer learning method for robot pose error prediction and compensation

Robots are widely employed in industrial settings owing to their efficiency, flexibility, and extensive operational ranges. However, their application in high-precision scenarios is limited owing to their low absolute accuracies. Existing methods suffer from high measurement costs, and limited applicability and accuracy. To address these issues, an active semi supervised transfer learning method (ASTL) is introduced. The pose error prediction problem was modelled as a transfer learning paradigm for the first time. It leverages the proposed multi-stage greedy sampling (MGS) for informed sample labelling combined with coarse calibration and semi supervised transfer learning (STL) to embed theoretical knowledge for globally accurate predictions. The proposed method is compared with other prediction and compensation approaches for four robotic motion tasks. It significantly reduces the time consumption by approximately 89.3% compared with direct measurements and achieves a maximum reduction of approximately 90% in robot pose errors. This substantial enhancement in pose accuracy promotes high-quality applications of robots in the industrial sector.

Sentiment classification of time-sync comments: A semi-supervised hierarchical deep learning method

Time-sync comment (TSC) has emerged as a new type of textual comment for real-time user interactions on online video platforms. The sentiment classification of TSCs provides considerable potential for platforms to optimize operation strategies but inevitably faces great challenges due to the TSCs’ often uninformative and informal text. Considering the contextual dependency among TSCs posted within the same video clip, this study posits that contextual TSCs may benefit the sentiment classification of a target TSC. To address the challenges of leveraging contextual TSCs, such as their semantic representation and fusion, we propose a semi-supervised hierarchical deep learning method for the sentiment classification of TSCs. We design a hierarchical architecture to capture the semantics of TSCs at the word, comment, and context levels. Considering the varying importance of words and comments, we also design attention mechanisms to focus on important sentiment information and fuse semantic representations. Empirical evaluation shows that the proposed method outperforms benchmarked sentiment classification methods. This study advances our knowledge of contextual information indicative of TSC sentiment, and contributes to improving the service operation of online video platforms.

Customization of latent space in semi-supervised Variational AutoEncoder

We propose a novel semi-supervised learning method of Variational AutoEncoder (VAE), which yields a customized latent space through our EXplainable encoder Network (EXoN). The customization involves a manual design of the interpolation and structural constraint, such as proximity, which enhances the interpretability of the latent space. To improve the classification performance, we introduce a new semi-supervised classification method called SCI (Soft-label Consistency Interpolation). Combining the classification loss and the Kullback–Leibler divergence is crucial in constructing an explainable latent space. Additionally, the variability of the generated samples is determined by an active latent subspace, which effectively captures distinctive characteristics. We conduct experiments using the MNIST, SVHN, and CIFAR-10 datasets, and the results demonstrate that our approach yields an explainable latent space while significantly reducing the effort required to analyze representation patterns within the latent space.

EPCO-34. IMAGE-BASED PHENOTYPIC HIGH-CONTENT CRISPR-CAS9 SCREEN FOR CHARACTERIZING CANCER DRIVERS IN PEDIATRIC HIGH-GRADE GLIOMA

Abstract The genomic landscape of pediatric high-grade glioma is characterized by a high level of aneuploidy but few focal amplifications or homozygous deletions. Cancer aneuploidies are the principal source of tumor heterogeneity driving tumor growth and therapy resistance. Here, we deployed an image-based phenotypic high-content CRISPR-Cas9 screen to identify aneuploidy-associated pediatric glioma driver genes. Through integrated computational analyses combining whole-genome sequencing datasets from 106 pediatric gliomas and 81 canine gliomas, we identified a set of 460 genes present in regions of aneuploidy in both diseases. We designed a custom guideRNA library to systemically knockout the 460 gene set using an arrayed CRISPR-Cas9 knockout screen in two pediatric high-grade glioma cell lines. By deploying 72 of 384-well plates on automated confocal Opera Phenix High-Content Screening System and dyes staining cell membrane, nucleus, and apoptotic cells, we established 5 million live-cell images across three timepoints (day 7, 8, 9 post-transduction). Each gene was randomized to 32 randomized wells across four 384-well plates to gather robust morphological profiles. Through image-based nucleus and cell segmentation workflow, we characterized 50 million single cells across 27,648 wells over three time points. Using semi-supervised machine learning methods, we derived 1,050 cell morphological features to characterize knockout phenotypes, e.g., viability, growth, and morphology (size, shape, texture of nucleus and cells) across cells. The top ranked genes were then linked to oncogenes and tumor suppressors based on pathway and ontology analysis as well as functional in vitro and in silico validation using Pediatric Cancer Dependency Map. We find convergence of the most impactful molecular abnormalities - based on their knockout phenotypes - on candidate signaling pathways for the development of new drugs and repurposing of existing drugs for pediatric high-grade glioma.

SPLAL: Similarity-based pseudo-labeling with alignment loss for semi-supervised medical image classification

Medical image classification presents significant challenges due to limited labeled samples and class imbalance resulting from varying disease prevalence. In many real-world scenarios, a scarcity of labeled images pertaining to specific ailments coexists with an abundance of unlabeled ones. The process of annotating these unlabeled medical images by domain experts is both time-intensive and financially burdensome. Consequently, medical image classification remains an active and evolving research area. Semi-supervised learning (SSL) techniques hold promise in addressing these challenges by effectively leveraging both labeled and unlabeled data. However, the application of SSL to medical image classification necessitates overcoming two primary obstacles: (1) the reliable estimation of pseudo-labels for unlabeled images and (2) the mitigation of biases stemming from class imbalance. In this work, we introduce a novel semi-supervised learning approach, denoted as SPLAL, which adeptly addresses these challenges. SPLAL leverages the concept of class prototypes and employs a weighted combination of classifiers to predict dependable pseudo-labels for a subset of unlabeled medical images. Additionally, we introduce an alignment loss term designed to alleviate model biases, particularly those arising from the overrepresentation of majority classes. This alignment loss mechanism ensures the model’s consistency in delivering the same classification output for diverse augmented images of the same disease. To evaluate the effectiveness of our proposed approach, we conducted comprehensive experiments on two publicly accessible benchmark datasets for medical image classification. Our empirical findings unequivocally demonstrate that SPLAL outperforms several state-of-the-art semi-supervised learning methods across various evaluation metrics.

ConvNeXt based semi-supervised approach with consistency regularization for weeds classification

Weed recognition is an essential step for automatic weed control systems. Identifying weeds enables targeted control measures to be implemented, minimizing the use of chemicals and reducing the impact on the environment. Deep learning-based approaches proved to be effective for addressing various complex classification problems. However, to benefit fully from their capabilities, large amounts of labeled data are required, which represents a limitation for agricultural applications, consequence of the tedious and time-consuming process of data labeling. Conversely, unlabeled data could be acquired in large quantities, with relative ease. Hence, our aim is to develop robust and precise deep learning models, to carry-out the recognition and identification of weed species, using both types of data. To this end, we propose a method, that adopts the semi-supervised learning paradigm, to optimally combine labeled and unlabeled data. The method is based on a new deep neural networks architecture, which consists of a modernized convolutional encoder belonging to the family ConvNeXt and a thoroughly designed deep decoder network. This architecture, enables a successful integration of consistency regularization. The conducted experiments on DeepWeeds and 4-Weeds, showed that the semi-supervised models trained through our proposed method provide a stable and high classification performance, compared to other state-of-the-art deep learning models, which were affected negatively by the amount of labeled data available, and the presence of noise during inference. Furthermore, the effectiveness of the proposed method was demonstrated in comparison to other semi-supervised learning methods. The results obtained demonstrate the benefits of adopting the semi-supervised learning paradigm, especially in scenarios with very limited labeled data.

Joint loss learning-enabled semi-supervised autoencoder for bearing fault diagnosis under limited labeled vibration signals

Rolling bearing is a critical component of machinery that has been widely applied in manufacturing, transportation, aerospace, and power and energy industries. The timely and accurate bearing fault detection thus is of vital importance. Computational data-driven deep learning has recently become a prevailing approach for bearing fault detection. Despite the progress of the deep learning approach, the deep learning performance is hinged upon the size of labeled data, the acquisition of which is expensive in actual implementation. Unlabeled data, on the other hand, are inexpensive. In this research, we develop a new semi-supervised learning method built upon the autoencoder to fully utilize a large amount of unlabeled data together with limited labeled data to enhance fault detection performance. Compared with the state-of-the-art semi-supervised learning methods, this proposed method can be more conveniently implemented with fewer hyperparameters to be tuned. In this method, a joint loss is established to account for the effects of labeled and unlabeled data, which is subsequently used to direct the backpropagation training. Systematic case studies using the Case Western Reserve University (CWRU) rolling bearing dataset are carried out, in which the effectiveness of this new method is verified by comparing it with other well-established baseline methods. Specifically, nearly all emulation runs using the proposed methodology can lead to around 2%–5% accuracy increase, indicating its robustness in performance enhancement.

Mapping user behaviors to identify professional accounts in Ethereum using semi-supervised learning

Ethereum is one of the largest blockchain platforms currently that has become a digital business environment. This platform allows for decentralized transactions between anonymous users. Thus, the development of methods to identify users’ behaviors and keep them anonymous can potentially leverage business on this platform. In this work, we aim to combine different categories of machine learning approaches, namely, unsupervised and semi-supervised, to map the behaviors of users’ owned accounts and identify users with professional activities in Ethereum. In addition, we provide here data to the community and analyze different machine learning techniques to characterize the users of Ethereum. These are challenging tasks due to the small fraction of publicly labeled data referring to users’ accounts that provide services on this platform, such as exchange, payment, and entertainment, among most casual behavior users. Initially, we use unsupervised learning techniques to cluster the unlabeled users’ accounts and to identify a set of them with casual behavior. As an outcome, a dataset containing labeled (casual or professional) and unlabeled instances is obtained. Semi-supervised learning methods are then applied (i) to generate models that classify accounts’ behaviors into casual or professional ones and (ii) to discover accounts with professional behaviors among the unlabeled ones. Computational experiments were conducted, and the results obtained by the proposed procedure are compared to those achieved by supervised learning techniques from the literature. The proposal outperformed those from the literature and reached values higher than 95% for the accuracy, precision, recall, Fβ-scores, MCC, and AUC-ROC.

An Integrated Semi-supervised Software Defect Prediction Model

<p>A novel semi-supervised software defect prediction model FFeSSTri (Filtered Feature Selecting, Sample and Tri-training) is proposed to address the problem that class imbalance and too many irrelevant or redundant features in labelled samples lower the accuracy of semi-supervised software defect prediction. Its innovation lies in that the construction of FFeSSTri integrates an oversampling technique, a new feature selection method, and a Tri-training algorithm, thus it can effectively improve the accuracy. Firstly, the oversampling technique is applied to expand the class of inadequate samples, thus it solves the unbalanced classification of the labelled samples. Secondly, a new filtered feature selection method based on relevance and redundancy is proposed, which can exclude those irrelevant or redundant features from labelled samples. Finally, the Tri-training algorithm is used to learn the labelled training samples to build the defect prediction model FFeSSTri. The experiments conducted on the NASA software defect prediction dataset show that FFeSSTri outperforms the existing four supervised learning methods and one semi-supervised learning method in terms of F-Measure values and AUC values.</p> <p> </p>