Semi-supervised Learning Approach Research Articles

Leveraging Structure for Improved Classification of Grouped Biased Data

We consider semi-supervised binary classification for applications in which data points are naturally grouped (e.g., survey responses grouped by state) and the labeled data is biased (e.g., survey respondents are not representative of the population). The groups overlap in the feature space and consequently the input-output patterns are related across the groups. To model the inherent structure in such data, we assume the partition-projected class-conditional invariance across groups, defined in terms of the group-agnostic feature space. We demonstrate that under this assumption, the group carries additional information about the class, over the group-agnostic features, with provably improved area under the ROC curve. Further assuming invariance of partition-projected class-conditional distributions across both labeled and unlabeled data, we derive a semi-supervised algorithm that explicitly leverages the structure to learn an optimal, group-aware, probability-calibrated classifier, despite the bias in the labeled data. Experiments on synthetic and real data demonstrate the efficacy of our algorithm over suitable baselines and ablative models, spanning standard supervised and semi-supervised learning approaches, with and without incorporating the group directly as a feature.

T5-Based Model for Abstractive Summarization: A Semi-Supervised Learning Approach with Consistency Loss Functions

Text summarization is a prominent task in natural language processing (NLP) that condenses lengthy texts into concise summaries. Despite the success of existing supervised models, they often rely on datasets of well-constructed text pairs, which can be insufficient for languages with limited annotated data, such as Chinese. To address this issue, we propose a semi-supervised learning method for text summarization. Our method is inspired by the cycle-consistent adversarial network (CycleGAN) and considers text summarization as a style transfer task. The model is trained by using a similar procedure and loss function to those of CycleGAN and learns to transfer the style of a document to its summary and vice versa. Our method can be applied to multiple languages, but this paper focuses on its performance on Chinese documents. We trained a T5-based model and evaluated it on two datasets, CSL and LCSTS, and the results demonstrate the effectiveness of the proposed method.

Robust Detection, Segmentation, and Metrology of High Bandwidth Memory 3D Scans Using an Improved Semi-Supervised Deep Learning Approach

Recent advancements in 3D deep learning have led to significant progress in improving accuracy and reducing processing time, with applications spanning various domains such as medical imaging, robotics, and autonomous vehicle navigation for identifying and segmenting different structures. In this study, we employ the latest developments in 3D semi-supervised learning to create cutting-edge models for the 3D object detection and segmentation of buried structures in high-resolution X-ray semiconductors scans. We illustrate our approach to locating the region of interest of the structures, their individual components, and their void defects. We showcase how semi-supervised learning is utilized to capitalize on the vast amounts of available unlabeled data to enhance both detection and segmentation performance. Additionally, we explore the benefit of contrastive learning in the data pre-selection step for our detection model and multi-scale Mean Teacher training paradigm in 3D semantic segmentation to achieve better performance compared with the state of the art. Our extensive experiments have shown that our method achieves competitive performance and is able to outperform by up to 16% on object detection and 7.8% on semantic segmentation. Additionally, our automated metrology package shows a mean error of less than 2 μm for key features such as Bond Line Thickness and pad misalignment.

SSMD-UNet: semi-supervised multi-task decoders network for diabetic retinopathy segmentation

Diabetic retinopathy (DR) is a diabetes complication that can cause vision loss among patients due to damage to blood vessels in the retina. Early retinal screening can avoid the severe consequences of DR and enable timely treatment. Nowadays, researchers are trying to develop automated deep learning-based DR segmentation tools using retinal fundus images to help Ophthalmologists with DR screening and early diagnosis. However, recent studies are unable to design accurate models due to the unavailability of larger training data with consistent and fine-grained annotations. To address this problem, we propose a semi-supervised multitask learning approach that exploits widely available unlabelled data (i.e., Kaggle-EyePACS) to improve DR segmentation performance. The proposed model consists of novel multi-decoder architecture and involves both unsupervised and supervised learning phases. The model is trained for the unsupervised auxiliary task to effectively learn from additional unlabelled data and improve the performance of the primary task of DR segmentation. The proposed technique is rigorously evaluated on two publicly available datasets (i.e., FGADR and IDRiD) and results show that the proposed technique not only outperforms existing state-of-the-art techniques but also exhibits improved generalisation and robustness for cross-data evaluation.

Advancements in AI for Brain Tumor Detection and Classification

Abstract: This research paper explores the development of artificial intelligence (AI) in brain tumor detection and classification from 2010 to the present. It examines the significance of brain tumors, the role of MRI in their analysis, and the motivation behind utilizing AI in this context. The objectives and methodology of the research are outlined. The paper discusses various AI techniques, such as machine learning and deep learning, as well as supervised, unsupervised, and semi-supervised learning approaches. The importance of feature extraction and selection is highlighted, and a comparison is made between traditional methods and AI-based approaches. The evolution of AI techniques in brain tumor detection and classification is examined, including notable research papers and their contributions. The impact of AI algorithms on accuracy, efficiency, and clinical applications is analyzed. The paper concludes with a discussion of challenges, future directions, and the potential integration of emerging technologies.

Gaussian transformation enhanced semi-supervised learning for sleep stage classification

Sleep disorders are significant health concerns affecting a large population. Related clinical studies face the deficiency in sleep data and challenges in data analysis, which requires enormous human expertise and labor. Moreover, in current clinical practice, sleep data acquisition processes usually cover only one night’s sleep history, which is too short to recognize long-term sleep patterns. To address these challenges, we propose a semi-supervised learning (cluster-then-label) approach for sleep stage classification, integrating clustering algorithms into the supervised learning pipeline. We test the effectiveness of the proposed semi-supervised learning approach on two architectures: an advanced architecture using deep learning for classification and k-means for clustering, and a relatively naive Gaussian-based architecture. Also, we introduce two novel Gaussian transformations to improve the robustness and accuracy of the Gaussian-based architecture: assembled-fixed transformation and neural network based transformation. We reveal the effectiveness of the proposed algorithm via experiments on whole-night electroencephalogram (EEG) data. The experiments demonstrate that the proposed learning strategy improves the accuracy and F1 score over the state-of-the-art baseline on out-of-distribution human subjects. The experiments also confirm that the proposed Gaussian transformations can significantly gain normality to EEG band-power features and in turn facilitate the semi-supervised learning process. This cluster-then-label learning approach, combined with novel Gaussian transformations, can significantly improve the accuracy and efficiency of sleep stage classification, enabling more effective diagnosis of sleep disorders.

MIM-Graph: A multi-sensor network approach for fault diagnosis of HSR Bogie bearings at the IoT edge via mutual information maximization

The Internet of Things (IoT) is crucial in developing next-generation high-speed railways (HSRs). HSR IoT enables intelligent diagnosis of trains using multi-sensor data, which is critical for maintaining high speeds and ensuring passenger safety. Graph neural network (GNN)-based methods have gained popularity in HSR IoT research due to the ability to represent the sensor network as intuitive graphs. However, labeling monitoring data in the HSR scenario takes time and effort. To address this challenge, we propose a semi-supervised graph-level representation learning approach called MIM-Graph, which uses mutual information maximization to learn from a large amount of unlabeled data. First, the multi-sensor data is converted into association graphs based on their spatial topology. The unsupervised encoder is trained using global–local mutual maximization. The teacher–student framework transfers knowledge from the unsupervised encoder learned to the supervised encoder, which is trained using a small amount of labeled data. As a result, the supervised encoder learns distinguishable representations for intelligent diagnosis of HSR. We evaluate the proposed method using CWRU dataset and data from HSR Bogie test platform, and the experimental results demonstrate the effectiveness and superiority of MIM-Graph.

KISL: knowledge-injected semi-supervised learning for biological co-expression network modules.

The exploration of important biomarkers associated with cancer development is crucial for diagnosing cancer, designing therapeutic interventions, and predicting prognoses. The analysis of gene co-expression provides a systemic perspective on gene networks and can be a valuable tool for mining biomarkers. The main objective of co-expression network analysis is to discover highly synergistic sets of genes, and the most widely used method is weighted gene co-expression network analysis (WGCNA). With the Pearson correlation coefficient, WGCNA measures gene correlation, and uses hierarchical clustering to identify gene modules. The Pearson correlation coefficient reflects only the linear dependence between variables, and the main drawback of hierarchical clustering is that once two objects are clustered together, the process cannot be reversed. Hence, readjusting inappropriate cluster divisions is not possible. Existing co-expression network analysis methods rely on unsupervised methods that do not utilize prior biological knowledge for module delineation. Here we present a method for identification of outstanding modules in a co-expression network using a knowledge-injected semi-supervised learning approach (KISL), which utilizes apriori biological knowledge and a semi-supervised clustering method to address the issue existing in the current GCN-based clustering methods. To measure the linear and non-linear dependence between genes, we introduce a distance correlation due to the complexity of the gene-gene relationship. Eight RNA-seq datasets of cancer samples are used to validate its effectiveness. In all eight datasets, the KISL algorithm outperformed WGCNA when comparing the silhouette coefficient, Calinski-Harabasz index and Davies-Bouldin index evaluation metrics. According to the results, KISL clusters had better cluster evaluation values and better gene module aggregation. Enrichment analysis of the recognition modules demonstrated their effectiveness in discovering modular structures in biological co-expression networks. In addition, as a general method, KISL can be applied to various co-expression network analyses based on similarity metrics. Source codes for the KISL and the related scripts are available online at https://github.com/Mowonhoo/KISL.git.

MsDroid: Identifying Malicious Snippets for Android Malware Detection

Machine learning has shown promise for improving the accuracy of Android malware detection in the literature. However, it is challenging to (1) stay robust towards real-world scenarios and (2) provide interpretable explanations for experts to analyse. In this paper, we propose MsDroid, an Android malware detection system that makes decisions by identifying malicious snippets with interpretable explanations. We mimic a common practice of security analysts, i.e., filtering APIs before looking through each method, to focus on local snippets around sensitive APIs instead of the whole program. Each snippet is represented with a graph encoding both code attributes and domain knowledge and then classified by GNN. To identify malicious snippets, we present a semi-supervised learning approach that only requires app labeling. To make malicious snippets less opaque, we design an explanation mechanism to show the importance of control flows and to retrieve similarly implemented snippets from known malwares. A comprehensive comparison with 5 baseline methods is conducted on a dataset of more than 81K apps in 3 real-world scenarios. The experimental results show that MsDroid is more robust than state-of-the-art systems in all cases, with 5.37% to 49.52% advantage in F1-score. Besides, we demonstrate how the explanations facilitate malware analysis

Simultaneous multi-crop land suitability prediction from remote sensing data using semi-supervised learning

Land suitability models for Canada are currently based on single-crop inventories and expert opinion. We present a data-driven multi-layer perceptron that simultaneously predicts the land suitability of several crops in Canada, including barley, peas, spring wheat, canola, oats, and soy. Available crop yields from 2013–2020 are downscaled to the farm level by masking the district level crop yield data to focus only on areas where crops are cultivated and leveraging soil-climate-landscape variables obtained from Google Earth Engine for crop yield prediction. This new semi-supervised learning approach can accommodate data from different spatial resolutions and enables training with unlabelled data. The incorporation of a crop indicator function further allows for the training of a multi-crop model that can capture the interdependences and correlations between various crops, thereby leading to more accurate predictions. Through k-fold cross-validation, we show that compared to the single crop models, our multi-crop model could produce up to a 2.82 fold reduction in mean absolute error for any particular crop. We found that barley, oats, and mixed grains were more tolerant to soil-climate-landscape variations and could be grown in many regions of Canada, while non-grain crops were more sensitive to environmental factors. Predicted crop suitability was associated with a region’s growing season length, which supports climate change projections that regions of northern Canada will become more suitable for agricultural use. The proposed multi-crop model could facilitate assessment of the suitability of northern lands for crop cultivation and be incorporated into cost-benefit analyses.

Identifying Fraudsters and Fraudulent Strategies in Mobile Social Network

Modern communication technologies have developed quickly, especially communications through (mobile) phones, which has greatly aided in the sharing of information and social connections between people. Nonetheless, the rise of telemarketing scams has the potential to seriously deplete communal and private wealth, slowing down or harming the economy. With an emphasis on exposing the "precise fraud" phenomena and the techniques employed by fraudsters to precisely choose targets, we propose to identify telemarketing scams in this study. We utilise a one-month comprehensive dataset of telecommunication information from Shanghai, which includes 698 million call logs and 54 million customers, to explore this issue. During our research, we have discovered that user information may have been substantially compromised, and that fraudsters prefer to target users who are younger and more active on mobile networks. To further separate fraudsters from non-fraudsters, we provide a unique semi-supervised learning approach. Our technique beats various cutting-edge algorithms in terms of accuracy of identifying fraudsters, according to experimental findings on real-world data. We think that our research may help governments and mobile service providers make better policy decisions.

Adaptive inverse mapping: a model-free semi-supervised learning approach towards robust imaging through dynamic scattering media.

Imaging through scattering media is a useful and yet demanding task since it involves solving for an inverse mapping from speckle images to object images. It becomes even more challenging when the scattering medium undergoes dynamic changes. Various approaches have been proposed in recent years. However, none of them are able to preserve high image quality without either assuming a finite number of sources for dynamic changes, assuming a thin scattering medium, or requiring access to both ends of the medium. In this paper, we propose an adaptive inverse mapping (AIP) method, which requires no prior knowledge of the dynamic change and only needs output speckle images after initialization. We show that the inverse mapping can be corrected through unsupervised learning if the output speckle images are followed closely. We test the AIP method on two numerical simulations: a dynamic scattering system formulated as an evolving transmission matrix and a telescope with a changing random phase mask at a defocused plane. Then we experimentally apply the AIP method to a multimode-fiber-based imaging system with a changing fiber configuration. Increased robustness in imaging is observed in all three cases. AIP method's high imaging performance demonstrates great potential in imaging through dynamic scattering media.

SpaDecon: cell-type deconvolution in spatial transcriptomics with semi-supervised learning

Spatially resolved transcriptomics (SRT) has advanced our understanding of the spatial patterns of gene expression, but the lack of single-cell resolution in spatial barcoding-based SRT hinders the inference of specific locations of individual cells. To determine the spatial distribution of cell types in SRT, we present SpaDecon, a semi-supervised learning approach that incorporates gene expression, spatial location, and histology information for cell-type deconvolution. SpaDecon was evaluated through analyses of four real SRT datasets using knowledge of the expected distributions of cell types. Quantitative evaluations were performed for four pseudo-SRT datasets constructed according to benchmark proportions. Using mean squared error and Jensen-Shannon divergence with the benchmark proportions as evaluation criteria, we show that SpaDecon performance surpasses that of published cell-type deconvolution methods. Given the accuracy and computational speed of SpaDecon, we anticipate it will be valuable for SRT data analysis and will facilitate the integration of genomics and digital pathology.

Semisupervised Learning for Noise Suppression Using Deep Reinforcement Learning of Contrastive Features

In this work, we present DeDRLSSL a generic semi-supervised noise suppression framework. The proposed model is based on a reinforcement learning (RL) system for learning contrastive features to refine the features utilized in consistency matching for semi-supervised learning (SSL). The proposed method outperforms the state-of-the-art supervised models in terms of error compensation for Inertial Measurement Unit (IMU) data from various evaluation metrics and improves the baselines for yaw estimation on average by 38% and 28% across the benchmarks for 30% and 50% of labeled data, respectively. Our approach can be adapted to any SSL approach to compensate for the problem of label scarcity.

SEBD: A Stream Evolving Bot Detection Framework with Application of PAC Learning Approach to Maintain Accuracy and Confidence Levels

A simple supervised learning model can predict a class from trained data based on the previous learning process. Trust in such a model can be gained through evaluation measures that ensure fewer misclassification errors in prediction results for different classes. This can be applied to supervised learning using a well-trained dataset that covers different data points and has no imbalance issues. This task is challenging when it integrates a semi-supervised learning approach with a dynamic data stream, such as social network data. In this paper, we propose a stream-based evolving bot detection (SEBD) framework for Twitter that uses a deep graph neural network. Our SEBD framework was designed based on multi-view graph attention networks using fellowship links and profile features. It integrates Apache Kafka to enable the Twitter API stream and predict the account type after processing. We used a probably approximately correct (PAC) learning framework to evaluate SEBD’s results. Our objective was to maintain the accuracy and confidence levels of our framework to enable successful learning with low misclassification errors. We assessed our framework results via cross-domain evaluation using test holdout, machine learning classifiers, benchmark data, and a baseline tool. The overall results show that SEBD is able to successfully identify bot accounts in a stream-based manner. Using holdout and cross-validation with a random forest classifier, SEBD achieved an accuracy score of 0.97 and an AUC score of 0.98. Our results indicate that bot accounts participate highly in hashtags on Twitter.

Classification of operational states in porous journal bearings using a semi-supervised multi-sensor Machine Learning approach

Tribosystems are designed for long-lasting operation necessitating condition monitoring to identify critical regimes in time. On the example of porous journal bearings we propose a new approach for the classification of the operational states.Tribometer data from accelerated lifetime tests were used for developing a semi-supervised multi-sensor Machine Learning (ML) classifier. To advance the state-of-the art and gain higher classification accuracy, data from multiple sensors were combined by calculating time and frequency feature sets. The frequencies found in the sensor data were traced using a 3D scanning vibrometer and originated from eigenmodes and vibration states of the system. Relevant features were selected using Sequential Forward Floating Selection (SFFS) to enhance the performance of the model. Labeling was assisted by explorative data analysis techniques, i.e., principal component analysis (PCA) and subsequent hierarchical (Ward) clustering of the feature sets.A tree-based ensemble classifier (Extra-Trees) differentiating four tribological regimes from ‘Run-in’ to ‘Critical’ yielded a classification accuracy of 0.98. The algorithm was able to reliably detect states even when applied to data from previously unseen experiments and to predict terminal failure of a bearing up to 50 h before it occurred.

Anomaly Detection for Sensor Signals Utilizing Deep Learning Autoencoder-Based Neural Networks.

Anomaly detection is a significant task in sensors' signal processing since interpreting an abnormal signal can lead to making a high-risk decision in terms of sensors' applications. Deep learning algorithms are effective tools for anomaly detection due to their capability to address imbalanced datasets. In this study, we took a semi-supervised learning approach, utilizing normal data for training the deep learning neural networks, in order to address the diverse and unknown features of anomalies. We developed autoencoder-based prediction models to automatically detect anomalous data recorded by three electrochemical aptasensors, with variations in the signals' lengths for particular concentrations, analytes, and bioreceptors. Prediction models employed autoencoder networks and the kernel density estimation (KDE) method for finding the threshold to detect anomalies. Moreover, the autoencoder networks were vanilla, unidirectional long short-term memory (ULSTM), and bidirectional LSTM (BLSTM) autoencoders for the training stage of the prediction models. However, the decision-making was based on the result of these three networks and the integration of vanilla and LSTM networks' results. The accuracy as a performance metric of anomaly prediction models showed that the performance of vanilla and integrated models were comparable, while the LSTM-based autoencoder models showed the least accuracy. Considering the integrated model of ULSTM and vanilla autoencoder, the accuracy for the dataset with the lengthier signals was approximately 80%, while it was 65% and 40% for the other datasets. The lowest accuracy belonged to the dataset with the least normal data in its dataset. These results demonstrate that the proposed vanilla and integrated models can automatically detect abnormal data when there is sufficient normal data for training the models.

A multi-task and multi-channel convolutional neural network for semi-supervised neonatal artefact detection

Objective. Automated artefact detection in the neonatal electroencephalogram (EEG) is crucial for reliable automated EEG analysis, but limited availability of expert artefact annotations challenges the development of deep learning models for artefact detection. This paper proposes a semi-supervised deep learning approach for artefact detection in neonatal EEG that requires few labelled data by training a multi-task convolutional neural network (CNN). Approach. An unsupervised and a supervised objective were jointly optimised by combining an autoencoder and an artefact classifier in one multi-output model that processes multi-channel EEG inputs. The proposed semi-supervised multi-task training strategy was compared to a classical supervised strategy and other existing state-of-the-art models. The models were trained and tested separately on two different datasets, which contained partially annotated multi-channel neonatal EEG. Models were evaluated using the F1-statistic and the relevance of the method was investigated in the context of a functional brain age (FBA) prediction model. Main results. The proposed multi-task and multi-channel CNN methods outperformed state-of-the-art methods, reaching F1 scores of 86.2% and 95.7% on two separate datasets. The proposed semi-supervised multi-task training strategy was shown to be superior to a classical supervised training strategy when the amount of labels in the dataset was artificially reduced. Finally, we found that the error of a brain age prediction model correlated with the amount of automatically detected artefacts in the EEG segment. Significance. Our results show that the proposed semi-supervised multi-task training strategy can train CNNs successfully even when the amount of labels in the dataset is limited. Therefore, this method is a promising semi-supervised technique for developing deep learning models with scarcely labelled data. Moreover, a correlation between the error of FBA estimates and the amount of detected artefacts in the corresponding EEG segments indicates the relevance of artefact detection for robust automated EEG analysis.

Deep reinforcement learning for data-efficient weakly supervised business process anomaly detection

The detection of anomalous behavior in business process data is a crucial task for preventing failures that may jeopardize the performance of any organization. Supervised learning techniques are impracticable because of the difficulties of gathering huge amounts of labeled business process anomaly data. For this reason, unsupervised learning techniques and semi-supervised learning approaches trained on entirely labeled normal data have dominated this domain for a long time. However, these methods do not work well because of the absence of prior knowledge of true anomalies. In this study, we propose a deep weakly supervised reinforcement learning-based approach to identify anomalies in business processes by leveraging limited labeled anomaly data. The proposed approach is intended to use a small collection of labeled anomalous data while exploring a huge set of unlabeled data to find new classes of anomalies that are outside the scope of the labeled anomalous data. We created a unique reward function that combined the supervisory signal supplied by a variational autoencoder trained on unlabeled data with the supervisory signal provided by the environment’s reward. To further reduce data deficiency, we introduced a sampling method to allow the effective exploration of the unlabeled data and to address the imbalanced data problem, which is a common problem in the anomaly detection field. This approach depends on the proximity between the data samples in the latent space of the variational autoencoder. Furthermore, to efficiently model the sequential nature of business process data and to handle the long-term dependences, we used a long short-term memory network combined with a self-attention mechanism to develop the agent of our reinforcement learning model. Multiple scenarios were used to test the proposed approach on real-world and synthetic datasets. The findings revealed that the proposed approach outperformed five competing approaches by efficiently using the few available anomalous examples.

Automatic classification of construction safety reports using semi-supervised YAKE-Guided LDA approach

Literature on supervised Machine-Learning (ML) approaches for classifying text-based safety reports for the construction sector has been growing. Recent studies have emphasized the need to build ML approaches that balance high classification accuracy and performance on management criteria, such as resource intensiveness. However, despite being highly accurate, the extensively focused, supervised ML approaches may not perform well on management criteria as many factors contribute to their resource intensiveness. Alternatively, the potential for semi-supervised ML approaches to achieve balanced performance has rarely been explored in the construction safety literature. The current study contributes to the scarce knowledge on semi-supervised ML approaches by demonstrating the applicability of a state-of-the-art semi-supervised learning approach, i.e., Yet, Another Keyword Extractor (YAKE) integrated with Guided Latent Dirichlet Allocation (GLDA) for construction safety report classification. Construction-safety-specific knowledge is extracted as keywords through YAKE, relying on accessible literature with minimal manual intervention. Keywords from YAKE are then seeded in the GLDA model for the automatic classification of safety reports without requiring a large quantity of prelabeled datasets. The YAKE-GLDA classification performance (F1 score of 0.66) is superior to existing unsupervised methods for the benchmark data containing injury narratives from Occupational Health and Safety Administration (OSHA). The YAKE-GLDA approach is also applied to near-miss safety reports from a construction site. The study demonstrates a high degree of generality of the YAKE-GLDA approach through a moderately high F1 score of 0.86 for a few categories in the near-miss data. The current research demonstrates that, unlike the existing supervised approaches, the semi-supervised YAKE-GLDA approach can achieve a novel possibility of consistently achieving reasonably good classification performance across various construction-specific safety datasets yet being resource-efficient. Results from an objective comparative and sensitivity analysis contribute to much-required knowledge-contesting insights into the functioning and applicability of the YAKE-GLDA. The results from the current study will help construction organizations implement and optimize an efficient ML-based knowledge-mining strategy for domains beyond safety and across sites where the availability of a pre-labeled dataset is a significant limitation.