Semi-supervised Learning Algorithm Research Articles

A Semi-Supervised Learning Algorithm for High and Low-Frequency Variable Imbalances in Industrial Data

A Semi-Supervised Learning Algorithm for High and Low-Frequency Variable Imbalances in Industrial Data

Wi-Fi sensing gesture control algorithm based on semi-supervised generative adversarial network

A Wi-Fi-sensing gesture control system for smart homes has been developed based on a theoretical investigation of the Fresnel region sensing model, addressing the need for non-contact gesture control in household environments. The system collects channel state information (CSI) related to gestures from Wi-Fi signals transmitted and received by network cards within a specific area. The collected data undergoes preprocessing to eliminate environmental interference, allowing for the extraction of complete gesture sets. Dynamic feature extraction is then performed, followed by the identification of unknown gestures using pattern recognition techniques. An improved dynamic double threshold gesture interception algorithm is introduced, achieving a gesture interception accuracy of 98.20%. Furthermore, dynamic feature extraction is enhanced using the Gramian Angular Summation Field (GASF) transform, which converts CSI data into GASF graphs for more effective gesture recognition. An enhanced generative adversarial network (GAN) algorithm with an embedded classifier is employed to classify unknown gestures, enabling the simultaneous recognition of multiple gestures. A semi-supervised learning algorithm designed to perform well even with limited labeled data demonstrates high performance in cross-scene gesture recognition. Compared to traditional fully-supervised algorithms like linear discriminant analysis (LDA), Light Gradient Boosting Machine (LightGBM), and support vector machine (SVM), the semi-supervised GAN algorithm achieves an average accuracy of 95.67%, significantly outperforming LDA (58.20%), LightGBM (78.20%), and SVM (75.67%). In conclusion, this novel algorithm maintains an accuracy of over 94% across various scenarios, offering both faster training times and superior accuracy, even with minimal labeled data.

How the marketing of real estate properties explains mortgage applicants by race and income

In this article, we study how the marketing of single-family homes explains the racial and income makeup of mortgage applicants in a neighborhood. We use a case study of the robust housing market of Charlotte, North Carolina, and annual, longitudinal real estate listing advertisements alongside mortgage lending data, to demonstrate how the share of properties advertised a certain way in a neighborhood in 1 year explains shares of mortgage applicants by race and income the following year. We classify property advertisement text using a semi-supervised learning algorithm into five categories following a housing investment and disinvestment to renewal continuum. We find stark racial disparities in mortgage applicants by housing type, even after controlling for income. We find that Black applicants nearly exclusively apply for mortgages in neighborhoods with a high share of properties advertised as disinvested with little profit-making promise. High-income White applicants rise as the share of advertised properties becomes more homogenous.

Semi-supervised Multi-view Ensemble Learning with Consensus

This paper is devoted to enchasing existing multi-view semi-supervised ensemble learning algorithms by introducing a cross-view consensus. A detailed overview of three state-of-the-art methods is given, with relevant steps of the training highlighted. A problem statement is formed to introduce both semi-supervised framework and consider the semi-supervised learning in the context of optimization problem. A novel multi-view semi-supervised ensemble learning algorithm called multi-view semi-supervised cross consensus (MSSXC) is introduced. The algorithm is tested against 5 synthetic datasets designed for semi-supervised learning challenges. The results indicate improvement in the average accuracy of up to 10% in comparison to existing methods, especially in low-volume, high density scenarios.

A semi-supervised learning algorithm for multi-label classification and multi-assignment clustering problems based on a Multivariate Data Analysis

Classically, in classification and clustering problems, an individual is exclusively assigned to a class (labeled data) or a cluster (unlabeled data). However, the assignment to a single class or cluster may be too strict in several real-world domains. In many cases, an individual may belong to multiple classes or clusters at the same time. On the other hand, we are starting to see machine learning algorithms that solve the problem of multi-label classification and multiple cluster assignment, but there are no algorithms that solve both problems simultaneously. Classic semi-supervised algorithms can work with labeled and unlabeled data simultaneously, but these types of algorithms assign individuals to a single class or a single cluster. Today, artificial intelligence faces the challenge of developing semi-supervised learning algorithms to work with semi-labeled data that can be assigned to different classes or clusters at the same time. In particular, this paper proposes a semi-supervised learning algorithm to fill this gap, which can solve the problems of multi-label classification and multiple cluster assignment simultaneously, on a semi-labeled dataset. This proposal is based on the LAMDA algorithm (Learning Algorithm for Multivariate Data Analysis), which calculates the degree of membership of a data to a group/class. In this work, a membership threshold is defined, which allows individuals to be assigned to classes or clusters that have a membership greater than this membership threshold. Thus, the main contribution of this work is the development of a semi-supervised algorithm that can process semi-labeled datasets to assign them to multiple classes and/or clusters. Furthermore, the work defines a metric to evaluate its efficiency in a semi-supervised context, called Multi Label-Cluster Index (MULCI). This proposal is tested on several datasets from the domains of multi-label classification or multi-assignment clustering, or a combination of both, showing very encouraging results. Very good quality metrics results are achieved in multiclass and multicluster tasks.

A novel masking model for Buddhist literature understanding by using Generative Adversarial Networks

This paper is focused on ancient Chinese Buddhist literature understanding. Buddhist literature incorporates a plethora of dialects and slang, which makes it challenging to extract semantic meaning seamlessly. To address this issue, a Generative Adversarial Network Masking Model (GAN-MM) is proposed to pre-train BERT models. This method focuses on optimizing the Masked Language Model (MLM) and takes advantage of the rich semantics of Buddhist terminologies as well as the absence of functional words in Buddhist literature. Furthermore, a semi-supervised learning algorithm was developed to train the GAN-MM. The model was evaluated by assessing its performance on two private tasks related to Buddhist literature understanding: sentiment classification and text segmentation, as well as two public tasks pertaining to the ancient Chinese understanding. The experimental results demonstrated a significant enhancement in the pre-training of BERT models when utilizing GAN-MM, as compared to conventional MLM methods. A large-scale Buddhist dataset, including 20,075 utterance documents and 146 million tokens, is public released at https://data.mendeley.com/datasets/5hzs8w46jh/1.

Detection of human movement by combining supervised machine learning and an embroidered textile capacitance sensor

This study contributes to respiratory pattern detection by introducing a fabric sensor utilizing capacitance measurement and a semi-supervised machine learning algorithm known as an AI-based autoencoder. The sensor, consisting of two embroidered electrodes composed of silver-coated conductive nylon filaments, leverages the body as a dielectric material. In the research, a garment-type respiratory sensor was employed to continuously monitor respiratory data during both static (standing) and dynamic (walking, brisk walking, running) actions. The sparse autoencoder algorithm was particularly employed for individual static and dynamic actions, effectively distinguishing respiratory patterns corresponding to various movements. In addition, the sparse autoencoder helps prevent overfitting, fundamentally minimizing errors between the compression and reconstruction of signals. The maximum number of epochs was set to 2000, and the target error was set at 0.005. All data were compared against the static walking as the training baseline. Ultimately, the root mean squared error (RMSE) between static postures averaged 0.1, while the RMSEs between dynamic actions of walking, brisk walking, and running were 0.61, 0.91, and 2.78, respectively. These results suggest that movement detection through error detection is practically feasible and possesses discernible capabilities.

GraphixMatch: Improving semi-supervised learning for graph classification with FixMatch

Graph classification is a critical problem that predicts the properties of an entire graph representing complicated relationships. This problem has been extensively studied in a supervised manner in various industries via graph convolutional networks that required large numbers of labeled graphs. However, labeled graphs are typically expensive and time-consuming to obtain in the real-world. Training graph classification models is difficult because of the sparseness of labeled data. Therefore, semi-supervised learning (SSL) algorithms that make full use of unlabeled data and overcome the lack of labels have led to the expansion of graph classification studies. In this study, we propose GraphixMatch to learn the graph properties of both labeled and unlabeled graphs based on the use of FixMatch for graph classification. To effectively capture the characteristics of a graph, we newly define and apply weak and strong augmentation of the graph. We evaluate the proposed method on seven benchmarks from a public dataset. We show that compared with various SSL methods, the simplified proposed method with weak and strong augmentation is superior. Furthermore, we conduct extensive additional experiments to derive the important parameters of GraphixMatch.

A topological approach for semi-supervised learning

Nowadays, Machine Learning and Deep Learning methods have become the state-of-the-art approach to solve data classification tasks. In order to use those methods, it is necessary to acquire and label a considerable amount of data; however, this is not straightforward in some fields, since data annotation is time consuming and might require expert knowledge. This challenge can be tackled by means of semi-supervised learning methods that take advantage of both labelled and unlabelled data. In this work, we present new semi-supervised learning methods based on techniques from Topological Data Analysis (TDA). In particular, we have created two semi-supervised learning methods following two topological approaches. In the former, we have used a homological approach that consists in studying the persistence diagrams associated with the data using the bottleneck and Wasserstein distances. In the latter, we have considered the connectivity of the data. In addition, we have carried out a thorough analysis of the developed methods using 9 tabular datasets with low and high dimensionality. The results show that the developed semi-supervised methods outperform the results obtained with models trained with only manually labelled data, and are an alternative to other classical semi-supervised learning algorithms.

MF-Match: A Semi-Supervised Model for Human Action Recognition.

Human action recognition (HAR) technology based on radar signals has garnered significant attention from both industry and academia due to its exceptional privacy-preserving capabilities, noncontact sensing characteristics, and insensitivity to lighting conditions. However, the scarcity of accurately labeled human radar data poses a significant challenge in meeting the demand for large-scale training datasets required by deep model-based HAR technology, thus substantially impeding technological advancements in this field. To address this issue, a semi-supervised learning algorithm, MF-Match, is proposed in this paper. This algorithm computes pseudo-labels for larger-scale unsupervised radar data, enabling the model to extract embedded human behavioral information and enhance the accuracy of HAR algorithms. Furthermore, the method incorporates contrastive learning principles to improve the quality of model-generated pseudo-labels and mitigate the impact of mislabeled pseudo-labels on recognition performance. Experimental results demonstrate that this method achieves action recognition accuracies of 86.69% and 91.48% on two widely used radar spectrum datasets, respectively, utilizing only 10% labeled data, thereby validating the effectiveness of the proposed approach.

Enhancing identification performance of cognitive impairment high-risk based on a semi-supervised learning method

BackgroundCognitive assessment plays a pivotal role in the early detection of cognitive impairment, particularly in the prevention and management of cognitive diseases such as Alzheimer’s and Lewy body dementia. Large-scale screening relies heavily on cognitive assessment scales as primary tools, with some low sensitivity and others expensive. Despite significant progress in machine learning for cognitive function assessment, its application in this particular screening domain remains underexplored, often requiring labor-intensive expert annotations. AimsThis paper introduces a semi-supervised learning algorithm based on pseudo-label with putback (SS-PP), aiming to enhance model efficiency in predicting the high risk of cognitive impairment (HR-CI) by utilizing the distribution of unlabeled samples. DataThe study involved 189 labeled samples and 215,078 unlabeled samples from real world. A semi-supervised classification algorithm was designed and evaluated by comparison with supervised methods composed by 14 traditional machine-learning methods and other advanced semi-supervised algorithms. ResultsThe optimal SS-PP model, based on GBDT, achieved an AUC of 0.947. Comparative analysis with supervised learning models and semi-supervised methods demonstrated an average AUC improvement of 8% and state-of-art performance, repectively. ConclusionThis study pioneers the exploration of utilizing limited labeled data for HR-CI predictions and evaluates the benefits of incorporating physical examination data, holding significant implications for the development of cost-effective strategies in relevant healthcare domains.

Clustering Network Traffic Using Semi-Supervised Learning

Clustering algorithms play a crucial role in early warning cybersecurity systems. They allow for the detection of new attack patterns and anomalies and enhance system performance. This paper discusses the problem of clustering data collected by a distributed system of network honeypots. In the proposed approach, when a network flow matches an attack signature, an appropriate label is assigned to it. This enables the use of semi-supervised learning algorithms and improves the quality of clustering results. The article compares the results of learning algorithms conducted with and without partial supervision, particularly non-negative matrix factorization and semi-supervised non-negative matrix factorization. Our results confirm the positive impact of labeling a portion of flows on the quality of clustering.

CPSS: Fusing consistency regularization and pseudo-labeling techniques for semi-supervised deep cardiovascular disease detection using all unlabeled electrocardiograms

Background and objective:Deep learning usually achieves good performance in the supervised way, which requires a large amount of labeled data. However, manual labeling of electrocardiograms (ECGs) is laborious that requires much medical knowledge. Semi-supervised learning (SSL) provides an effective way of leveraging unlabeled data to improve model performance, providing insight for solving this problem. The objective of this study is to improve the performance of cardiovascular disease (CVD) detection by fully utilizing unlabeled ECG. Methods:A novel SSL algorithm fusing consistency regularization and pseudo-labeling techniques (CPSS) is proposed. CPSS consists of supervised learning and unsupervised learning. For supervised learning, the labeled ECGs are mapped into prediction vectors by the classifier. The cross-entropy loss function is used to optimize the classifier. For unsupervised learning, the unlabeled ECGs are weakly and strongly augmented, and a consistency loss is used to minimize the difference between the classifier’s predictions for the two augmentations. Pseudo-labeling techniques include positive pseudo-labeling (PL) and ranking-based negative pseudo-labeling (RNL). PL introduces pseudo-labels for data with high prediction confidence. RNL assigns negative pseudo-labels to the lower-ranked categories in the prediction vectors to leverage data with low prediction confidence. In this study, VGGNet and ResNet are used as classifiers, which are jointly optimized by labeled and unlabeled ECGs. Results:CPSS has been validated on several databases. With the same number of labeled ECGs (10%), it improves the accuracies over pure supervised learning by 13.59%, 4.60%, and 5.38% in the CPSC2018, PTB-XL, and Chapman databases, respectively. CPSS achieves comparable results to the fully supervised method with only 10% of labeled ECGs, which reduces the labeling workload by 90%. In addition, to verify the practicality of CPSS, a cardiovascular disease monitoring system is designed by heterogeneously deploying the trained classifiers on an SoC (system-on-a-chip), which can detect CVD in real time. Conclusion:The results of this study indicate that the proposed CPSS can significantly improve the performance of CVD detection using unlabeled ECG, which reduces the burden of ECG labeling in deep learning. In addition, the designed monitoring system makes the proposed CPSS promising for real-world applications.

Few-Shot Steel Defect Detection Based on a Fine-Tuned Network with Serial Multi-Scale Attention

Detecting defects on a steel surface is crucial for the quality enhancement of steel, but its effectiveness is impeded by the limited number of high-quality samples, diverse defect types, and the presence of interference factors such as dirt spots. Therefore, this article proposes a fine-tuned deep learning approach to overcome these obstacles in unstructured few-shot settings. Initially, to address steel surface defect complexities, we integrated a serial multi-scale attention mechanism, concatenating attention and spatial modules, to generate feature maps that contain both channel information and spatial information. Further, a pseudo-label semi-supervised learning algorithm (SSL) based on a variant of the locally linear embedding (LLE) algorithm was proposed, enhancing the generalization capability of the model through information from unlabeled data. Afterwards, the refined model was merged into a fine-tuned few-shot object detection network, which applied extensive base class samples for initial training and sparsed new class samples for fine-tuning. Finally, specialized datasets considering defect diversity and pixel scales were constructed and tested. Compared with conventional methods, our approach improved accuracy by 5.93% in 7-shot detection tasks, markedly reducing manual workload and signifying a leap forward for practical applications in steel defect detection.

Triplet Adaptation Framework for Robust Semi-Supervised Learning.

Semi-supervised learning (SSL) suffers from severe performance degradation when labeled and unlabeled data come from inconsistent and imbalanced distribution. Nonetheless, there is a lack of theoretical guidance regarding a remedy for this issue. To bridge the gap between theoretical insights and practical solutions, we embark to an analysis of generalization bound of classic SSL algorithms. This analysis reveals that distribution inconsistency between unlabeled and labeled data can cause a significant generalization error bound. Motivated by this theoretical insight, we present a Triplet Adaptation Framework (TAF) to reduce the distribution divergence and improve the generalization of SSL models. TAF comprises three adapters: Balanced Residual Adapter, aiming to map the class distribution of labeled and unlabeled data to a uniform distribution for reducing class distribution divergence; Representation Adapter, aiming to map the representation distribution of unlabeled data to labeled one for reducing representation distribution divergence; and Pseudo-Label Adapter, aiming to align the predicted pseudo-labels with the class distribution of unlabeled data, thereby preventing erroneous pseudo-labels from exacerbating representation divergence. These three adapters collaborate synergistically to reduce the generalization bound, ultimately achieving a more robust and generalizable SSL model. Extensive experiments across various robust SSL scenarios validate the efficacy of our method.

Developing an atmospheric aging evaluation model of acrylic coatings: A semi-supervised machine learning algorithm

Developing an atmospheric aging evaluation model of acrylic coatings: A semi-supervised machine learning algorithm

Al-BERT: a semi-supervised denoising technique for disease prediction.

Medical records are a valuable source for understanding patient health conditions. Doctors often use these records to assess health without solely depending on time-consuming and complex examinations. However, these records may not always be directly relevant to a patient's current health issue. For instance, information about common colds may not be relevant to a more specific health condition. While experienced doctors can effectively navigate through unnecessary details in medical records, this excess information presents a challenge for machine learning models in predicting diseases electronically. To address this, we have developed 'al-BERT', a new disease prediction model that leverages the BERT framework. This model is designed to identify crucial information from medical records and use it to predict diseases. 'al-BERT' operates on the principle that the structure of sentences in diagnostic records is similar to regular linguistic patterns. However, just as stuttering in speech can introduce 'noise' or irrelevant information, similar issues can arise in written records, complicating model training. To overcome this, 'al-BERT' incorporates a semi-supervised layer that filters out irrelevant data from patient visitation records. This process aims to refine the data, resulting in more reliable indicators for disease correlationsand enhancing the model's predictive accuracy and utility in medical diagnostics. To discern noise diseases within patient records, especially those resembling influenza-like illnesses, our approach employs a customized semi-supervised learning algorithm equipped with a focused attention mechanism. This mechanism is specifically calibrated to enhance the model's sensitivity to chronic conditions while concurrently distilling salient features from patient records, thereby augmenting the predictive accuracy and utility of the model in clinical settings. We evaluate the performance of al-BERT using real-world health insurance data provided by Taiwan's National Health Insurance. In our study, we evaluated our model against two others: one based on BERT that uses complete disease records, and another variant that includes extra filtering techniques. Our findings show that models incorporating filtering mechanisms typically perform better than those using the entire, unfiltered dataset. Our approach resulted in improved outcomes across several key measures: AUC-ROC (an indicator of a model's ability to distinguish between classes), precision (the accuracy of positive predictions), recall (the model's ability to find all relevant cases), and overall accuracy. Most notably, our model showed a 15% improvement in recall compared to the current best-performing method in the field of disease prediction. The conducted ablation study affirms the advantages of our attention mechanism and underscores the crucial role of the selection module within al-BERT.

PiCO+: Contrastive Label Disambiguation for Robust Partial Label Learning.

Partial label learning (PLL) is an important problem that allows each training example to be labeled with a coarse candidate set with the ground-truth label included. However, in a more practical but challenging scenario, the annotator may miss the ground-truth and provide a wrong candidate set, which is known as the noisy PLL problem. To remedy this problem, we propose the PiCO+ framework that simultaneously disambiguates the candidate sets and mitigates label noise. Core to PiCO+, we develop a novel label disambiguation algorithm PiCO that consists of a contrastive learning module along with a novel class prototype-based disambiguation method. Theoretically, we show that these two components are mutually beneficial, and can be rigorously justified from an expectation-maximization (EM) algorithm perspective. To handle label noise, we extend PiCO to PiCO+, which further performs distance-based clean sample selection, and learns robust classifiers by a semi-supervised contrastive learning algorithm. Beyond this, we further investigate the robustness of PiCO+ in the context of out-of-distribution noise and incorporate a novel energy-based rejection method for improved robustness. Extensive experiments demonstrate that our proposed methods significantly outperform the current state-of-the-art approaches in standard and noisy PLL tasks and even achieve comparable results to fully supervised learning.

Triple-task mutual consistency for semi-supervised 3D medical image segmentation

Semi-supervised deep learning algorithm is an effective means of medical image segmentation. Among these methods, multi-task learning with consistency regularization has achieved outstanding results. However, most of the existing methods usually simply embed the Signed Distance Map (SDM) task into the network, which underestimates the potential ability of SDM in edge awareness and leads to excessive dependence between tasks. In this work, we propose a novel triple-task mutual consistency (TTMC) framework to enhance shape and edge awareness capabilities, and overcome the task dependence problem underestimated in previous work. Specifically, we innovatively construct the Signed Attention Map (SAM), a novel fusion image with attention mechanism, and use it as an auxiliary task for segmentation to enhance the edge awareness ability. Then we implement a triple-task deep network, which jointly predicts the voxel-wise classification map, the Signed Distance Map and the Signed Attention Map. In our proposed framework, an optimized differentiable transformation layer associates SDM with voxel-wise classification map and SAM prediction, while task-level consistency regularization utilizes unlabeled data in an unsupervised manner. Evaluated on the public Left Atrium dataset and NIH Pancreas dataset, our proposed framework achieves significant performance gains by effectively utilizing unlabeled data, outperforming recent state-of-the-art semi-supervised segmentation methods. Code is available at https://github.com/Saocent/TTMC.

ABAE: Auxiliary Balanced AutoEncoder for class-imbalanced semi-supervised learning

Semi-supervised learning has achieved extraordinary success in prevalent image-classification benchmarks. However, a class-balanced distribution that differs notably from real-world data distribution is required. In general, models trained under class-imbalanced semi-supervised learning conditions are severely biased towards the majority classes. To address this issue, we propose a novel framework called ABAE by implanting an Auxiliary Balanced AutoEncoder branch into existing semi-supervised learning algorithms. Considering that adaptive feature augmentation for different classes can alleviate confirmation bias, we devise a class-aware reconstruction loss to train the AutoEncoder module. To further smooth the output, we adopt a graph-based label propagation scheme at the end of the AutoEncoder. Extensive experiments on CIFAR-10/100-LT, SVHN-LT and Small ImageNet-127 demonstrate the effectiveness of ABAE.