Semi-supervised Learning Algorithm Research Articles

RETRACTED ARTICLE: A Semi-supervised Riemannian Kernel Dictionary Learning Algorithm Based on Locality-Constrained for Image Classification

RETRACTED ARTICLE: A Semi-supervised Riemannian Kernel Dictionary Learning Algorithm Based on Locality-Constrained for Image Classification

Semi-supervised learning for person re-identification based on style-transfer-generated data by CycleGANs

Person re-identification (re-ID) is an exceedingly significant branch in the field of computer vision, especially for video surveillance. It is still a challenge to obtain more labeled training data and use them reasonably for more precise matching, though the person re-ID performance has been improved significantly. In order to solve this challenge, this study proposes a semi-supervised learning algorithm for data augmentation, the style-transfer-generated data as an extra class (STGDEC), which is aided by the Cycle-Consistent Adversarial Networks (CycleGANs) in generating extra unlabeled training data. Specifically, the algorithm firstly trains the CycleGANs and Deep Convolutional Generative Adversarial Networks so as to generate large amounts of unlabeled data. Secondly, we propose an adaptive receptive field module to expand the size of receptive fields and select the appropriate receptive field features dynamically in order to learn more contextual information and discriminative feature representation and embed the module in the backbone network easily. Thirdly, we use the combination of label smoothing regularization for outliers and an extra class loss to regularize the generated data and encourage the network not to be too confident to the ground-truth. Finally, this paper proposes three training strategies for the combination of standard dataset and generated samples. Comprehensive experiments based on the STGDEC are conducted, and these results show that the proposed algorithm gains a significant improvement over the baseline, the Basel. + LSRO and state-of-the-art approaches of person re-ID in many cases.

The MultiRank Bootstrap Algorithm: Semi-Supervised Political Blog Classification and Ranking Using Semi-Supervised Link Classification

We present a new semi-supervised learning algorithm for classifying political blogs in a blog network and ranking them within predicted classes. We test our algorithm on two datasets and achieve classification accuracy of 81.9% and 84.6% using only 2 seed blogs.

A Payload Based Malicious HTTP Traffic Detection Method Using Transfer Semi-Supervised Learning

Malicious HTTP traffic detection plays an important role in web application security. Most existing work applies machine learning and deep learning techniques to build the malicious HTTP traffic detection model. However, they still suffer from the problems of huge training data collection cost and low cross-dataset generalization ability. Aiming at these problems, this paper proposes DeepPTSD, a deep learning method for payload based malicious HTTP traffic detection. First, it treats the malicious HTTP traffic detection as a text classification problem and trains the initial detection model using TextCNN on a public dataset, and then adapts the initial detection model to the target dataset based on a transfer learning algorithm. Second, in the transfer learning procedure, it uses a semi-supervised learning algorithm to accomplish the model adaptation task. The semi-supervised learning algorithm enhances the target dataset based on a HTTP payload data augmentation mechanism to exploit both the labeled and unlabeled data. We evaluate DeepPTSD on two real HTTP traffic datasets. The results show that DeepPTSD has competitive performance under the small data condition.

A Comparative Analysis of Semi-Supervised Learning in Detecting Burst Header Packet Flooding Attack in Optical Burst Switching Network

This paper presents a comparative analysis of four semi-supervised machine learning (SSML) algorithms for detecting malicious nodes in an optical burst switching (OBS) network. The SSML approaches include a modified version of K-means clustering, a Gaussian mixture model (GMM), a classical self-training (ST) model, and a modified version of self-training (MST) model. All the four approaches work in semi-supervised fashion, while the MST uses an ensemble of classifiers for the final decision making. SSML approaches are particularly useful when a limited number of labeled data is available for training and validation of the classification model. Manual labeling of a large dataset is complex and time consuming. It is even worse for the OBS network data. SSML can be used to leverage the unlabeled data for making a better prediction than using a smaller set of labelled data. We evaluated the performance of four SSML approaches for two (Behaving, Not-behaving), three (Behaving, Not-behaving, and Potentially Not-behaving), and four (No-Block, Block, NB- wait and NB-No-Block) class classifications using precision, recall, and F1 score. In case of the two-class classification, the K-means and GMM-based approaches performed better than the others. In case of the three-class classification, the K-means and the classical ST approaches performed better than the others. In case of the four-class classification, the MST showed the best performance. Finally, the SSML approaches were compared with two supervised learning (SL) based approaches. The comparison results showed that the SSML based approaches outperform when a smaller sized labeled data is available to train the classification models.

Improving fairness of artificial intelligence algorithms in Privileged-Group Selection Bias data settings

An increasing number of decisions regarding the daily lives of human beings are being controlled by artificial intelligence (AI) algorithms. Since they now touch on many aspects of our lives, it is crucial to develop AI algorithms that are not only accurate but also objective and fair. Recent studies have shown that algorithmic decision-making may be inherently prone to unfairness, even when there is no intention for it. In this paper, we study the fairness of AI algorithms in data settings for which unprivileged groups are extremely under-represented compared to privileged groups. A typical domain which often presents such Privileged Group Selection Bias (PGSB) is AI-based hiring, which stems from an inherent lack of labeled information for rejected applicants. We first demonstrate that such a selection bias can lead to a high algorithmic bias, even if privileged and unprivileged groups are treated exactly the same. We then propose several methods to overcome this type of bias. In particular, we suggest three in-process and pre-process fairness mechanisms, combined with both supervised and semi-supervised learning algorithms. An extensive evaluation that was conducted using two real-world datasets, reveals that the proposed methods are able to improve fairness considerably, with only a minimal compromise in accuracy. This is despite the limited information available for unprivileged groups and the inherent trade-off between fairness and accuracy.

A novel sub-Kmeans based on co-training approach by transforming single-view into multi-view

Semi-supervised learning is dedicated to solving the problem of poor model performance caused by the scarcity of labeled samples. Co-training algorithm, as a representative of semi-supervised learning algorithms, through constructing the diversity of classifiers, is used to solve the shortage of label samples that leads to low classifier accuracy. However, traditional Co-training style algorithms have strong constraints: (1) there are multiple views of data, and (2) the randomness of data views is relatively large, which leads to the low stability of the Co-training style algorithms. To optimize and solve the shortcomings of traditional Co-training style algorithms, a novel Co-training method based on sub-Kmeans named Op-FSCO is proposed. The proposed approach uses an optimal subspace construct method to extend Co-training to single-view data, greatly improving the application field of Co-training style algorithm. The clustering subspace is generated by plotting the probability of the correlation between the feature and the mutual information measure between the feature and the class label. For the resulting cluster subspace, the top two are selected to construct different views. Finally, experiments on a synthetic dataset, UCI dataset, and real-world dataset demonstrate that the proposed approach is more effective than other Co-training style algorithms. On the UCI benchmark dataset, the performance of the Op-FSCO algorithm on 15 UCI datasets is significantly better than the traditional Co-training algorithm. And on the cere bral stroke dataset, our approach is 2%–5% higher than that of traditional Co-training style algorithms.

Target Detection Network for SAR Images Based on Semi-Supervised Learning and Attention Mechanism

The existing Synthetic Aperture Radar (SAR) image target detection methods based on convolutional neural networks (CNNs) have achieved remarkable performance, but these methods require a large number of target-level labeled training samples to train the network. Moreover, some clutter is very similar to targets in SAR images with complex scenes, making the target detection task very difficult. Therefore, a SAR target detection network based on a semi-supervised learning and attention mechanism is proposed in this paper. Since the image-level label simply marks whether the image contains the target of interest or not, which is easier to be labeled than the target-level label, the proposed method uses a small number of target-level labeled training samples and a large number of image-level labeled training samples to train the network with a semi-supervised learning algorithm. The proposed network consists of a detection branch and a scene recognition branch with a feature extraction module and an attention module shared between these two branches. The feature extraction module can extract the deep features of the input SAR images, and the attention module can guide the network to focus on the target of interest while suppressing the clutter. During the semi-supervised learning process, the target-level labeled training samples will pass through the detection branch, while the image-level labeled training samples will pass through the scene recognition branch. During the test process, considering the help of global scene information in SAR images for detection, a novel coarse-to-fine detection procedure is proposed. After the coarse scene recognition determining whether the input SAR image contains the target of interest or not, the fine target detection is performed on the image that may contain the target. The experimental results based on the measured SAR dataset demonstrate that the proposed method can achieve better performance than the existing methods.

Semi-supervised generative adversarial networks for closed-angle detection on anterior segment optical coherence tomography images: an empirical study with a small training dataset.

BackgroundSemi-supervised learning algorithms can leverage an unlabeled dataset when labeling is limited or expensive to obtain. In the current study, we developed and evaluated a semi-supervised generative adversarial networks (GANs) model that detects closed-angle on anterior segment optical coherence tomography (AS-OCT) images using a small labeled dataset.MethodsIn this cross-sectional study, a semi-supervised GANs model was developed for automatic closed-angle detection training on a small labeled and large unsupervised training dataset collected from the Joint Shantou International Eye Center of Shantou University and the Chinese University of Hong Kong (JSIEC). The closed-angle was defined as iris-trabecular contact beyond the scleral spur in AS-OCT images. We further developed two supervised deep learning (DL) models training on the same supervised dataset and the whole dataset separately. The semi-supervised GANs model and supervised DL models’ performance were compared on two independent testing datasets from JSIEC (515 images) and the Department of Ophthalmology (84 images), National University Health System, respectively. The diagnostic performance was assessed by evaluation matrices, including the accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve (AUC).ResultsFor closed-angle detection using clinician grading of AS-OCT imaging as the reference standard, the semi-supervised GANs model showed comparable performance, with AUCs of 0.97 (95% CI, 0.96–0.99) and 0.98 (95% CI, 0.94–1.00), compared with the supervised DL model (using the whole dataset) [AUCs of 0.97 (95% CI, 0.96–0.99), and 0.97 (95% CI, 0.94–1.00)]. When training on the same small supervised dataset, the semi-supervised GANs achieved performance at least as well as, if not better than, the supervised DL model [AUCs of 0.90 (95% CI: 0.84–0.96), and 0.92 (95% CI, 0.86–0.97)].ConclusionsThe semi-supervised GANs method achieves diagnostic performance at least as good as a supervised DL model when trained on small labeled datasets. Further development of semi-supervised learning methods could be useful within clinical and research settings.Trial registration numberChiCTR2000037892.

Study on the semi-supervised learning-based patient similarity from heterogeneous electronic medical records

BackgroundA new learning-based patient similarity measurement was proposed to measure patients’ similarity for heterogeneous electronic medical records (EMRs) data.MethodsWe first calculated feature-level similarities according to the features’ attributes. A domain expert provided patient similarity scores of 30 randomly selected patients. These similarity scores and feature-level similarities for 30 patients comprised the labeled sample set, which was used for the semi-supervised learning algorithm to learn the patient-level similarities for all patients. Then we used the k-nearest neighbor (kNN) classifier to predict four liver conditions. The predictive performances were compared in four different situations. We also compared the performances between personalized kNN models and other machine learning models. We assessed the predictive performances by the area under the receiver operating characteristic curve (AUC), F1-score, and cross-entropy (CE) loss.ResultsAs the size of the random training samples increased, the kNN models using the learned patient similarity to select near neighbors consistently outperformed those using the Euclidean distance to select near neighbors (all P values < 0.001). The kNN models using the learned patient similarity to identify the top k nearest neighbors from the random training samples also had a higher best-performance (AUC: 0.95 vs. 0.89, F1-score: 0.84 vs. 0.67, and CE loss: 1.22 vs. 1.82) than those using the Euclidean distance. As the size of the similar training samples increased, which composed the most similar samples determined by the learned patient similarity, the performance of kNN models using the simple Euclidean distance to select the near neighbors degraded gradually. When exchanging the role of the Euclidean distance, and the learned patient similarity in selecting the near neighbors and similar training samples, the performance of the kNN models gradually increased. These two kinds of kNN models had the same best-performance of AUC 0.95, F1-score 0.84, and CE loss 1.22. Among the four reference models, the highest AUC and F1-score were 0.94 and 0.80, separately, which were both lower than those for the simple and similarity-based kNN models.ConclusionsThis learning-based method opened an opportunity for similarity measurement based on heterogeneous EMR data and supported the secondary use of EMR data.

Fault diagnosis of rolling bearing based on Laplacian regularization

How to design a reasonable classification method to identify the states is a critical step in rolling bearing fault diagnosis. Along with labeled samples, the Laplacian regularization (LapR) classification method, a graph-based semi-supervised learning algorithm, can also exploit the wealth of numerous cheap unlabeled samples to obtain acceptable performance. Based on the LapR classification method, a novel fault diagnosis method of rolling bearing is put forward. First, the vibration datasets or feature datasets are constructed into an undirected and weighted K-nearest neighbor graph, which can fully reflect the similarity between dataset elements. Then, the labels of all the dataset elements are interpreted as graph signals which are indexed by the represented graph’s vertices. Finally, when the constraint given by the known dataset elements is satisfied, the labels of the unknown dataset elements can be determined by finding a graph signal with minimal total variation. Experimental results reveal that whether the vibration datasets or feature datasets are analyzed, the LapR classification method is obviously superior to the popularly known classification methods in identifying the rolling bearing states especially when there are very few known samples.

Construction of Urban Water Source Circulating Water Pollution Prevention System Based on Semi-supervised Learning and Bayesian Algorithm

Construction of Urban Water Source Circulating Water Pollution Prevention System Based on Semi-supervised Learning and Bayesian Algorithm

Regularized Semi-Supervised Metric Learning with Latent Structure Preserved

Metric learning is a critical problem in classification. Most classifiers are based on a metric, the simplest one is the KNN classifier, whose outcome is directly decided by the given metric. This paper will discuss semi-supervised metric learning. Most traditional semi-supervised metric learning algorithms preserve the local structure of all the samples (including labeled and unlabeled) in the input space, when making the same labeled samples together and separating different labeled samples. In most existing methods, the local structure is calculated by the Euclidean distance which uses all the features. As we all know, high dimensional data lies on a low dimension manifold, and not all the features are discriminative. Thus, in this paper, we try to explore the latent structure of the samples and use the more discriminative features to calculate the local structure. The latent structure is learned by clustering random forest and cast into similarity between samples. Based on the hierarchical structure of the trees and the split function, the similarity is obtained from discriminant features. Experimental results on public data sets show our algorithm outperforms the traditional similar related algorithms.

Fast Detection of Deceptive Reviews by Combining the Time Series and Machine Learning

With the rapid growth of online product reviews, many users refer to others’ opinions before deciding to purchase any product. However, unfortunately, this fact has promoted the constant use of fake reviews, resulting in many wrong purchase decisions. The effective identification of deceptive reviews becomes a crucial yet challenging task in this research field. The existing supervised learning methods require a large number of labeled examples of deceptive and truthful opinions by domain experts, while the available unsupervised learning methods are inefficient because they depend on the features of reviewers to detect each fake review. Therefore, by focusing on the detection efficiency problem and the limitation of large amount of labeled examples dependence, in this paper, we proposed an effective semisupervised learning approach for detecting spam reviews. Firstly, a time series model of all the reviews of a product is constructed, and then the suspected time intervals are captured based on the burst review increases in these intervals. Secondly, a co-training two-view semisupervised learning algorithm was performed in each captured interval, in which linguistic cues, metadata, and user purchase behaviors were synthetically employed to classify the reviews and check whether they are spam ones or not. A series of numerical experiments on a real dataset acquired from Taobao.com have confirmed the effectiveness of the proposed model, not only reaping benefits in terms of time efficiency and high accuracy but also overcoming the shortcomings of supervised learning methods, which depend on large amounts of labeled examples. And a trade-off balance was obtained between accuracy and efficiency.

Porphyry Cu-Au prospectivity modelling using semi-supervised learning algorithm in Dehsalm district, eastern Iran

Porphyry Cu-Au prospectivity modelling using semi-supervised learning algorithm in Dehsalm district, eastern Iran

Semi-supervised Medical Image Segmentation through Dual-task Consistency

Deep learning-based semi-supervised learning (SSL) algorithms have led to promising results in medical images segmentation and can alleviate doctors' expensive annotations by leveraging unlabeled data. However, most of the existing SSL algorithms in literature tend to regularize the model training by perturbing networks and/or data. Observing that multi/dual-task learning attends to various levels of information which have inherent prediction perturbation, we ask the question in this work: can we explicitly build task-level regularization rather than implicitly constructing networks- and/or data-level perturbation and then regularization for SSL? To answer this question, we propose a novel dual-task-consistency semi-supervised framework for the first time. Concretely, we use a dual-task deep network that jointly predicts a pixel-wise segmentation map and a geometry-aware level set representation of the target. The level set representation is converted to an approximated segmentation map through a differentiable task transform layer. Simultaneously, we introduce a dual-task consistency regularization between the level set-derived segmentation maps and directly predicted segmentation maps for both labeled and unlabeled data. Extensive experiments on two public datasets show that our method can largely improve the performance by incorporating the unlabeled data. Meanwhile, our framework outperforms the state-of-the-art semi-supervised learning methods.

SALNet: Semi-supervised Few-Shot Text Classification with Attention-based Lexicon Construction

We propose a semi-supervised bootstrap learning framework for few-shot text classification. From a small amount of the initial dataset, our framework obtains a larger set of reliable training data by using the attention weights from an LSTM-based trained classifier. We first train an LSTM-based text classifier from a given labeled dataset using the attention mechanism. Then, we collect a set of words for each class called a lexicon, which is supposed to be a representative set of words for each class based on the attention weights calculated for the classification task. We bootstrap the classifier using the new data that are labeled by the combination of the classifier and the constructed lexicons to improve the prediction accuracy. As a result, our approach outperforms the previous state-of-the-art methods including semi-supervised learning algorithms and pretraining algorithms for few-shot text classification task on four publicly available benchmark datasets. Moreover, we empirically confirm that the constructed lexicons are reliable enough and substantially improve the performance of the original classifier.

Contrastive and Generative Graph Convolutional Networks for Graph-based Semi-Supervised Learning

Graph-based Semi-Supervised Learning (SSL) aims to transfer the labels of a handful of labeled data to the remaining massive unlabeled data via a graph. As one of the most popular graph-based SSL approaches, the recently proposed Graph Convolutional Networks (GCNs) have gained remarkable progress by combining the sound expressiveness of neural networks with graph structure. Nevertheless, the existing graph-based methods do not directly address the core problem of SSL, \emph{i.e.}, the shortage of supervision, and thus their performances are still very limited. To accommodate this issue, this paper presents a novel GCN-based SSL algorithm which aims to enrich the supervision signals by utilizing both data similarities and graph structure. Firstly, by designing a semi-supervised contrastive loss, the improved node representations can be generated via maximizing the agreement between different views of the same data or the data from the same class. Therefore, the rich unlabeled data and the scarce yet valuable labeled data can jointly provide abundant supervision information for learning discriminative node representations, which helps improve the subsequent classification result. Secondly, the underlying determinative relationship between the input graph topology and data features is extracted as supplementary supervision signals for SSL via using a graph generative loss related to input features. Intensive experimental results on a variety of real-world datasets firmly verify the effectiveness of our algorithm when compared with other state-of-the-art methods.

Bagging-based positive-unlabeled learning algorithm with Bayesian hyperparameter optimization for three-dimensional mineral potential mapping

Mineralization is a rare event. Hence, the geosciences datasets used for three-dimensional (3D) mineral potential mapping (MPM) are often imbalanced, consisting of scarce positive samples and abundant unlabeled data. Compared with selecting positive samples, it is challenging to select reliable negative samples in 3D MPM. However, the application of supervised machine learning algorithms in 3D MPM requires balanced positive and negative samples. Consequently, semi-supervised machine learning algorithms, which are only trained on positive samples, are widely used in 3D mineral prospecting. In this study, the bagging-based positive-unlabeled learning (PUL) algorithm that utilizes positive samples and unlabeled data in the training process was developed and applied to produce a 3D gold (Au) potential map in the Wulong Au district, China. This study employed Bayesian hyperparameter optimization to tune the hyperparameters of the bagging-based PUL algorithm. The performance of the bagging-based PUL algorithm was further compared with that of the widely used random forest, weights-of-evidence, one-class support vector machine, and continuous weighting approach in 3D MPM. The results demonstrated that the bagging-based PUL algorithm outperformed the aforementioned widely used predictive methods. The 3D mineral targets obtained by the bagging-based PUL algorithm can be beneficial for subsurface Au exploration in the Wulong Au district of China.

NIA-Network: Towards improving lung CT infection detection for COVID-19 diagnosis

During pandemics (e.g., COVID-19) physicians have to focus on diagnosing and treating patients, which often results in that only a limited amount of labeled CT images is available. Although recent semi-supervised learning algorithms may alleviate the problem of annotation scarcity, limited real-world CT images still cause those algorithms producing inaccurate detection results, especially in real-world COVID-19 cases. Existing models often cannot detect the small infected regions in COVID-19 CT images, such a challenge implicitly causes that many patients with minor symptoms are misdiagnosed and develop more severe symptoms, causing a higher mortality. In this paper, we propose a new method to address this challenge. Not only can we detect severe cases, but also detect minor symptoms using real-world COVID-19 CT images in which the source domain only includes limited labeled CT images but the target domain has a lot of unlabeled CT images. Specifically, we adopt Network-in-Network and Instance Normalization to build a new module (we term it NI module) and extract discriminative representations from CT images from both source and target domains. A domain classifier is utilized to implement infected region adaptation from source domain to target domain in an Adversarial Learning manner, and learns domain-invariant region proposal network (RPN) in the Faster R-CNN model. We call our model NIA-Network (Network-in-Network, Instance Normalization and Adversarial Learning), and conduct extensive experiments on two COVID-19 datasets to validate our approach. The experimental results show that our model can effectively detect infected regions with different sizes and achieve the highest diagnostic accuracy compared with existing SOTA methods.