Unlabeled Dataset Research Articles

Self-Writer: Clusterable Embedding Based Self-Supervised Writer Recognition from Unlabeled Data

Writer recognition based on a small amount of handwritten text is one of the most challenging deep learning problems because of the implicit characteristics of handwriting styles. In a deep convolutional neural network, writer recognition based on supervised learning has shown great success. These supervised methods typically require a lot of annotated data. However, collecting annotated data is expensive. Although unsupervised writer recognition methods may address data annotation issues significantly, they often fail to capture sufficient feature relationships and usually perform less efficiently than supervised learning methods. Self-supervised learning may solve the unlabeled dataset issue and train the unsupervised datasets in a supervised manner. This paper introduces Self-Writer, a self-supervised writer recognition approach dealing with unlabeled data. The proposed scheme generates clusterable embeddings from a small fixed-length image frame such as a text block. The training strategy presumes that a small image frame of handwritten text should include the writer’s handwriting characteristics. We construct pairwise constraints and nongenerative augmentation to train Siamese architecture to generate embeddings depending on such an assumption. Self-Writer is evaluated on the two most widely used datasets, IAM and CVL, on pairwise and triplet architecture. We find Self-Writer to be convincing in achieving satisfactory performance using pairwise architectures.

Unsupervised Adversarial Domain Adaptation for Agricultural Land Extraction of Remote Sensing Images

Agricultural land extraction is an essential technical means to promote sustainable agricultural development and modernization research. Existing supervised algorithms rely on many finely annotated remote-sensing images, which is both time-consuming and expensive. One way to reduce the annotation cost approach is to migrate models trained on existing annotated data (source domain) to unannotated data (target domain). However, model generalization capability is often unsatisfactory due to the limit of the domain gap. In this work, we use an unsupervised adversarial domain adaptation method to train a neural network to close the gap between the source and target domains for unsupervised agricultural land extraction. The overall approach consists of two phases: inter-domain and intra-domain adaptation. In the inter-domain adaptation, we use a generative adversarial network (GAN) to reduce the inter-domain gap between the source domain (labeled dataset) and the target domain (unlabeled dataset). The transformer with robust long-range dependency modeling acts as the backbone of the generator. In addition, the multi-scale feature fusion (MSFF) module is designed in the generator to accommodate remote sensing datasets with different spatial resolutions. Further, we use an entropy-based approach to divide the target domain. The target domain is divided into two subdomains, easy split images and hard split images. By training against each other between the two subdomains, we reduce the intra-domain gap. Experiments results on the “DeepGlobe → LoveDA”, “GID → LoveDA” and “DeepGlobe → GID” unsupervised agricultural land extraction tasks demonstrate the effectiveness of our method and its superiority to other unsupervised domain adaptation techniques.

Imbalanced tabular data modelization using CTGAN and machine learning to improve IoT Botnet attacks detection

The Internet of Things (IoT) has been trending in the past few years, posing so many security problems. IoT Botnets are one of the most serious attacks that threaten the reliability of IoT systems because of IoT devices resources constraints. Most of the classical and even the intelligent solutions such as ML and DL for botnet detection are trained on unlabeled dataset, in other cases, dataset trustability is not approved but it is used, which may cause degradation of performance in case those models were implemented in security tools in order to deal with zero-day threats. Another two critical problems are the limits of classical oversampling methods while generating samples and the limits of understanding complex datasets and modeling real tabular data by the existing GAN models. The aim of this paper is to implement the CTGAN model, the state-of-the-art of Generative Adversarial Networks models in tabular data modeling and generation in order to overcome all previous mentioned limits. The results are promising. After data augmentation using CTGAN, MLP achieves an accuracy of 98.93%, F1-Score equal to 0.9907, Geometric-Score equal to 0.9874, sensitivity and specificity achieve resp. values of 0.9893 and 0.9856.

Semi-Supervised Portrait Matting via the Collaboration of Teacher–Student Network and Adaptive Strategies

In the portrait matting domain, existing methods rely entirely on annotated images for learning. However, delicate manual annotations are time-consuming and there are few detailed datasets available. To reduce complete dependency on labeled datasets, we design a semi-supervised network (ASSN) with two kinds of innovative adaptive strategies for portrait matting. Three pivotal sub-modules are embedded in our architecture, including a static teacher network (S-TN), a static student network (S-SN), and an adaptive student network (A-SN). S-TN and S-SN are modules that need to be trained with a small number of high-quality labeled datasets. Moreover, A-SN and S-SN share the same module parameters. When processing unlabeled datasets, A-SN adopts the adaptive strategies designed by us to discard the dependence on labeled datasets. The adaptive strategies include: (i) An auxiliary adaption: The teacher network with complicated design not only provides alpha mattes for the adaptive student network but also transmits rough segmentation results and edge graphs as optimization reference standards. (ii) A self-adjusting adaption: The adaptive network can make self-supervised to the characteristics of different layers. In addition, we have produced a finely annotated dataset for scholars in the field. Compared with existing datasets, our dataset complements the following two types of data neglected in previous datasets: (i) Images taken by multiple people. (ii) Images under low light conditions.

Efficient Staining-Invariant Nuclei Segmentation Approach Using Self-Supervised Deep Contrastive Network.

Existing nuclei segmentation methods face challenges with hematoxylin and eosin (H&E) whole slide imaging (WSI) due to the variations in staining methods and nuclei shapes and sizes. Most existing approaches require a stain normalization step that may cause losing source information and fail to handle the inter-scanner feature instability problem. To mitigate these issues, this article proposes an efficient staining-invariant nuclei segmentation method based on self-supervised contrastive learning and an effective weighted hybrid dilated convolution (WHDC) block. In particular, we propose a staining-invariant encoder (SIE) that includes convolution and transformers blocks. We also propose the WHDC block allowing the network to learn multi-scale nuclei-relevant features to handle the variation in the sizes and shapes of nuclei. The SIE network is trained on five unlabeled WSIs datasets using self-supervised contrastive learning and then used as a backbone for the downstream nuclei segmentation network. Our method outperforms existing approaches in challenging multiple WSI datasets without stain color normalization.

Semisupervised Deep Learning for Image Classification With Distribution Mismatch: A Survey

Deep learning methodologies have been employed in several different fields, with an outstanding success in image recognition applications, such as material quality control, medical imaging, autonomous driving, etc. Deep learning models rely on the abundance of labeled observations to train a prospective model. These models are composed of millions of parameters to estimate, increasing the need of more training observations. Frequently, it is expensive to gather labeled observations of data, making the usage of deep learning models not ideal, as the model might overfit data. In a semisupervised setting, unlabeled data are used to improve the levels of accuracy and generalization of a model with small labeled datasets. Nevertheless, in many situations different unlabeled data sources might be available. This raises the risk of a significant distribution mismatch between the labeled and unlabeled datasets. Such phenomena can cause a considerable performance hit to typical semisupervised deep learning (SSDL) frameworks, which often assume that both labeled and unlabeled datasets are drawn from similar distributions. Therefore, in this article we study the latest approaches for SSDL for image recognition. Emphasis is made in SSDL models designed to deal with a distribution mismatch between the labeled and unlabeled datasets. We address open challenges with the aim to encourage the community to tackle them, and overcome the high data demand of traditional deep learning pipelines under real-world usage settings.

Image Clustering Using an Augmented Generative Adversarial Network and Information Maximization.

Image clustering has recently attracted significant attention due to the increased availability of unlabeled datasets. The efficiency of traditional clustering algorithms heavily depends on the distance functions used and the dimensionality of the features. Therefore, performance degradation is often observed when tackling either unprocessed images or high-dimensional features extracted from processed images. To deal with these challenges, we propose a deep clustering framework consisting of a modified generative adversarial network (GAN) and an auxiliary classifier. The modification employs Sobel operations prior to the discriminator of the GAN to enhance the separability of the learned features. The discriminator is then leveraged to generate representations as to the input to an auxiliary classifier. An objective function is utilized to train the auxiliary classifier by maximizing the mutual information between the representations obtained via the discriminator model and the same representations perturbed via adversarial training. We further improve the robustness of the auxiliary classifier by introducing a penalty term into the objective function. This minimizes the divergence across multiple transformed representations generated by the discriminator model with a low dropout rate. The auxiliary classifier is implemented with a group of multiple cluster-heads, where a tolerance hyper-parameter is used to tackle imbalanced data. Our results indicate that the proposed method achieves competitive results compared with state-of-the-art clustering methods on a wide range of benchmark datasets including CIFAR-10, CIFAR-100/20, and STL10.

Knowledge-Based Representation Learning for Nucleus Instance Classification From Histopathological Images.

The classification of nuclei in H&E-stained histopathological images is a fundamental step in the quantitative analysis of digital pathology. Most existing methods employ multi-class classification on the detected nucleus instances, while the annotation scale greatly limits their performance. Moreover, they often downplay the contextual information surrounding nucleus instances that is critical for classification. To explicitly provide contextual information to the classification model, we design a new structured input consisting of a content-rich image patch and a target instance mask. The image patch provides rich contextual information, while the target instance mask indicates the location of the instance to be classified and emphasizes its shape. Benefiting from our structured input format, we propose Structured Triplet for representation learning, a triplet learning framework on unlabelled nucleus instances with customized positive and negative sampling strategies. We pre-train a feature extraction model based on this framework with a large-scale unlabeled dataset, making it possible to train an effective classification model with limited annotated data. We also add two auxiliary branches, namely the attribute learning branch and the conventional self-supervised learning branch, to further improve its performance. As part of this work, we will release a new dataset of H&E-stained pathology images with nucleus instance masks, containing 20,187 patches of size 1024 ×1024 , where each patch comes from a different whole-slide image. The model pre-trained on this dataset with our framework significantly reduces the burden of extensive labeling. We show a substantial improvement in nucleus classification accuracy compared with the state-of-the-art methods.

Learning Deep Representations of Cardiac Structures for 4D Cine MRI Image Segmentation through Semi-Supervised Learning.

Learning good data representations for medical imaging tasks ensures the preservation of relevant information and the removal of irrelevant information from the data to improve the interpretability of the learned features. In this paper, we propose a semi-supervised model-namely, combine-all in semi-supervised learning (CqSL)-to demonstrate the power of a simple combination of a disentanglement block, variational autoencoder (VAE), generative adversarial network (GAN), and a conditioning layer-based reconstructor for performing two important tasks in medical imaging: segmentation and reconstruction. Our work is motivated by the recent progress in image segmentation using semi-supervised learning (SSL), which has shown good results with limited labeled data and large amounts of unlabeled data. A disentanglement block decomposes an input image into a domain-invariant spatial factor and a domain-specific non-spatial factor. We assume that medical images acquired using multiple scanners (different domain information) share a common spatial space but differ in non-spatial space (intensities, contrast, etc.). Hence, we utilize our spatial information to generate segmentation masks from unlabeled datasets using a generative adversarial network (GAN). Finally, to reconstruct the original image, our conditioning layer-based reconstruction block recombines spatial information with random non-spatial information sampled from the generative models. Our ablation study demonstrates the benefits of disentanglement in holding domain-invariant (spatial) as well as domain-specific (non-spatial) information with high accuracy. We further apply a structured similarity loss along with a mutual information minimizer (MIM) to improve the adversarially trained generative models for better reconstruction. Experimental results achieved on the STACOM 2017 ACDC cine cardiac magnetic resonance (MR) dataset suggest that our proposed (CqSL) model outperforms fully supervised and semi-supervised models, achieving an 83.2% performance accuracy even when using only 1% labeled data. We hypothesize that our proposed model has the potential to become an efficient semantic segmentation tool that may be used for domain adaptation in data-limited medical imaging scenarios, where annotations are expensive. Code, and experimental configurations will be made available publicly.

Semi-supervised portrait matting using transformer

Transformer is successfully applied in many tasks but is not sensible to directly embed in the portrait matting. Such an operation can effectively consider global features ignoring local features, which leads to the difficulty of identifying the complete and edge-refined portraits. What is more, the existing supervised architectures are feeble against the unlabeled images. To achieve the effective excavation of both local and global features, we design a semi-supervised network that leverages Transformer to capture the global features. The arduous task of compensating more local features is left to the portrait detailed decoding module (PDDM) that we designed. In addition, to provide the possibility of improving effectiveness when faced with unlabeled images, we design an intelligent pseudo-label generation strategy to embed in our semi-supervised network. This strategy can generate more detailed pseudo-labels than predicted results through redundant foreground filtering and edge adjustment. Compared to existing portrait matting methods, our network successfully achieves performance improvements with a small number of datasets and has the ability to train on unlabeled datasets. The training models and the code will be released at https://github.com/XinyueZhangqdu/SSPMTransformer/.

Survey on Recent Trends in Medical Image Classification Using Semi-Supervised Learning

Training machine learning and deep learning models for medical image classification is a challenging task due to a lack of large, high-quality labeled datasets. As the labeling of medical images requires considerable time and effort from medical experts, models need to be specifically designed to train on low amounts of labeled data. Therefore, an application of semi-supervised learning (SSL) methods provides one potential solution. SSL methods use a combination of a small number of labeled datasets with a much larger number of unlabeled datasets to achieve successful predictions by leveraging the information gained through unsupervised learning to improve the supervised model. This paper provides a comprehensive survey of the latest SSL methods proposed for medical image classification tasks.

Detecting Emotions behind the Screen

Students’ emotional health is a major contributor to educational success. Hence, to support students’ success in online learning platforms, we contribute with the development of an analysis of the emotional orientations and triggers in their text messages. Such analysis could be automated and used for early detection of the emotional status of students. In our approach, we relied on transfer learning to train the model, using the pre-trained Bidirectional Encoder Representations from Transformers model (BERT). The model classified messages as positive, negative, or neutral. The transfer learning model was then used to classify a larger unlabeled dataset and fine-grained emotions in the negative messages only, using NRC lexicon. In our analysis to the results, we focused in discovering the dominant negative emotions expressed and the most common words students used to express them. We believe this can be an important clue or first line of detection that may assist mental health practitioners to develop targeted programs for students, especially with the massive shift to online education due to the COVID-19 pandemic. We compared our model to a state-of-the-art ML-based model and found our model outperformed the other by achieving a 91% accuracy compared to an 86%. To the best of our knowledge, this is the first study to focus on a mental health analysis of students in online educational platforms other than massive open online courses (MOOCs).

Remote Sensing Image Scene Classification with Self-Supervised Learning Based on Partially Unlabeled Datasets

In recent years, supervised learning, represented by deep learning, has shown good performance in remote sensing image scene classification with its powerful feature learning ability. However, this method requires large-scale and high-quality handcrafted labeled datasets, which leads to a high cost of obtaining annotated samples. Self-supervised learning can alleviate this problem by using unlabeled data to learn the image’s feature representation and then migrate to the downstream task. In this study, we use an encoder–decoder structure to construct a self-supervised learning architecture. In the encoding stage, the image mask is used to discard some of the image patches randomly, and the image’s feature representation can be learned from the remaining image patches. In the decoding stage, the lightweight decoder is used to recover the pixels of the original image patches according to the features learned in the encoding stage. We constructed a large-scale unlabeled training set using several public scene classification datasets and Gaofen-2 satellite data to train the self-supervised learning model. In the downstream task, we use the encoder structure with the masked image patches that have been removed as the backbone network of the scene classification task. Then, we fine-tune the pre-trained weights of self-supervised learning in the encoding stage on two open datasets with complex scene categories. The datasets include NWPU-RESISC45 and AID. Compared with other mainstream supervised learning methods and self-supervised learning methods, our proposed method has better performance than the most state-of-the-art methods in the task of remote sensing image scene classification.

Semi-supervised 3D-InceptionNet for segmentation and survival prediction of head and neck primary cancers

Cancers, known collectively as head and neck cancers, usually begin in the squamous cells that line the head and neck’s mucosal surfaces, forming a tumour mass. It usually develops in the salivary glands, nose and sinuses, voice box (larynx), throat (pharynx), or muscles or nerves in the head and neck, but these types of cancer are much less common than squamous cell carcinomas. Nearly 4% of all cancers are H&N cancers with a very low survival rate (a five-year survival rate of 64.7%). The most commonly used molecular imaging procedure for diagnosing or guiding the treatment of head and neck cancer is Fluorodeoxyglucose-positron emission tomography scanning (FDG-PET/CT), which is often used in conjunction with computed tomography (CT) scanning, and sentinel node biopsy. This work presents a semi-supervised 3D Inception-Residual framework with 3D depth-wise convolution and squeeze and excitation block. In the first phase, we performed pre-training of 3D-auto-encoder using both train and test unlabelled dataset. We, then used pre-trained weight to fine-tune the later network which is aided with depth-wise convolution-inception encoder consisting of an additional 3D squeeze and excitation block and a 3D depth-wise convolution-based residual learning decoder under deep supervision (Semi 3D-IncNet). The proposed network not only helps to recalibrate the channel-wise features adaptively through explicit inter-dependencies modelling but also integrates the coarse and fine features resulting in accurate tumour segmentation. We further demonstrate the effectiveness of semi-supervised inception-residual encoder–decoder architecture in achieving better dice scores and the impact of depth-wise convolution in lowering the computational cost. For survival prediction, we applied random forest on deep, clinical, and radiomics features. Experiments were conducted on the benchmark HECKTOR2021 and HECKTOR2022 challenge showed significantly better performance by surpassing the state-of-the-artwork and achieved 0.824/0.836 and 0.754/0.678 Dice/Concordance index for HECKTOR2021 and HECKTOR2022 respectively. We made the model and code publicly available. 11https://github.com/RespectKnowledge/HeadandNeck21_3D_Segmentation.

A Semi-Supervised Machine Learning Approach in Predicting High-Risk Pregnancies in the Philippines.

Early risk tagging is crucial in maternal health, especially because it threatens both the mother and the long-term development of the baby. By tagging high-risk pregnancies, mothers would be given extra care before, during, and after pregnancies, thus reducing the risk of complications. In the Philippines, where the fertility rate is high, especially among the youth, awareness of risks can significantly contribute to the overall outcome of the pregnancy and, to an extent, the Maternal mortality rate. Although supervised machine learning models have ubiquity as predictors, there is a gap when data are weak or scarce. Using limited collected data from the municipality of Daraga in Albay, the study first compared multiple supervised machine learning algorithms to analyze and accurately predict high-risk pregnancies. Through hyperparameter tuning, supervised learning algorithms such as Decision Tree, Random Forest, Support Vector Machine, K-Nearest Neighbors, Naïve Bayes, and Multilayer Perceptron were evaluated by using 10-fold cross validation to obtain the best parameters with the best scores. The results show that Decision Tree bested other algorithms and attained a test score of 93.70%. To address the gap, a semi-supervised approach using a Self-Training model was applied to the modified Decision Tree, which was then used as the base estimator with a 30% unlabeled dataset and achieved a 97.01% accuracy rate which outweighs similar studies.

Crystal twins: self-supervised learning for crystalline material property prediction

Machine learning (ML) models have been widely successful in the prediction of material properties. However, large labeled datasets required for training accurate ML models are elusive and computationally expensive to generate. Recent advances in Self-Supervised Learning (SSL) frameworks capable of training ML models on unlabeled data mitigate this problem and demonstrate superior performance in computer vision and natural language processing. Drawing inspiration from the developments in SSL, we introduce Crystal Twins (CT): a generic SSL method for crystalline materials property prediction that can leverage large unlabeled datasets. CT adapts a twin Graph Neural Network (GNN) and learns representations by forcing graph latent embeddings of augmented instances obtained from the same crystalline system to be similar. We implement Barlow Twins and SimSiam frameworks in CT. By sharing the pre-trained weights when fine-tuning the GNN for downstream tasks, we significantly improve the performance of GNN on 14 challenging material property prediction benchmarks.

Semi-supervised underwater acoustic source localization based on residual convolutional autoencoder

Passive localization of underwater targets was a thorny problem in underwater acoustics. For traditional model-driven passive localization methods, the main challenges are the inevitable environmental mismatch and the presence of interference and noise everywhere. In recent years, data-driven machine learning approaches have opened up new possibilities for passive localization of underwater acoustics. However, the acquisition and processing of underwater acoustics data are more restricted than other scenarios, and the lack of data is one of the most enormous difficulties in the application of machine learning to underwater acoustics. To take full advantage of the relatively easy accessed unlabeled data, this paper proposes a framework for underwater acoustic source localization based on a two-step semi-supervised learning classification model. The first step is trained in unsupervised mode with the whole available dataset (labeled and unlabeled dataset), and it consists of a convolutional autoencoder (CAE) for feature extraction and self-attention (RA) mechanism for picking more useful features by applying constraints on the CAE. The second step is trained in supervised mode with the labeled dataset, and it consists of a multilayer perceptron connected to an encoder from the first step and is used to perform the source location task. The proposed framework is validated on uniform vertical line array data of SWellEx-96 event S5. Compared with the supervised model and the model without the RA, the proposed framework maintains good localization performance with the reduced labeled dataset, and the proposed framework is more robust when the training dataset and the test dataset of the second step are distributed differently, which is called “data mismatch.”

Unsupervised Image-to-Image Translation: A Review.

Supervised image-to-image translation has been proven to generate realistic images with sharp details and to have good quantitative performance. Such methods are trained on a paired dataset, where an image from the source domain already has a corresponding translated image in the target domain. However, this paired dataset requirement imposes a huge practical constraint, requires domain knowledge or is even impossible to obtain in certain cases. Due to these problems, unsupervised image-to-image translation has been proposed, which does not require domain expertise and can take advantage of a large unlabeled dataset. Although such models perform well, they are hard to train due to the major constraints induced in their loss functions, which make training unstable. Since CycleGAN has been released, numerous methods have been proposed which try to address various problems from different perspectives. In this review, we firstly describe the general image-to-image translation framework and discuss the datasets and metrics involved in the topic. Furthermore, we revise the current state-of-the-art with a classification of existing works. This part is followed by a small quantitative evaluation, for which results were taken from papers.

Detecting Cheating in Large-Scale Assessment: The Transfer of Detectors to New Tests.

Recent approaches to the detection of cheaters in tests employ detectors from the field of machine learning. Detectors based on supervised learning algorithms achieve high accuracy but require labeled data sets with identified cheaters for training. Labeled data sets are usually not available at an early stage of the assessment period. In this article, we discuss the approach of adapting a detector that was trained previously with a labeled training data set to a new unlabeled data set. The training and the new data set may contain data from different tests. The adaptation of detectors to new data or tasks is denominated as transfer learning in the field of machine learning. We first discuss the conditions under which a detector of cheating can be transferred. We then investigate whether the conditions are met in a real data set. We finally evaluate the benefits of transferring a detector of cheating. We find that a transferred detector has higher accuracy than an unsupervised detector of cheating. A naive transfer that consists of a simple reuse of the detector increases the accuracy considerably. A transfer via a self-labeling (SETRED) algorithm increases the accuracy slightly more than the naive transfer. The findings suggest that the detection of cheating might be improved by using existing detectors of cheating at an early stage of an assessment period.

Semi-supervised learning based framework for urban level building electricity consumption prediction

The spatial feature of building energy consumption in a city is essential for urban level energy planning and policy making. With the increasing availability of urban level building energy benchmarking datasets, machine learning has shown a powerful capability of making data-driven predictions on urban level building energy consumption. However, the building energy benchmarking datasets usually only cover large buildings, which are not sufficient representations of all buildings in a city. Besides building energy benchmarking datasets, many other urban level open datasets are also valuable to building energy prediction, but they do not contain building energy use data, in other words, they are unlabeled. This study proposes a novel framework based on semi-supervised learning to make effective use of the unlabeled datasets to develop more generic urban level data-driven building energy prediction models, and energy mapping with higher space resolution. The framework consists of preliminary labeling, selection of pseudo labeled samples and predictive modelling. Several machine learning algorithms are proposed and compared for generating pseudo labels of building electricity consumption for unlabeled datasets of small and medium-sized buildings. A selection process consisting of convergence testing and screening is designed to select pseudo labeled samples with high credibility to enlarge the labeled dataset. A novel two-level performance evaluation method is proposed to evaluate the performance of the framework at both urban level and district level to enhance the spatial resolution of the predictions. The framework is implemented to model and map the individual electricity consumptions of all buildings in two years in the districts of New York City using multiple open datasets. The results show significant improvement in terms of prediction accuracy at both levels. In addition, the applicability of the model to various buildings in the city is remarkably enhanced.