Unlabeled Dataset Research Articles

A Visual Representation-Guided Framework With Global Affinity for Weakly Supervised Salient Object Detection

Fully supervised salient object detection (SOD) methods have made considerable progress in performance, yet these models rely heavily on expensive pixel-wise labels. Recently, to achieve a trade-off between labeling burden and performance, scribble-based SOD methods have attracted increasing attention. Previous scribble-based models directly implement the SOD task only based on SOD training data with limited information, it is extremely difficult for them to understand the image and further achieve a superior SOD task. In this paper, we propose a simple yet effective framework guided by general visual representations with rich contextual semantic knowledge for scribble-based SOD. These general visual representations are generated by self-supervised learning based on large-scale unlabeled datasets. Our framework consists of a task-related encoder, a general visual module, and an information integration module to efficiently combine the general visual representations with task-related features to perform the SOD task based on understanding the contextual connections of images. Meanwhile, we propose a novel global semantic affinity loss to guide the model to perceive the global structure of the salient objects. Experimental results on five public benchmark datasets demonstrate that our method, which only utilizes scribble annotations without introducing any extra label, outperforms the state-of-the-art weakly supervised SOD methods. Specifically, it outperforms the previous best scribble-based method on all datasets with an average gain of 5.5% for max f-measure, 5.8% for mean f-measure, 24% for MAE, and 3.1% for E-measure. Moreover, our method achieves comparable or even superior performance to the state-of-the-art fully supervised models.

CLOP-hERG: The Contrastive Learning Optimized Pre-trained Model for Representation Learning in Predicting Drug-Induced hERG Channel Blockers

During drug development, ensuring that drug molecules do not block the hERG (human Ether-à-go-go-Related Gene) channel is critical. If this channel is blocked, it can cause many cardiovascular-related diseases. However, traditional experimental detection methods are expensive and time-consuming. In this work, we proposed a novel deep learning framework CLOP-hERG, that combines contrastive learning with the RoBERTa pre-trained model to predict whether drug molecules will block the hERG channel. We employed data augmentation techniques on molecular structures to ensure that our model can capture the multifaceted information of the molecules. Besides, we used a contrastive learning strategy to enable the model to learn meaningful molecular features from large unlabeled datasets. The RoBERTa pre-trained model played a pivotal role in this process, giving our model with a robust representational learning capability. The model, obtained through contrastive learning, was further fine-tuned to achieve high-precision prediction of hERG blockers. Through a series of experiments, we demonstrated the effectiveness of CLOP-hERG. This work provides a novel and effective strategy for predicting hERG blockers and provides some insights for other similar pharmaceutical tasks.

Cross-database and cross-channel electrocardiogram arrhythmia heartbeat classification based on unsupervised domain adaptation

The classification of electrocardiogram (ECG) plays a crucial role in the development of an automatic cardiovascular diagnostic system. However, the considerable variances in ECG signals between individuals pose a significant challenge. Changes in data distribution limit cross-domain utilization of a model. In this study, we propose a solution to classify ECG in an unlabeled dataset by leveraging knowledge obtained from labeled source domain. We present a domain-adaptive deep network based on cross-domain feature discrepancy optimization. Our method comprises three stages: pre-training, cluster-centroid computing, and adaptation. In pre-training, we employ a Distributionally Robust Optimization (DRO) technique to deal with the vanishing worst-case training loss. To enhance the richness of the features, we concatenate three temporal features with the deep learning features. The cluster computing stage involves computing centroids of distinctly separable clusters for the source using true labels, and for the target using confident predictions. We propose a novel technique for selecting confident predictions in the target domain. In the adaptation stage, we minimize compacting loss within the same cluster, separating loss across different clusters, inter-domain cluster discrepancy loss, and running combined loss to produce a domain-robust model. Experiments conducted in both cross-domain and cross-channel paradigms show the efficacy of the proposed method. Our method achieves superior performance compared to other state-of-the-art approaches in detecting ventricular ectopic beats (V), supraventricular ectopic beats (S), and fusion beats (F). Our method achieves an average improvement of 11.78% in overall accuracy over the non-domain-adaptive baseline method on the three test datasets.

Knowledge-guided pre-training and fine-tuning: Video representation learning for action recognition

Video-based action recognition is an important task in the computer vision community, aiming to extract rich spatial–temporal information to recognize human actions from videos. Many approaches adopt self-supervised learning in large-scale unlabeled datasets and exploit transfer learning in the downstream action recognition task. Though much progress has been made for action recognition with video representation learning, two main issues remain for most existing methods. Firstly, the pre-training with self-supervised pretext tasks usually learns neutral and not much informative representations for the downstream action recognition task. Secondly, the valuable learned knowledge from large-scaled pre-training datasets will be gradually forgotten in the fine-tuning stage. To address such issues, in this paper, we propose a novel video representation learning method with knowledge-guided pre-training and fine-tuning for action recognition, which incorporates external human parsing knowledge for generating informative representation in the pre-training, and preserves the pre-trained knowledge in the fine-tuning stage to avoid catastrophic forgetting via self-distillation. Our model, with contributions from the external human parsing knowledge, video-level contrastive learning, and knowledge preserving self-distillation, achieves state-of-the-art performance on two popular benchmarks, i.e., UCF101 and HMDB51, verifying the effectiveness of the proposed method.

Semi-Supervised Knowledge Distillation Via Teaching Assistant

As deep neural networks are widely used with computer vision, Model Compression Methods for knowledge distillation are being actively investigated in order to deploy them into smaller devices. However, when there are significant differences between models for students and teachers, a large amount of labeling wastes a great quantity of manpower while the performance of student learning decreases. In this paper, we propose an approach that uses semi-supervised teacher assistants to fuse knowledge distillation with teachers, effectively bridging the large teacher-student gap on a large number of unlabeled datasets and a small number of labeled datasets. We enable unsupervised teacher pre-training, fine-tuning, followed by teacher-assistant offline distillation, student-teacher-assistant oline distillation, and student-teacher offline distillation. We validate the effectiveness of the proposed method for the classification task using CIFAR10, CIFAR-100, and ImageNet.

Training multi-source domain adaptation network by mutual information estimation and minimization

We address the problem of Multi-Source Domain Adaptation (MSDA), which trains a neural network using multiple labeled source datasets and an unlabeled target dataset, and expects the trained network to well classify the unlabeled target data. The main challenge in this problem is that the datasets are generated by relevant but different joint distributions. In this paper, we propose to address this challenge by estimating and minimizing the mutual information in the network latent feature space, which leads to the alignment of the source joint distributions and target joint distribution simultaneously. Here, the estimation of the mutual information is formulated into a convex optimization problem, such that the global optimal solution can be easily found. We conduct experiments on several public datasets, and show that our algorithm statistically outperforms its competitors. Video and code are available at https://github.com/sentaochen/Mutual-Information-Estimation-and-Minimization.

DiffQuant: Reducing Compression Difference for Neural Network Quantization

Deep neural network quantization is a widely used method in the deployment of mobile or edge devices to effectively reduce memory overhead and speed up inference. However, quantization inevitably leads to a reduction in the performance and equivalence of models. Moreover, access to labeled datasets is often denied as they are considered valuable assets for companies or institutes. Consequently, performing quantization training becomes challenging without sufficient labeled datasets. To address these issues, we propose a novel quantization pipeline named DiffQuant, which can perform quantization training using unlabeled datasets. The pipeline includes two cores: the compression difference (CD) and model compression loss (MCL). The CD can measure the degree of equivalence loss between the full-precision and quantized models, and the MCL supports fine-tuning the quantized models using unlabeled data. In addition, we design a quantization training scheme that allows the quantization of both the batch normalization (BN) layer and the bias. Experimental results show that our method outperforms state-of-the-art methods on ResNet18/34/50 networks, maintaining performance with a reduced CD. We achieve Top-1 accuracies of 70.08%, 74.11%, and 76.16% on the ImageNet dataset for the 8-bit quantized ResNet18/34/50 models and reduce the gap to 0.55%, 0.61%, and 0.71% with the full-precision network, respectively. We achieve CD values of only 7.45%, 7.48%, and 8.52%, which allows DiffQuant to further exploit the potential of quantization.

Utilization of deep learning tools to map and monitor biological soil crusts

Biological soil crusts (biocrusts) form a layer of only one to few centimeters depth on the soil surface and occur mostly in hot and cold deserts. Biocrusts have a major impact on different processes in these ecosystems, like carbon and nitrogen cycling, biodiversity preservation, erosion protection and soil dust emission reduction, but also react highly sensitive upon climate alterations and land use intensification. Therefore, monitoring tools are required to keep track of the changes of these specialized communities in an altering environment. In the current study, we applied a semantic image segmentation approach, using neural networks. One main problem to be solved was, that the training data and target data, on which the model is applied, are often recorded with different camera devices. This leads to different statistical properties of the image data, like different scale, resolution, brightness etc., which could significantly affect the model's performance. To solve this problem, we propose a new domain adaption method using a joint energy-based approach. To test a semantic segmentation approach in general, we utilized biocrust imagery taken in Utah (United States of America) and two sub datasets from the National Park Gesäuse (Austria). Here, we achieved highly reliable results with an overall classification accuracy of 85.9% for the USA data and 88.6% and 91.4%, respectively, for the two sub datasets of the National Park Gesäuse. To test our joint energy-based domain adaption approach, we used the two sub datasets from the National Park Gesäuse, which were recorded with different camera devices. With this newly established approach, we improved the accuracy of our segmentation on the unlabeled sub dataset from 70.4% to 75.3%. The results suggest that joint energy-based modelling is a well-suited domain adaption method for semantic segmentation that could be applied to face various deep learning and image-based biomonitoring challenges.

A semi-supervised deep learning approach for cropped image detection

Deep learning algorithms have shown great performance in multimedia forensics applications using supervised learning on large-scale labeled datasets. However, constructing such extensive labeled datasets can be challenging and costly in several multimedia forensics scenarios. Additionally, heavyweight deep learning models with complex architectures and a large number of parameters require significant hardware resources for training. To address these challenges in the context of image cropping detection, a common multimedia forensics application, we propose a semi-supervised deep learning framework capable of training on a large amount of unlabeled image samples. In this framework, we leverage a teacher model, trained on a small set of labeled image samples, to rank the confidence scores of image samples in a large-scale unlabeled dataset. By utilizing the ranked image samples, we train a student network successfully. To validate the effectiveness of our collaborative training framework across various image cropping detection scenarios, we conduct extensive experiments on a large-scale dataset. The experimental results clearly demonstrate that our semi-supervised learning approach achieved a state-of-the-art performance compared to existing supervised detection frameworks, achieving an accuracy of 91.79% on the BOSSbase dataset and 89.23% on the Alaska dataset. Furthermore, we conducted in-depth research on various factors that influence detection performance in the context of semi-supervised learning. These factors include pairings of teacher–student models, the top-K selection approach, the number of unlabeled samples, the number of iterations in self-training, and the proportion of high-confidence samples using in semi-supervised learning.

Expert teacher based on foundation image segmentation model for object detection in aerial images

Despite the remarkable progress of general object detection, the lack of labeled aerial images limits the robustness and generalization of the detector. Teacher–student learning is a feasible solution on natural image domain, but few works focus on unlabeled aerial images. Inspired by foundation models with the powerful generalization in computer vision field, we propose an expert teacher framework based on foundation image segmentation model called ET-FSM. Our approach provides the performance gains for the student detector by generating high-quality pseudo-labels for unlabeled aerial images. In the ET-FSM, we design the binary detector with expert guidance mechanism to sufficiently leverage the extra knowledge obtained from the foundation image segmentation model, which accurately detects object positions in the complex backgrounds. Also, we present the momentum contrast classification module to distinguish confused object categories in aerial images. To demonstrate the effectiveness of the proposed method, we construct an unlabeled aerial image dataset covering various scenes. The experiments are conducted on diverse types of student detectors. The results show that the proposed approach achieves superior performance compared to existing methods, and allows the student detector to achieve fully supervised performance with much less labeled aerial images. Our dataset and code are available at https://github.com/cq100/ET-FSM.

Machine learning‐based approach for maritime target classification and anomaly detection using millimetre wave radar Doppler signatures

AbstractThe authors present multiple machine learning‐based methods for distinguishing maritime targets from sea clutter. The main goal for this classification framework is to aid future millimetre wave radar system design for marine autonomy. Availability of empirical data at this frequency range in the literature is scarce. The classification and anomaly detection techniques reported here use experimental data collected from three different field trials from three different millimetre wave radars. Two W‐band radars operating at 77 and 94 GHz and a G‐band radar operating at 207 GHz were used for the field trial data collection. The dataset encompasses eight classes including sea clutter returns. The other targets are boat, stand up paddleboard/kayak, swimmer, buoy, pallet, stationary solid object (i.e. rock) and sea lion. The Doppler signatures of the targets have been investigated to generate feature values. Five feature values have been extracted from Doppler spectra and four feature values from Doppler spectrograms. The features were trained on a supervised learning model for classification as well as an unsupervised model for anomaly detection. The supervised learning was performed for both multi‐class and 2‐class (sea clutter and target) classification. The classification based on spectrum features provided an 84.3% and 80.1% validation and test accuracy respectively for the multi‐class classification. For the spectrogram feature‐based learning, the validation and test accuracy for multi‐class increased to 93.3% and 88.7% respectively. For the 2‐class classification, the spectrum feature‐based training accuracies are 88.1% and 86.8%, whereas with the spectrogram feature‐based model, the values are 95% and 94.1% for validation and test accuracies respectively. A one class support vector machine was also applied to an unlabelled dataset for anomaly detection training, with 10% outlier data. The cross‐validation accuracy has shown very good agreement with the expected anomaly detection rate.

SHIP DETECTION IN COSMO-SKYMED SAR IMAGERY USING A NOVEL CNN-BASED DETECTOR: A CASE STUDY FROM THE SUEZ CANAL

Abstract. The Suez Canal, strategically located as the shortest international sea route, plays a crucial role in facilitating the transportation of goods between Asia and Europe. However, the occurrence of traffic disruptions within the Canal poses a serious threat to global trade, as evidenced by the recent incident of the container ship Ever Given, which ran aground in the Suez Canal on March 23, 2021. This event led to a complete blockade of the Canal that lasted for six days, resulting in a fleet of ships waiting to pass through the Canal. This highlights the need to monitor the Canal to prevent similar disturbances in the future. In this paper, we propose a CNN-based attention-guided self-learning framework for ship detection from 3m high-resolution COSMO-SkyMed SAR imagery acquired in April 2021 via the Egyptian Suez Canal. We introduce a self-learning augmented segmentation (SLAS) technique to augment the dataset with new ship samples by pseudo-labeling an unlabeled dataset. We also present the Attention-guided Feature Refinement (AFR) module to extract more significant semantic features and contextual information, especially for ships of varying sizes in SAR images. Finally, the AFR module is fed into a Region Proposal Network (RPN) to generate a set of proposal anchors, which are later used in a Deep Detection Network (DDN) for ship classification and localization. Our experimental results demonstrate that the proposed method outperforms current state-of-the-art detection models in terms of detection accuracy, particularly in complex coastal scenes, with an overall accuracy of up to 87% mean average precision (mAP).

Learning Gait Representation from Massive Unlabelled Walking Videos: A Benchmark.

Gait depicts individuals' unique and distinguishing walking patterns and has become one of the most promising biometric features for human identification. As a fine-grained recognition task, gait recognition is easily affected by many factors and usually requires a large amount of completely annotated data that is costly and insatiable. This paper proposes a large-scale self-supervised benchmark for gait recognition with contrastive learning, aiming to learn the general gait representation from massive unlabelled walking videos for practical applications via offering informative walking priors and diverse real-world variations. Specifically, we collect a large-scale unlabelled gait dataset GaitLU-1M consisting of 1.02M walking sequences and propose a conceptually simple yet empirically powerful baseline model GaitSSB. Experimentally, we evaluate the pre-trained model on four widely-used gait benchmarks, CASIA-B, OU-MVLP, GREW and Gait3D with or without transfer learning. The unsupervised results are comparable to or even better than the early model-based and GEI-based methods. After transfer learning, GaitSSB outperforms existing methods by a large margin in most cases, and also showcases the superior generalization capacity. Further experiments indicate that the pre-training can save about 50% and 80% annotation costs of GREW and Gait3D. Theoretically, we discuss the critical issues for gait-specific contrastive framework and present some insights for further study. As far as we know, GaitLU-1M is the first large-scale unlabelled gait dataset, and GaitSSB is the first method that achieves remarkable unsupervised results on the aforementioned benchmarks. The source code of GaitSSB and anonymous data of GaitLU-1M is available at https://github.com/ShiqiYu/OpenGait.

Unsupervised feature selection by learning exponential weights

Unsupervised feature selection has gained considerable attention for extracting valuable features from unlabeled datasets. Existing approaches typically rely on sparse mapping matrices to preserve local neighborhood structures. However, this strategy favors large-weight features, potentially overlooking smaller yet valuable ones and distorting data distribution and feature structure. Besides, some methods focus on local structure information, failing to explore global information. To address these limitations, we introduce an exponential weighting mechanism to induce a rational feature distribution and explore data structure in the feature subspace. Specifically, we propose a unified framework incorporating local structure learning and exponentially weighted sparse regression for optimal feature combinations, preserving global and local information. Experimental results demonstrate the superiority of our approach over existing unsupervised feature selection methods.

A novel and effective method based deep learning model for detecting non-technical electricity losses

<p>This study focused on non-technical electricity loss detection. As mentioned, non-technical losses (NTLs) affect utilities and economies financially. Electricity theft, fraud, and metering issues can create NTLs. NTL generate most distribution losses in electrical power networks, costing utilities a lot. NTL detection approaches are data-focused, network-oriented, or hybrid. Data-oriented writing dominated this analysis. After data collection and cleaning and labeling the unlabeled dataset with a target, a methodology was supplied that used four machine learning techniques random forest, decision tree, KNN, and logistic regression and four neural network models-DNN, CNN, CNN-LSTM, and CNN-GRU. The CNN and DNN model have the best accuracy, stability, fast learning, and training time.</p>

YoloCurvSeg: You only label one noisy skeleton for vessel-style curvilinear structure segmentation.

Weakly-supervised learning (WSL) has been proposed to alleviate the conflict between data annotation cost and model performance through employing sparsely-grained (i.e., point-, box-, scribble-wise) supervision and has shown promising performance, particularly in the image segmentation field. However, it is still a very challenging task due to the limited supervision, especially when only a small number of labeled samples are available. Additionally, almost all existing WSL segmentation methods are designed for star-convex structures which are very different from curvilinear structures such as vessels and nerves. In this paper, we propose a novel sparsely annotated segmentation framework for curvilinear structures, named YoloCurvSeg. A very essential component of YoloCurvSeg is image synthesis. Specifically, a background generator delivers image backgrounds that closely match the real distributions through inpainting dilated skeletons. The extracted backgrounds are then combined with randomly emulated curves generated by a Space Colonization Algorithm-based foreground generator and through a multilayer patch-wise contrastive learning synthesizer. In this way, a synthetic dataset with both images and curve segmentation labels is obtained, at the cost of only one or a few noisy skeleton annotations. Finally, a segmenter is trained with the generated dataset and possibly an unlabeled dataset. The proposed YoloCurvSeg is evaluated on four publicly available datasets (OCTA500, CORN, DRIVE and CHASEDB1) and the results show that YoloCurvSeg outperforms state-of-the-art WSL segmentation methods by large margins. With only one noisy skeleton annotation (respectively 0.14%, 0.03%, 1.40%, and 0.65% of the full annotation), YoloCurvSeg achieves more than 97% of the fully-supervised performance on each dataset. Code and datasets will be released at https://github.com/llmir/YoloCurvSeg.

Learning feature representations from unlabeled data for volcano-seismic event classification

Accurately identifying and labeling seismic events is essential for understanding the internal dynamics of volcanoes and predicting volcanic eruptions. However, the manual labeling of seismic events is a labor-intensive and resource-demanding process that requires the expertise of highly skilled vulcanologist professionals. Unsupervised learning approaches provide an alternative solution by automatically discovering hidden structures in the data, thereby enabling the characterization of seismic events without requiring manual labeling. We compare two widely recognized unsupervised learning algorithms, k-means and the Gaussian mixture model (GMM), which employ different assignment strategies (i.e., hard assignment for k-means and soft assignment for GMM), to learn a feature representation from an unlabeled dataset collected from the Cotopaxi volcano. The learned feature representations are then evaluated on a supervised task to classify LP and VT events using labeled datasets from the Cotopaxi and Llaima volcanoes. Additionally, we compare our approach to a baseline composed of hand-crafted features engineered by domain experts over the years. Our findings highlight the effectiveness of unsupervised techniques in learning a wide range of relevant features that exhibit a high correlation with the baseline. Furthermore, when assessed in a classification task, our unsupervised approach achieves a statistically similar performance compared to the baseline features. Notably, the GMM feature representation outperforms the baseline features in effectively handling VT and LP noisy signals from the BREF station at Cotopaxi volcano.

Self-supervised enhanced thyroid nodule detection in ultrasound examination video sequences with multi-perspective evaluation

Objective. Ultrasound is the most commonly used examination for the detection and identification of thyroid nodules. Since manual detection is time-consuming and subjective, attempts to introduce machine learning into this process are ongoing. However, the performance of these methods is limited by the low signal-to-noise ratio and tissue contrast of ultrasound images. To address these challenges, we extend thyroid nodule detection from image-based to video-based using the temporal context information in ultrasound videos. Approach. We propose a video-based deep learning model with adjacent frame perception (AFP) for accurate and real-time thyroid nodule detection. Compared to image-based methods, AFP can aggregate semantically similar contextual features in the video. Furthermore, considering the cost of medical image annotation for video-based models, a patch scale self-supervised model (PASS) is proposed. PASS is trained on unlabeled datasets to improve the performance of the AFP model without additional labelling costs. Main results. The PASS model is trained by 92 videos containing 23 773 frames, of which 60 annotated videos containing 16 694 frames were used to train and evaluate the AFP model. The evaluation is performed from the video, frame, nodule, and localization perspectives. In the evaluation of the localization perspective, we used the average precision metric with the intersection-over-union threshold set to 50% (AP@50), which is the area under the smoothed Precision-Recall curve. Our proposed AFP improved AP@50 from 0.256 to 0.390, while the PASS-enhanced AFP further improved the AP@50 to 0.425. AFP and PASS also improve the performance in the valuations of other perspectives based on the localization results. Significance. Our video-based model can mitigate the effects of low signal-to-noise ratio and tissue contrast in ultrasound images and enable the accurate detection of thyroid nodules in real-time. The evaluation from multiple perspectives of the ablation experiments demonstrates the effectiveness of our proposed AFP and PASS models.

Discordant pair analysis for sample efficient model evaluation

We present a new technique for assessing the effectiveness of a classification algorithm using discordant pair analysis. This method utilizes a known performance baseline algorithm and a large unlabeled dataset with an assumed class distribution to obtain overall performance estimates by only assessing the subset of examples that the algorithms classify discordantly. Our approach offers an efficient way to evaluate the performance of an algorithm that minimizes the human adjudications needed while also maintaining precision in the evaluation and in some cases improving the evaluation quality by reducing human adjudication errors. This approach is a computationally efficient alternative to the traditional exhaustive method of performance evaluation and has the potential to improve the accuracy of performance estimates. Simulation studies show that the discordant pair method reduces the number of adjudications by over 90%, while maintaining the same level of sensitivity and specificity.

BatmanNet: bi-branch masked graph transformer autoencoder for molecular representation.

Although substantial efforts have been made using graph neural networks (GNNs) for artificial intelligence (AI)-driven drug discovery, effective molecular representation learning remains an open challenge, especially in the case of insufficient labeled molecules. Recent studies suggest that big GNN models pre-trained by self-supervised learning on unlabeled datasets enable better transfer performance in downstream molecular property prediction tasks. However, the approaches in these studies require multiple complex self-supervised tasks and large-scale datasets , which are time-consuming, computationally expensive and difficult to pre-train end-to-end. Here, we design a simple yet effective self-supervised strategy to simultaneously learn local and global information about molecules, and further propose a novel bi-branch masked graph transformer autoencoder (BatmanNet) to learn molecular representations. BatmanNet features two tailored complementary and asymmetric graph autoencoders to reconstruct the missing nodes and edges, respectively, from a masked molecular graph. With this design, BatmanNet can effectively capture the underlying structure and semantic information of molecules, thus improving the performance of molecular representation. BatmanNet achieves state-of-the-art results for multiple drug discovery tasks, including molecular properties prediction, drug-drug interaction and drug-target interaction, on 13 benchmark datasets, demonstrating its great potential and superiority in molecular representation learning.