Soft Labels Research Articles

One-shot Retinal Artery and Vein Segmentation via Cross-modality Pretraining

To perform one-shot retinal artery and vein segmentation with cross-modality artery-vein (AV) soft-label pretraining. Cross-sectional study. The study included 6479 color fundus photography (CFP) and arterial-venous fundus fluorescein angiography (FFA) pairs from 1964 participants for pretraining and 6 AV segmentation data sets with various image sources, including RITE, HRF, LES-AV, AV-WIDE, PortableAV, and DRSplusAV for one-shot finetuning and testing. We structurally matched the arterial and venous phase of FFA with CFP, the AV soft labels were automatically generated by utilizing the fluorescein intensity difference of the arterial and venous-phase FFA images, and the soft labels were then used to train a generative adversarial network to learn to generate AV soft segmentations using CFP images as input. We then finetuned the pretrained model to perform AV segmentation using only one image from each of the AV segmentation data sets and test on the remainder. To investigate the effect and reliability of one-shot finetuning, we conducted experiments without finetuning and by finetuning the pretrained model on an iteratively different single image for each data set under the same experimental setting and tested the models on the remaining images. The AV segmentation was assessed by area under the receiver operating characteristic curve (AUC), accuracy, Dice score, sensitivity, and specificity. After the FFA-AV soft label pretraining, our method required only one exemplar image from each camera or modality and achieved similar performance with full-data training, with AUC ranging from 0.901 to 0.971, accuracy from 0.959 to 0.980, Dice score from 0.585 to 0.773, sensitivity from 0.574 to 0.763, and specificity from 0.981 to 0.991. Compared with no finetuning, the segmentation performance improved after one-shot finetuning. When finetuned on different images in each data set, the standard deviation of the segmentation results across models ranged from 0.001 to 0.10. This study presents the first one-shot approach to retinal artery and vein segmentation. The proposed labeling method is time-saving and efficient, demonstrating a promising direction for retinal-vessel segmentation and enabling the potential for widespread application. Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Union label smoothing adversarial training: Recognize small perturbation attacks and reject larger perturbation attacks balanced

Recently, several adversarial training methods have been proposed for rejecting perturbation-based adversarial examples, which enhance the robustness of deep neural networks to larger perturbations. However, they often perform unsatisfactorily when dealing with examples with lower level perturbations. To address these issues, we introduce a novel adversarial training approach called the union label smoothing adversarial training (ULSAT), which employs a new label smoothing curve and a union strategy for adversarial training. The label smoothing curve assigns soft labels to perturbed examples, allowing for a more reasonable adjustment of calibration. The union strategy adds interpolation examples and combines adversarial examples generated under different perturbation tolerances into the training stage, which improves the rejection ability of the model and balances it with the classification ability. Through theoretical analysis and ablation study, we demonstrate the effectiveness of our proposed approach. Numerical experiments show that ULSAT can accurately classifies less disturbed examples while maintaining a good rejection ability for adversarial examples with higher levels of perturbation. Moreover, we introduce an evaluation index that comprehensively considers the classification ability and rejection ability of the model. Under this index, ULSAT achieves state-of-the-art results.

Unsupervised Vehicle Re-Identification Based on Cross-Style Semi-Supervised Pre-Training and Feature Cross-Division

Vehicle Re-Identification (Re-ID) based on Unsupervised Domain Adaptation (UDA) has shown promising performance. However, two main issues still exist: (1) existing methods that use Generative Adversarial Networks (GANs) for domain gap alleviation combine supervised learning with hard labels of the source domain, resulting in a mismatch between style transfer data and hard labels; (2) pseudo label assignment in the fine-tuning stage is solely determined by similarity measures of global features using clustering algorithms, leading to inevitable label noise in generated pseudo labels. To tackle these issues, this paper proposes an unsupervised vehicle re-identification framework based on cross-style semi-supervised pre-training and feature cross-division. The framework consists of two parts: cross-style semi-supervised pre-training (CSP) and feature cross-division (FCD) for model fine-tuning. The CSP module generates style transfer data containing source domain content and target domain style using a style transfer network, and then pre-trains the model in a semi-supervised manner using both source domain and style transfer data. A pseudo-label reassignment strategy is designed to generate soft labels assigned to the style transfer data. The FCD module obtains feature partitions through a novel interactive division to reduce the dependence of pseudo-labels on global features, and the final similarity measurement combines the results of partition features and global features. Experimental results on the VehicleID and VeRi-776 datasets show that the proposed method outperforms existing unsupervised vehicle re-identification methods. Compared with the last best method on each dataset, the method proposed in this paper improves the mAP by 0.63% and the Rank-1 by 0.73% on the three sub-datasets of VehicleID on average, and it improves mAP by 0.9% and Rank-1 by 1% on VeRi-776 dataset.

Labeling confidence for uncertainty-aware histology image classification.

Deep learning-based models applied to digital pathology require large, curated datasets with high-quality (HQ) annotations to perform correctly. In many cases, recruiting expert pathologists to annotate large databases is not feasible, and it is necessary to collect additional labeled data with varying label qualities, e.g., pathologists-in-training (henceforth, non-expert annotators). Learning from datasets with noisy labels is more challenging in medical applications since medical imaging datasets tend to have instance-dependent noise and suffer from high inter/intra-observer variability. In this paper, we design an uncertainty-driven labeling strategy with which we generate soft labels from 10 non-expert annotators for multi-class skin cancer classification. Based on this soft annotation, we propose an uncertainty estimation-based framework to handle these noisy labels. This framework is based on a novel formulation using a dual-branch min-max entropy calibration to penalize inexact labels during the training. Comprehensive experiments demonstrate the promising performance of our labeling strategy. Results show a consistent improvement by using soft labels with standard cross-entropy loss during training (∼4.0% F1-score) and increases when calibrating the model with the proposed min-max entropy calibration (∼6.6% F1-score). These improvements are produced at negligible cost, both in terms of annotation and calculation.

Class-Wise Subspace Alignment-Based Unsupervised Adaptive Land Cover Classification in Scene-Level Using Deep Siamese Network.

In this article, an unsupervised domain adaptation strategy has been investigated using a deep Siamese neural network in scene-level land cover classification using remotely sensed images. At the onset, the soft class label and probability scores of each target sample have been obtained using a pretrained model of a deep convolutional neural network. Thereafter, a semiautomatic threshold selection algorithm along with a graph-based approach has been explored to obtain the ``most-confident'' target samples. Furthermore, the deep Siamese network has been incorporated by training the source and ``most-confident'' target samples to generate the classwise cross domain common subspace. To assess the effectiveness of the proposed framework, experiments are carried out using three aerial image datasets. The results are found to be encouraging for the proposed scheme in comparison with the other state-of-art techniques.

Adaptive Collaborative Soft Label Learning for Unsupervised Multi-View Feature Selection

Unsupervised multi-view feature selection aims to select informative features with multi-view features and unsupervised learning. It is a challenging problem due to the absence of explicit semantic supervision. Recently, graph theory and hard pseudo-label learning have been adopted to solve multi-view feature selection problems under the unsupervised learning paradigm. However, graph-based methods are difficult to support large-scale real scenarios due to the high computational complexity of graph construction. Moreover, existing methods based on hard pseudo-label learning generally result in significant information loss. In this article, we propose an Adaptive Collaborative Soft Label Learning (ACSLL) model for unsupervised multi-view feature selection. In this model, collaborative soft label learning and multi-view feature selection are integrated into a unified framework. Specifically, we learn the pseudo soft labels from each view feature by a simple and efficient method and fuse them with an adaptive weighting strategy into a joint soft label matrix. This matrix is further used for guiding the feature selection process to identify valuable features. An effective optimization strategy guaranteed with proven convergence is derived to iteratively solve this problem. Experiments demonstrate the superiority of the proposed method in both feature selection accuracy and efficiency.

IterDE: An Iterative Knowledge Distillation Framework for Knowledge Graph Embeddings

Knowledge distillation for knowledge graph embedding (KGE) aims to reduce the KGE model size to address the challenges of storage limitations and knowledge reasoning efficiency. However, current work still suffers from the performance drops when compressing a high-dimensional original KGE model to a low-dimensional distillation KGE model. Moreover, most work focuses on the reduction of inference time but ignores the time-consuming training process of distilling KGE models. In this paper, we propose IterDE, a novel knowledge distillation framework for KGEs. First, IterDE introduces an iterative distillation way and enables a KGE model to alternately be a student model and a teacher model during the iterative distillation process. Consequently, knowledge can be transferred in a smooth manner between high-dimensional teacher models and low-dimensional student models, while preserving good KGE performances. Furthermore, in order to optimize the training process, we consider that different optimization objects between hard label loss and soft label loss can affect the efficiency of training, and then we propose a soft-label weighting dynamic adjustment mechanism that can balance the inconsistency of optimization direction between hard and soft label loss by gradually increasing the weighting of soft label loss. Our experimental results demonstrate that IterDE achieves a new state-of-the-art distillation performance for KGEs compared to strong baselines on the link prediction task. Significantly, IterDE can reduce the training time by 50% on average. Finally, more exploratory experiments show that the soft-label weighting dynamic adjustment mechanism and more fine-grained iterations can improve distillation performance.

Rethinking Label Refurbishment: Model Robustness under Label Noise

A family of methods that generate soft labels by mixing the hard labels with a certain distribution, namely label refurbishment, are widely used to train deep neural networks. However, some of these methods are still poorly understood in the presence of label noise. In this paper, we revisit four label refurbishment methods and reveal the strong connection between them. We find that they affect the neural network models in different manners. Two of them smooth the estimated posterior for regularization effects, and the other two force the model to produce high-confidence predictions. We conduct extensive experiments to evaluate related methods and observe that both effects improve the model generalization under label noise. Furthermore, we theoretically show that both effects lead to generalization guarantees on the clean distribution despite being trained with noisy labels.

Dual Label-Guided Graph Refinement for Multi-View Graph Clustering

With the increase of multi-view graph data, multi-view graph clustering (MVGC) that can discover the hidden clusters without label supervision has attracted growing attention from researchers. Existing MVGC methods are often sensitive to the given graphs, especially influenced by the low quality graphs, i.e., they tend to be limited by the homophily assumption. However, the widespread real-world data hardly satisfy the homophily assumption. This gap limits the performance of existing MVGC methods on low homophilous graphs. To mitigate this limitation, our motivation is to extract high-level view-common information which is used to refine each view's graph, and reduce the influence of non-homophilous edges. To this end, we propose dual label-guided graph refinement for multi-view graph clustering (DuaLGR), to alleviate the vulnerability in facing low homophilous graphs. Specifically, DuaLGR consists of two modules named dual label-guided graph refinement module and graph encoder module. The first module is designed to extract the soft label from node features and graphs, and then learn a refinement matrix. In cooperation with the pseudo label from the second module, these graphs are refined and aggregated adaptively with different orders. Subsequently, a consensus graph can be generated in the guidance of the pseudo label. Finally, the graph encoder module encodes the consensus graph along with node features to produce the high-level pseudo label for iteratively clustering. The experimental results show the superior performance on coping with low homophilous graph data. The source code for DuaLGR is available at https://github.com/YwL-zhufeng/DuaLGR.

Active consistency network for multi-source domain generalization in brain tumor segmentation

Recent advances in deep learning for brain tumor segmentation demonstrate good performance when the training and test data share the same distribution. However, medical images from different medical institutions often face distribution shifts in clinical applications. To deal with domain shift in multi-source domain generalization, we propose Active Consistency Network (ACN) with a single encoder and dual decoders to learn domain invariant features. A causal feature learning strategy and uncertainty-based refinery strategy are proposed to obtain reliable soft labels of reliable samples (Active Consistency Labels, ACL). Active consistency labels remove the noisy and unreliable samples for model consistency. Causality-based Random Masking (CRM) is proposed to obtain more stable causal features by adversarial learning. Causal features help achieve stable uncertainty estimation. Besides, two decoders are supervised by the active consistency labels learned from each other’s decoders, which can help decoders learn domain invariant features. Experimental results show that ACN utilizes causal feature learning and active consistency labels improving segmentation performance on multi-institutional brain tumor segmentation and achieving state-of-the-art performance compared to other domain generalization methods.

Dual Distillation Discriminator Networks for Domain Adaptive Few-Shot Learning

Domain Adaptive Few-Shot Learning (DA-FSL) aims at accomplishing few-shot classification tasks on a novel domain with the aid of a large number of source-style samples and several target-style samples. It is essential for DA-FSL to transfer task knowledge from the source domain to the target domain and overcome the asymmetry amount of labeled data in both domains. To this end, we propose Dual Distillation Discriminator Networks (D3Net) from the perspective of the lack of labeled target domain style samples in DA-FSL. Specifically, we employ the idea of distillation discrimination to avoid the over-fitting caused by the unequal number of samples in the target and source domains, which trains the student discriminator by the soft labels from the teacher discriminator. Meanwhile, we design the task propagation stage and the mixed domain stage respectively from the level of feature space and instances to generate more target-style samples, which apply the task distributions and the sample diversity of the source domain to enhance the target domain. Our D3Net realizes the distribution alignment between the source domain and the target domain and constraints the FSL task distribution by prototype distributions on the mixed domain. Extensive experiments on three DA-FSL benchmark datasets, i.e., mini-ImageNet, tiered-ImageNet, and DomainNet, demonstrate that our D3Net achieves competitive performance.

MuAt-Va: Multi-Attention and Video-Auxiliary Network for Device-Free Action Recognition

With the growing popularity of internet of things (IoT) systems, device-free action recognition begins to attract extensive attentions due to its friendly feasibility in broad applications such as human-computer interaction and smart elderly care. Considering abundant information in the vision modality, existing methods adopt the cross-model methods for performance enhancement. However, the dependency of synchronous multimodal data in the collection and recognition stage brings into the vision weaknesses such as sensitivity to occlusion and privacy invasion. In this paper, we integrate multi-attention structure and auxiliary video information into a novel end-to-end deep learning framework named MuAt-Va, in which video soft labels learned in advance are utilized to teach the multi-attention WiFi feature training process without vision information involved during test. Specifically, in order to enlarge the application scope and reduce the data cost, we beforehand acquire videos under a satisfactory condition only once, and then leverage teacher-student mechanism to guide the WiFi stream. Instead of straightforwardly concatenating multi-antenna channel state information (CSI) from homogeneous wireless signals as previous works, we design a CSI subcarrier-wise, temporal-wise and view-wise attention module to assign different weights on the basis of data characteristics for the sensing task. Our experiments with multiple subjects data in two scenes demonstrate that MuAt-Va can accurately recognize human actions with more superior performances.

Joint Latent Space and Label Inference Estimation with Adaptive Fused Data and Label Graphs

Recently, structured computing has become an interesting topic in the world of artificial intelligence, especially in the field of machine learning, as most researchers focus on the development of graph-based semi-supervised learning models. In this article, we present a new framework for graph-based semi-supervised learning. We present a powerful method for simultaneous label inference and linear transform estimation. The targeted linear transformation is used to obtain a discriminant subspace. To improve semi-supervised learning, our framework focuses on exploiting the data structure and soft labels of the available unlabeled samples. In the iterative optimization scheme used, the prior estimation of the label increases the supervision information indirectly through an introduced informative matrix called the label graph, thus avoiding the use of hard confidence-based decisions as used in self-supervised methods. In addition, the estimation of labels and projected data is made more robust by using smoothing concepts based on hybrid graphs. For each type of smoothing, the hybrid graph is an adaptive fusion of the two graphs encoding the similarity of the data and the similarity of the labels. The proposed method leads to an improved discriminative linear transformation. Several experimental results on real image datasets confirm the effectiveness of the proposed method. They also show superior performance compared to semi-supervised methods that use integration and label inference simultaneously.

SSDLog: a semi-supervised dual branch model for log anomaly detection

With versatility and complexity of computer systems, warning and errors are inevitable. To effectively monitor system’s status, system logs are critical. To detect anomalies in system logs, deep learning is a promising way to go. However, abnormal system logs in the real world are often difficult to collect, and effectively and accurately categorize the logs is an even time-consuming project. Thus, the data incompleteness is not conducive to the deep learning for this practical application. In this paper, we put forward a novel semi-supervised dual branch model that alleviate the need for large scale labeled logs for training a deep system log anomaly detector. Specifically, our model consists of two homogeneous networks that share the same parameters, one is called weak augmented teacher model and the other is termed as strong augmented student model. In the teacher model, the log features are augmented with small Gaussian noise, while in the student model, the strong augmentation is injected to force the model to learn a more robust feature representation with the guidance of teacher model provided soft labels. Furthermore, to further utilize unlabeled samples effectively, we propose a flexible label screening strategy that takes into account the confidence and stability of pseudo-labels. Experimental results show favorable effect of our model on prevalent HDFS and Hadoop Application datasets. Precisely, with only 30% training data labeled, our model can achieve the comparable results as the fully supervised version.

Facial expression recognition of masked faces using deep learning

Facial expression recognition (FER) represents one of the most prevalent forms of interpersonal communication, which contains rich emotional information. But it became even more challenging during the times of COVID, where face masks became a mandatory protection measure, leading to the challenge of occluded lower-face during facial expression recognition. In this study, deep convolutional neural network (DCNN) represents the core of both our full-face FER system and our masked face FER model. The focus was on incorporating knowledge distillation in transfer learning between a teacher model, which is the full-face FER DCNN, and the student model, which is the masked face FER DCNN via the combination of both the loss from the teacher soft-labels vs the student soft labels and the loss from the dataset hard-labels vs the student hard-labels. The teacher-student architecture used FER2013 and a masked customized version of FER2013 as datasets to generate an accuracy of 69% and 61% respectively. Therefore, the study proves that the process of knowledge distillation may be used as a way for transfer learning and enhancing accuracy as a regular DCNN model (student only) would result in 46% accuracy compared to our approach (61% accuracy).

A CNN-Based Born-Again TSK Fuzzy Classifier Integrating Soft Label Information and Knowledge Distillation

This paper proposes a CNN-based born-again Takagi-Sugeno-Kang (TSK) fuzzy classifier denoted as CNNBaTSK. CNNBaTSK achieves the following distinctive characteristics: 1) CNNBaTSK provides a new perspective of knowledge distillation with a non-iterative learning method (least learning machine with knowledge distillation, LLM-KD) to solve the consequent parameters of fuzzy rule, where consequent parameters are trained jointly on the ground-truth label loss, knowledge distillation loss, and regularization term; 2) with the inherent advantage of the fuzzy rule, CNNBaTSK has the capability to express the dark knowledge acquired from the CNN in an interpretable manner. Specifically, the dark knowledge (soft label information) is partitioned into five fixed antecedent fuzzy spaces. The centers of each soft label information in different fuzzy rules are {0, 0.25, 0.5, 0.75, 1}, which may have corresponding linguistic explanations: { <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">very low, low, medium, high, very high</i> }. For the consequent part of the fuzzy rule, the original features are employed to train the consequent parameters that ensure the direct interpretability in the original feature space. The experimental results on the benchmark datasets and the CHB-MIT EEG dataset demonstrate that CNNBaTSK can simultaneously improve the classification performance and model interpretability.

Combining layered label correction and mixup supervised contrastive learning to learn noisy labels

Deep neural networks trained using standard cross-entropy loss often induce overfitting of noisy labels that degrades performance. Label correction and contrastive learning have been theoretically demonstrated to be mutually beneficial in noisy label learning. However, the fusion of these two modules in practice has not been researched adequately, and the influence of each module on performance improvement has not been clarified. In this paper, we propose an effective noisy label learning framework that combines layered label correction with mixup supervised contrastive learning, based on hierarchical clustering using first neighbor relations and mixup contrastive loss. In particular, the framework consists of two modules—Layered Label Correction (LLC) and Mixup Supervised Contrastive Loss (MSCL). The LLC module aggregates the first neighbor's soft labels via label propagation among multilayer partitions based on the hierarchical clustering of the feature representation. Its purpose is to capture the underlying structure of learning representations at different levels of granularity. The MSCL module incorporates pseudo-labels into mixup contrastive learning directly to learn class-related representations of all instances of each class. Ablation studies are conducted to evaluate the contribution of individual modules to the overall performance enhancement of the framework. Moreover, experiments on synthetic datasets comprising noise of various patterns and levels demonstrate that the proposed framework yields a superior classification accuracy in comparison with existing alternatives.

ConvNeXt-based anchor-free object detection model for infrared image of power equipment

Infrared thermography has been widely used in the monitoring and diagnosis of power equipment to ensure the reliable functioning of power systems because of its non-contact and rapid capabilities. With the promotion of intelligent detection schemes in daily patrol inspection of power equipment by utilities, the amount of infrared images of power equipment (IIOPE) is growing geometrically. The research community has begun using deep learning technology to automatically detect objects in the IIOPE. However, the performance of existing detection methods is still not satisfactory because of the inherent attributes of power equipment infrared images, such as low contrast, complex background, and multi-scale object distribution. This research proposes an anchor-free detection model based on ConvNeXt to address these issues. An enhanced feature pyramid network (FPN) is presented to address the issue of multi-scale object distribution in the IIOPE. In addition, a dynamic soft label assignment strategy is proposed for precise object localization. Extensive experimental results on the self-built IIOPE dataset and the PASCAL VOC 2007 dataset demonstrate that the proposed method yields positive results. When compared to the current anchor-based as well as anchor-free infrared image detectors for power equipment, the proposed model offers improved accuracy while ensuring the detection speed that may satisfy the practical requirements.

DHSEGATs: distance and hop-wise structures encoding enhanced graph attention networks

Numerous works prove that existing neighbor-averaging graph neural networks (GNNs) cannot efficiently catch structure features, and many works show that injecting structure, distance, position, or spatial features can significantly improve the performance of GNNs, however, injecting high-level structure and distance into GNNs is an intuitive but untouched idea. This work sheds light on this issue and proposes a scheme to enhance graph attention networks (GATs) by encoding distance and hop-wise structure statistics. Firstly, the hop-wise structure and distributional distance information are extracted based on several hop-wise ego-nets of every target node. Secondly, the derived structure information, distance information, and intrinsic features are encoded into the same vector space and then added together to get initial embedding vectors. Thirdly, the derived embedding vectors are fed into GATs, such as GAT and adaptive graph diffusion network (AGDN) to get the soft labels. Fourthly, the soft labels are fed into correct and smooth (C&S) to conduct label propagation and get final predictions. Experiments show that the distance and hop-wise structures encoding enhanced graph attention networks (DHSEGATs) achieve a competitive result.

Multitask learning for acoustic scene classification with topic-based soft labels and a mutual attention mechanism

Acoustic scene classification (ASC) is a fundamental task of computational sound scene analysis that aims to identify the acoustic environment via audio. Many multitask learning (MTL) models have been proposed in computational sound scene analysis, but most of them are for acoustic event detection (AED). Existing MTL models for ASC usually leverage the knowledge of the primary and auxiliary tasks only via the shared layers and train the network using hard labels. They do not take advantage of the information contained in the primary and auxiliary tasks to improve the generalization performance, and ignore modeling the relationship between events, scenes or groups. Moreover, some models have the problem of subjectivity since they generate labels via observations, and subjectivity can create unreasonable information, which may restrict the improvement of system performance. To address these issues, we propose a novel MTL scheme for ASC that employs a mutual attention mechanism to explore the information contained in the primary and auxiliary tasks and employs a neural topic model to generate soft group labels automatically. The proposed method can model the relationship between groups and allows the primary and auxiliary tasks to make full use of each other’s information to improve generalization performance. Experimental results on two real-world datasets show that our MTL scheme can make full use of the auxiliary task to improve the performance of the ASC primary task and achieves significant improvements compared to baselines.