Distribution Of Label Research Articles

Label distribution learning for compound facial expression recognition in‐the‐wild: A comparative study

AbstractHuman emotional states encompass both basic and compound facial expressions. However, current works primarily focus on basic expressions, consequently neglecting the broad spectrum of human emotions encountered in practical scenarios. Compound facial expressions involve the simultaneous manifestation of multiple emotions on an individual's face. This phenomenon reflects the complexity and richness of human states, where facial features dynamically convey a combination of feelings. This study embarks on a pioneering exploration of Compound Facial Expression Recognition (CFER), with a distinctive emphasis on leveraging the Label Distribution Learning (LDL) paradigm. This strategic application of LDL aims to address the ambiguity and complexity inherent in compound expressions, marking a significant departure from the dominant Single Label Learning (SLL) and Multi‐Label Learning (MLL) paradigms. Within this framework, we rigorously investigate the potential of LDL for a critical challenge in Facial Expression Recognition (FER): recognizing compound facial expressions in uncontrolled environments. We utilize the recently introduced RAF‐CE dataset, meticulously designed for compound expression assessment. By conducting a comprehensive comparative analysis pitting LDL against conventional SLL and MLL approaches on RAF‐CE, we aim to definitively establish LDL's superiority in handling this complex task. Furthermore, we assess the generalizability of LDL models trained on RAF‐CE by evaluating their performance on the EmotioNet and RAF‐DB Compound datasets. This demonstrates their effectiveness without domain adaptation. To solidify these findings, we conduct a comprehensive comparative analysis of 12 cutting‐edge LDL algorithms on RAF‐CE, S‐BU3DFE, and S‐JAFFE datasets, providing valuable insights into the most effective LDL techniques for FER in‐the‐wild.

A Multitask Dynamic Graph Attention Autoencoder for Imbalanced Multilabel Time Series Classification.

Graph learning is widely applied to process various complex data structures (e.g., time series) in different domains. Due to multidimensional observations and the requirement for accurate data representation, time series are usually represented in the form of multilabels. Accurately classifying multilabel time series can provide support for personalized predictions and risk assessments. It requires effectively capturing complex label relevance and overcoming imbalanced label distributions of multilabel time series. However, the existing methods are unable to model label relevance for multilabel time series or fail to fully exploit it. In addition, the existing multilabel classification balancing strategies suffer from limitations, such as disregarding label relevance, information loss, and sampling bias. This article proposes a dynamic graph attention autoencoder-based multitask (DGAAE-MT) learning framework for multilabel time series classification. It can fully and accurately model label relevance for each instance by using a dynamic graph attention-based graph autoencoder to improve multilabel classification accuracy. DGAAE-MT employs a dual-sampling strategy and cooperative training approach to improve the classification accuracy of low-frequency classes while maintaining the classification accuracy of high-frequency and mid-frequency classes. It avoids information loss and sampling bias. DGAAE-MT achieves a mean average precision (mAP) of 0.955 and an F1 score of 0.978 on a mixed medical time series dataset. It outperforms state-of-the-art works in the past two years.

Eliminating quantization errors in classification-based sound source localization

Sound Source Localization (SSL) involves estimating the Direction of Arrival (DOA) of sound sources. Since the DOA estimation output space is continuous, regression might be more suitable for DOA, offering higher precision. However, in practice, classification often outperforms regression, exhibiting greater robustness. Conversely, classification’s drawback is inherent quantization error. Within the classification paradigm, the DOA output space is discretized into several intervals, each treated as a class. These classes show strong inter-class correlations, being inherently ordered, with higher similarity as intervals grow closer. Nevertheless, this characteristic has not been fully exploited. To address this, we propose Unbiased Label Distribution (ULD) to eliminate quantization error in training targets. Furthermore, we introduce Weighted Adjacent Decoding (WAD) to overcome quantization error during the decoding stage. Finally, we tailor two loss functions for the soft labels: Negative Log Absolute Error (NLAE) and Mean Squared Error without activation (MSE(wo)). Experimental results show our approach surpasses classification quantization limits, achieving state-of-the-art performance. Our code and supplementary material are available at https://github.com/linfeng-feng/ULD.

Adaptive Deep Neural Network for Click-Through Rate estimation

Over the past few years, deep learning networks have been applied in the estimation of the Click-Through Rate (CTR). The main task of the CTR model lies in predicting a user’s positive/negative response to an item. Most of existing CTR models are constituted of two parts: a deep neural network (DNN) and a wide model (e.g., deep cross network). The wide models are generally designed to learn the feature interaction and contain much more parameters than the DNN. As a consequence, it is quite difficult for existing CTR models to work on the resource-limited devices. Blindly removing the wide model will significantly encumber the predictive performance of the algorithm. In this paper, we adopt the knowledge distillation to train a single multi-branch network and assemble all branches as a teacher network. Each branch network is a simple DNN which not only matches the ground-truth label distribution but also aligns the prediction distribution of the teacher network. One of branch networks, termed as Adaptive Deep Neural Network (ADNN), is trained independently and further combined with a wide model to learn feature interactions. Our method does not require pre-training any high-capacity teacher models, which endows our method with higher efficiency compared with existing ones. The experimental results tested on Criteo and Avazu datasets show that the hybrid model outperforms state-of-the-art methods, and the light model ADNN also has a considerable performance accuracy over certain modern complex models, demonstrating the superiority of the methodology.

Fusing spatial and frequency features for compositional zero-shot image classification

Compositional Zero-Shot Learning (CZSL) is a particular Zero-Shot Learning (ZSL) task that aims to utilize known concepts (e.g., states and objects) to identify novel state-object compositions for Image Classification. Previous works have primarily focused on disentangling concept compositions or exploring the complex interactions between the states and objects while neglecting the critical fact that the inference of many states and compositions is related to different frequency components, which should be analyzed from the global perspective. Therefore, we propose a Spatial-frequency Feature Fusion Network (SFFNet) to introduce a new branch that utilizes a frequency-domain filtering encoder to enhance key frequency components and capture non-local interactions adaptively. Besides, we also find that the widely used backbone in conventional CZSL settings behaves superior in perceiving local features. Thus, we construct a fusion block to combine both strengths to capture the local and non-local information. In addition, the traditional one-hot ground-truth distribution in the training phase does not reflect the accurate relationships between compositions, so we propose a composition-relation based label distribution regularization to encourage the model to actively learn the inner relationships between compositions, and extend this method to construct unseen composition pseudo distribution to further enhance the model’s generalization ability to unseen compositions. Extensive experiments and detailed analysis are conducted on three popular datasets, and the results show that our method can achieve state-of-the-art performance, which reveals its superiority in identifying novel compositions. Code is available at https://github.com/lisuyi/SFFNet_czsl.

Learning cluster-wise label distribution for label enhancement

Learning cluster-wise label distribution for label enhancement

Noise-Disruption-Inspired Neural Architecture Search with Spatial–Spectral Attention for Hyperspectral Image Classification

In view of the complexity and diversity of hyperspectral images (HSIs), the classification task has been a major challenge in the field of remote sensing image processing. Hyperspectral classification (HSIC) methods based on neural architecture search (NAS) is a current attractive frontier that not only automatically searches for neural network architectures best suited to the characteristics of HSI data, but also avoids the possible limitations of manual design of neural networks when dealing with new classification tasks. However, the existing NAS-based HSIC methods have the following limitations: (1) the search space lacks efficient convolution operators that can fully extract discriminative spatial–spectral features, and (2) NAS based on traditional differentiable architecture search (DARTS) has performance collapse caused by unfair competition. To overcome these limitations, we proposed a neural architecture search method with receptive field spatial–spectral attention (RFSS-NAS), which is specifically designed to automatically search the optimal architecture for HSIC. Considering the core needs of the model in extracting more discriminative spatial–spectral features, we designed a novel and efficient attention search space. The core component of this innovative space is the receptive field spatial–spectral attention convolution operator, which is capable of precisely focusing on the critical information in the image, thus greatly enhancing the quality of feature extraction. Meanwhile, for the purpose of solving the unfair competition issue in the traditional differentiable architecture search (DARTS) strategy, we skillfully introduce the Noisy-DARTS strategy. The strategy ensures the fairness and efficiency of the search process and effectively avoids the risk of performance crash. In addition, to further improve the robustness of the model and ability to recognize difficult-to-classify samples, we proposed a fusion loss function by combining the advantages of the label smoothing loss and the polynomial expansion perspective loss function, which not only smooths the label distribution and reduces the risk of overfitting, but also effectively handles those difficult-to-classify samples, thus improving the overall classification accuracy. Experiments on three public datasets fully validate the superior performance of RFSS-NAS.

Learning subjective time-series data via Utopia Label Distribution Approximation

Subjective time-series regression (STR) tasks have gained increasing attention recently. However, most existing methods overlook the label distribution bias in STR data, which results in biased models. Emerging studies on imbalanced regression tasks, such as age and depth estimations, hypothesize the label distribution is uniform and known. But in reality, the label distribution of test set in STR tasks is usually non-uniform and unknown. Moreover, the time-series data exhibits continuity in both temporal context and label spaces, which has not been addressed by existing methods. To tackle these issues, we propose a Utopia Label Distribution Approximation (ULDA) method, which approximates the training label distribution to the real-world but unknown (utopia) label distribution for calibrating the training and test sets. The utopia label distribution is generated by convolving the original one using a Gaussian kernel. ULDA also has two new devised modules (Time-slice Normal Sampling (TNS) generating required new samples and Convolution Weighted Loss (CWL) lowering learning weights for redundant samples), which not only assist the model training, but also maintain the sample continuity in temporal context space. Extensive experiments demonstrate that ULDA lifts the state-of-the-art performance on STR tasks and shows a considerable generalization ability to other time-series tasks.

Ototoxicity-related changes in GABA immunolabeling within the rat inferior colliculus

Several studies suggest that hearing loss results in changes in the balance between inhibition and excitation in the inferior colliculus (IC). The IC is an integral nucleus within the auditory brainstem. The majority of ascending pathways from the lateral lemniscus (LL), superior olivary complex (SOC), and cochlear nucleus (CN) synapse in the IC before projecting to the thalamus and cortex. Many of these ascending projections provide inhibitory innervation to neurons within the IC. However, the nature and the distribution of this inhibitory input have only been partially elucidated in the rat. The inhibitory neurotransmitter, gamma aminobutyric acid (GABA), from the ventral nucleus of the lateral lemniscus (VNLL), provides the primary inhibitory input to the IC of the rat with GABA from other lemniscal and SOC nuclei providing lesser, but prominent innervation.There is evidence that hearing related conditions can result in dysfunction of IC neurons. These changes may be mediated in part by changes in GABA inputs to IC neurons. We have previously used gene micro-arrays in a study of deafness-related changes in gene expression in the IC and found significant changes in GAD as well as the GABA transporters and GABA receptors (Holt 2005). This is consistent with reports of age and trauma related changes in GABA (Bledsoe et al., 1995; Mossop et al., 2000; Salvi et al., 2000). Ototoxic lesions of the cochlea produced a permanent threshold shift. The number, intensity, and density of GABA positive axon terminals in the IC were compared in normal hearing and deafened rats. While the number of GABA immunolabeled puncta was only minimally different between groups, the intensity of labeling was significantly reduced. The ultrastructural localization and distribution of labeling was also examined. In deafened animals, the number of immuno gold particles was reduced by 78 % in axodendritic and 82 % in axosomatic GABAergic puncta. The affected puncta were primarily associated with small IC neurons. These results suggest that reduced inhibition to IC neurons contribute to the increased neuronal excitability observed in the IC following noise or drug induced hearing loss. Whether these deafness diminished inhibitory inputs originate from intrinsic or extrinsic CNIC sources awaits further study.

Convergence analysis of a novel high order networks model based on entropy error function

It is generally known that the error function is one of the key factors that determine the convergence, stability and generalization ability of neural networks. For most feedforward neural networks, the squared error function is usually chosen as the error function to train the network. However, networks based on the squared error function can lead to slow convergence and easily fall into local optimum in the actual training process. Recent studies have found that, compared to the squared error function, the gradient method based on the entropy error function measures the difference between the probability distribution of the model output and the probability distribution of the true labels during the iterative process, which can be more able to handle the uncertainty in the classification problem, less likely to fall into a local optimum and can learn to converge more rapidly. In this paper, we propose a batch gradient method for Sigma-Pi-Sigma neural networks based on the entropy error function and rigorously demonstrate the weak and strong convergence of the new algorithm in the batch input mode. Finally, the theoretical results and effectiveness of the algorithm are verified by simulation.

A graph residual generation network for node classification based on multi-information aggregation

The key to improving the performance of graph convolutional networks (GCN) is to fully explore the correlation between neighboring and distant information. Aiming at the over-smoothing problem of GCN, in order to make full use of the relationship among features, graphs and labels, a graph residual generation network based on multi-information aggregation (MIA-GRGN) is proposed. Firstly, aiming at the defects of GCN, we design a deep initial residual graph convolution network (DIRGCN), which connects the initial input through residuals, so that each layer node retains part of the information of the initial features, ensuring the localization of the graph structure and effectively alleviating the problem of over-smoothing. Secondly, we propose a random graph generation method (RGGM) by utilizing graph edge sampling and negative edge sampling, and optimize the supervision loss function of DIRGCN in the form of generation framework. Finally, applying RGGM and DIRGCN as inference modules for modeling hypotheses and obtaining approximate posterior distributions of unknown labels, an optimized loss function is obtained, we construct a multi-information aggregation MIA-GRGN that combines graph structure, node characteristics and label joint distribution. Experiments on benchmark graph classification datasets show that MIA-GRGN achieves better classification results compared with the benchmark models and mainstream models, especially for datasets with less dense edge relationships between nodes.

Label distribution learning via second-order self-representation

Label distribution learning via second-order self-representation

Data-free knowledge distillation via generator-free data generation for Non-IID federated learning

Data heterogeneity (Non-IID) on Federated Learning (FL) is currently a widely publicized problem, which leads to local model drift and performance degradation. Because of the advantage of knowledge distillation, it has been explored in some recent work to refine global models. However, these approaches rely on a proxy dataset or a data generator. First, in many FL scenarios, proxy dataset do not necessarily exist on the server. Second, the quality of data generated by the generator is unstable and the generator depends on the computing resources of the server. In this work, we propose a novel data-Free knowledge distillation approach via generator-Free Data Generation for Non-IID FL, dubbed as FedF2DG. Specifically, FedF2DG requires only local models to generate pseudo datasets for each client, and can generate hard samples by adding an additional regularization term that exploit disagreements between local model and global model. Meanwhile, FedF2DG enables flexible utilization of computational resources by generating pseudo dataset locally or on the server. And to address the label distribution shift in Non-IID FL, we propose a Data Generation Principle that can adaptively control the label distribution and number of pseudo dataset based on client current state, and this allows for the extraction of more client knowledge. Then knowledge distillation is performed to transfer the knowledge in local models to the global model. Extensive experiments demonstrate that our proposed method significantly outperforms the state-of-the-art FL methods and can serve as plugin for existing Federated Learning methds such as FedAvg, FedProx, etc, and improve their performance.

Disentangled variational auto-encoder for multimodal fusion performance analysis in multimodal sentiment analysis

Multimodal Sentiment Analysis (MSA) holds extensive applicability owing to its capacity to analyze and interpret users' emotions, feelings, and perspectives by integrating complementary information from multiple modalities. However, inefficient and unbalanced cross-modal information fusion substantially undermines the accuracy and reliability of MSA models. Consequently, a critical challenge in the field now lies in effectively assessing the information integration capabilities of these models to ensure balanced and equitable processing of multimodal data. In this paper, a Disentanglement-based Variable Auto-Encoder (DVAE) is proposed for systematically assessing fusion performance and investigating the factors that facilitate multimodal fusion. Specifically, a distribution constraint module is presented to decouple the fusion matrices and generate multiple low-dimensional and trustworthy disentangled latent vectors that adhere to the authentic unimodal input distribution. In addition, a combined loss term is modified to effectively balance inductive bias, signal reconstruction, and distribution constraint items to facilitate the optimization of neural network weights and parameters. Utilizing the proposed evaluation method, we can evaluate the fusion performance of multimodal models by contrasting the classification degradation ratio derived from disentangled hidden representations and joint representations. Experiments conducted with eight state-of-the-art multimodal fusion methods on the CMU-MOSEI and CMU-MOSEI benchmark datasets demonstrate that DVAE is capable of effectively evaluating the effects of multimodal fusion. Moreover, the comparative experimental results indicate that the equalizing effect among various advanced mechanisms in multimodal sentiment analysis, as well as the single-peak characteristic of the ground label distribution, both contribute significantly to multimodal data fusion.

MLAWSMOTE: Oversampling in Imbalanced Multi-label Classification with Missing Labels by Learning Label Correlation Matrix

Missing labels in multi-label datasets are a common problem, especially for minority classes, which are more likely to occur. This limitation hinders the performance of classifiers in identifying and extracting information from minority classes. Oversampling is an effective method for addressing imbalanced multi-label problems by generating synthetic instances to create a class-balanced dataset. However, the existing oversampling algorithms mainly focus on the location of the generated data, and there is a lack of design on how to complete the labels of the synthetic data. To address this issue, we propose MLAWSMOTE, a synthetic data generation algorithm based on matrix factorization weights. We introduce a weak supervised learning method in the oversampling method, optimize the weights of features and labels by using label correlation, and iteratively learn the ideal label weights. The mapping relationship between features and labels is learned from the dataset and the label correlation matrix. The oversampling ratio is defined based on the discrepancy between observed labels and the ideal label of synthetic instances. It mitigates the impact of missing minority labels on the model’s predictions. The labeling of synthetic instances is performed based on label prediction, and the potential labeling distribution is complemented. Experimental results on multiple multi-label datasets under different label missing ratios demonstrate the effectiveness of the proposed method in terms of ACC, Hamming loss, MacroF1 and MicroF1. In the validation of the four classifiers, MacroF1 decreased by 24.78%, 17.81%, 3.8% and 19.56%, respectively, with the increase of label loss rate. After applying MLAWSMOTE only decreased by 15.79%, 13.63%, 3.78% and 15.21%.

Domain generalized open-set intelligent fault diagnosis based on feature disentanglement meta-learning

Abstract Existing domain generalization-based intelligent fault diagnosis methods mainly focus on learning domain-invariant features. However, in practical scenarios, these features are difficult to extract and effectively distinguish from class-related features. Moreover, these methods often assume identical label distributions between the source and target domain, making it challenging to handle scenarios where unknown classes exist in the target domain. To address these issues, this paper proposes a domain generalized open-set intelligent fault diagnosis method based on feature disentanglement meta-learning. A binary mask feature disentanglement module is constructed to overcome the information loss caused by feature reconstruction, enabling the separation of domain-specific and class-related features. Additionally, a meta-purification loss function is defined, incorporating a correlation loss term to remove impurity features from the class-related features, and further purifying class information through feature combination pairing. The method is trained on multiple source domains using a meta-learning strategy and generalized to target domains with unknown classes. The method is utilized for bearing fault diagnosis, designing multi-task experimental scenarios under different rotational speeds, and compared with existing domain generalization methods. Experimental results show that the proposed method exhibits excellent generalization ability and effectively addresses the issue of domain generalized open-set fault diagnosis.

Adaptive Weighted Ranking-Oriented Label Distribution Learning.

Label distribution learning (LDL) is a novel machine-learning paradigm generalized from multilabel learning (MLL). LDL attaches a label distribution to each instance, giving the description degree of different labels. In many real-world applications, key labels, that is, labels with relatively higher description degrees, are preferable to be better predicted. Unfortunately, existing LDL metrics measure the distance or similarity between label distributions from a global perspective, failing to give sufficient attention to key labels. Therefore, we design a novel LDL metric, the description-degree percentile average (DPA), which simultaneously integrates both the exact ranking value and the description degree of each label. The DPA can enhance accuracy in predicting key labels. Furthermore, noting the shape characteristics of the label distributions, we minimize the variance distance between the predicted and the ground-truth label distributions, to better maintain the distinguishability of labels. Finally, we propose an adaptive weighted ranking-oriented LDL algorithm, which is more suitable for realistic LDL problems that require higher accuracy in predicting key labels. We conduct extensive comparison experiments on various types of LDL datasets. Experimental results on both traditional and newly introduced metrics demonstrate the effectiveness of our proposal.

Structure-aware sign language recognition with spatial–temporal scene graph

Continuous sign language recognition (CSLR) is essential for the social participation of deaf individuals. The structural information of sign language motion units plays a crucial role in semantic representation. However, most existing CSLR methods treat motion units as a whole appearance in the video sequence, neglecting the exploitation and explanation of structural information in the models. This paper proposes a Structure-Aware Graph Convolutional Neural Network (SA-GNN) model for CSLR. This model constructs a spatial–temporal scene graph, explicitly capturing motion units’ spatial structure and temporal variation. Furthermore, to effectively train the SA-GNN, we propose an adaptive bootstrap strategy that enhances weak supervision using dense pseudo labels. This strategy incorporates a confidence cross-entropy loss to adjust the distribution of pseudo labels adaptively. Extensive experiments validate the effectiveness of our proposed method, achieving competitive results on popular CSLR datasets.

Incomplete label distribution learning via label correlation decomposition

Label distribution learning (LDL) has garnered increased attention in recent studies on label ambiguity. However, collecting complete annotations for LDL tasks is often time-consuming and labor-intensive compared to traditional learning paradigms. Therefore, designing effective incomplete LDL algorithms is crucial to broaden LDL’s application scope. In this paper, we propose a novel LDL algorithm, called Incomplete Label Distribution Learning via Label Correlation Decomposition (IncomLDL-LCD), which simultaneously learns label distributions and recovers missing description degrees of labels through label correlations. Specifically, we decompose the label correlation into sparse local label correlation and low-rank global label correlation using a soft-thresholding operator and a singular value thresholding operator, respectively. The former is utilized to capture the related label subsets necessary for reconstructing each possible label, while the latter focuses on extracting the coarse-grained semantic concepts from all labels and exploring the groupings of labels. Additionally, we develop an alternating solution with the accelerated proximal gradient descent method for optimization. Extensive experiments on 16 real-world data sets with varying degrees of missing annotations validate that our algorithm effectively handles incomplete LDL tasks and outperforms state-of-the-art algorithms.

Mitigating data imbalance and noise: A divergence-based approach with enhanced sample selection

Label noise and distribution imbalance are common challenges in real-world datasets. To address noisy labels, sample selection methods are typically used. These methods identify small-loss data as correctly labeled and utilize only these clean samples for updating parameters during training. However, our thorough analysis shows that in scenarios with a long-tailed distribution, methods based on small-loss tend to favor samples from dominant classes, leading to a significant drop in performance. To tackle this issue, we introduce a divergence-based separation mechanism coupled with prototype distance for clean sample identification across each class, effectively mitigating the adverse effects of distribution imbalance on sample selection. Furthermore, drawing on recent theoretical insights that highlight the benefits of pseudo-labels in managing class imbalances, we enhance our approach by integrating semi-supervised and contrastive learning techniques. Comprehensive tests conducted on two synthetic noisy long-tailed datasets and one real-world dataset demonstrate the efficacy of our method in dealing with label noise and long-tailed distributions.