Label Ambiguity Research Articles

Re-Weighting Large Margin Label Distribution Learning for Classification.

Label ambiguity has attracted quite some attention among the machine learning community. The latterly proposed Label Distribution Learning (LDL) can handle label ambiguity and has found wide applications in real classification problems. In the training phase, an LDL model is learned first. In the test phase, the top label(s) in the label distribution predicted by the learned LDL model is (are) then regarded as the predicted label(s). That is, LDL considers the whole label distribution in the training phase, but only the top label(s) in the test phase, which likely leads to objective inconsistency. To avoid such inconsistency, we propose a new LDL method Re-Weighting Large Margin Label Distribution Learning (RWLM-LDL). First, we prove that the expected L1-norm loss of LDL bounds the classification error probability, and thus apply L1-norm loss as the learning metric. Second, re-weighting schemes are put forward to alleviate the inconsistency. Third, large margin is introduced to further solve the inconsistency. The theoretical results are presented to showcase the generalization and discrimination of RWLM-LDL. Finally, experimental results show the statistically superior performance of RWLM-LDL against other comparing methods.

Fast Label Enhancement for Label Distribution Learning

Label Distribution Learning (LDL) has attracted increasing research attentions due to its potential to address the label ambiguity problem in machine learning and success in many real-world applications. In LDL, it is usually expensive to obtain the ground-truth label distributions of data, but it is relatively easy to obtain the logical labels of data. How to use training instances only with logical labels to learn an effective LDL model is a challenging problem. In this paper, we propose a two-step framework to address this problem. Specifically, we firstly design an efficient recovery model to recover the latent label distributions of training instances, named Fast Label Enhancement (FLE). Our idea is to use non-negative matrix factorization (NMF) to mine the label distribution information from the feature space. Moreover, we take the instance-class similarities into consideration to discover the importance of each label to training instances, which is useful for learning precise label distributions. Then, we train a predictive model for testing instances based on generated label distributions of training instances and an existing LDL method (e.g., SA-BFGS). Experimental results on fifteen benchmark datasets show the effectiveness of the proposed two-step framework and verify the superiority of FLE over several state-of-the-art approaches.

Deep Conditional Distribution Learning for Age Estimation

Age estimation is a challenging task not only because face appearance is affected by illumination, pose, and expression, but also because there exists age label ambiguity among different demographic groups. In this work, we first revisit different label distribution learning (LDL) based age estimation methods and propose a more general formulation, which can unify individual LDL-based age estimation methods, as well as the traditional regression, classification, and ranking based age estimation methods. Based on such a general formulation, we propose a novel deep conditional distribution learning (DCDL) method, which can flexibly leverage a varying number of auxiliary face attributes to achieve adaptive age-related feature learning and improve age estimation robustness against the challenges above. Experimental results on multiple age estimation datasets (MORPH II, AgeDB, FG-NET, MegaAge-Asian, CLAP2016, UTK-Face, and LFW+) show that the proposed approach outperforms the state-of-the-art age estimation methods by a large margin. In addition, the proposed approach can generalize well to other human attributes estimation tasks, like height, weight, and body mass index (BMI) estimation.

Uncertainty in Bayesian deep label distribution learning

Though widely used, most deep convolutional neural networks fail to capture prediction uncertainty, which can be crucial in scenarios such as automotive applications and disease diagnosis. Aleatoric and epistemic uncertainties have been proposed in the Bayesian deep learning framework for regression and classification, which require training images with unambiguous labels for success. Some situations do not have precise labels by nature, such as age estimation or lesion contour annotation by different physicians in the real world. Label distribution learning (LDL) has been proposed to account for the label ambiguity. However, uncertainty estimation has not been studied for LDL. This study presents a Bayesian deep label distribution learning (BLDL) to obtain the uncertainties of LDL tasks. We define semantic uncertainty to account for the incompleteness of data acquisition and inaccurate data labeling and unify the three types of uncertainties’ mathematical expressions for classification and LDL. We further demonstrate that applying the three types of uncertainties in the loss functions significantly improves model performance, and using semantic uncertainty as a criterion to select the training samples substantially promotes active learning.

Incorporating Phase-Encoded Spectrum Masking into Speaker-Independent Monaural Source Separation

Abstract Typical speech separation systems usually operate in the time-frequency (T-F) domain by enhancing the magnitude response and leaving the phase response unaltered. Recent studies, however, suggest that phase is important for perceptual quality, leading many researchers to consider magnitude and phase spectrum enhancements. The complex-valued Fourier spectrum and real-valued shifted real spectrum (SRS) of a discrete-time speech signal sequence carry information both in magnitude and phase functions, which can reconstruct the speech signal without loss. In this paper, we propose novel methods called the pcIRM and pSRSM to solve the problem of speaker-independent monaural source separation and to recover the phase of source speech signal. Specifically, the pcIRM achieves the complex ideal ratio mask (cIRM) estimation based on the complex-valued phase-encoded Fourier spectrogram, and the pSRSM obtains the shifted real spectrum mask (SRSM) estimation based on the real-valued phase-encoded shifted real spectrum, both of which are creatively implemented with the deep bi-directional Long Short-Term Memory networks. Furthermore, the merging of these masking estimation methods and the utterance-level permutation invariant training (uPIT) approach has been proved to be an effective way to improve speech separation. Typically, the uPIT addresses the label ambiguity problem well, which is the major barrier for speaker-independent multi-talker source separation. Finally, we report separation results for the proposed methods and compare them with that of the state-of-the-art methods, which are evaluated with the WSJ0-2mix datasets. Extensive experiment results demonstrate the advantages of our proposed pcIRM method in terms of the signal-to-distortion ratio (SDR) and the perceptual evaluation of speech quality (PESQ) metrics, and the pSRSM method obtains comparable performance to the pcIRM in the opposite gender speakers source separation circumstance with smaller model complexity.

Label distribution feature selection for multi-label classification with rough set

Multi-label learning deals with cases where every instance corresponds to multiple labels. The objective is to learn mapping from an instance to a relevant label set. Existing multi-label learning approaches assume that the significance for all related labels is same for every instance. Several problems of label ambiguity can be dealt with using multi-label learning, but some practical applications with significance among related labels for every instance cannot be effectively processed. To achieve superior results by conducting different significance of labels, label distribution learning is used for such applications. First, the probability model and rough set are embedded in the labeling significance, thus more supervised information can be obtained from original multi-label data. Subsequently, to resolve the feature selection problem of label distribution data, according to the feature dependency and the rough set, a novel feature selection algorithm for multi-label classification is designed. Finally, to verify the effectiveness of the proposed algorithms, an extensive experiment is conducted on 15 real-world multiple label data sets. The performance of the proposed algorithm through the multi-label classifier is compared with seven state-of-the-art approaches, thereby indicating the applicability and effectiveness of label distribution feature selection.

Deep Semantic Parsing of Freehand Sketches With Homogeneous Transformation, Soft-Weighted Loss, and Staged Learning

In this paper, we propose a novel deep framework for part-level semantic parsing of freehand sketches, which makes three main contributions that are experimentally shown to have substantial practical merit. First, we propose a homogeneous transformation method to address the problem of domain adaptation. For the task of sketch parsing, there is no available data of labeled freehand sketches that can be directly used for model training. An alternative solution is to learn from datasets of real image parsing, while the domain adaptation is an inevitable problem. Unlike existing methods that utilize the edge maps of real images to approximate freehand sketches, the proposed homogeneous transformation method transforms the data from domains of real images and freehand sketches into a homogeneous space to minimize the semantic gap. Second, we design a soft-weighted loss function as guidance for the training process, which gives attention to both the ambiguous label boundary and class imbalance. Third, we present a staged learning strategy to improve the parsing performance of the trained model, which takes advantage of the shared information and specific characteristic from different sketch categories. Extensive experimental results demonstrate the effectiveness of the above three methods. Specifically, to evaluate the generalization ability of our homogeneous transformation method, additional experiments for the task of sketch-based image retrieval are conducted on the QMUL FG-SBIR dataset. Finally, by integrating the proposed three methods into a unified framework of deep semantic sketch parsing (DeepSSP), we achieve the state-of-the-art on the public SketchParse dataset.

General Partial Label Learning via Dual Bipartite Graph Autoencoder

We formulate a practical yet challenging problem: General Partial Label Learning (GPLL). Compared to the traditional Partial Label Learning (PLL) problem, GPLL relaxes the supervision assumption from instance-level — a label set partially labels an instance — to group-level: 1) a label set partially labels a group of instances, where the within-group instance-label link annotations are missing, and 2) cross-group links are allowed — instances in a group may be partially linked to the label set from another group. Such ambiguous group-level supervision is more practical in real-world scenarios as additional annotation on the instance-level is no longer required, e.g., face-naming in videos where the group consists of faces in a frame, labeled by a name set in the corresponding caption. In this paper, we propose a novel graph convolutional network (GCN) called Dual Bipartite Graph Autoencoder (DB-GAE) to tackle the label ambiguity challenge of GPLL. First, we exploit the cross-group correlations to represent the instance groups as dual bipartite graphs: within-group and cross-group, which reciprocally complements each other to resolve the linking ambiguities. Second, we design a GCN autoencoder to encode and decode them, where the decodings are considered as the refined results. It is worth noting that DB-GAE is self-supervised and transductive, as it only uses the group-level supervision without a separate offline training stage. Extensive experiments on two real-world datasets demonstrate that DB-GAE significantly outperforms the best baseline over absolute 0.159 F1-score and 24.8% accuracy. We further offer analysis on various levels of label ambiguities.

Cascading Convolutional Color Constancy

Regressing the illumination of a scene from the representations of object appearances is popularly adopted in computational color constancy. However, it's still challenging due to intrinsic appearance and label ambiguities caused by unknown illuminants, diverse reflection properties of materials and extrinsic imaging factors (such as different camera sensors). In this paper, we introduce a novel algorithm – Cascading Convolutional Color Constancy (in short, C4) to improve robustness of regression learning and achieve stable generalization capability across datasets (different cameras and scenes) in a unique framework. The proposed C4 method ensembles a series of dependent illumination hypotheses from each cascade stage via introducing a weighted multiply-accumulate loss function, which can inherently capture different modes of illuminations and explicitly enforce coarse-to-fine network optimization. Experimental results on the public Color Checker and NUS 8-Camera benchmarks demonstrate superior performance of the proposed algorithm in comparison with the state-of-the-art methods, especially for more difficult scenes.

A Transfer Learning-Based Multi-Instance Learning Method With Weak Labels.

In multi-instance learning (MIL), labels are associated with bags rather than the instances in the bag. Most of the previous MIL methods assume that each bag has the actual label in the training set. However, from the process of labeling work, the label of a bag is always evaluated by the calculation of the labels obtained from a number of labelers. In the calculation, the weight of each labeler is always unknown and people always assign the weight for each labeler by random or equally, and this may result in the ambiguous labels for the bags, which is called weak labels here. In addition, we always meet the problem of knowledge transfer from the source task to the target task, and this leads to the study of multiple instance transfer learning. In this article, we propose a new framework called transfer learning-based multiple instance learning (TMIL) framework to address the problem of multiple instance transfer learning in which both the source task and the target task contain the weak labels. We first construct a TMIL model with weak labels, which can transfer knowledge from the source task to the target task where both source and target tasks contain weak labels. We then put forward an iterative framework to solve the transfer learning model with weak labels so that we can update the label of the bag to improve the performance of multiple instance learning. We then present the convergence analysis of the proposed method. The experiments show that the proposed method outperforms the existing multiple instance learning methods and can correct the initial labels to obtain the actual labels for the bags.

Mutual information-based label distribution feature selection for multi-label learning

Feature selection used for dimensionality reduction of the feature space plays an important role in multi-label learning where high-dimensional data are involved. Although most existing multi-label feature selection approaches can deal with the problem of label ambiguity which mainly focuses on the assumption of uniform distribution with logical labels, it cannot be applied to many practical applications where the significance of related label for every instance tends to be different. To deal with this issue, in this study, label distribution learning covered with a certain real number of labels is introduced to design a model for the labeling-significance. Nevertheless, multi-label feature selection is limited to handling only labels consisting of logical relations. In order to solve this problem, combining the random variable distribution with granular computing, we first propose a label enhancement algorithm to transform logical labels in multi-label data into label distribution with more supervised information, which can mine the hidden label significance from every instance. On this basis, to remove some redundant or irrelevant features in multi-label data, a label distribution feature selection algorithm using mutual information and label enhancement is developed. Finally, the experimental results show that the performance of the proposed method is superior to the other state-of-the-art approaches when dealing with multi-label data.

Label distribution learning: A local collaborative mechanism

Label distribution learning (LDL) is a generalized machine learning framework for dealing with label ambiguity, as it can explore the relative importance levels of different labels in the description of each sample. Although several algorithms have been proposed to solve LDL problems, they usually destroy the consistency of geometric structures between feature space and label space to a certain extent, which frequently plays a significant role in learning tasks. Meanwhile, most existing LDL algorithms only take predictive performances into consideration, while ignoring the computational cost and robustness to noises. To remedy above deficiencies, we propose a novel algorithm, i.e., Local Collaborative Representation based Label Distribution Learning, shortly LCR-LDL. In LCR-LDL, an unlabeled sample is treated as the collaborative representation of the local dictionary constructed by the neighborhood of the unlabeled sample, and the discriminatory information of representation coefficients is used to reconstruct the label distribution of the unlabeled sample. Experimental results on 20 real-world LDL data sets compared with results produced by 11 state-of-the-art algorithms show that, the proposed LCR-LDL algorithm can not only effectively improve the predictive performances for LDL tasks, but also exhibit higher robustness and a lightweight computational overhead. This study suggests new trends for considering the computational cost and robustness issues in the LDL community.

A Structure-Induced Framework for Multi-Label Feature Selection With Highly Incomplete Labels

Feature selection has shown significant promise in improving the effectiveness of multi- label learning by constructing a reduced feature space. Previous studies typically assume that label assignment is complete or partially complete; however, missing-label and unlabeled data are commonplace and accompanying occurrences in real applications due to the high expense of manual annotation and label ambiguity. We call this “highly incomplete labels” problem. Such label incompleteness severely damages the inherent label structures and masks true label correlations. In this paper, we propose a novel structure-induced feature selection model to simultaneously identify the most discriminative features and recover the highly incomplete labels. To our best knowledge, it is the first attempt to explore the local density structure of data to capture the intricate feature-label dependency in the highly incomplete learning scenarios. Feature selection is guided by the label structure reconstruction, and highly incomplete labels are recovered via the structure transferred from feature space. In this elegant manner, the processes of selecting discriminative features and recovering incomplete labels are coupled in a unified optimization framework. Comprehensive experiments on public benchmark datasets demonstrate the superiority of the proposed approach.

A new self-paced method for multiple instance boosting learning

Multi-instance learning is a useful tool for solving label ambiguity. MILBoost is one of the algorithms, which uses boosting method to handle the multiple instance learning problems. Although MILBoosting has achieved good effect on multiple instance learning, little work has been done on the problem of multiple instance learning where a small number of bags are labeled. In this paper, we propose a new approach by incorporating the SPL and boosting into the procedure of multiple instance learning, called Self-Paced Boost Multiple Instance Learning (SP-B-MIL). The proposed approach can improve the effectiveness and robustness of multi-instance learning when a small number of bags are labeled. We first reformulate the multiple instance boosting model with a self-paced loss formulation. Then we propose a self-paced function for realizing desired self-paced scheme, which makes it possible to select instances from different bags during each iteration. Finally, we design a simple and effective algorithm to solve the optimization problem. Experimental results show that the proposed algorithm is comparable to the classical algorithms in some multi-instance learning benchmark data sets.

P.788 Associative learning and resting-state connectivity changes in the adult brain

Label distribution learning (LDL) is one of the paradigms for dealing with label ambiguity, and it can learn the relative importance of each label to a particular instance. Most of the existing LDL approaches require strong supervision information, which always involves high cost of the data-labeling process. To this end, we propose a novel transductive weakly supervised LDL algorithm based on matrix completion, which simultaneously explores the discriminant knowledge of both training instances and testing instances. Moreover, the manifold regularization is exploited to capture the samples’ relevance to enhance the performance of matrix completion. Experimental results on real-world data sets with various missing percentages validate the effectiveness of our method.

Audio–visual domain adaptation using conditional semi-supervised Generative Adversarial Networks

Accessing large, manually annotated audio databases in an effort to create robust models for emotion recognition is a notably difficult task, handicapped by the annotation cost and label ambiguities. On the contrary, there are plenty of publicly available datasets for emotion recognition which are based on facial expressivity due to the prevailing role of computer vision in deep learning research, nowadays. Thereby, in the current work, we performed a study on cross-modal transfer knowledge between audio and facial modalities within the emotional context. More concretely, we investigated whether facial information from videos could be used to boost the awareness and the prediction tracking of emotions in audio signals. Our approach was based on a simple hypothesis: that the emotional state’s content of a person’s oral expression correlates with the corresponding facial expressions. Research in the domain of cognitive psychology was affirmative to our hypothesis and suggests that visual information related to emotions fused with the auditory signal is used from humans in a cross-modal integration schema to better understand emotions. In this regard, a method called dacssGAN (which stands for Domain Adaptation Conditional Semi-Supervised Generative Adversarial Networks) is introduced in this work, in an effort to bridge these two inherently different domains. Given as input the source domain (visual data) and some conditional information that is based on inductive conformal prediction, the proposed architecture generates data distributions that are as close as possible to the target domain (audio data). Through experimentation, it is shown that classification performance of an expanded dataset using real audio enhanced with generated samples produced using dacssGAN (50.29% and 48.65%) outperforms the one obtained merely using real audio samples (49.34% and 46.90%) for two publicly available audio–visual emotion datasets.

Label Distribution Learning with Label Correlations on Local Samples

Label distribution learning (LDL) is proposed for solving the label ambiguity problem in recent years, which can be seen as an extension of multi-label learning. To improve the performance of label distribution learning, some existing algorithms exploit label correlations in a global manner that assumes the label correlations are shared by all instances. However, the instances in different groups may share different label correlations, and few label correlations are globally applicable in real-world tasks. In this paper, two novel label distribution learning algorithms are proposed by exploiting label correlations on local samples, which are called GD-LDL-SCL and Adam-LDL-SCL, respectively. To utilize the label correlations on local samples, the influence of local samples is encoded, and a local correlation vector is designed as the additional features for each instance, which is based on the different clustered local samples. Then, the label distribution for an unseen instance can be predicted by exploiting the original features and the additional features simultaneously. Extensive experiments on some real-world data sets validate that our proposed methods can address the label distribution problems effectively and outperform state-of-the-art methods.

Large Margin Partial Label Machine.

Partial label learning (PLL) is a multi-class weakly supervised learning problem where each training instance is associated with a set of candidate labels but only one label is the ground truth. The main challenge of PLL is how to deal with the label ambiguities. Among various disambiguation techniques, large margin (LM)-based algorithms attract much attention due to their powerful discriminative performance. However, existing LM-based algorithms either neglect some potential candidate labels in constructing the margin or introduce auxiliary estimation of class capacities which is generally inaccurate. As a result, their generalization performances are deteriorated. To address the above-mentioned drawbacks, motivated by the optimistic superset loss, we propose an LM Partial LAbel machiNE (LM-PLANE) by extending multi-class support vector machines (SVM) to PLL. Compared with existing LM-based disambiguation algorithms, LM-PLANE considers the margin of all potential candidate labels without auxiliary estimation of class capacities. Furthermore, an efficient cutting plane (CP) method is developed to train LM-PLANE in the dual space. Theoretical insights into the effectiveness and convergence of our CP method are also presented. Extensive experiments on various PLL tasks demonstrate the superiority of LM-PLANE over existing LM based and other representative PLL algorithms in terms of classification accuracy.

Structured and Sparse Annotations for Image Emotion Distribution Learning

Label distribution learning methods effectively address the label ambiguity problem and have achieved great success in image emotion analysis. However, these methods ignore structured and sparse information naturally contained in the annotations of emotions. For example, emotions can be grouped and ordered due to their polarities and degrees. Meanwhile, emotions have the character of intensity and are reflected in different levels of sparse annotations. Motivated by these observations, we present a convolutional neural network based framework called Structured and Sparse annotations for image emotion Distribution Learning (SSDL) to tackle two challenges. In order to utilize structured annotations, the Earth Mover’s Distance is employed to calculate the minimal cost required to transform one distribution to another for ordered emotions and emotion groups. Combined with Kullback-Leibler divergence, we design the loss to penalize the mispredictions according to the dissimilarities of same emotions and different emotions simultaneously. Moreover, in order to handle sparse annotations, sparse regularization based on emotional intensity is adopted. Through combined loss and sparse regularization, SSDL could effectively leverage structured and sparse annotations for predicting emotion distribution. Experiment results demonstrate that our proposed SSDL significantly outperforms the state-of-the-art methods.

Indoor Crowd Counting by Mixture of Gaussians Label Distribution Learning.

In this paper, we tackle the problem of crowd counting in indoor videos, where people often stay almost static for a long time. The label distribution, which covers a certain number of crowd counting labels, representing the degree to which each label describes the video frame, is previously adopted to model the label ambiguity of the crowd number. However, since the label ambiguity is significantly affected by the crowd number of the scene, we initialize the label distribution of each frame by the discretized Gaussian distribution with adaptive variance instead of the original single static Gaussian distribution. Moreover, considering the gradual change of crowd numbers in the adjacent frames, a mixture of Gaussian models is proposed to generate the final label distribution representation for each frame. The weights of the Gaussian models rely on the frame and feature distances between the current frame and the adjacent frames. The mixed l2,1 -norm is adopted to restrict the weights of predicting the adjacent crowd numbers to be locally correlated. We collect three new indoor video datasets with frame number annotation for further research. The proposed approach achieves state-of-the-art performance on seven challenging indoor videos and cross-scene experiments.