Label Space Research Articles

Dual Learning for Joint Facial Landmark Detection and Action Unit Recognition

Facial landmark detection and action unit~(AU) recognition are two essential tasks in facial analysis. Previous works rarely consider the relationship between these complementary tasks. In this paper, we introduce a novel multi-task dual learning framework to exploit the relationship between facial landmark detection and AU recognition while simultaneously addressing both tasks. When both tasks share middle-level features, common patterns can be exploited and middle- and high-level features can be used to perform facial landmark detection and AU recognition, respectively. In addition, a dual learning mechanism is designed to convert the predicted landmarks and AUs of the label space to the corresponding facial image of the image space, further exploring the strong correlations between the tasks. By jointly training the proposed method at both the feature and label levels, each task improves the other. Experiments on two benchmark databases demonstrate that the proposed method can leverage dependencies to boost the generalization of both tasks.

Domain adaptation meta-learning network with discard-supplement module for few-shot cross-domain rotating machinery fault diagnosis

Intelligent diagnostic methods based on deep learning have proven to be effective in equipment management and maintenance. However, in practical industrial applications in which data is scarce and equipment, load, and operating conditions are variable, the performance of well-trained laboratory models degrades significantly. To this end, this study proposes a domain adaptation meta-learning network with feature-oriented discard-supplement module (FD-DAML) for few-shot cross-domain rotating machinery fault diagnosis. This method addresses the diagnosis issues of severe domain distribution discrepancy, label space mismatch, and scarcity of labeled samples in the target domain within a unified framework. Specifically, the proposed method attempts a training mode that alternates the execution of the source and target domains meta-learning, and combines it with domain adversarial training. Such a training mode not only contributes to the accumulation of domain-invariant meta-knowledge from the source domain for the model, but also effectively learns the discriminative model for the target domain and achieves good generalization over it. Moreover, a plug-and-play feature-oriented discard-supplement module is designed to perform discard and supplement operations on extracted features against the context, to improve the generalization of the model. Extensive comparative experiments on public datasets, experimental datasets, and actual wind turbine datasets validate the effectiveness of the proposed FD-DAML and the feasibility of engineering diagnostics. The code will be published at https://github.com/zhangyu-ysu/FD-DAML.

Research on the construction of event logic knowledge graph of supply chain management

Knowledge graph technology plays an important role in knowledge supporting for efficient supply chain management (SCM) of manufacturing enterprises, a SCM knowledge graph can be constructed based on the relevant case corpus. In order to solve the problems of coarse concept granularity of event ontology knowledge in SCM cases, potential words of characters in the existing entity recognition models may not be matched or matched incorrectly, key features of entities with different lengths may interfere with each other in the expression of attention mechanism, and lack of a large number of annotation training samples, a top-down construction method of SCM event logic knowledge graph (ELKG) is proposed. Firstly, SCM event argument classes and class relations are defined, an event logic ontology model is built, and the event argument entities according to event logic ontology are labeled. Then, an active learning event argument entity recognition (EAER) model based on two-stage generative adversarial network (GAN) is proposed. In GAN generator, an EAER model based on binocular attention-based stacked BiLSTM with CNN (BACSBN) is proposed, which combines word-level character feature attention mechanism and n-gram pooling feature attention mechanism to improve the attention to character features of constituent words and highlight entity key information with different lengths, respectively. Two-stage GAN adversarial training and label space attention mechanism are introduced to select the correct predicted label samples for active learning training. The experimental results show that BACSBN can improve the entity recognition accuracy, and the two-stage GAN’s active learning can further improve the recognition effect on the basis of full annotation sample training, and can still maintain a high accuracy in the absence of a large number of manual annotation data. Further, according to the sentence pattern and keyword matching, the matching relations of argument entities are completed, and the SCM ELKG is constructed to provide knowledge support for autonomous SCM.

Signal differentiation models for multiple microRNA detection: a critical review.

MicroRNAs (miRNAs) are a class of small, single-stranded non-coding RNAs which have critical functions in various biological processes. Increasing evidence suggested that abnormal miRNA expression was closely related to many human diseases, and they are projected to be very promising biomarkers for non-invasive diagnosis. Multiplex detection of aberrant miRNAs has great advantages including improved detection efficiency and enhanced diagnostic precision. Traditional miRNA detection methods do not meet the requirements of high sensitivity or multiplexing. Some new techniques have opened novel paths to solve analytical challenges of multiple miRNA detection. Herein, we give a critical overview of the current multiplex strategies for the simultaneous detection of miRNAs from the perspective of two different signal differentiation models, including label differentiation and space differentiation. Meanwhile, recent advances of signal amplification strategies integrated into multiplex miRNA methods are also discussed. We hope this review provides the reader with future perspectives on multiplex miRNA strategies in biochemical research and clinical diagnostics.

Prior Knowledge Regularized Self-Representation Model for Partial Multilabel Learning.

Partial multilabel learning (PML) aims to learn from training data, where each instance is associated with a set of candidate labels, among which only a part is correct. The common strategy to deal with such a problem is disambiguation, that is, identifying the ground-truth labels from the given candidate labels. However, the existing PML approaches always focus on leveraging the instance relationship to disambiguate the given noisy label space, while the potentially useful information in label space is not effectively explored. Meanwhile, the existence of noise and outliers in training data also makes the disambiguation operation less reliable, which inevitably decreases the robustness of the learned model. In this article, we propose a prior label knowledge regularized self-representation PML approach, called PAKS, where the self-representation scheme and prior label knowledge are jointly incorporated into a unified framework. Specifically, we introduce a self-representation model with a low-rank constraint, which aims to learn the subspace representations of distinct instances and explore the high-order underlying correlation among different instances. Meanwhile, we incorporate prior label knowledge into the above self-representation model, where the prior label knowledge is regarded as the complement of features to obtain an accurate self-representation matrix. The core of PAKS is to take advantage of the data membership preference, which is derived from the prior label knowledge, to purify the discovered membership of the data and accordingly obtain more representative feature subspace for model induction. Enormous experiments on both synthetic and real-world datasets show that our proposed approach can achieve superior or comparable performance to state-of-the-art approaches.

Open Set Domain Adaptation With Soft Unknown-Class Rejection.

The goal of domain adaptation (DA) is to train a good model for a target domain, with a large amount of labeled data in a source domain but only limited labeled data in the target domain. Conventional closed set domain adaptation (CSDA) assumes source and target label spaces are the same. However, this is not quite practical in real-world applications. In this work, we study the problem of open set domain adaptation (OSDA), which only requires the target label space to partially overlap with the source label space. Consequently, the solution to OSDA requires unknown classes detection and separation, which is normally achieved by introducing a threshold for the prediction of target unknown classes; however, the performance can be quite sensitive to that threshold. In this article, we tackle the above issues by proposing a novel OSDA method to perform soft rejection of unknown target classes and simultaneously match the source and target domains. Extensive experiments on three standard datasets validate the effectiveness of the proposed method over the state-of-the-art competitors.

Dynamic Modeling Cross-Modal Interactions in Two-Phase Prediction for Entity-Relation Extraction.

Joint extraction of entities and their relations benefits from the close interaction between named entities and their relation information. Therefore, how to effectively model such cross-modal interactions is critical for the final performance. Previous works have used simple methods, such as label-feature concatenation, to perform coarse-grained semantic fusion among cross-modal instances but fail to capture fine-grained correlations over token and label spaces, resulting in insufficient interactions. In this article, we propose a dynamic cross-modal attention network (CMAN) for joint entity and relation extraction. The network is carefully constructed by stacking multiple attention units in depth to dynamic model dense interactions over token-label spaces, in which two basic attention units and a novel two-phase prediction are proposed to explicitly capture fine-grained correlations across different modalities (e.g., token-to-token and label-to-token). Experiment results on the CoNLL04 dataset show that our model obtains state-of-the-art results by achieving 91.72% F1 on entity recognition and 73.46% F1 on relation classification. In the ADE and DREC datasets, our model surpasses existing approaches by more than 2.1% and 2.54% F1 on relation classification. Extensive analyses further confirm the effectiveness of our approach.

A Multi-Source Weighted Deep Transfer Network for Open-Set Fault Diagnosis of Rotary Machinery

In real industries, there often exist application scenarios where the target domain holds fault categories never observed in the source domain, which is an open-set domain adaptation (DA) diagnosis issue. Existing DA diagnosis methods under the assumption of sharing identical label space across domains fail to work. What is more, labeled samples can be collected from different sources, where multisource information fusion is rarely considered. To handle this issue, a multisource open-set DA diagnosis approach is developed. Specifically, multisource domain data of different operation conditions sharing partial classes are adopted to take advantage of fault information. Then, an open-set DA network is constructed to mitigate the domain gap across domains. Finally, a weighting learning strategy is introduced to adaptively weigh the importance on feature distribution alignment between known class and unknown class samples. Extensive experiments suggest that the proposed approach can substantially boost the performance of open-set diagnosis issues and outperform existing diagnosis approaches.

Open set domain adaptation with latent structure discovery and kernelized classifier learning

Numerous visual domain adaptation methods have been proposed for transferring knowledge from a well-labeled source domain to an unlabeled but related target domain. Most of existing works are only geared to closed set domain adaptation, where an identical label space is shared between two domains. In this paper, we focus on a more realistic but challenging scenario, open set domain adaptation, where the target domain contains unknown classes that do not appear in the label space of source domain. The main task of open set domain adaptation is to simultaneously recognize the target images of known classes and those of unknown classes correctly. To achieve this goal, in this paper, we propose a novel open set domain adaptation method, which consists of two parts: latent structure discovery and kernelized classifier learning. In the first part, we employ an adaptive discriminative graph learning strategy to capture the intrinsic manifold structure of the source and target domain data in the latent feature space, such that the boundaries among all classes will be delineated more clearly. In the second part, the samples from the latent feature space are mapped into a high-dimensional kernel space to make them linearly separable, and a linear classifier is learned by jointly operating unknown target samples separating, known samples matching and local structure preserving. As the optimization problem is not convex with all variables, we devise an efficient iterative algorithm to solve it. The extensive experimental results on five image datasets confirm the superiority of the proposed method compared with the state-of-the-art traditional and deep competitors.

Prior Knowledge Constrained Adaptive Graph Framework for Partial Label Learning

Partial label learning (PLL) aims to learn a robust multi-class classifier from the ambiguous data, where each instance is given with several candidate labels, among which only one label is real. Most existing methods usually cope with such problem by utilizing a feature similarity graph to conduct label disambiguation. However, these methods construct the feature graph by only employing original features, while the influences of latent outliers and the contributions of label space are regrettably ignored. To tackle these issues, in this article, we propose a P rior Kn O wledge Cons T rained A daptive G raph Fram E work ( POTAGE ) for partial label learning, which utilizes an adaptive graph fused with label information to accurately describe the instance relationship and guide the desired model training. Compared with the feature-induced fixed graph, the adaptive graph is deemed to be more robust and accurate to reveal the intrinsic manifold structure within the data, and the embedding label information is expected to effectively alleviate the label ambiguities and enlarge the gap of label confidences between two instances from different classes. Extensive experiments demonstrate that POTAGE achieves state-of-the-art performance.

Multi-sensor open-set cross-domain intelligent diagnostics for rotating machinery under variable operating conditions

Domain adaptation techniques have the proven ability to deal with fault diagnosis issues under variable operating conditions. They can achieve a superb diagnostic performance in single-sensor monitoring scenarios where the training and test data share the same label space. However, in practical engineering, fault modes are usually mixed with each other and new failure modes may appear during operation, which poses a challenge to the effectiveness of existing cross-domain fault diagnosis methods. Furthermore, with the increasing complexity of modern industrial systems, multi-sensor collaborative monitoring has been increasingly deployed for comprehensive measurement and detection of the complicated system. Unfortunately, there is less attention paid to multi-sensor cross-domain diagnosis in the current literature. To bridge this research gap, this paper aims to develop a novel multi-sensor open-set cross-domain fault diagnosis method. First, a convolutional neural network-based single-sensor feature extraction module and a Transformer-based multi-sensor feature fusion module are constructed for discriminative feature extraction and fusion. Second, a weighted adversarial learning scheme is built to conduct domain-invariant learning of the shared fault modes between the source and target domains. Then, a threshold-based supervised contrastive loss is defined to realize instance-level domain alignment, together with an entropy max–min loss to identify unknown class samples. The effectiveness and practicability of the proposed method are validated by a series of experiments conducted on two different types of gearboxes.

Generalized Knowledge Distillation via Relationship Matching.

The knowledge of a well-trained deep neural network (a.k.a. the teacher) is valuable for learning similar tasks. Knowledge distillation extracts knowledge from the teacher and integrates it with the target model (a.k.a. the student), which expands the student's knowledge and improves its learning efficacy. Instead of restricting the teacher from working on the same task as the student, we borrow the knowledge of a teacher trained from a general label space --- in this Generalized Knowledge Distillation (GKD), the classes of the teacher and the student may be the same, completely different, or partially overlapped. We claim that the comparison ability between instances acts as an essential factor threading knowledge across tasks, and propose the Relationship Facilitated Local Classifier Distillation (ReFilled) approach, which decouples the GKD flow of the embedding and the top-layer classifier. In particular, different from reconciling the instance-label confidence between models, ReFilled requires the teacher to reweight the hard tuples push forwarded by the student adaptively and then matches the similarity comparison levels between instances. ReFilled demonstrates strong discriminative ability when the classes of the teacher vary from the same to a fully non-overlapped set w.r.t. the student.

Learning to Adapt Across Dual Discrepancy for Cross-Domain Person Re-Identification.

Thanks to the advent of deep neural networks, recent years have witnessed rapid progress in person re-identification (re-ID). Deep-learning-based methods dominate the leadership of large-scale benchmarks, some of which even surpass the human-level performance. Despite their impressive performance under the single-domain setup, current fully-supervised re-ID models degrade significantly when transplanted to an unseen domain. According to the characteristics of the re-ID task, such degradation is mainly attributed to the dramatic variation within the target domain and the severe shift between the source and target domain, which we call dual discrepancy in this paper. To achieve a model that generalizes well to the target domain, it is desirable to take such dual discrepancy into account. In terms of the former issue, a prevailing solution is to enforce consistency between nearest-neighbors in the embedding space. However, we find that the search of neighbors is highly biased in our case due to the discrepancy across cameras. For this reason, we equip the vanilla neighborhood invariance approach with a camera-aware learning scheme. As for the latter issue, we propose a novel cross-domain mixup scheme. It works in conjunction with virtual prototypes which are employed to handle the disjoint label space between the two domains. In this way, we can realize the smooth transfer by introducing the interpolation between the two domains as a transition state. Extensive experiments on four public benchmarks demonstrate the superiority of our method. Without any auxiliary models and offline clustering procedure, it achieves competitive performance against existing state-of-the-art methods. The code is available at https://github.com/LuckyDC/generalizing-reid-improved.

Multi-Dimensional Classification via Decomposed Label Encoding

In multi-dimensional classification (MDC), a number of class variables are assumed in the output space with each of them specifying the class membership w.r.t. one heterogeneous class space. One major challenge in learning from MDC examples lies in the heterogeneity of class spaces, where the modeling outputs from different class spaces are not directly comparable. To tackle this problem, we propose a new strategy named <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">decomposed label encoding,</i> which enables modeling alignment for MDC in an encoded label space derived from one-versus-one (OvO) decomposition. Specifically, the original MDC output space is transformed into a ternary encoded label space by conducting OvO decomposition w.r.t. each class space. Then, the manifold structure in the feature space is exploited to enrich the labeling information in the encoded label space. Finally, the predictive model is induced by fitting the metric-aligned modeling outputs with enriched labeling information. Extensive experiments over twenty benchmark data sets clearly show the superiority of the proposed MDC strategy against state-of-the-art approaches.

Robust fused hypergraph neural networks for multi-label classification

Deep neural networks have been adopted in multi-label classification for their excellent performance, however, existing methods fail to comprehensively utilize the high-order correlations between instances and the high-order correlations between labels, and these methods are difficult to deal with label noise effectively. We propose a novel end-to-end deep framework named Robust Fused Hypergraph Neural Networks for Multi-Label Classification (RFHNN), which can effectively utilize the two kinds of high-order correlations and adopt them to mitigate the impact of label noise. In RFHNN, Hypergraph Neural Networks (HNNs) are adopted to mine and utilize the high-order correlations of the instances in the feature space and the label space respectively. The high-order correlations of the instances can not only improve the accuracy of the classification and the discrimination of the proposed model, but also lay the foundation for the subsequent noise correction module. Meanwhile, a hypergraph construction method based on the Apriori algorithm is proposed to realize Hypergraph Neural Networks (HNNs), which can mine robust second-order and high-order label correlations effectively. Effective classifiers are learned based on the correlations between the labels, which will not only improve the accuracy of the model, but can also enhance the subsequent noise correction module. In addition, we have designed a noise correction module in the networks. With the help of the high-order correlations among the instances and the effective classifier, the framework can effectively correct the noise and improve the robustness of the model. Extensive experimental results on datasets demonstrate that our proposed approach is better than the state-of-the-art multi-label classification algorithms. When dealing with the multi-label training datasets with noise in the label space, our proposed method also has great performance.

Multi-omics integration method based on attention deep learning network for biomedical data classification

Background and objectiveIntegrating multi-omics data for the comprehensive analysis of the biological processes in human diseases has become one of the most challenging tasks of bioinformatics. Deep learning (DL) algorithms have recently become one of the most promising multi-omics data integration analysis methods. However, existing DL-based studies almost integrate the multi-omics data by concatenation in the input data space or the learned feature space, ignoring the correlations between patients and omics. MethodsWe propose a novel multi-omics integration method, called Multi-omics Attention Deep Learning Network (MOADLN), which is used for biomedical data classification. Firstly, for each type of omics data, we use three fully-connected layers and the self-attention mechanism to reduce dimensionality, and construct the correlations between patients, respectively. Then, we apply the feature vector learned from self-attention to generate the initial category labels. Secondly, for the initial label predicted of each omics data, we use an effective Multi-Omics Correlation Discovery Network (MOCDN) to learn the cross-omic correlations in the label space. Finally, we use the softmax classifier for label prediction. ResultsWe demonstrate that our method outperforms several state-of-the-art methods on two datasets with mRNA expression data, DNA methylation data, and miRNA expression data. In addition, we identified essential biomarkers of relevant diseases by MOADLN, and the generality of MOADLN is also demonstrated in the KIRP and KIRC datasets. ConclusionsMOADLN jointly explores correlations between patients in intra-omics and correlations of cross-omics in label space, which is an effective DL-based classification of biomedical data.

Multi-dimensional multi-label classification: Towards encompassing heterogeneous label spaces and multi-label annotations

In traditional classification framework, the semantics of each object is usually characterized by annotating a single class label from one homogeneous label space. Nonetheless, objects with rich semantics naturally arise in real-world applications whose properties need to be characterized in a more sophisticated manner. In this paper, a new classification framework named Multi-Dimensional Multi-Label (MDML) classification is investigated which models objects with rich semantics by encompassing heterogeneous label spaces and multi-label annotations. Specifically, MDML generalizes the traditional classification framework by assuming a number of heterogeneous label spaces to characterize semantics from different dimensions, where each object is further annotated with multiple class labels from each heterogeneous label space. To learn from MDML training examples, a first attempt named CLIM is proposed based on an augmented stacking strategy. Firstly, CLIM induces a base multi-label predictive model w.r.t. each label space by maximizing the likelihood of the observed multiple class labels. Secondly, the thresholding predictions from all base models are used to augment the original feature space to yield stacked multi-label predictive models. The two-level models are refined alternately via empirical threshold tuning. Experiments on four real-world MDML data sets validate the effectiveness of CLIM in learning from training examples with heterogeneous label spaces and multi-label annotations.

Multiple extended target tracking based on distributed multi‐sensor fusion and shape estimation

AbstractDistributed multi‐sensor fusion based on the generalised covariance intersection (GCI) fusion has been widely integrated into the Random Finite Set theory, which is promising for multi‐target tracking with an unknown number of targets. However, it has not been widely investigated in the multiple extend target tracking (METT) field, and there is still an open problem on how to solve the inconsistency of label space among the sensors. For these problems, we first introduce the GCI fusion into the METT and proposed an association algorithm by considering the estimated shapes and the target positions to avoid the phenomenon of the label inconsistency as well as to reduce the computational burden. Simulation results show that the proposed algorithm has a better tracking performance than the traditional METT algorithms.

Open Set Recognition for Malware Traffic via Predictive Uncertainty

Existing machine learning-based malware traffic recognition techniques can effectively detect abnormal behaviors in the network. However, almost all of them focus on a closed-set scenario in which the data used for training and testing come from the same label space. Since sophisticated malware and advanced persistent threats are evolving, it is impossible to exhaust all attacks to train a complete recognition model under the existing technical conditions. Therefore, recognition in the real network is an open-set problem, i.e., the recognition system should identify unknown and unseen attacks at test time. In this paper, we propose an uncertainty-aware method to identify known malicious traffic accurately and handle unknown traffic effectively. This method employs predictive uncertainty in deep learning as an indicator for unknown class detection. The predictive uncertainty represents the confidence in neural network predictions. In particular, the Deep Evidence Malware Traffic Recognition (DEMTR) model is presented to provide the multi-classification probability and predictive uncertainty in open-set scenarios using evidential deep learning. We demonstrate the performance of DEMTR on the MCFP dataset. Experimental results indicate that the proposed model outperforms the baseline methods in accuracy and F1-score.

Category-Stitch Learning for Union Domain Generalization

Domain generalization aims at generalizing the network trained on multiple domains to unknown but related domains. Under the assumption that different domains share the same classes, previous works can build relationships across domains. However, in realistic scenarios, the change of domains is always followed by the change of categories, which raises a difficulty for collecting sufficient aligned categories across domains. Bearing this in mind, this article introduces union domain generalization (UDG) as a new domain generalization scenario, in which the label space varies across domains, and the categories in unknown domains belong to the union of all given domain categories. The absence of categories in given domains is the main obstacle to aligning different domain distributions and obtaining domain-invariant information. To address this problem, we propose category-stitch learning (CSL), which aims at jointly learning the domain-invariant information and completing missing categories in all domains through an improved variational autoencoder and generators. The domain-invariant information extraction and sample generation cross-promote each other to better generalizability. Additionally, we decouple category and domain information and propose explicitly regularizing the semantic information by the classification loss with transferred samples. Thus our method can breakthrough the category limit and generate samples of missing categories in each domain. Extensive experiments and visualizations are conducted on MNIST, VLCS, PACS, Office-Home, and DomainNet datasets to demonstrate the effectiveness of our proposed method.