Domain Label Research Articles

Conditional Independence Induced Unsupervised Domain Adaptation

Learning domain-adaptive features is important to tackle the dataset bias problem, where data distributions in the labeled source domain and the unlabeled target domain can be different. The critical issue is to identify and then reduce the redundant information including class-irrelevant and domain-specific features. In this paper, a conditional independence induced unsupervised domain adaptation (CIDA) method is proposed to tackle the challenges. It aims to find the low-dimensional and transferable feature representation of each observation, namely the latent variable in the domain-adaptive subspace. Technically, two mutual information terms are optimized at the same time. One is the mutual information between the latent variable and the class label, and the other is the mutual information between the latent variable and the domain label. Note that the key module can be approximately reformulated as a conditional independence/dependence based optimization problem, and thus, it has a probabilistic interpretation with the Gaussian process. Temporary labels of the target samples and the model parameters are alternatively optimized. The objective function can be incorporated with deep network architectures, and the algorithm is implemented iteratively in an end-to-end manner. Extensive experiments are conducted on several benchmark datasets, and the results show effectiveness of CIDA.

Class Overwhelms: Mutual Conditional Blended-Target Domain Adaptation

Current methods of blended targets domain adaptation (BTDA) usually infer or consider domain label information but underemphasize hybrid categorical feature structures of targets, which yields limited performance, especially under the label distribution shift. We demonstrate that domain labels are not directly necessary for BTDA if categorical distributions of various domains are sufficiently aligned even facing the imbalance of domains and the label distribution shift of classes. However, we observe that the cluster assumption in BTDA does not comprehensively hold. The hybrid categorical feature space hinders the modeling of categorical distributions and the generation of reliable pseudo labels for categorical alignment. To address these, we propose a categorical domain discriminator guided by uncertainty to explicitly model and directly align categorical distributions P(Z|Y). Simultaneously, we utilize the low-level features to augment the single source features with diverse target styles to rectify the biased classifier P(Y|Z) among diverse targets. Such a mutual conditional alignment of P(Z|Y) and P(Y|Z) forms a mutual reinforced mechanism. Our approach outperforms the state-of-the-art in BTDA even compared with methods utilizing domain labels, especially under the label distribution shift, and in single target DA on DomainNet.

Domain-Invariant Feature Learning for Domain Adaptation

Unsupervised domain adaptation (UDA) mainly explores how to learn domain-invariant features from the source domain when the target domain label is unknown. To learn domain-invariant features requires aligning the distribution of samples from two domains in the feature space, which can be achieved by minimizing the maximum mean discrepancy (MMD) of samples from the two domains. However, there is still no effective way to find the best parameter values of MMD. Such a problem is addressed in the MMD with deep kernels (MMD-D), whose optimal parameters can be obtained through training. This study proposes a method of domain-invariant feature learning for UDA, whose architecture, named MMDDCDA, comprises an MMD-D module and a cross domain adaptation (CDA) module. MMDDCDA performs alternating training similar to adversarial training to alternately boost the power of the two modules. To our knowledge, this is the first UDA method that performs such alternating training on a UDA architecture using MMD with deep kernels. Experimental validation showed that the proposed method yields state-of-the-art results among UDA methods using other MMD variants and some UDA benchmarks.

Research on Shaft Current Damage Identification of Variable Condition Motor Bearings Based on Multiscale Feature Label Propagation and Manifold Metric Transfer

The current damage is the most stubborn and difficult fault of high-power motor bearings because its vibration characteristics are easily confused with those of ordinary bearing mechanical faults. If it is discriminated as an ordinary mechanical fault without electrical insulation protection, the current damage of bearing shafts will still repeatedly appear. Aiming at the problem that it is difficult to identify the bearing current damage fault under variable working conditions, a bearing shaft current damage identification method based on multiscale feature label propagation and manifold metric transfer (MFLP-MMT) is proposed. Firstly, the multiscale sub-band signal is obtained by wavelet packet decomposition, and the multiscale sub-band fuzzy entropy is obtained by calculating its fuzzy entropy. Then, according to the extracted features, a neighbor graph is constructed on the source domain of the known fault label to obtain the pseudo label of the target domain sample, and the source domain label information is gradually diffused by way of the graph label propagation. The multiscale sub-band fuzzy entropy of the sample is mapped to the low-dimensional manifold space by locality preserving projections (LPP), and the source domain samples close to the target domain are given higher weights by cross-domain density ratio estimation to solve the problem of domain offset. Combined with the label samples of the target domain in label propagation, the manifold distance metric is learned to minimize the intra-class distance and maximize the inter-class distance in the domain and eliminate the overlapping phenomenon in the domain. By increasing the range of label propagation after each iteration, the label propagation error of the leading graph is gradually reduced, and unsupervised metric transfer learning is realized. The experimental results show that the new method is superior to the semi-supervised transfer learning method in fault identification ability; the highest fault identification accuracy can reach 100% and it has a good robustness.

Research on fault diagnosis method of MS-CNN rolling bearing based on local central moment discrepancy

Transfer learning is an excellent approach to deal with the problem that the target domain label can not be adequately obtained when rolling bearing cross-condition fault detection. A transfer learning fault diagnosis method of multi-scale CNN rolling bearings based on local central moment discrepancy is presented in this research. The method maps bearing vibration data to a shared space by building a shared multi-scale feature extraction structure and fully connected layers. The source domain label and target domain pseudo-label are used to divide the category subspace in the shared space. And then the local central moment discrepancy is used to match source and target domain in the category subspace to realize fault knowledge transfer under different conditions. The experimental findings reveal that multi-scale CNN migration diagnosis based on local central moment discrepancy has superior accuracy and stability in diverse diagnostic tasks when compared to classic transfer learning approaches.

Manifold discrimination partial adversarial domain adaptation

Most traditional domain adaptations promote knowledge transfer with the assumption that source domain and target domain have the same label space. In a big data environment, we usually obtain the source dataset with a larger label space than the target dataset. Under this background, partial domain adaptation (PDA) comes into being naturally. In PDA, when the target domain label space is a subset of the source domain label space, cross-domain adaptation can be achieved by matching shared class samples in cross-domain. Many researchers make good progress in this field and propose different kinds of PDA methods. But in the existing PDA methods, the samples of the unknown class are still used for distribution alignment, and this makes the model suffer from the risk of negative transfer. In order to solve these problems, we propose manifold discrimination partial adversarial domain adaptation (MDPDA) to align the relevant classes across two domains and obtain effective domain adaptation. MDPDA combines manifold learning and adversarial learning. In MDPDA, we design the manifold discrimination module to map high-dimensional data into low-dimensional space and learn low-dimensional data information of source domain and target domain. We construct discriminative source domain feature space for model discrimination in MDPDA. MDPDA distinguishes and confuses the two domains through manifold learning and similarity structure, and generates discriminant feature space to promote the auxiliary network to better recognize the source domain samples. The comprehensive experiments on several benchmark datasets evaluate the superior performance of the proposed method compared to the state-of-the-art methods.

Memory-Guided Multi-View Multi-Domain Fake News Detection

The wide spread of fake news is increasingly threatening both individuals and society. Great efforts have been made for automatic fake news detection on a <i>single</i> domain (e.g., politics). However, correlations exist commonly across multiple news domains, and thus it is promising to simultaneously detect fake news of <i>multiple</i> domains. Based on our analysis, we pose two challenges in multi-domain fake news detection: 1) <bold/><i>domain shift</i><bold/>, caused by the discrepancy among domains in terms of words, emotions, styles, etc. 2) <bold/><i>domain labeling incompleteness</i><bold/>, stemming from the real-world categorization that only outputs one single domain label, regardless of topic diversity of a news piece. In this paper, we propose a Memory-guided Multi-view Multi-domain Fake News Detection Framework (M <inline-formula><tex-math notation="LaTeX">$^{3}$</tex-math></inline-formula> FEND) to address these two challenges. We model news pieces from a multi-view perspective, including semantics, emotion, and style. Specifically, we propose a Domain Memory Bank to enrich domain information which could discover potential domain labels based on seen news pieces and model domain characteristics. Then, with enriched domain information as input, a Domain Adapter could adaptively aggregate discriminative information from multiple views for news in various domains. Extensive offline experiments on English and Chinese datasets demonstrate the effectiveness of M <inline-formula><tex-math notation="LaTeX">$^{3}$</tex-math></inline-formula> FEND, and online tests verify its superiority in practice. Our code is available at <uri>https://github.com/ICTMCG/M3FEND</uri>.

Partial Transfer Learning of Multidiscriminator Deep Weighted Adversarial Network in Cross-Machine Fault Diagnosis

Deep transfer learning provides a feasible fault diagnosis method for intelligent mechanical systems. However, this method usually assumes that the source domain and the target domain have the same label space, which greatly limits its application in the actual industry. Therefore, this article proposes a multidiscriminator deep weighted adversarial network (MDWAN) method, which is especially suitable for partial transfer learning where the number of target domain categories is less than the source domain. The proposed method is mainly composed of three parts: feature extractor, multidiscriminator, and classifier. In the feature extraction section, a deep separable convolutional neural network model (DSC) is proposed, which can greatly reduce parameter calculation amount under the premise of ensuring the extraction accuracy. In the multidiscriminator section, a multidiscriminator weighted learning strategy is proposed. This strategy comprehensively considers the domain information and source domain label information, and introduces a weight function to quantify the contribution of source domain samples to the domain discriminator and classifier, which can effectively identify and filter outlier source samples to promote the positive transfer of shared samples. In order to verify the effectiveness and feasibility of the proposed method, the method is applied to three types of bearing datasets of different machines. Comparing the classification results of different methods, the conclusion shows that this method is more beneficial for bearing fault classification.

A Prototypical Knowledge Oriented Adaptation Framework for Semantic Segmentation.

A prevalent family of fully convolutional networks are capable of learning discriminative representations and producing structural prediction in semantic segmentation tasks. However, such supervised learning methods require a large amount of labeled data and show inability of learning cross-domain invariant representations, giving rise to overfitting performance on the source dataset. Domain adaptation, a transfer learning technique that demonstrates strength on aligning feature distributions, can improve the performance of learning methods by providing inter-domain discrepancy alleviation. Recently introduced output-space based adaptation methods provide significant advances on cross-domain semantic segmentation tasks, however, a lack of consideration for intra-domain divergence of domain discrepancy remains prone to over-adaptation results on the target domain. To address the problem, we first leverage prototypical knowledge on the target domain to relax its hard domain label to a continuous domain space, where pixel-wise domain adaptation is developed upon a soft adversarial loss. The development of prototypical knowledge allows to elaborate specific adaptation strategies on under-aligned regions and well-aligned regions of the target domain. Furthermore, aiming to achieve better adaptation performance, we employ a unilateral discriminator to alleviate implicit uncertainty on prototypical knowledge. At last, we theoretically and experimentally demonstrate that the proposed prototypical knowledge oriented adaptation approach provides effective guidance on distribution alignment and alleviation on over-adaptation. The proposed approach shows competitive performance with state-of-the-art methods on two cross-domain segmentation tasks.

Majorities help minorities: Hierarchical structure guided transfer learning for few-shot fault recognition

To ensure the operational safety and reliability, fault recognition of complex systems is becoming an essential process in industrial systems. However, the existing recognition methods mainly focus on common faults with enough data, which ignore that many faults are lack of samples in engineering practice. Transfer learning can be helpful, but irrelevant knowledge transfer can cause performance degradation, especially in complex systems. To address the above problem, a hierarchy guided transfer learning framework (HGTL) is proposed in this paper for fault recognition with few-shot samples. Firstly, we fuse domain knowledge, label semantics and inter-class distance to calculate the affinity between categories, based on which a category hierarchical tree is constructed by hierarchical clustering. Then, guided by the hierarchical structure, the samples in most similar majority classes are selected from the source domain to pre-train the hierarchical feature learning network (HFN) and extract the transferable fault information. For the fault knowledge extracted from the child nodes of one parent node are similar and can be transferred with each other, so the trained HFN can extract better features of few samples classes with the help of the information from similar faults, and used to address few-shot fault recognition problems. Finally, a dataset of a nuclear power system with 65 categories and the widely used Tennessee Eastman dataset are analyzed respectively via the proposed method, as well as state-of-the-art recognition methods for comparison. The experimental results demonstrate the effectiveness and superiority of the proposed method in fault recognition with few-shot problem.

Distribution Aligned Multimodal and Multi-domain Image Stylization

Multimodal and multi-domain stylization are two important problems in the field of image style transfer. Currently, there are few methods that can perform multimodal and multi-domain stylization simultaneously. In this study, we propose a unified framework for multimodal and multi-domain style transfer with the support of both exemplar-based reference and randomly sampled guidance. The key component of our method is a novel style distribution alignment module that eliminates the explicit distribution gaps between various style domains and reduces the risk of mode collapse. The multimodal diversity is ensured by either guidance from multiple images or random style codes, while the multi-domain controllability is directly achieved by using a domain label. We validate our proposed framework on painting style transfer with various artistic styles and genres. Qualitative and quantitative comparisons with state-of-the-art methods demonstrate that our method can generate high-quality results of multi-domain styles and multimodal instances from reference style guidance or a random sampled style.

Domain Invariant and Agnostic Adaptation

Domain adaptation addresses the prediction problem in which the source and target data are sampled from different but related probability distributions. The key problem here lies in properly matching the distributions and learning general feature representation for training the prediction model. In this article, we introduce a Domain Invariant and Agnostic Adaptation (DIAA) solution, which matches the source and target joint distributions, and simultaneously aligns the feature and domain label joint distribution to its marginal product. In particular, DIAA matches and aligns the distributions via a feature transformation, and compares the two kinds of distribution disparities uniformly under the Kullback–Leibler (KL) divergence. To approximate the two corresponding KL divergences from observed samples, we derive a linear-regression-like technique that fits linear models to different ratio functions under the quadratic loss. With the estimated KL divergences, learning the DIAA feature transformation is formulated as solving a Grassmannian minimization problem. Experiments on text and image classification tasks with varied nature demonstrate the success of our approach.

Generalizable Representation Learning for Mixture Domain Face Anti-Spoofing

Face anti-spoofing approach based on domain generalization (DG) has drawn growing attention due to its robustness for unseen scenarios. Existing DG methods assume that the domain label is known. However, in real-world applications, the collected dataset always contains mixture domains, where the domain label is unknown. In this case, most of existing methods may not work. Further, even if we can obtain the domain label as existing methods, we think this is just a sub-optimal partition. To overcome the limitation, we propose domain dynamic adjustment meta-learning (D$^2$AM) without using domain labels, which iteratively divides mixture domains via discriminative domain representation and trains a generalizable face anti-spoofing with meta-learning. Specifically, we design a domain feature based on Instance Normalization (IN) and propose a domain representation learning module (DRLM) to extract discriminative domain features for clustering. Moreover, to reduce the side effect of outliers on clustering performance, we additionally utilize maximum mean discrepancy (MMD) to align the distribution of sample features to a prior distribution, which improves the reliability of clustering. Extensive experiments show that the proposed method outperforms conventional DG-based face anti-spoofing methods, including those utilizing domain labels. Furthermore, we enhance the interpretability through visualization.

Adversarial domain adaptation network for tumor image diagnosis

In medical fields it is very difficult and time-consuming to label samples. Only a small number of labeled samples or unlabeled samples are often encountered in medical fields. How to deal with this problem in medical diagnosis? Domain adaptation is an effective machine learning method to solve the scarce or no labeled samples problem. In this paper, an improved adversarial domain adaptation network is presented to solve this problem in tumor image diagnosis. We construct the tumor image diagnosis adversarial domain adaptation network framework. The framework consists of three parts: a feature extractor, a domain discriminator and a label predictor. The loss function of framework is designed in the paper. We design a feature extractor to acquire the domain invariant feature between the source domain and target domain. The feature extractor is an improved neural network consisting of several convolutional layers and sampling layers. The feature extractor pre-processes the input samples and gets the initial feature of the input space. The domain discriminator predicts the domain label and works adversarial with the feature extractor. The adversarial learning between the feature extractor and the domain discriminator acts as a regularization in the framework. The adversarial game can minimize the distance between the two domains and make sure the feature extractor learns the domain invariant feature. A label predictor is designed to classify the tumor image of the target domain based on the domain-invariant feature. The experiments on tumor image dataset are presented to validate the performance of proposed method. The experimental results show that the proposed method is superior to the other domain adaptation methods.

Imitating targets from all sides: an unsupervised transfer learning method for person re-identification

Person re-identification (Re-ID) models usually present a limited performance when they are trained on one dataset and tested on another dataset due to the inter-dataset bias (e.g. completely different identities and backgrounds) and the intra-dataset difference (e.g. camera and pose changes). In other words, the absence of identity labels (who the person is) and pairwise labels (whether a pair of images belongs to the same person or not) leads to failures in unsupervised person Re-ID problem. We argue that synchronous consideration of these two aspects can improve the performance of unsupervised person Re-ID model. In this work, we introduce a Classification and Latent Commonality (CLC) method based on transfer learning for the unsupervised person Re-ID problem. Our method has three characteristics: (1) proposing an imitate model to generate an imitated target domain with estimated identity labels and create a pseudo target domain to compensate the pairwise labels across camera views; (2) formulating a dual classification loss on both the source domain and imitated target domain to learn a discriminative representation and diminish the inter-domain bias; (3) investigating latent commonality and reducing the intra-domain difference by constraining triplet loss on the source domain, imitated target domain and pairwise label target domain (composed of pseudo target domain and target domain). Extensive experiments are conducted on three widely employed benchmarks, including Market-1501, DukeMTMC-reID and MSMT17, and experimental results demonstrate that the proposed method can achieve a competitive performance against other state-of-the-art unsupervised Re-ID approaches.

The role of response domain and scale label in the quantitative interpretation of patient-reported outcome measure response options

PurposeUncertainties exist in how respondents interpret response options in patient-reported outcome measures (PROMs), particularly across different domains and for different scale labels. The current study assessed how respondents quantitatively interpret common response options.MethodsMembers of the general public were recruited to this study via an online panel, stratified by age, gender, and having English as a first language. Participants completed background questions and were randomised to answer questions on one of three domains (i.e. loneliness (negatively phrased), happiness or activities (positively phrased)). Participants were asked to provide quantitative interpretations of response options (e.g. how many times per week is equal to “often”) and to order several common response options (e.g. occasionally, sometimes) on a 0–100 slider scale. Chi-squared tests and regression analyses were used to assess whether response options were interpreted consistently across domains and respondent characteristics.ResultsData from 1377 participants were analysed. There was general consistency in quantifying the number of times over the last 7 days to which each response option referred. Response options were consistently assigned a lower value in the loneliness than happiness and activities domains. Individual differences, such as age and English as a second language, explained some significant variation in responses, but less than domain.ConclusionMembers of the public quantify common response options in a similar way, but their quantification is not equivalent across domains or every type of respondent. Recommendations for the use of certain scale labels over others in PROM development are provided.

Adversarial Learning of Disentangled and Generalizable Representations of Visual Attributes.

Recently, a multitude of methods for image-to-image translation have demonstrated impressive results on problems, such as multidomain or multiattribute transfer. The vast majority of such works leverages the strengths of adversarial learning and deep convolutional autoencoders to achieve realistic results by well-capturing the target data distribution. Nevertheless, the most prominent representatives of this class of methods do not facilitate semantic structure in the latent space and usually rely on binary domain labels for test-time transfer. This leads to rigid models, unable to capture the variance of each domain label. In this light, we propose a novel adversarial learning method that: 1) facilitates the emergence of latent structure by semantically disentangling sources of variation and 2) encourages learning generalizable, continuous, and transferable latent codes that enable flexible attribute mixing. This is achieved by introducing a novel loss function that encourages representations to result in uniformly distributed class posteriors for disentangled attributes. In tandem with an algorithm for inducing generalizable properties, the resulting representations can be utilized for a variety of tasks such as intensity-preserving multiattribute image translation and synthesis, without requiring labeled test data. We demonstrate the merits of the proposed method by a set of qualitative and quantitative experiments on popular databases such as MultiPIE, RaFD, and BU-3DFE, where our method outperforms other state-of-the-art methods in tasks such as intensity-preserving multiattribute transfer and synthesis.

Towards fully domain‐independent gesture recognition using COTS WiFi device

This letter proposes a fully domain-independent WiFi-based gesture recognition system based on the multi-label adversarial network. Unlike pioneer machine learning works, our system does not require retraining in the target domain, which benefits from our key idea of eliminating fully domain information such as orientations, locations, environment information. The system proposed by us includes three parts: Feature extractor, domain discriminator, and gesture recogniser. The feature extractor attempts to deceive the domain discriminator based on the adversarial network structure so that it is difficult to judge the input domain label, thereby obtaining domain-independent features and realising fully domain-independent gesture recognition. Extensive experiments are conducted on the Widar 3.0 dataset and our dataset to evaluate system performance. The results show that our system can achieve an in-domain recognition accuracy of 93.2% and a cross-domain recognition accuracy of 87.1%, which is superior to other state-of-the-art works.

An enhanced dynamic interaction network for claim verification

Claim verification aims to verify the truthfulness of claims according to the evidence. In the real-world scenarios, the claim verification methods are required 1) to identify and filter out the noise; 2) to enable the interaction among evidence; and 3) to predict the labels in mixed label sets. However, the prior studies adopt either the implicit or cumbersome interaction processes, which disable them from capturing the commonalities among evidence nor reducing the impact of noise, and few of them consider the label prediction in mixed label sets. To alleviate these issues, we propose an Enhanced Dynamic Interaction Network (EDIN). The EDIN contains two main modules: A Gate-based Dynamic Interaction Module to promote the interaction of evidence and summarize the commonality to verify the claim, which employs hub node and gates to control the information transmission and denoise. An Enhancement Module to assist the EDIN with predicting labels in a mixed label sets environment, where we introduce the auxiliary domain detection task and label transition component to enhance the expressiveness of label representation. Experimental results on multiple public datasets reveal the superiority of our method, and further analysis validates the effectiveness of our proposed modules.

Domain Adversarial Graph Convolutional Network for Fault Diagnosis Under Variable Working Conditions

Unsupervised domain adaptation (UDA)-based methods have made great progress in mechanical fault diagnosis under variable working conditions. In UDA, three types of information, including class label, domain label, and data structure, are essential to bridging the labeled source domain and unlabeled target domain. However, most existing UDA-based methods use only the former two information and ignore the modeling of data structure, which make the information contained in the features extracted by the deep network incomplete. To tackle this issue, a domain adversarial graph convolutional network (DAGCN) is proposed to model the three types of information in a unified deep network and achieving UDA. The first two types of information are modeled by the classifier and the domain discriminator, respectively. In data structure modeling, a convolutional neural network (CNN) is first employed to exact features from input signals. After that, the CNN features are input to the proposed graph generation layer to construct instance graphs by mining the relationship of structural characteristics of samples. Then, the instance graphs are modeled by a graph convolutional network, and the maximum mean discrepancy metric is leveraged to estimate the structure discrepancy of instance graphs from different domains. Experimental results conducted on two case studies demonstrate that the proposed DAGCN can not only obtain the best performance among the comparison methods, but also can extract transferable features for domain adaptation. The code library is available at: https://github.com/HazeDT/DAGCN .