Unlabeled Target Domain Research Articles

Hierarchical Connectivity-Centered Clustering for Unsupervised Domain Adaptation on Person Re-Identification.

Unsupervised domain adaptation (UDA) on person Re-Identification (ReID) aims to transfer the knowledge from a labeled source domain to an unlabeled target domain. Recent works mainly optimize the ReID models with pseudo labels generated by unsupervised clustering on the target domain. However, the pseudo labels generated by the unsupervised clustering methods are often unreliable, due to the severe intra-person variations and complicated cluster structures in the practical application scenarios. In this work, to handle the complicated cluster structures, we propose a novel learnable Hierarchical Connectivity-Centered (HCC) clustering scheme by Graph Convolutional Networks (GCNs) to generate more reliable pseudo labels. Our HCC scheme learns the complicated cluster structure by hierarchically estimating the connectivity among samples from the vertex level to cluster level in a graph representation, and thereby progressively refines the pseudo labels. Additionally, to handle the intra-person variations in clustering, we propose a novel relation feature for HCC clustering, which exploits the identities from the source domain as references to represent target domain samples. Experiments demonstrate that our method is able to achieve state-of-the art performance on three challenging benchmarks.

Learning Causal Representations for Robust Domain Adaptation

Domain adaptation solves the learning problem in a target domain by leveraging the knowledge in a relevant source domain. While remarkable advances have been made, almost all existing domain adaptation methods heavily require large amounts of unlabeled target domain data for learning domain invariant representations to achieve good generalizability on the target domain. In fact, in many real-world applications, target domain data may not always be available. In this paper, we study the cases where at the training phase the target domain data is unavailable and only well-labeled source domain data is available, called robust domain adaptation. To tackle this problem, under the assumption that causal relationships between features and the class variable are robust across domains, we propose a novel Causal AutoEncoder (CAE), which integrates deep autoencoder and causal structure learning into a unified model to learn causal representations only using data from a single source domain. Specifically, a deep autoencoder model is adopted to learn low-dimensional representations, and a causal structure learning model is designed to separate the low-dimensional representations into two groups: causal representations and task-irrelevant representations. Using three real-world datasets the extensive experiments have validated the effectiveness of CAE compared to eleven state-of-the-art methods.

Graph Domain Adversarial Transfer Network for Cross-Domain Sentiment Classification

In the text sentiment classification task, some words are seemingly unrelated to the classification task, but they have a direct impact on the performance of classification model. For example, in the sentences “I have terminal cancer” and “Cancer is a very common disease”, it can be clearly found that the word “cancer” has two different sentiment tendencies in the daily life domain and the medical domain. In the daily life domain, the word “cancer” shows an extremely negative sentiment tendency. While in the medical domain, the word “cancer” is just a simple term with a relatively neutral sentiment tendency. Although current deep learning models have already achieved good performance through their powerful feature learning capabilities, there are serious deficiencies in dealing with the above problem. Therefore, from a new perspective, this paper proposes the Graph Domain Adversarial Transfer Network (GDATN) based on the idea of adversarial learning, which uses the labeled source domain data to predict the sentiment label of unlabeled target domain data. Firstly, GDATN extracts feature representations through the Bidirectional Long Short-Term Memory (BiLSTM) Network and Graph Attention Network (GAT) successively. Then, GDATN introduces the domain classifier to capture the domain-shared text feature representation with the Gradient Reversal Layer (GRL). In addition, an auxiliary task named the projection mechanism is constructed to further capture the domain-specific text feature representation in response to the text domain problem. Extensive experimental results on two benchmark datasets show that GDATN proposed in this paper outperforms the other six benchmark sentiment classification models, and GDATN has a better stability on different cross-domain pairs.

A Novel Remaining Useful Life Prediction Method of Rolling Bearings Based on Deep Transfer Auto-Encoder

Deep learning (DL) has shown effectiveness in the field of Prognostic and Health Management (PHM) of rolling bearings and successfully solved many problems with its powerful feature learning and function fitting capabilities. However, it is still very challenging to transfer a remaining useful life (RUL) prediction model trained by historical data to bearings running under new operating conditions. Domain adaptation (DA) is an effective solution to obtain cross-domain features of bearings. This article presents an RUL prediction method for rolling bearings based on a deep transfer auto-encoder. Two strategies, parameter transfer and feature representation transfer, are used to transfer the feature extraction model and RUL regression model trained in the labeled source domain to the unlabeled target domain. In this way, the full utilization of the historical model and the rapid adaptation to the new operating conditions are realized. A case study on the IEEE PHM Challenge 2012 bearing data set verifies the effectiveness of the proposed method. The comparison with other nontransfer and transfer machine learning models shows the advantages of this method in precision and accuracy.

Meta-reweighted Regularization for Unsupervised Domain Adaptation

Unsupervised domain adaptation enables knowledge transfer from a labeled source domain to an unlabeled target domain by reducing the cross-domain distribution discrepancy, and the adversarial learning based paradigm has achieved remarkable success. On top of this, recent works seeks to further regularize the classification decision boundary via self-training to learn target adaptive classifier with pseudo-labeled target samples. However, since pseudo labels are inevitably noisy, most of prior methods focus on manually designing elaborate target selection algorithms or optimization objectives. Different from them, we propose a meta-learning based target-reweighting regularization algorithm called MetaReg. Specifically, MetaReg is motivated by the intuition that an ideal target classifier trained on correct target pseudo labels should make small classification errors on target-like source samples. Therefore, we explicitly define a meta reweighting problem that aims to find optimal weights for different samples by minimizing the classification loss on a class-balanced set consisting of source samples that are most similar to target ones. The optimization problem is solved efficiently with a simplified approximation technique. As a result, the automatically learned optimal weights are utilized to reweight pseudo-labeled target samples and regularize the model learning. Comprehensive experiments verify that MetaReg outperforms the non-regularized UDA counterparts with state-of-the-art performance.

Unsupervised Domain Adaptation Network With Category-Centric Prototype Aligner for Biomedical Image Segmentation

With the widespread success of deep learning in biomedical image segmentation, domain shift becomes a critical and challenging problem, as the gap between two domains can severely affect model performance when deployed to unseen data with heterogeneous features. To alleviate this problem, we present a novel unsupervised domain adaptation network, for generalizing models learned from the labeled source domain to the unlabeled target domain for cross-modality biomedical image segmentation. Specifically, our approach consists of two key modules, a conditional domain discriminator (CDD) and a category-centric prototype aligner (CCPA). The CDD, extended from conditional domain adversarial networks in classifier tasks, is effective and robust in handling complex cross-modality biomedical images. The CCPA, improved from the graph-induced prototype alignment mechanism in cross-domain object detection, can exploit precise instance-level features through an elaborate prototype representation. In addition, it can address the negative effect of class imbalance via entropy-based loss. Extensive experiments on a public benchmark for the cardiac substructure segmentation task demonstrate that our method significantly improves performance on the target domain.

Exploring Category Attention for Open Set Domain Adaptation

Great success has been achieved in the area of unsupervised domain adaptation, which learns to generalize from the labeled source domain to the unlabeled target domain. However, most of the existing techniques can only handle the closed-set scenario, which requires both the source domain and the target domain to have a shared category label set. In this work, we propose a two-stage method to deal with the more challenging task of open set domain adaptation, where the target domain contains categories unseen to the source domain. Our first stage formulates the alignment of two domains as a semi-supervised clustering problem, and initially associates each target-domain sample x <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">t</sup> ∈ X <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">t</sup> with a source-domain category label ℓ <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">s</sup> ∈ L <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">s</sup> . To this end, we use the self-training strategy to learn a teacher network and a student network, both of which adopt the self-attention mechanism. Our second stage refines the resulting clusters by identifying the negative associations (x <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">t</sup> , ℓ <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">s</sup> ) and labeling the involved xt as unknown. For this purpose, we investigate the compatibility of each association by replacing the self-attention maps in the last convolutional layers with the newly proposed category attention maps (CAMs), which locate the informative feature pixels for a given category. Experimental results on three public datasets show the effectiveness and robustness our method in adaptation across various domain pairs.

Sentiment Classification for Financial Texts Based on Deep Learning.

Sentiment classification for financial texts is of great importance for predicting stock markets and financial crises. At present, with the popularity of applications in the field of natural language processing (NLP) adopting deep learning, the application of automatic text classification and text-based sentiment classification has become more and more extensive. However, in the field of financial text-based sentiment classification, due to a lack of labeled samples, such applications are limited. A domain-adaptation-based financial text sentiment classification method is proposed in this paper, which can adopt source domain (SD) text data with sentiment labels and a large amount of unlabeled target domain (TD) financial text data as training samples for the proposed neural network. The proposed method is a cross-domain transfer-learning-based method. The domain classification subnetwork is added to the original neural network, and the domain classification loss function is also added to the original training loss function. Therefore, the network can simultaneously adapt to the target domain and then accomplish the classification task. The experiment of the proposed sentiment classification transfer learning method is carried out through an open-source dataset. The proposed method in this paper uses the reviews of Amazon Books, DVDs, electronics, and kitchen appliances as the source domain for cross-domain learning, and the classification accuracy rates can reach 65.0%, 61.2%, 61.6%, and 66.3%, respectively. Compared with nontransfer learning, the classification accuracy rate has improved by 11.0%, 7.6%, 11.4%, and 13.4%, respectively.

Metric Information Matrix for Maximum Mean Discrepancy for Domain Adaptation

In this paper, we focus the problem of unsupervised domain adaptation which transfers knowledge from a well-labeled source domain to an unlabeled target domain with distinctive distributions. Based on Gromov-Hausdorff’s theory, we proposed two kinds of feature mappings in the model of joint distribution adaptation by embedding the original feature subspace to a common subspace. It can been seen as a part of feature embedding used for the models based feature alignment. Our experiments show that constructed mappings have the abilities to alleviate the feature discrepancy and mitigate the distribution shift between source domain and target domains.

Unsupervised Cross Domain Person Re-Identification by Multi-Loss Optimization Learning.

Unsupervised cross domain (UCD) person re-identification (re-ID) aims to apply a model trained on a labeled source domain to an unlabeled target domain. It faces huge challenges as the identities have no overlap between these two domains. At present, most UCD person re-ID methods perform "supervised learning" by assigning pseudo labels to the target domain, which leads to poor re-ID performance due to the pseudo label noise. To address this problem, a multi-loss optimization learning (MLOL) model is proposed for UCD person re-ID. In addition to using the information of clustering pseudo labels from the perspective of supervised learning, two losses are designed from the view of similarity exploration and adversarial learning to optimize the model. Specifically, in order to alleviate the erroneous guidance brought by the clustering error to the model, a ranking-average-based triplet loss learning and a neighbor-consistency-based loss learning are developed. Combining these losses to optimize the model results in a deep exploration of the intra-domain relation within the target domain. The proposed model is evaluated on three popular person re-ID datasets, Market-1501, DukeMTMC-reID, and MSMT17. Experimental results show that our model outperforms the state-of-the-art UCD re-ID methods with a clear advantage.

Minimizing-Entropy and Fourier Consistency Network for Domain Adaptation on Optic Disc and Cup Segmentation

Automated segmentation of the optic disc (OD) and optic cup (OC) from different datasets plays an important role in the diagnosis of glaucoma and greatly saves human resources in both data annotation and image segmentation. However, the domain shift between different datasets suppresses the generalization ability of the segmentation network, especially damaging the performance of segmentation in the target domain, which is unlabeled. Therefore, using a transfer learning algorithm or domain adaptation method to enhance the migration ability of segmentation models has become an essential step and has attracted the attention of many researchers. In this paper, we propose an unsupervised domain adaptation network, called the Minimizing-entropy and Fourier Domain Adaptation network (MeFDA), to narrow the discrepancy between the source and target domains and prevent the degradation of segmentation performance. First, we perform adversarial optimization on the entropy maps of the predicted segmentation results to alleviate the domain shift. Then, direct entropy-minimization optimization is applied to the unlabeled target domain data to improve the credibility of the prediction segmentation maps. To enhance the prediction consistency of the target domain data, we augment the target domain dataset through the Fourier transform by replacing the low-frequency part in the target images with that of the source images. Then, a semantic consistency constraint is imposed on the raw images and augmented images of the target domain to improve the prediction consistency of the segmentation model, thereby further narrowing the discrepancy between the source and target domains. Experiments on several public retinal fundus image datasets prove the superiority of MeFDA compared with state-of-the-art methods, and the ablation study analyzes the importance of the different proposed components.

Generative Self-training for Cross-domain Unsupervised Tagged-to-Cine MRI Synthesis.

Self-training based unsupervised domain adaptation (UDA) has shown great potential to address the problem of domain shift, when applying a trained deep learning model in a source domain to unlabeled target domains. However, while the self-training UDA has demonstrated its effectiveness on discriminative tasks, such as classification and segmentation, via the reliable pseudo-label selection based on the softmax discrete histogram, the self-training UDA for generative tasks, such as image synthesis, is not fully investigated. In this work, we propose a novel generative self-training (GST) UDA framework with continuous value prediction and regression objective for cross-domain image synthesis. Specifically, we propose to filter the pseudo-label with an uncertainty mask, and quantify the predictive confidence of generated images with practical variational Bayes learning. The fast test-time adaptation is achieved by a round-based alternative optimization scheme. We validated our framework on the tagged-to-cine magnetic resonance imaging (MRI) synthesis problem, where datasets in the source and target domains were acquired from different scanners or centers. Extensive validations were carried out to verify our framework against popular adversarial training UDA methods. Results show that our GST, with tagged MRI of test subjects in new target domains, improved the synthesis quality by a large margin, compared with the adversarial training UDA methods.

Contradistinguisher: A Vapnik's Imperative to Unsupervised Domain Adaptation.

Recent domain adaptation works rely on an indirect way of first aligning the source and target domain distributions and then train a classifier on the labeled source domain to classify the target domain. However, the main drawback of this approach is that obtaining a near-perfect domain alignment in itself might be difficult/impossible (e.g., language domains). To address this, inspired by how humans use supervised-unsupervised learning to perform tasks seamlessly across multiple domains or tasks, we follow Vapnik's imperative of statistical learning that states any desired problem should be solved in the most direct way rather than solving a more general intermediate task and propose a direct approach to domain adaptation that does not require domain alignment. We propose a model referred to as Contradistinguisher that learns contrastive features and whose objective is to jointly learn to contradistinguish the unlabeled target domain in an unsupervised way and classify in a supervised way on the source domain. We achieve the state-of-the-art on Office-31, Digits and VisDA-2017 datasets in both single-source and multi-source settings. We demonstrate that performing data augmentation results in an improvement in the performance over vanilla approach. We also notice that the contradistinguish-loss enhances performance by increasing the shape bias.

Unsupervised cross-domain named entity recognition using entity-aware adversarial training

The success of neural network based methods in named entity recognition (NER) is heavily relied on abundant manual labeled data. However, these NER methods are unavailable when the data is fully-unlabeled in a new domain. To address the problem, we propose an unsupervised cross-domain model which leverages labeled data from source domain to predict entities in unlabeled target domain. To relieve the distribution divergence when transferring knowledge from source to target domain, we apply adversarial training. Furthermore, we design an entity-aware attention module to guide the adversarial training to reduce the discrepancy of entity features between different domains. Experimental results demonstrate that our model outperforms other methods and achieves state-of-the-art performance.

Unsupervised adversarial domain adaptation with similarity diffusion for person re-identification

Due to the scarcity of human identities labels, unsupervised person re-identification (re-ID) draws much attention recently, which attempts to learn discriminative person representation without labels. Domain adaptation based methods utilize the labeled sample in source domain and transfer the knowledge to the unlabeled target domain. However, for person re-ID, no identities overlapping between source and target domain leads to the difficulty during adaptation. To address this problem, we propose an unsupervised adversarial domain adaptation method for person re-ID which exploits the pixel level and feature level alignments. Specifically, we utilize CycleGAN to transfer the target domain style to the source domain for the pixel level, and we propose an adversarial metric adaptation method which aligns between the source domain and target domain for the feature level. We further explore the feature similarity laying on manifold structure revealed by the features through the similarity diffusion. To verify the efficacy of our proposed method, we conduct extensive experiments on three benchmark datasets: Market1501, Duke-MTMC, and MSMT17. Comparing with state-of-the-art unsupervised domain adaptation approaches, we have comparable performance on ranking metric and significant improvement on mAP metric, which validates the efficacy of the proposed technique for person re-identification tasks.

A new transferable bearing fault diagnosis approach with adaptive manifold embedded distribution alignment

The conditional and marginal distributions of bearing vibration signal data collected under different working conditions generally may not obey the same distribution. Moreover, feature distortions are difficult to eliminate when performing distribution alignment in the original space. Based on these challenges, this study proposes a novel transferable fault diagnosis approach with adaptive manifold embedded distribution alignment (AMEDA). AMEDA can learn transformed feature representations in the Grassmann manifold space by constructing a geodesic flow kernel to avoid feature distortions. In addition, this paper initially aims to construct an adaptive factor defined by -distance to adjust the relative importance of the conditional and marginal distributions dynamically. After manifold feature learning and dynamic distribution alignment, a cross-domain classifier f with structural risk minimization can be learned to predict unlabeled target domain data. Experimental results from multiple indicators based on two datasets demonstrate the superiority of AMEDA over the other existing methods.

A Triple Adversary Network Driven by Hybrid High-Order Attention for Domain Adaptation

How to bridge the knowledge gap between the annotated source domain and the unlabeled target domain is a basic challenge to domain adaptation. The existing approaches can relieve this gap by feature alignments across domains; however, aligning non-transferable features may lead to negative shift confusing the knowledge learning on target domains. In this paper, a triple adversary network is proposed on the basis of a high-order attention, hopefully to solve the problem. The proposed architecture focuses on the detailed feature alignment by a hybrid high-order attention using a fast iteration algorithm. In addition, an orthogonal loss of two complementary modules is applied to constrain the mutual exclusion of foreground and background features. Finally, a triple adversarial strategy is introduced to further improve the training convergence for the composed architectures. Numeric experiments on datasets of Digits, Office-31 and Office-home illuminate that the proposed network can effectively improve the state-of-art domain adaptations with superior transferring performance.

Compact class-conditional domain invariant learning for multi-class domain adaptation

Neural network-based models have recently shown excellent performance in various kinds of tasks. However, a large amount of labeled data is required to train deep networks, and the cost of gathering labeled training data for every kind of domain is prohibitively expensive. Domain adaptation tries to solve this problem by transferring knowledge from labeled source domain data to unlabeled target domain data. Previous research tried to learn domain-invariant features of source and target domains to address this problem, and this approach has been used as a key concept in various methods. However, domain-invariant features do not mean that a classifier trained on source data can be directly applied to target data because it does not guarantee that data distribution of the same classes will be aligned across two domains. In this paper, we present novel generalization upper bounds for domain adaptation that motivates the need for class-conditional domain invariant learning. Based on this theoretical framework, we then propose a class-conditional domain invariant learning method that can learn a feature space in which features in the same class are expected to be mapped nearby. We empirically experimented that our model showed state-of-the-art performance on standard datasets and showed effectiveness by visualization of latent space.

TScatNet: An Interpretable Cross-Domain Intelligent Diagnosis Model With Antinoise and Few-Shot Learning Capability

In a real industrial scenario, domain shift frequently occurred due to working loads variation, operation speeds variation, and environmental noise interference, severely degrading intelligent fault diagnosis models' performance. Currently, domain adaptation-based models eliminate domain shift by calibrating unlabeled target-domain data using labeled source-domain data. Nevertheless, these models may fail when encountering unseen working conditions, lacking unlabeled target-domain data for learning domain-invariant features. Besides, the existing deep domain adaptation-based models lack a few-shot learning capability and interpretability. This article develops a cross-domain diagnosis model named time-scattering convolutional network (TScatNet) to remedy these gaps. TScatNet extracts domain-invariant features using Morlet wavelet as the predefined convolutional kernel, modulus as nonlinearity, and scaling averaging as pooling layer. This predefined architecture eliminates domain shift without any domain adaptation, endows TScatNet few-shot learning capability, simplifies the hyperparameter tuning process, and brings interpretability. Both the CWRU and DDS data sets are used to verify the proposed model, which shows that TScatNet could stably realize 100% accuracy on transfer tasks across working loads and 100% accuracy across operation speeds. Moreover, even though the SNR value descends to -4, TScatNet achieved 96% accuracy on ten categories tasks and 99.8% accuracy on four categories tasks. Besides, TScatNet achieved nearly 100% accuracy both under training samples' sparsity and sparsity of working conditions.

Unsupervised domain adaptation for person re-identification with iterative soft clustering

In this work, we propose to address the unsupervised domain adaptive (UDA) person re-id problem in which the model learns from an unlabeled target domain using a fully annotated source domain. Current approaches mainly address domain shift problem or the inter/intra-domain variation of the two domains. However, they have neglected to integrate the-easy-to-learn label distribution of the target domain into the model to improve its performance. Moreover, the automatic label assignment for the unlabeled target data currently used in UDA methods does not reflect the underlying data. To address these issues, we introduce a technique that enforces three properties: (1) target instance invariance that considers the target data and uses a key–value memory to guess the label distribution that is later used as the supervision signal. (2) a camera invariance, formed by unlabeled target images, and their camera-style transferred. Here, a new loss function is proposed to control overconfident predictions on the styled images. Lastly, (3) a hierarchical clustering-based optimization technique that exploits the similarities between the target images to constrain the supervision information of the first property. Here, we randomly allocate each target image to a separate cluster, then gradually incorporate similarity within each identity as we group similar images into clusters and use the cluster-IDs as the new target labels. We iteratively refine the guessed label distribution of the target domain by making predictions on the unlabeled target domain and then train the network with these new samples. Extensive experimental results on the concurrent use of these three properties demonstrate that the proposed model can achieve the state-of-the-art on unsupervised domain adaptive person re-id. Our work is important for knowledge discovery and knowledge transfer.