Unsupervised Cross-domain Research Articles

RFA-Net: Reconstructed Feature Alignment Network for Domain Adaptation Object Detection in Remote Sensing Imagery

With the development of deep learning, great progress has been made in object detection of remote sensing (RS) imagery. However, the object detector is hard to generalize well from one labeled dataset (source domain) to another unlabeled dataset (target domain) due to the discrepancy of data distribution (domain shift). Currently, adversarial-based domain adaptation methods align the semantic features of source and target domain features to alleviate the domain shift. But they fail to avoid the alignment of noisy background features and neglect the instance-level features, which are inappropriate for detection models that focus on instance location and classification. To mitigate domain shift existing in object detection, we propose a reconstructed feature alignment network (RFA-Net) for unsupervised cross-domain object detection in remote sensing imagery. The RFA-Net includes one sequential data augmentation module (SDA) deployed on data level for providing solid gains on unlabeled data, one sparse feature reconstruction module (SFR) deployed on feature level to intensify instance feature for feature alignment, and one pseudo-label generation module (PLG) deployed on label level for the supervision of the unlabeled target domain. Extensive experiments illustrate that our proposed RFA-Net is effective to alleviate the domain shift problem in domain adaptation object detection of RS imagery.

Joint Distribution Alignment via Adversarial Learning for Domain Adaptive Object Detection

Unsupervised domain adaptive object detection aims to adapt a well-trained detector from its original source domain with rich labeled data to a new target domain with unlabeled data. Recently, mainstream approaches perform this task through adversarial learning, yet still suffer from two limitations. First, they mainly align marginal distribution by unsupervised cross-domain feature matching, and ignore each feature's categorical and positional information that can be exploited for conditional alignment; Second, they treat all classes as equally important for transferring cross-domain knowledge and ignore that different classes usually have different transferability. In this article, we propose a joint adaptive detection framework (JADF) to address the above challenges. First, an end-to-end joint adversarial adaptation framework for object detection is proposed, which aligns both marginal and conditional distributions between domains without introducing any extra hyper-parameter. Next, to consider the transferability of each object class, a metric for class-wise transferability assessment is proposed, which is incorporated into the JADF objective for domain adaptation. Further, an extended study from unsupervised domain adaptation (UDA) to unsupervised few-shot domain adaptation (UFDA) is conducted, where only a few unlabeled training images are available in unlabeled target domain. Extensive experiments validate that JADF is effective in both the UDA and UFDA settings, achieving significant performance gains over existing state-of-the-art cross-domain detection methods.

Unsupervised cross-domain person re-identification by instance and distribution alignment

Most existing person re-identification (re-id) methods assume supervised model training on a separate large set of training samples from the target domain. While performing well in the training domain, such trained models are seldom generalisable to a new independent unsupervised target domain without further labelled training data from the target domain. To solve this scalability limitation, we develop a novel Hierarchical Unsupervised Domain Adaptation (HUDA) method. It can transfer labelled information of an existing dataset (a source domain) to an unlabelled target domain for unsupervised person re-id. Specifically, HUDA is designed to model jointly global distribution alignment and local instance alignment in a two-level hierarchy for discovering transferable source knowledge in unsupervised domain adaptation. Crucially, this approach aims to overcome the under-constrained learning problem of existing unsupervised domain adaptation methods. Extensive evaluations show the superiority of HUDA for unsupervised cross-domain person re-id over a wide variety of state-of-the-art methods on four re-id benchmarks: Market-1501, DukeMTMC, MSMT17 and CUHK03.

Tri-training for Dependency Parsing Domain Adaptation

In recent years, the research on dependency parsing focuses on improving the accuracy of the domain-specific (in-domain) test datasets and has made remarkable progress. However, there are innumerable scenarios in the real world that are not covered by the dataset, namely, the out-of-domain dataset. As a result, parsers that perform well on the in-domain data usually suffer from significant performance degradation on the out-of-domain data. Therefore, to adapt the existing in-domain parsers with high performance to a new domain scenario, cross-domain transfer learning methods are essential to solve the domain problem in parsing. This paper examines two scenarios for cross-domain transfer learning: semi-supervised and unsupervised cross-domain transfer learning. Specifically, we adopt a pre-trained language model BERT for training on the source domain (in-domain) data at the subword level and introduce self-training methods varied from tri-training for these two scenarios. The evaluation results on the NLPCC-2019 shared task and universal dependency parsing task indicate the effectiveness of the adopted approaches on cross-domain transfer learning and show the potential of self-learning to cross-lingual transfer learning.

HQA‐Trans: An end‐to‐end high‐quality‐awareness image translation framework for unsupervised cross‐domain pedestrian detection

Unsupervised cross-domain pedestrian detection has attracted attention in recent years. Although some works adopted unsupervised image translation frameworks to generate an intermediate domain to narrow the gap between source and target domains, the images in the intermediate domain tend to be distorted due to the instability of the generation network. In this work, we propose a new framework to improve the image quality of the generated intermediate domain via an end-to-end translation framework. First, an image quality assessment index is adopted and adjusted appropriately. The part that controls the image quality is kept, and the part that adversely affects the domain style translation is discarded. Secondly, the adjusted image quality assessment index is integrated into the unsupervised image translation framework, where a new loss with the index's weight is proposed. An end-to-end high-quality-awareness image translation framework is constructed to generate a high-quality intermediate domain directly through this process. Finally, the intermediate domain with high-quality images is applied for cross-domain pedestrian detection. Experimental results on benchmark datasets show that the proposed framework can effectively improve unsupervised cross-domain pedestrian detection performance. Compared with some state-of-the-art works, the proposed framework can also achieve superior performance under miss rate metrics.

IAS-NET: Joint intraclassly adaptive GAN and segmentation network for unsupervised cross-domain in neonatal brain MRI segmentation.

In neonatal brain magnetic resonance image (MRI) segmentation, the model we trained on the training set (source domain) often performs poorly in clinical practice (target domain). As the label of target-domain images is unavailable, this cross-domain segmentation needs unsupervised domain adaptation (UDA) to make the model adapt to the target domain. However, the shape and intensity distribution of neonatal brain MRI images across the domains are largely different from adults'. Current UDA methods aim to make synthesized images similar to the target domain as a whole. But it is impossible to synthesize images with intraclass similarity because of the regional misalignment caused by the cross-domain difference. This will result in generating intraclassly incorrect intensity information from target-domain images. To address this issue, we propose an IAS-NET (joint intraclassly adaptive generative adversarial network (GAN) (IA-NET) and segmentation) framework to bridge the gap between the two domains for intraclassalignment. Our proposed IAS-NET is an elegant learning framework that transfers the appearance of images across the domains from both image and feature perspectives. It consists of the proposed IA-NET and a segmentation network (S-NET). The proposed IA-NET is a GAN-based adaptive network that contains one generator (including two encoders and one shared decoder) and four discriminators for cross-domain transfer. The two encoders are implemented to extract original image, mean, and variance features from source and target domains. The proposed local adaptive instance normalization algorithm is used to perform intraclass feature alignment to the target domain in the feature-map level. S-NET is a U-net structure network that is used to provide semantic constraint by a segmentation loss for the training of IA-NET. Meanwhile, it offers pseudo-label images for calculating intraclass features of the target domain. Source code (in Tensorflow) is available at https://github.com/lb-whu/RAS-NET/. Extensive experiments are carried out on two different data sets (NeoBrainS12 and dHCP), respectively. There exist great differences in the shape, size, and intensity distribution of magnetic resonance (MR) images in the two databases. Compared to baseline, we improve the average dice score of all tissues on NeoBrains12 by 6% through adaptive training with unlabeled dHCP images. Besides, we also conduct experiments on dHCP and improved the average dice score by 4%. The quantitative analysis of the mean and variance of the synthesized images shows that the synthesized image by the proposed is closer to the target domain both in the full brain or within each class than that of the comparedmethods. In this paper, the proposed IAS-NET can improve the performance of the S-NET effectively by its intraclass feature alignment in the target domain. Compared to the current UDA methods, the synthesized images by IAS-NET are more intraclassly similar to the target domain for neonatal brain MR images. Therefore, it achieves state-of-the-art results in the compared UDA models for the segmentation task.

Generative and self-supervised domain adaptation for one-stage object detection

Unsupervised cross-domain object detection has recently attracted considerable attention because of its ability to significantly reduce annotation costs. For two-stage detectors, several improvements have been made in feature-level adaptations. However, this approach is not suitable for one-stage detectors that do not have access to instance-level features. Although other approaches are often used for one-stage detectors, their performance is insufficient compared to domain adaptation methods for two-stage detectors. In this study, we propose a generative and self-supervised domain adaptation method for one-stage detectors. The proposed method is composed of an adversarial generative method and a self-supervision-based method. We tested our method on three evaluation datasets, and an improvement in the mean average precision was achieved using this method. We also confirmed the complementary effects of an adversarial generative method and a self-supervision-based method.

Statistical Scattering Component-Based Subspace Alignment for Unsupervised Cross-Domain PolSAR Image Classification

Increasing amounts of polarimetric synthetic aperture radar (PolSAR) images from different sensors covering different scenes are available, but limited labeled samples and trained models can hardly work well in these cross-domain data interpretations. Fortunately, domain adaptation (DA) can transfer knowledge in existing images to new yet related images. DA shows attractive potential for PolSAR classification, and it is still challenging due to more complex domain shifts caused by different sensors, imaging conditions, and distributions. Inspired by the widely applicable polarimetric scattering mechanisms and DA ability of subspace alignment (SA), this article is devoted to constructing a robust unsupervised cross-domain PolSAR classification framework, by exploring scattering and statistical characteristics mapping between the source and target domains. First, classical scattering components of both source and target data were extracted, and Wishart clustering was adopted to derive the statistical information of scattering components at patch level. Second, the intrinsic polarimetric scattering components were estimated and extracted, which were called statistical scattering components (SSCs). Third, by applying SA, the source SSC was aligned with target SSC, and domain shift was further reduced. Finally, the target PolSAR image was classified based on labeled samples from source domain, and unsupervised cross-domain classification was achieved by SSC-based SA (SSC-SA). The unsupervised cross-domain experiments are conducted on 49 units among 11 data sets, including Radarsat-2, Gaofen-3, AIRSAR, and Pi-SAR images. With randomly selected labeled samples (about 2%–10%) from source domain, the accuracies of the proposed cross-domain classifications range between 80.20% and 95.64%. Also, the proposed SSC feature pattern is proved extensible for other polarimetric basis and decompositions.

Learning Cycle-Consistent Cooperative Networks via Alternating MCMC Teaching for Unsupervised Cross-Domain Translation

This paper studies the unsupervised cross-domain translation problem by proposing a generative framework, in which the probability distribution of each domain is represented by a generative cooperative network that consists of an energy-based model and a latent variable model. The use of generative cooperative network enables maximum likelihood learning of the domain model by MCMC teaching, where the energy-based model seeks to fit the data distribution of domain and distills its knowledge to the latent variable model via MCMC. Specifically, in the MCMC teaching process, the latent variable model parameterized by an encoder-decoder maps examples from the source domain to the target domain, while the energy-based model further refines the mapped results by Langevin revision such that the revised results match to the examples in the target domain in terms of the statistical properties, which are defined by the learned energy function. For the purpose of building up a correspondence between two unpaired domains, the proposed framework simultaneously learns a pair of cooperative networks with cycle consistency, accounting for a two-way translation between two domains, by alternating MCMC teaching. Experiments show that the proposed framework is useful for unsupervised image-to-image translation and unpaired image sequence translation.

Source data‐free domain adaptation of object detector through domain‐specific perturbation

The current unsupervised cross-domain detection methods need source domain data to retrain the detection model in target domain. However, the source domain data may be unavailable due to privacy, decentralization, or computation resource restrictions. A natural idea is to optimize the parameters of the source domain model by self-supervised learning based on pseudo labels. We propose another approach from the viewpoint of noise perturbation without pseudo-labeling. It can be assumed that the source and target domains are actually derived from a domain invariant space through domain-specific perturbations, respectively. A super target domain can be constructed by augmenting more target domain perturbations to the target domain images. The optimal direction of the target domain to the domain invariant space can be approximated as the alignment direction from the super target domain to the target domain. Based on this idea, we propose a novel method called SOAP (SOurce data-free domain Adaptation through domain Perturbation) which can remove domain perturbation from the target domain. The image-level, instance-level, and category consistency regularizations based on Mean Teacher structure are proposed to learn the correct alignment direction. Specifically, the category consistency can also further improve the classification accuracy. Extensive experiments on multiple domain adaptation scenarios demonstrate that SOAP achieves better performance surpassing the baseline (Faster R-CNN) and multiple state-of-the-art domain adaptation methods which need to access source domain data.

Unsupervised cross-domain person re-identification with self-attention and joint-flexible optimization

Unsupervised domain adaptation (UDA) for person re-identication (ReID) remains a challenging task, as the trained ReID system often fails to adapting to a new dataset. Due to the lack of supervision of real labels, the performance of the UDA models suffers from inefficient feature learning and inevitable pseudo label noise. In this work, we tackle the problems by designing an effective dual-path mutual-learning framework which can capture effective information for better feature learning and mitigate the impact of label noise. Firstly, to reduce the impact of occlusion and viewpoints, we introduce the self-attention mechanism in a two-stage strategy making the models focus on the key areas of identifying people. Secondly, considering that UDA is an open-set task, we leverage density-based spatial clustering of applications with noise (DBSCAN) to avoid manually setting the number of classes of the target domain. Thirdly, for realizing joint and flexible optimization under the supervision of soft pseudo labels and hard pseudo labels, a joint and flexible loss (JFL) is proposed to train the network. Experiments on three large-scale datasets show that our model outperforms the state-of-the-art UDA methods in both mAP and top-1 evaluation protocols by large margins. Especially on task of Duke-to-Market, our method outperforms the state-of-the-art by 6.9% mAP.

A General Feature Paradigm for Unsupervised Cross-Domain PolSAR Image Classification

Limited labels and increasing multisource data promote domain adaptation (DA) problem as a challenging study for polarimetric synthetic aperture radar (PolSAR) interpretation. Existing DAs for optical images cannot generalize over PolSAR imagery due to its special side-imaging characteristics and complex distribution shifts. In this letter, a general feature paradigm (GFP) is proposed for unsupervised cross-domain PolSAR image classification. The GFP is based on a key observation that interclass aggregation is optimized after four-step feature transformations. This key observation leads to GFP that not only reduces the domain shifts but also compatible with typical DA methods. The GFPs are conducted on both source and target domain by unsupervised manner, including polarimetric basis extraction, the Wishart clustering, histogram statistics, and dimensionality reduction. After these transformations, the unlabeled target PolSAR image can be classified based on obtained GFP, DA, and limited labeled samples only from the source domain. Extensive unsupervised cross-domain experiments on 27 scenarios verified that GFP leads to at most 93.76% accuracy for full- and dual-polarized synthetic aperture radar (SAR) images’ classification. Moreover, the GFP shed light on extensive cross-domain PolSAR applications about built-up areas, vegetation, and bare land analysis.

Joint distribution adaptation network with adversarial learning for rolling bearing fault diagnosis

Numerous intelligent methods have been developed to approach the challenges of fault diagnosis. However, due to the different distributions of training samples and test samples, and the lack of information on test samples, most of these methods cannot directly handle the unsupervised cross-domain fault diagnosis issues. In this paper, a joint distribution adaptation network with adversarial learning is developed to effectively tackle the mentioned fault diagnosis issues. Firstly, deep convolutional neural network (CNN) is constructed to extract the features of training samples and test samples. Secondly, since the joint maximum mean discrepancy (JMMD) cannot precisely measure the joint distribution discrepancy between different domains, an improved joint maximum mean discrepancy (IJMMD) is proposed to accurately match the feature distributions. Finally, adversarial domain adaptation is also developed to help the constructed CNN to extract the domain-invariant features. Therefore, the proposed method can achieve precisely distribution matching, and extract the category-discriminative and domain-invariant features between the source and target domains. Substantial transfer fault diagnosis cases based on three rolling bearing datasets fully demonstrate the effectiveness and generalization ability of the proposed method.

Cross-view similarity exploration for unsupervised cross-domain person re-identification

Due to the existence of a domain gap between different domains, when a model trained on one domain is applied to other domain, performance will drop dramatically. For the moment, some of the solutions are concentrating on reducing data distribution discrepancy in different domains, but they ignore unlabeled samples in the target domain. To address this problem, we propose the cross-view similarity exploration (CVSE) method, which combines style-transferred samples to optimize the CNN model and the relationship between samples. It mainly includes two stages. In stage-I, we use starGAN to train a style transfer model, which generates images of multiple camera styles for increasing the quantity and diversity of samples. In stage-II, we propose incremental optimization learning, which iterates between similarity grouping and CNN model optimization to progressively explore the potential similarities of all training samples. Furthermore, with the purpose of reducing the impact of label noise on performance, we propose a new ranking-guided triplet loss, which is on the basis of similarity and does not require any label to select reliable triple samples. We perform a mass of experiments on Market-1501, and DukeMTMC-reID datasets prove that the proposed CVSE is competitive to the most advanced methods.

Learning domain invariant and specific representation for cross-domain person re-identification

Person re-identification (re-ID) aims to match person images under different cameras with disjoint views. Although supervised re-ID has achieved great progress, unsupervised cross-domain re-ID remains a challenging work due to domain bias. In this work, we divide cross-domain re-ID task into two phases: domain-invariant features learning and domain-specific features learning. Our contributions are twofold. (i) To achieve domain-invariant features learning, a novel model called Pedestrian General Similarity (PGS) is proposed, which can eliminate two main factors that cause domain bias: image style and background. Compared with the existing re-ID models, PGS has better generalization ability. (ii) A novel pseudo label assignment method named Mutual Nearest Neighbors Pseudo Labeling (MNNPL) is proposed, which calculates pseudo labels based on the similarity between samples in the target domain, and the resulting pseudo labels are used to guide domain-specific feature learning. Extensive experiments are conducted on several large scale datasets, the results show that our method outperforms most published unsupervised cross-domain methods by a large margin.

Unsupervised Cross Domain Person Re-Identification by Multi-Loss Optimization Learning.

Unsupervised cross domain (UCD) person re-identification (re-ID) aims to apply a model trained on a labeled source domain to an unlabeled target domain. It faces huge challenges as the identities have no overlap between these two domains. At present, most UCD person re-ID methods perform "supervised learning" by assigning pseudo labels to the target domain, which leads to poor re-ID performance due to the pseudo label noise. To address this problem, a multi-loss optimization learning (MLOL) model is proposed for UCD person re-ID. In addition to using the information of clustering pseudo labels from the perspective of supervised learning, two losses are designed from the view of similarity exploration and adversarial learning to optimize the model. Specifically, in order to alleviate the erroneous guidance brought by the clustering error to the model, a ranking-average-based triplet loss learning and a neighbor-consistency-based loss learning are developed. Combining these losses to optimize the model results in a deep exploration of the intra-domain relation within the target domain. The proposed model is evaluated on three popular person re-ID datasets, Market-1501, DukeMTMC-reID, and MSMT17. Experimental results show that our model outperforms the state-of-the-art UCD re-ID methods with a clear advantage.

Unsupervised Horizontal Pyramid Similarity Learning for Cross-Domain Adaptive Person Re-Identification

Although person re-identification has made great progress, unsupervised cross-domain adaptive person re-identification is still a challenging problem. With no labeled data in target domain, the performance may have a significant drop. In this paper, we propose an unsupervised cross-domain adaptive person re-identification framework based on horizontal pyramid similarity learning (UHPS). Firstly, horizontal pyramid features are extracted by dividing the deep feature maps into different number of partial feature bins. These feature bins with diverse scales can incorporate not only the global information but also local information in different spatial scales, making the framework more robust in complex environment. Then, horizontal pyramid similarity learning is proposed with the mechanism of fusing together the internal similarity of the target domain and the similarity between the source domain and target domain. Finally, the unsupervised clustering algorithm DBSCAN embeded with the horizontal pyramid similarity is employed to select training data in the target domain and estimate the pseudo labels in each training iteration, for the purpose of adapting the framework to the target domain. The results on Market1501 and DukeMTMC-reID confirm that the proposed framework can adapt to the target domain effectively and outperforms the state-of-the-art unsupervised cross domain person re-identification approaches.

Self-Training With Progressive Representation Enhancement for Unsupervised Cross-Domain Person Re-Identification.

In recent years, person re-identification (re-ID) has achieved relatively good performance, benefiting from the revival of deep neural networks. However, due to the existence of domain bias which refers to the different data distributions between two domains, it remains challenging to directly deploy a model trained on a labeled source domain to a target domain only with unlabeled data available. In this paper, a Self-Training with Progressive Representation Enhancement (PREST) framework, which comprises a multi-scale self-training method and a view-invariant representation learning module, is proposed to promote re-ID performance on the target domain in an unsupervised manner. More specifically, multi-scale representations, including the global body and local parts of pedestrian images, are utilized to obtain pseudo-labels. Then, some images are selected according to the pseudo-labels to create a new dataset for supervising the fine-tuning process, which is operated iteratively to progressively promote the performance. Furthermore, to mitigate the influence of different styles among sub-domains, in cases where a single sub-domain is captured by one camera, a classifier with a gradient reverse layer is first employed to learn view-invariant representation for pedestrian images with the same identity taken by different cameras; this can further enhance the reliability of the predicted labels and improve the cross-domain re-ID performance. Extensive experiments on three large-scale re-ID datasets demonstrate that our framework achieves significantly better performance than existing approaches.

Unsupervised cross-domain named entity recognition using entity-aware adversarial training

The success of neural network based methods in named entity recognition (NER) is heavily relied on abundant manual labeled data. However, these NER methods are unavailable when the data is fully-unlabeled in a new domain. To address the problem, we propose an unsupervised cross-domain model which leverages labeled data from source domain to predict entities in unlabeled target domain. To relieve the distribution divergence when transferring knowledge from source to target domain, we apply adversarial training. Furthermore, we design an entity-aware attention module to guide the adversarial training to reduce the discrepancy of entity features between different domains. Experimental results demonstrate that our model outperforms other methods and achieves state-of-the-art performance.

Unsupervised Cross-Domain Fault Diagnosis Using Feature Representation Alignment Networks for Rotating Machinery

In this article, the problem of the cross-domain fault diagnosis of rotating machinery is considered. In a practical setting of this approach, the operating platform of the machine may have a different setup and conditions compared to the experimental platform that is used to collect the training data. This can lead to significant data variations, specifically domain shifts. Conventional data-driven approaches are known to adapt poorly to these domain shifts, resulting in a significant drop in the diagnosis accuracy when the pretrained model is applied in the actual operating situation. In this article, an unsupervised domain adaptation approach is developed to mitigate the domain shifts between the data gathered from the experimental platform (the source domain) and the operating platform (the target domain) by aligning the features extracted from the two data domains. The mutual information between the target feature space and the entire feature space is maximized to improve the knowledge transferability of the labeled data in the source domain. Furthermore, the feature-level discrepancy between the two domains is minimized to further improve diagnosis accuracy. The experiments using public datasets and real-world adaptation scenarios demonstrate the feasibility and the superior performance of the proposed method.