Domain Shift Problem Research Articles

Bearing Fault Diagnosis Based on Multiscale Convolutional Neural Network Using Data Augmentation

Bearings are one of the most important parts of a rotating machine. Bearing failure can lead to mechanical failure, financial loss, and even personal injury. In recent years, various deep learning techniques have been used to diagnose bearing faults in rotating machines. However, deep learning technology has a data imbalance problem because it requires huge amounts of data. To solve this problem, we used data augmentation techniques. In addition, Convolutional Neural Network, one of the deep learning models, is a method capable of performing feature learning without prior knowledge. However, since conventional fault diagnosis based on CNN can only extract single‐scale features, not only useful information may be lost but also domain shift problems may occur. In this paper, we proposed a Multiscale Convolutional Neural Network (MSCNN) to extract more powerful and differentiated features from raw signals. MSCNN can learn more powerful feature expression than conventional CNN through multiscale convolution operation and reduce the number of parameters and training time. The proposed model proved better results and validated the effectiveness of the model compared to 2D‐CNN and 1D‐CNN.

Domain Shift Preservation for Zero-Shot Domain Adaptation.

In learning-based image processing a model that is learned in one domain often performs poorly in another since the image samples originate from different sources and thus have different distributions. Domain adaptation techniques alleviate the problem of domain shift by learning transferable knowledge from the source domain to the target domain. Zero-shot domain adaptation (ZSDA) refers to a category of challenging tasks in which no target-domain sample for the task of interest is accessible for training. To address this challenge, we propose a simple but effective method that is based on the strategy of domain shift preservation across tasks. First, we learn the shift between the source domain and the target domain from an irrelevant task for which sufficient data samples from both domains are available. Then, we transfer the domain shift to the task of interest under the hypothesis that different tasks may share the domain shift for a specified pair of domains. Via this strategy, we can learn a model for the unseen target domain of the task of interest. Our method uses two coupled generative adversarial networks (CoGANs) to capture the joint distribution of data samples in dual-domains and another generative adversarial network (GAN) to explicitly model the domain shift. The experimental results on image classification and semantic segmentation demonstrate the satisfactory performance of our method in transferring various kinds of domain shifts across tasks.

Generative Self-training for Cross-domain Unsupervised Tagged-to-Cine MRI Synthesis.

Self-training based unsupervised domain adaptation (UDA) has shown great potential to address the problem of domain shift, when applying a trained deep learning model in a source domain to unlabeled target domains. However, while the self-training UDA has demonstrated its effectiveness on discriminative tasks, such as classification and segmentation, via the reliable pseudo-label selection based on the softmax discrete histogram, the self-training UDA for generative tasks, such as image synthesis, is not fully investigated. In this work, we propose a novel generative self-training (GST) UDA framework with continuous value prediction and regression objective for cross-domain image synthesis. Specifically, we propose to filter the pseudo-label with an uncertainty mask, and quantify the predictive confidence of generated images with practical variational Bayes learning. The fast test-time adaptation is achieved by a round-based alternative optimization scheme. We validated our framework on the tagged-to-cine magnetic resonance imaging (MRI) synthesis problem, where datasets in the source and target domains were acquired from different scanners or centers. Extensive validations were carried out to verify our framework against popular adversarial training UDA methods. Results show that our GST, with tagged MRI of test subjects in new target domains, improved the synthesis quality by a large margin, compared with the adversarial training UDA methods.

Zero-Shot Learning via Discriminative Dual Semantic Auto-Encoder

Zero-shot learning (ZSL) is an effective method to perform the recognition task without any training samples of specific classes. Most existing ZSL models put emphasis on learning an embedding between visual space and semantic space directly. However, few ZSL models research whether the human-designed semantic features are discriminative enough to recognize different classes. Moreover, one-way mapping suffers from the project domain shift problem. In this article, we propose to learn a Discriminative Dual Semantic Auto-encoder (DDSA) based on the encoder-decoder paradigm to solve this problem. DDSA attempts to construct two bidirectional embeddings to connect the visual space and the semantic space with the help of the learned aligned space which includes discriminative information of the visual features and semantic features. Based on the DDSA, we additionally propose a Deep DDSA to capture deep aligned features that are more conducive to zero-shot classification. The key to the proposed framework is that it implicitly exact the principal information from visual space and semantic space to construct aligned features, which is not only semantic-preserving but also discriminative. Extensive experiments on five benchmarks (SUN, CUB, AWA1, AWA2 and aPY) demonstrate the effectiveness of the proposed framework with state-of-the-art performance obtained on both conventional ZSL and generalized ZSL settings.

Stable Median Centre Clustering for Unsupervised Domain Adaptation Person Re-Identification.

The current unsupervised domain adaptation person re-identification (re-ID) method aims to solve the domain shift problem and applies prior knowledge learned from labelled data in the source domain to unlabelled data in the target domain for person re-ID. At present, the unsupervised domain adaptation person re-ID method based on pseudolabels has obtained state-of-the-art performance. This method obtains pseudolabels via a clustering algorithm and uses these pseudolabels to optimize a CNN model. Although it achieves optimal performance, the model cannot be further optimized due to the existence of noisy labels in the clustering process. In this paper, we propose a stable median centre clustering (SMCC) for the unsupervised domain adaptation person re-ID method. SMCC adaptively mines credible samples for optimization purposes and reduces the impact of label noise and outliers on training to improve the performance of the resulting model. In particular, we use the intracluster distance confidence measure of the sample and its K-reciprocal nearest neighbour cluster proportion in the clustering process to select credible samples and assign different weights according to the intracluster sample distance confidence of samples to measure the distances between different clusters, thereby making the clustering results more robust. The experiments show that our SMCC method can select credible and stable samples for training and improve performance of the unsupervised domain adaptation model. Our code is available at https://github.com/sunburst792/SMCC-method/tree/master.

Cut-and-Paste Dataset Generation for Balancing Domain Gaps in Object Instance Detection

Training an object instance detector where only a few training object images are available is a challenging task. One solution is a cut-and-paste method that generates a training dataset by cutting object areas out of training images and pasting them onto other background images. A detector trained on a dataset generated with a cut-and-paste method suffers from the conventional domain shift problem, which stems from a discrepancy between the source domain (generated training dataset) and the target domain (real test dataset). Though state-of-the-art domain adaptation methods are able to reduce this gap, it is limited because they do not consider the difference of domain gaps of foreground and background. In this study, we present that the conventional domain gap can be divided into two sub-domain gaps for foreground and background. Then, we show that the original cut-and-paste approach suffers from a new domain gap problem, an unbalanced domain gaps, because it has two separate source domains for foreground and background, unlike the conventional domain shift problem. Then, we introduce an advanced cut-and-paste method to balance the unbalanced domain gaps by diversifying the foreground with GAN (generative adversarial network)-generated seed images and simplifying the background using image processing techniques. Experimental results show that our method is effective for balancing domain gaps and improving the accuracy of object instance detection in a cluttered indoor environment using only a few seed images. Furthermore, we show that balancing domain gaps can improve the detection accuracy of state-of-the-art domain adaptation methods.

Multi-perspective cross-class domain adaptation for open logo detection

Existing logo detection methods mostly rely on supervised learning with a large quantity of labelled training data in limited classes. This restricts their scalability to a large number of logo classes subject to limited labelling budget. In this work, we consider a more scalable open logo detection problem where only a fraction of logo classes are fully labelled whilst the remaining classes are only annotated with a clean icon image (e.g. 1-shot icon supervised). To generalise and transfer knowledge of fully supervised logo classes to other 1-shot icon supervised classes, we propose a Multi-Perspective Cross-Class (MPCC) domain adaptation method. In a data augmentation principle, MPCC conducts feature distribution alignment in two perspectives. Specifically, we align the feature distribution between synthetic logo images of 1-shot icon supervised classes and genuine logo images of fully supervised classes, and that between logo images and non-logo images, concurrently. This allows for mitigating the domain shift problem between model training and testing on 1-shot icon supervised logo classes, simultaneously reducing the model overfitting towards fully labelled logo classes. Extensive comparative experiments show the advantage of MPCC over existing state-of-the-art competitors on the challenging QMUL-OpenLogo benchmark (Su et al., 2018).

Domain-aware Stacked AutoEncoders for zero-shot learning

Zero-shot learning (ZSL), which focuses on transferring the knowledge from the seen (source) classes to unseen (target) ones, is getting more and more attention in the computer vision community. However, there often has a large domain gap between the source and target classes, resulting in the projection domain shift problem. To this end, we propose a novel model, named Domain-aware Stacked AutoEncoders (DaSAE), that consists of two interactive stacked auto-encoders to learn the domain-aware projections for adapting source and target domains respectively. In each of them, the first-layer encoder aims to project a visual feature vector into the semantic space, and the second-layer encoder connects the semantic description of a sample with its label directly. Meanwhile, the two-layer decoders seek to reconstruct the visual representation from the label information and semantic description successively. Moreover, the manifold regularization that explores the manifold structure residing in the target data is integrated to the basic DaAE, which further improves the generalization ability of our model. Extensive experiments on the benchmark datasets clearly demonstrate that our DaSAE outperforms the state-of-the-art alternatives by the significant margins.

Pyramid Pooling Module-Based Semi-Siamese Network: A Benchmark Model for Assessing Building Damage from xBD Satellite Imagery Datasets

Most mainstream research on assessing building damage using satellite imagery is based on scattered datasets and lacks unified standards and methods to quantify and compare the performance of different models. To mitigate these problems, the present study develops a novel end-to-end benchmark model, termed the pyramid pooling module semi-Siamese network (PPM-SSNet), based on a large-scale xBD satellite imagery dataset. The high precision of the proposed model is achieved by adding residual blocks with dilated convolution and squeeze-and-excitation blocks into the network. Simultaneously, the highly automated process of satellite imagery input and damage classification result output is reached by employing concurrent learned attention mechanisms through a semi-Siamese network for end-to-end input and output purposes. Our proposed method achieves F1 scores of 0.90, 0.41, 0.65, and 0.70 for the undamaged, minor-damaged, major-damaged, and destroyed building classes, respectively. From the perspective of end-to-end methods, the ablation experiments and comparative analysis confirm the effectiveness and originality of the PPM-SSNet method. Finally, the consistent prediction results of our model for data from the 2011 Tohoku Earthquake verify the high performance of our model in terms of the domain shift problem, which implies that it is effective for evaluating future disasters.

Low-rank embedded orthogonal subspace learning for zero-shot classification

Zero-shot classification methods have attracted considerable attention in recent years. Existing ZSC methods encounter domain shift, hubness and visual–semantic gap problems. To address these problems, we propose a low-rank embedded orthogonal subspace learning method (LEOSL) for ZSC. Many previous works project visual features to the semantic space. However, they often suffer from the visual–semantic gap problem. To handle this problem, we project the visual representations and semantic representations to the common subspace. To address the domain shift problem, we restrict the mapping functions with a low-rank constraint. To handle the hubness problem, we introduce the class similarity term so that samples of the same class are located near each other, while samples of different classes are located far away. Furthermore, we restrict the shared representations in the subspace with an orthogonal constraint to remove the correlation between samples. The results show the superiority of LEOSL compared to many state-of-the-art methods.

Unsupervised domain adaptation for person re-identification with iterative soft clustering

In this work, we propose to address the unsupervised domain adaptive (UDA) person re-id problem in which the model learns from an unlabeled target domain using a fully annotated source domain. Current approaches mainly address domain shift problem or the inter/intra-domain variation of the two domains. However, they have neglected to integrate the-easy-to-learn label distribution of the target domain into the model to improve its performance. Moreover, the automatic label assignment for the unlabeled target data currently used in UDA methods does not reflect the underlying data. To address these issues, we introduce a technique that enforces three properties: (1) target instance invariance that considers the target data and uses a key–value memory to guess the label distribution that is later used as the supervision signal. (2) a camera invariance, formed by unlabeled target images, and their camera-style transferred. Here, a new loss function is proposed to control overconfident predictions on the styled images. Lastly, (3) a hierarchical clustering-based optimization technique that exploits the similarities between the target images to constrain the supervision information of the first property. Here, we randomly allocate each target image to a separate cluster, then gradually incorporate similarity within each identity as we group similar images into clusters and use the cluster-IDs as the new target labels. We iteratively refine the guessed label distribution of the target domain by making predictions on the unlabeled target domain and then train the network with these new samples. Extensive experimental results on the concurrent use of these three properties demonstrate that the proposed model can achieve the state-of-the-art on unsupervised domain adaptive person re-id. Our work is important for knowledge discovery and knowledge transfer.

Iterative Knowledge Distillation for Automatic Check-Out

Automatic Check-Out (ACO) provides an object detection based mechanism for retailers to process the purchases of customers automatically. However, it suffers a lot from the domain shift problem because of different data distribution between the single item in training exemplar images and mixed items in testing checkout images. In this paper, we propose a new iterative knowledge distillation method to solve the domain adaptation problem for this task. First, we develop a new augmentation data strategy to generate synthesized checkout images. It can extract segmented items from the training images by the coarse-to-fine strategy and filter items with unrealistic poses by pose pruning. Second, we propose a dual pyramid scale network (DPSNet) to exploit the multi-scale feature representation in joint detection and counting views. Third, the iterative knowledge distillation training strategy is developed to make full use of both image-level and instance-level samples to narrow the semantic gap between source domain and target domain. Extensive experiments on the large-scale Retail Product Checkout (RPC) dataset show the proposed DPSNet can achieve state-of-the-art performance compared with existing methods. The source codes can be found at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://isrc.iscas.ac.cn/gitlab/research/dpsnet</uri> .

Contrastive Adversarial Domain Adaptation for Machine Remaining Useful Life Prediction

Enabling precise forecasting of the remaining useful life (RUL) for machines can reduce maintenance cost, increase availability, and prevent catastrophic consequences. Data-driven RUL prediction methods have already achieved acclaimed performance. However, they usually assume that the training and testing data are collected from the same condition (same distribution or domain), which is generally not valid in real industry. Conventional approaches to address domain shift problems attempt to derive domain-invariant features, but fail to consider target-specific information, leading to limited performance. To tackle this issue, in this article, we propose a contrastive adversarial domain adaptation (CADA) method for cross-domain RUL prediction. The proposed CADA approach is built upon an adversarial domain adaptation architecture with a contrastive loss, such that it is able to take target-specific information into consideration when learning domain-invariant features. To validate the superiority of the proposed approach, comprehensive experiments have been conducted to predict the RULs of aeroengines across 12 cross-domain scenarios. The experimental results show that the proposed method significantly outperforms state-of-the-arts with over 21% and 38% improvements in terms of two different evaluation metrics.

Domain-Oriented Semantic Embedding for Zero-Shot Learning

Zero-Shot Learning (ZSL) targets to recognize images from new classes. Existing methods focus on learning a projection function to associate the visual features and category descriptions in the seen domain, which is directly transferred to the unseen domain. However, due to the inherent domain shift, a single shared projection cannot fully capture the domain difference and similarity, thereby making the unseen samples tend to be recognized as seen categories. In this paper, we propose a novel Domain-Oriented Semantic Embedding (DOSE) network that learns specific projections for different domains to better capture the domain characteristics for unbiased ZSL. Besides a domain-shared projection, DOSE learns two auxiliary domain-specific sub-projections to model the semantic-visual association in respective seen and unseen domains. Specifically, the domain-specific projections are learned in a cycle consistency way to capture domain characteristics, and a domain division constraint is developed to penalize the margin between two domain embeddings. Furthermore, to boost semantic-visual association, a semantic-visual dual attention module is designed to automatically remove trivial information in both visual and semantic embeddings under a co-guidance learning manner. Experiments on four public benchmarks prove that the proposed DOSE is robust to the domain shift problem in ZSL and obtains an averaged 5.6% improvement in terms of harmonic mean.

DOMAIN ADAPTATION WITH CYCLEGAN FOR CHANGE DETECTION IN THE AMAZON FOREST

Abstract. Deep learning classification models require large amounts of labeled training data to perform properly, but the production of reference data for most Earth observation applications is a labor intensive, costly process. In that sense, transfer learning is an option to mitigate the demand for labeled data. In many remote sensing applications, however, the accuracy of a deep learning-based classification model trained with a specific dataset drops significantly when it is tested on a different dataset, even after fine-tuning. In general, this behavior can be credited to the domain shift phenomenon. In remote sensing applications, domain shift can be associated with changes in the environmental conditions during the acquisition of new data, variations of objects’ appearances, geographical variability and different sensor properties, among other aspects. In recent years, deep learning-based domain adaptation techniques have been used to alleviate the domain shift problem. Recent improvements in domain adaptation technology rely on techniques based on Generative Adversarial Networks (GANs), such as the Cycle-Consistent Generative Adversarial Network (CycleGAN), which adapts images across different domains by learning nonlinear mapping functions between the domains. In this work, we exploit the CycleGAN approach for domain adaptation in a particular change detection application, namely, deforestation detection in the Amazon forest. Experimental results indicate that the proposed approach is capable of alleviating the effects associated with domain shift in the context of the target application.

Intensity Augmentation to Improve Generalizability of Breast Segmentation Across Different MRI Scan Protocols.

The segmentation of the breast from the chest wall is an important first step in the analysis of breast magnetic resonance images. 3D U-Nets have been shown to obtain high segmentation accuracy and appear to generalize well when trained on one scanner type and tested on another scanner, provided that a very similar MR protocol is used. There has, however, been little work addressing the problem of domain adaptation when image intensities or patient orientation differ markedly between the training set and an unseen test set. In this work we aim to address this domain shift problem. We propose to apply extensive intensity augmentation in addition to geometric augmentation during training. We explored both style transfer and a novel intensity remapping approach as intensity augmentation strategies. For our experiments, we trained a 3D U-Net on T1-weighted scans. We tested our network on T2-weighted scans from the same dataset as well as on an additional independent test set acquired with a T1-weighted TWIST sequence and a different coil configuration. By applying intensity augmentation we increased segmentation performance for the T2-weighted scans from a Dice of 0.71 to 0.88. This performance is very close to the baseline performance of training with T2-weighted scans (0.92). On the T1-weighted dataset we obtained a performance increase from 0.77 to 0.85. Our results show that the proposed intensity augmentation increases segmentation performance across different datasets. The proposed method can improve whole breast segmentation of clinical MR scans acquired with different protocols.

Learning domain invariant unseen features for generalized zero-shot classification

Generalized zero-shot classification is a challenging task to recognize test data which may come from seen or unseen classes. Existing methods suffer from the bias problem that the unseen images are easy to be misclassified to seen classes. Generating some fake unseen samples by Generative Adversarial Network has been a popular method. However, these models are not easy to train. In this paper, we proposed a method by learning domain invariant unseen features for generalized zero-shot classification. Specifically, we learn the support seen class set for each unseen class for transferring knowledge from source to target domain. The unseen samples of each class are generated based on the combinations of the samples from its support seen class set. In addition, for dealing with the domain shift problem between source and target domains, we learn domain invariant unseen features by minimizing the Maximum Mean Discrepancy distance of seen data, generated unseen data and then project target data to the common space. For dealing with the bias problem, we select some confident target unseen samples to augment training samples for training the classifier. In experiments, we demonstrate that the proposed method significantly outperforms other state-of-the-art methods.

Framework for role discovery using transfer learning

Discovering the node roles in a network helps to solve diverse social problems. Role discovery attempts to predict the node roles from a network structure, and this method has been extensively studied in various fields. Role discovery using transfer learning has many advantages, but methods using this approach face two kinds of problems: domain-shift problems and model selection. To address these problems, we propose a general framework that includes network representation learning, domain adversarial learning for suppressing domain-shift problems, and model selection without using target labels. As a result of computational experiments, we show on publicly available datasets that the proposed model outperforms conventional methods, the proposed model selection method performs well without using target labels, and the proposed method can be used in real-world datasets. Furthermore, we found that our framework suppressed domain-shift problems, worked well even with differences between networks, and could handle imbalanced classes.

Adversarial Multiple-Target Domain Adaptation for Fault Classification

Data-driven fault classification methods are receiving great attention as they can be applied to many real-world applications. However, they work under the assumption that training data and testing data are drawn from the same distribution. Practical scenarios have varying operating conditions, which results in a domain-shift problem that significantly deteriorates the diagnosis performance. Recently, domain adaptation (DA) has been explored to address the domain-shift problem by transferring the knowledge from labeled source domain (e.g., source working condition) to unlabeled target domain (e.g., target working condition). Yet, all the existing methods are working under single-source single-target (1S1T) settings. Hence, a new model needs to be trained for each new target domain. This shows limited scalability in handling multiple working conditions since different models should be trained for different target working conditions, which is clearly not a viable solution in practice. To address this problem, we propose a novel adversarial multiple-target DA (AMDA) method for single-source multiple-target (1SmT) scenario, where the model can generalize to multiple-target domains concurrently. Adversarial adaptation is applied to transform the multiple-target domain features to be invariant from the single-source-domain features. This leads to a scalable model with a novel capability of generalizing to multiple-target domains. Extensive experiments on two public datasets and one self-collected dataset have demonstrated that the proposed method outperforms state-of-the-art methods consistently. Our source codes and data are available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/mohamedr002/AMDA</uri> .

Target-Independent Domain Adaptation for WBC Classification Using Generative Latent Search.

Automating the classification of camera-obtained microscopic images of White Blood Cells (WBCs) and related cell subtypes has assumed importance since it aids the laborious manual process of review and diagnosis. Several State-Of-The-Art (SOTA) methods developed using Deep Convolutional Neural Networks suffer from the problem of domain shift - severe performance degradation when they are tested on data (target) obtained in a setting different from that of the training (source). The change in the target data might be caused by factors such as differences in camera/microscope types, lenses, lighting-conditions etc. This problem can potentially be solved using Unsupervised Domain Adaptation (UDA) techniques albeit standard algorithms presuppose the existence of a sufficient amount of unlabelled target data which is not always the case with medical images. In this paper, we propose a method for UDA that is devoid of the need for target data. Given a test image from the target data, we obtain its 'closest-clone' from the source data that is used as a proxy in the classifier. We prove the existence of such a clone given that infinite number of data points can be sampled from the source distribution. We propose a method in which a latent-variable generative model based on variational inference is used to simultaneously sample and find the 'closest-clone' from the source distribution through an optimization procedure in the latent space. We demonstrate the efficacy of the proposed method over several SOTA UDA methods for WBC classification on datasets captured using different imaging modalities under multiple settings.