Domain-invariant Representations Research Articles

Unsupervised domain adaptation via re-weighted transfer subspace learning with inter-class sparsity

Unsupervised domain adaptation is intended to respond to a challenging scenario where the training and testing samples are related yet distributed differently. A common technique to realize the alignment of distributions is feature transformation, which is gaining increasing attention in the transfer learning community. Many of existing studies concentrate on learning domain-invariant feature representations in order to transfer classifiers from the source to the target domain. However, the class discriminability of features is also crucial for the cross-domain learning, which could lead to decreased performance if ignored. To extract both the transferable and discriminative feature representation, a novel unsupervised domain adaptation method termed re-weighted transfer subspace learning with inter-class sparsity (ICS-RTSL) is proposed. Central to the proposed approach is the introduction of a class-wise sparsity regularization that aims to reduce the intra-class distances by maintaining the structural consistency of the source samples, thus making it possible to learn a more discriminative representation. In addition, a residual term through the least absolute criterion is constructed to mitigate the negative impact possibly brought by outliers. This is followed by the proposal of an efficient algorithm with iteratively re-weighted least squares to optimize the learning model. Supported by a set of comparative experiments, the effectiveness of the proposed ICS-RTSL is demonstrated through cross-domain image recognition tasks.

Meta-Generalization for Domain-Invariant Speaker Verification

Automatic speaker verification (ASV) exhibits unsatisfactory performance under domain mismatch conditions owing to intrinsic and extrinsic factors, such as variations in speaking styles and recording devices encountered in real-world applications. To ensure robust performance under unseen conditions, domain generalization has been explored. However, an inherent contradiction exists between model discrimination and domain generalization, in which the discrimination ability may be reduced while learning to generalize. In this paper, to extract discriminative yet domain-invariant representations, we propose the meta-generalized speaker verification (MGSV) via meta-learning. Specifically, we propose a metric-based distribution optimization and a gradient-based meta-optimization to simultaneously supervise the spatial relationship between embeddings and improve the generalization ability of the model on unseen domains. In addition, we design multiple-single (MS) and simulated speaker verification (SSV) sampling strategies based on single-domain (SD) and single-single (SS) strategies to simulate the train/test domain mismatch more relevantly, thereby mining transferable speaker-related knowledge. SSV is chosen as the most effective method, as it substantially improves the domain generalization by ensuring that the model has learned to discriminate efficiently. Additionally, to intuitively reflect the model performance on the unseen domains, the proposed method is validated on cross-genre, cross-device, and cross-dataset tasks. The experimental results demonstrate that our proposed method achieves remarkable performance in handling domain mismatch issues in speaker verification.

Meta-Learning Based Tasks Similarity Representation for Cross Domain Lifelong Learning

Deep neural networks perform better in most specific single tasks than humans, but it is hard to handle a sequence of new tasks from different domains. The deep learning-based models always need to remember the parameters of the learned tasks to perform well in the new tasks and forfeit the ability to generalize from previous data, which is inconsistent with human learning. We propose a novel lifelong learning framework that can guide the model to learn new knowledge without forgetting the old knowledge through learning the similarity representation based on meta-learning. Specifically, we employ a cross-domain triplets network (CDTN) by minimizing the maximum mean discrepancy (MMD) between the current task and the knowledge base to learn the domain invariant similarity representation between tasks in different domains. Furthermore, we add a self-attention module to enhance the extraction of similarity features. Secondly, a spatial attention network (SAN) which can assign different weights according to the learned similarity representation of tasks, is proposed in addition. The experimental results show that our method effectively reduces catastrophic forgetting compared with the state-of-the-art methods when learning many tasks. Moreover, we show that the proposed method can hardly forget the old knowledge while continuously enhancing the performance of the old tasks, which is more in line with the human way of learning.

Multi-Modal Cross-Domain Alignment Network for Video Moment Retrieval

As an increasingly popular task in multimedia information retrieval, video moment retrieval (VMR) aims to localize the target moment from an untrimmed video according to a given language query. Most previous methods depend heavily on numerous manual annotations ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> , moment boundaries), which are extremely expensive to acquire in practice. In addition, due to the domain gap between different datasets, directly applying these pre-trained models to an unseen domain leads to a significant performance drop. In this paper, we focus on a novel task: cross-domain VMR, where fully-annotated datasets are available in one domain (“source domain”), but the domain of interest (“target domain”) only contains unannotated datasets. As far as we know, we present the first study on cross-domain VMR. To address this new task, we propose a novel Multi-Modal Cross-Domain Alignment (MMCDA) network to transfer the annotation knowledge from the source domain to the target domain. However, due to the domain discrepancy between the source and target domains and the semantic gap between videos and queries, directly applying trained models to the target domain generally leads to a performance drop. To solve this problem, we develop three novel modules: (i) a domain alignment module is designed to align the feature distributions between different domains of each modality; (ii) a cross-modal alignment module aims to map both video and query features into a joint embedding space and to align the feature distributions between different modalities in the target domain; (iii) a specific alignment module tries to obtain the fine-grained similarity between a specific frame and the given query for optimal localization. By jointly training these three modules, our MMCDA can learn domain-invariant and semantic-aligned cross-modal representations. Extensive experiments on three challenging benchmarks (ActivityNet Captions, Charades-STA and TACoS) illustrate that our cross-domain method MMCDA outperforms all state-of-the-art single-domain methods. Impressively, MMCDA raises the performance by more than 7% in representative cases, which demonstrates its effectiveness.

Cross-Domain Contrastive Learning for Unsupervised Domain Adaptation

Unsupervised domain adaptation (UDA) aims to transfer knowledge learned from a fully-labeled source domain to a different unlabeled target domain. Most existing UDA methods learn domain-invariant feature representations by minimizing feature distances across domains. In this work, we build upon contrastive self-supervised learning to align features so as to reduce the domain discrepancy between training and testing sets. Exploring the same set of categories shared by both domains, we introduce a simple yet effective framework CDCL, for domain alignment. In particular, given an anchor image from one domain, we minimize its distances to cross-domain samples from the same class relative to those from different categories. Since target labels are unavailable, we use a clustering-based approach with carefully initialized centers to produce pseudo labels. In addition, we demonstrate that CDCL is a general framework and can be adapted to the data-free setting, where the source data are unavailable during training, with minimal modification. We conduct experiments on two widely used domain adaptation benchmarks, i.e., Office-31 and VisDA-2017, for image classification tasks, and demonstrate that CDCL achieves state-of-the-art performance on both datasets.

Collaborative and adversarial network for text‐independent speaker verification in domain adaptation

Speaker verification models have achieved good results on the single genre data. But the performance degrades when model training and testing are not in the same domain. The adversarial training method is proposed to solve this problem by minimizing domain distribution differences. However, the adversarial training ignores domain-specific information for the domain-invariant speaker representations. In this paper, an improved collaborative adversarial network for domain adaptation in speaker verification is performed. Compared to the adversarial training, a collaborative discriminator is newly incorporated that learns domain-specific information at the lower layers. Further, the projection block is added to the collaborative discriminator. It reduces the noise introduced by the collaborative discriminator. Experiments are conducted in different mismatch scenarios and using different speaker encoders. All the experimental results show that the performance of this method is better than the baseline and previous work using adversarial training.

A Novel Multiview Predictive Local Adversarial Network for Partial Transfer Learning in Cross-Domain Fault Diagnostics

In recent years, transfer learning technology has been widely developed in the field of fault diagnosis and detection. At present, most transfer learning methods are based on the assumption that the source domain and the target domain share the same label space. However, in the industrial production process, it is more common that the label space of the target domain is a subset of the label space of the source domain, which is usually called partial transfer learning. In this article, a new multiview predictive local adversarial network (MPLAN) is proposed to solve this problem. The proposed method aligns the source domain and the target domain by local adversarial training where the features of normal states in the source domain and target domain are used to implement adversarial training to learn domain-invariant feature representations and avoid negative transfer caused by outlier classes. Moreover, two classifiers, which have different viewpoints, are used to predict the same sample in order to map the raw data to a better feature space. Weights of two classifiers are made a constraint in order to guarantee that two classifiers own different viewpoints. In order to verify the effectiveness of the proposed method, two datasets, which contain one 1-D sample dataset and one multidimensional difficult dataset, are used to design experiments. Based on the experimental results on two datasets, the proposed MPLAN method can improve the baseline by about 10%.

Self-Supervised Deep Tensor Domain-Adversarial Regression Adaptation for Online Remaining Useful Life Prediction Across Machines

With deep transfer learning techniques, this paper focuses on the online remaining useful life (RUL) prediction problem across different machines, and tries to address the following concerns: 1) The effect of transfer learning decreases significantly due to considerable divergence of degradation characteristic; 2) A high computational cost is raised by re-training whole model with online data; 3) Error accumulation occurs because of lacking label information of online data. In this paper, a self-supervised deep tensor domain-adversarial regression adaptation approach is proposed. In the pre-training stage, a novel tensor domain-adversarial network, with a tensorized domain discriminator, is constructed using the offline whole-life degradation data and early fault data of the target machine. A new training algorithm with an alternating minimization scheme is then built to seek the optimal core tensor and domain-invariant feature representation. In the online stage, a new self-supervised fine-tuning strategy is designed for the target network initialized from the pre-trained network. The core tensor-formed self-supervised information, extracted from the monotonicity of online degradation process, and the pseudo-supervised information from the pre-trained network are integrated to realize fast and adaptive RUL prediction. This paper takes rolling bearing as an example, and runs both cross-conditions and cross-machines experiments on three rolling bearings datasets, i.e., IEEE PHM Challenge 2012, XJTU-SY and our test rig. The results verify the use of tensor representation can facilitate regression adversarial training, and demonstrate the proposed approach can effectively improve predictive accuracy and stability under unknown online conditions.

Failure to Achieve Domain Invariance With Domain Generalization Algorithms: An Analysis in Medical Imaging

One prominent issue in the application of deep learning is the failure to generalize to data that lies on a different distribution to the training data. While many methods have been proposed to address this, prior work has shown that when operating under the same conditions most algorithms perform almost equally. As such, more work needs to be done to validate past and future methods before they are put into important scenarios like medical imaging. Our work analyses eight domain generalization algorithms across four important medical imaging classification datasets along with three standard natural image classification problems to discover the differences in how these methods operate in these different contexts. We assess these algorithms in terms of generalization capability, domain invariance, and representational sensitivity. Through this, we show that despite the differences between domain and content variations between natural and medical imaging there is little deviation in the operation of each method between natural images and medical images. Additionally, we show that all tested algorithms retain significant amounts of domain-specific information in their feature representations despite explicit training to remove it. Thus, revealing the failure point of all these methods is a lack of class-discriminative features extracted from out-of-distribution data. While these results show that methods that work well on natural imaging work similarly in medical imaging, no method outperforms baseline methods, highlighting the continuing gap of achieving adequate domain generalization. Similarly, the results also question the efficacy of optimizing for domain invariant representations as a method for generalizing to unseen domains.

Learning Domain Invariant Representations for Generalizable Person Re-Identification.

Generalizable person Re-Identification (ReID) aims to learn ready-to-use cross-domain representations for direct cross-data evaluation, which has attracted growing attention in the recent computer vision (CV) community. In this work, we construct a structural causal model (SCM) among identity labels, identity-specific factors (clothing/shoes color etc.), and domain-specific factors (background, viewpoints etc.). According to the causal analysis, we propose a novel Domain Invariant Representation Learning for generalizable person Re-Identification (DIR-ReID) framework. Specifically, we propose to disentangle the identity-specific and domain-specific factors into two independent feature spaces, based on which an effective backdoor adjustment approximate implementation is proposed for serving as a causal intervention towards the SCM. Extensive experiments have been conducted, showing that DIR-ReID outperforms state-of-the-art (SOTA) methods on large-scale domain generalization (DG) ReID benchmarks.

Single-source Domain Expansion Network for Cross-Scene Hyperspectral Image Classification.

Currently, cross-scene hyperspectral image (HSI) classification has drawn increasing attention. It is necessary to train a model only on source domain (SD) and directly transferring the model to target domain (TD), when TD needs to be processed in real time and cannot be reused for training. Based on the idea of domain generalization, a Single-source Domain Expansion Network (SDEnet) is developed to ensure the reliability and effectiveness of domain extension. The method uses generative adversarial learning to train in SD and test in TD. A generator including semantic encoder and morph encoder is designed to generate the extended domain (ED) based on encoder-randomization-decoder architecture, where spatial randomization and spectral randomization are specifically used to generate variable spatial and spectral information, and the morphological knowledge is implicitly applied as domain invariant information during domain expansion. Furthermore, the supervised contrastive learning is employed in the discriminator to learn class-wise domain invariant representation, which drives intra-class samples of SD and ED. Meanwhile, adversarial training is designed to optimize the generator to drive intra-class samples of SD and ED to be separated. Extensive experiments on two public HSI datasets and one additional multispectral image (MSI) dataset demonstrate the superiority of the proposed method when compared with state-of-the-art techniques. The codes will be available from the website: https://github.com/YuxiangZhang-BIT/IEEE_TIP_SDEnet.

Cross-Domain Few-Shot Hyperspectral Image Classification With Class-Wise Attention

Few-shot learning (FSL) is an effective method to solve the problem of hyperspectral image (HSI) classification with few labeled samples. It learns transferable knowledge from sufficient labeled auxiliary data to classify unseen classes with limited labeled samples for training. However, the distribution difference between auxiliary data and unseen classes results in the learned transferable knowledge not being well applied to the new task. Therefore, a class-wise attentive cross-domain FSL (CA-CFSL) framework is proposed in this article, in which a feature extractor is learned to extract data features with discriminability and domain invariance. The class-wise attention metric module (CAMM) introduces class-wise attention on the FSL framework to learn more discriminative features, which improves the interclass decision boundaries. Furthermore, an asymmetric domain adversarial module (ADAM) is designed to enhance the ability of extracting domain-invariant representations, which combines asymmetric adversarial training with embedded domain-specific information. Experimental results on four public HSI datasets demonstrate that the proposed method outperforms the existing methods.

Fault Diagnosis of Unseen Modes in Chemical Processes Based on Labeling and Class Progressive Adversarial Learning

While deep neural network (DNN)-based fault diagnosis methods can monitor the faults that occur on the operating modes, they cannot perform well on the modes that are never experienced before. This limitation makes it challenging to ensure the production safety of chemical processes. In this article, this issue is formulated as domain generalization (DG), which aims to learn a universal fault diagnosis model from historical operating modes that can generalize well to unseen modes. Many existing DG approaches focus on learning a domain-invariant representation by aligning marginal distributions between domains, which ignore the conditional relationships and label information. Recently, some researches start to reduce the discrepancy of the class conditional distributions across domains, while the theoretical justifications of that are still missing. To address this issue, the theoretical analysis of DG is developed to investigate how to minimize the unseen domains’ risk, which is a theoretical guarantee that the DG methods can generalize well over unseen domains. This theory reveals that the unseen domain error can be bounded by the shift of the label and class conditional distributions across source domains. Then, this result motivates a novel labeling and class progressive adversarial learning (LCPAL) algorithm for fault diagnosis, which simultaneously controls the errors weighted based on label information and aligns the class conditional distributions between different historical operating modes, as well as reducing the adverse effect of the domain-specific feature. Empirical results on both the numerical example and the Tennessee Eastman process (TEP) demonstrate the effectiveness of the LCPAL approach.

Prototype-Guided Multitask Adversarial Network for Cross-Domain LiDAR Point Clouds Semantic Segmentation

Unsupervised domain adaptation (UDA) segmentation aims to leverage labeled source data to make accurate predictions on unlabeled target data. The key is to make the segmentation network learn domain-invariant representations. In this work, we propose a prototype-guided multitask adversarial network (PMAN) to achieve this. First, we propose an intensity-aware segmentation network (IAS-Net) that leverages the private intensity information of target data to substantially facilitate feature learning of the target domain. Second, the category-level cross-domain feature alignment strategy is introduced to flee the side effects of global feature alignment. It employs the prototype (class centroid) and includes two essential operations: 1) build an auxiliary nonparametric classifier to evaluate the semantic alignment degree of each point based on the prediction consistency between the main and auxiliary classifiers and 2) introduce two class-conditional point-to-prototype learning objectives for better alignment. One is to explicitly perform category-level feature alignment in a progressive manner, and the other aims to shape the source feature representation to be discriminative. Extensive experiments reveal that our PMAN outperforms state-of-the-art results on two benchmark datasets.

Domain-Class Correlation Decomposition for Generalizable Person Re-Identification

Domain generalization in person re-identification is a highly important meaningful and practical task in which a model trained with data from several source domains is expected to generalize well to unseen target domains. Domain adversarial learning is a promising domain generalization method that aims to remove domain information in the latent representation through adversarial training. However, in person re-identification, the domain and class are correlated, and we theoretically show that domain adversarial learning will lose certain information about class due to this domain-class correlation. Inspired by casual inference, we propose to perform interventions to the domain factor d, aiming to decompose the domain-class correlation. To achieve this goal, we proposed estimating the resulting representation z caused by the intervention through first- and second-order statistical characteristic matching. Specifically, we build a memory bank to restore the statistical characteristics of each domain. Then, we use the newly generated samples {z, y, d} to compute the loss function. These samples are domain-class correlation decomposed; thus, we can learn a domain-invariant representation that can capture more class-related features. Extensive experiments show that our model outperforms the state-of-the-art methods on the large-scale domain generalization Re-ID benchmark.

Style Uncertainty Based Self-Paced Meta Learning for Generalizable Person Re-Identification.

Domain generalizable person re-identification (DG ReID) is a challenging problem, because the trained model is often not generalizable to unseen target domains with different distribution from the source training domains. Data augmentation has been verified to be beneficial for better exploiting the source data to improve the model generalization. However, existing approaches primarily rely on pixel-level image generation that requires designing and training an extra generation network, which is extremely complex and provides limited diversity of augmented data. In this paper, we propose a simple yet effective feature based augmentation technique, named Style-uncertainty Augmentation (SuA). The main idea of SuA is to randomize the style of training data by perturbing the instance style with Gaussian noise during training process to increase the training domain diversity. And to better generalize knowledge across these augmented domains, we propose a progressive learning to learn strategy named Self-paced Meta Learning (SpML) that extends the conventional one-stage meta learning to multi-stage training process. The rationality is to gradually improve the model generalization ability to unseen target domains by simulating the mechanism of human learning. Furthermore, conventional person Re-ID loss functions are unable to leverage the valuable domain information to improve the model generalization. So we further propose a distance-graph alignment loss that aligns the feature relationship distribution among domains to facilitate the network to explore domain-invariant representations of images. Extensive experiments on four large-scale benchmarks demonstrate that our SuA-SpML achieves state-of-the-art generalization to unseen domains for person ReID.

Generative Inference Network for Imbalanced Domain Generalization.

Domain generalization (DG) aims to learn transferable knowledge from multiple source domains and generalize it to the unseen target domain. To achieve such expectation, the intuitive solution is to seek domain-invariant representations via generative adversarial mechanism or minimization of cross-domain discrepancy. However, the widespread imbalanced data scale problem across source domains and category in real-world applications becomes the key bottleneck of improving generalization ability of model due to its negative effect on learning the robust classification model. Motivated by this observation, we first formulate a practical and challenging imbalance domain generalization (IDG) scenario, and then propose a straightforward but effective novel method generative inference network (GINet), which augments reliable samples for minority domain/category to promote discriminative ability of the learned model. Concretely, GINet utilizes the available cross-domain images from the identical category and estimates their common latent variable, which derives to discover domain-invariant knowledge for unseen target domain. According to these latent variables, our GINet further generates more novel samples with optimal transport constraint and deploys them to enhance the desired model with more robustness and generalization ability. Considerable empirical analysis and ablation studies on three popular benchmarks under normal DG and IDG setups suggests the advantage of our method over other DG methods on elevating model generalization. The source code is available in GitHub https://github.com/HaifengXia/IDG.

A Novel Joint Adversarial Domain Adaptation Method for Rotary Machine Fault Diagnosis under Different Working Conditions.

In real-world applications of detecting faults, many factors-such as changes in working conditions, equipment wear, and environmental causes-can cause a significant mismatch between the source domain on which classifiers are trained and the target domain to which those classifiers are applied. As such, existing deep network algorithms perform poorly under different working conditions. To solve this problem, we propose a novel fault diagnosis method named Joint Adversarial Domain Adaptation (JADA) for fault detection under different working conditions. Our approach simultaneously aligns marginal distribution and conditional distribution across the source and target through a unified adversarial learning process. JADA aims to construct domain-invariant and category-discriminative feature representation that is effective and robust for substantial distribution difference caused by working conditions. We also introduce a supervision signal, namely center loss, that penalizes the distances between the deep features and their corresponding class centers. This makes the learned features better equipped with more discriminative structures and effectively prevents mode collapse. Twenty-four transfer fault diagnosis tasks based on two experimental platforms were conducted to evaluate the effectiveness of the proposed methods. Extensive experiments verified that the JADA can significantly outperform several popular methods under different transfer diagnosis tasks.

Domain Generalization for Activity Recognition via Adaptive Feature Fusion

Human activity recognition requires the efforts to build a generalizable model using the training datasets with the hope to achieve good performance in test datasets. However, in real applications, the training and testing datasets may have totally different distributions due to various reasons such as different body shapes, acting styles, and habits, damaging the model’s generalization performance. While such a distribution gap can be reduced by existing domain adaptation approaches, they typically assume that the test data can be accessed in the training stage, which is not realistic. In this article, we consider a more practical and challenging scenario: domain-generalized activity recognition (DGAR) where the test dataset cannot be accessed during training. To this end, we propose Adaptive Feature Fusion for Activity Recognition (AFFAR) , a domain generalization approach that learns to fuse the domain-invariant and domain-specific representations to improve the model’s generalization performance. AFFAR takes the best of both worlds where domain-invariant representations enhance the transferability across domains and domain-specific representations leverage the model discrimination power from each domain. Extensive experiments on three public HAR datasets show its effectiveness. Furthermore, we apply AFFAR to a real application, i.e., the diagnosis of Children’s Attention Deficit Hyperactivity Disorder (ADHD), which also demonstrates the superiority of our approach.

Rethinking adversarial domain adaptation: Orthogonal decomposition for unsupervised domain adaptation in medical image segmentation.

Medical image segmentation methods based on deep learning have made remarkable progress. However, such existing methods are sensitive to data distribution. Therefore, slight domain shifts will cause a decline of performance in practical applications. To relieve this problem, many domain adaptation methods learn domain-invariant representations by alignment or adversarial training whereas ignoring domain-specific representations. In response to this issue, this paper rethinks the traditional domain adaptation framework and proposes a novel orthogonal decomposition adversarial domain adaptation (ODADA) architecture for medical image segmentation. The main idea behind our proposed ODADA model is to decompose the input features into domain-invariant and domain-specific representations and then use the newly designed orthogonal loss function to encourage their independence. Furthermore, we propose a two-step optimization strategy to extract domain-invariant representations by separating domain-specific representations, fighting the performance degradation caused by domain shifts. Encouragingly, the proposed ODADA framework is plug-and-play and can replace the traditional adversarial domain adaptation module. The proposed method has consistently demonstrated effectiveness through comprehensive experiments on three publicly available datasets, including cross-site prostate segmentation dataset, cross-site COVID-19 lesion segmentation dataset, and cross-modality cardiac segmentation dataset. The source code is available at https://github.com/YonghengSun1997/ODADA.