Unseen Target Domain Research Articles

Visual state estimation in unseen environments through domain adaptation and metric learning.

In robotics, deep learning models are used in many visual perception applications, including the tracking, detection and pose estimation of robotic manipulators. The state of the art methods however are conditioned on the availability of annotated training data, which may in practice be costly or even impossible to collect. Domain augmentation is one popular method to improve generalization to out-of-domain data by extending the training data set with predefined sources of variation, unrelated to the primary task. While this typically results in better performance on the target domain, it is not always clear that the trained models are capable to accurately separate the signals relevant to solving the task (e.g., appearance of an object of interest) from those associated with differences between the domains (e.g., lighting conditions). In this work we propose to improve the generalization capabilities of models trained with domain augmentation by formulating a secondary structured metric-space learning objective. We concentrate on one particularly challenging domain transfer task—visual state estimation for an articulated underground mining machine—and demonstrate the benefits of imposing structure on the encoding space. Our results indicate that the proposed method has the potential to transfer feature embeddings learned on the source domain, through a suitably designed augmentation procedure, and on to an unseen target domain.

Better pseudo-label: Joint domain-aware label and dual-classifier for semi-supervised domain generalization

With the goal of directly generalizing trained model to unseen target domains, domain generalization (DG), a newly proposed learning paradigm, has attracted considerable attention. Previous DG models usually require a sufficient quantity of annotated samples from observed source domains during training. In this paper, we relax this requirement about full annotation and investigate semi-supervised domain generalization (SSDG) where only one source domain is fully annotated along with the other domains totally unlabeled in the training process. With the challenges of tackling the domain gap between observed source domains and predicting unseen target domains, we propose a novel deep framework via joint domain-aware labels and dual-classifier to produce high-quality pseudo-labels. Concretely, to predict accurate pseudo-labels under domain shift, a domain-aware pseudo-labeling module is developed. Also, considering inconsistent goals between generalization and pseudo-labeling: former prevents overfitting on all source domains while latter might overfit the unlabeled source domains for high accuracy, we employ a dual-classifier to independently perform pseudo-labeling and domain generalization in the training process. When accurate pseudo-labels are generated for unlabeled source domains, the domain mixup operation is applied to augment new domains between labeled and unlabeled domains, which is beneficial for boosting the generalization capability of the model. Extensive results on publicly available DG benchmark datasets show the efficacy of our proposed SSDG method.

Feature-Based Style Randomization for Domain Generalization

As a recent noticeable topic, domain generalization (DG) aims to first learn a generic model on multiple source domains and then directly generalize to an arbitrary unseen target domain without any additional adaption. In previous DG models, by generating virtual data to supplement observed source domains, the data augmentation based methods have shown its effectiveness. To simulate the possible unseen domains, most of them enrich the diversity of original data via image-level style transformation. However, we argue that the potential styles are hard to be exhaustively illustrated and fully augmented due to the limited referred styles, leading the diversity could not be always guaranteed. Unlike image-level augmentation, we in this paper develop a simple yet effective feature-based style randomization module to achieve feature-level augmentation, which can produce random styles via integrating random noise into the original style. Compared with existing image-level augmentation, our feature-level augmentation favors a more goal-oriented and sample-diverse way. Furthermore, to sufficiently explore the efficacy of the proposed module, we design a novel progressive training strategy to enable all parameters of the network to be fully trained. Extensive experiments on three standard benchmark datasets, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> , PACS, VLCS and Office-Home, highlight the superiority of our method compared to the state-of-the-art methods.

Semantic-Discriminative Mixup for Generalizable Sensor-based Cross-domain Activity Recognition

It is expensive and time-consuming to collect sufficient labeled data to build human activity recognition (HAR) models. Training on existing data often makes the model biased towards the distribution of the training data, thus the model might perform terribly on test data with different distributions. Although existing efforts on transfer learning and domain adaptation try to solve the above problem, they still need access to unlabeled data on the target domain, which may not be possible in real scenarios. Few works pay attention to training a model that can generalize well to unseen target domains for HAR. In this paper, we propose a novel method called Semantic-Discriminative Mixup (SDMix) for generalizable cross-domain HAR. Firstly, we introduce semantic-aware Mixup that considers the activity semantic ranges to overcome the semantic inconsistency brought by domain differences. Secondly, we introduce the large margin loss to enhance the discrimination of Mixup to prevent misclassification brought by noisy virtual labels. Comprehensive generalization experiments on five public datasets demonstrate that our SDMix substantially outperforms the state-of-the-art approaches with 6% average accuracy improvement on cross-person, cross-dataset, and cross-position HAR.

Single-Domain Generalization in Medical Image Segmentation via Test-Time Adaptation from Shape Dictionary

Domain generalization typically requires data from multiple source domains for model learning. However, such strong assumption may not always hold in practice, especially in medical field where the data sharing is highly concerned and sometimes prohibitive due to privacy issue. This paper studies the important yet challenging single domain generalization problem, in which a model is learned under the worst-case scenario with only one source domain to directly generalize to different unseen target domains. We present a novel approach to address this problem in medical image segmentation, which extracts and integrates the semantic shape prior information of segmentation that are invariant across domains and can be well-captured even from single domain data to facilitate segmentation under distribution shifts. Besides, a test-time adaptation strategy with dual-consistency regularization is further devised to promote dynamic incorporation of these shape priors under each unseen domain to improve model generalizability. Extensive experiments on two medical image segmentation tasks demonstrate the consistent improvements of our method across various unseen domains, as well as its superiority over state-of-the-art approaches in addressing domain generalization under the worst-case scenario.

Inter-domain curriculum learning for domain generalization

Domain generalization aims to learn a domain-invariant representation from multiple source domains so that a model can generalize well across unseen target domains. Such models are often trained with examples that are presented randomly from all source domains, which can make the training unstable due to optimization in conflicting gradient directions. Here, we explore inter-domain curriculum learning (IDCL) where source domains are exposed in a meaningful order to gradually provide more complex ones. The experiments show that significant improvements can be achieved in both PACS and Office–Home benchmarks, and ours improves the state-of-the-art method by 1.08%.

Hcdg: A Hierarchical Consistency Framework for Domain Generalization on Medical Image Segmentation

Modern deep neural networks struggle to transfer knowledge and generalize across domains when deploying to real-world applications. Domain generalization (DG) aims to learn a universal representation from multiple source domains to improve the network generalization ability on unseen target domains. Previous DG methods mostly focus on the data-level consistency scheme to advance the generalization capability of deep networks, without considering the synergistic regularization of different consistency schemes. In this paper, we present a novel Hierarchical Consistency framework for Domain Generalization (HCDG) by ensembling Extrinsic Consistency and Intrinsic Consistency. Particularly, for Extrinsic Consistency, we leverage the knowledge across multiple source domains to enforce data-level consistency. Also, we design a novel Amplitude Gaussian-mixing strategy for Fourier-based data augmentation to enhance such consistency. For Intrinsic Consistency, we perform task-level consistency for the same instance under the dual-task form. We evaluate the proposed HCDG framework on two medical image segmentation tasks, i.e., optic cup/disc segmentation on fundus images and prostate MRI segmentation. Extensive experimental results manifest the effectiveness and versatility of our HCDG framework. Code will be available once accept.

Pivot-Guided Embedding for Domain Generalization

Neural networks have suffered from a distribution gap between training and test data, known as domain shift. Domain generalization (DG) methods aim to learn domain invariant representations only with limited source domain data to cope with unseen target domains. The main assumption is that the model trained to extract semantically consistent features without any domain specific information is highly adaptable to the unseen target domain. Metric learning allows embedding representations to be class-separated and domain-mixed, which is an optimal condition for DG but has been downplayed in recent works. Even the most popular triplet embedding has limitations in forming an optimal embedding space for DG due to instability. In this paper, we present a novel deep metric learning method for domain invariant representations. Specifically, we propose Pivot-Guided Embedding (PGE), which explicitly forms the entire feature distribution of the embedding space with a novel pivot-guided attraction-repulsion mechanism, to address the instability problem that triplet embedding has. In particular, we leverage pivot features representing a coarse distribution of the entire space as reference points to guide other features toward domain invariant feature distribution. To this end, a pivot selection algorithm is presented to reliably reflect the entire feature distribution. Furthermore, we define Guide-Field, a subspace spanned by a subset of pivots chosen for individual samples, to guide each sample to domain invariant feature space. In a nutshell, the attraction-repulsion mechanism based on pivots, the reliable set of features representing the entire feature distribution, enables the model to extract domain invariant feature representations and also settles the instability problem of triplet loss. Experimental results on three different benchmarks validate the performance advantages of the proposed method over the state-of-the-art DG techniques.

Label Distribution Learning for Generalizable Multi-Source Person Re-Identification

Person re-identification (Re-ID) is a critical technique in the video surveillance system, which has achieved significant success in the supervised setting. However, it is difficult to directly apply the supervised model to arbitrary unseen domains due to the domain gap between the available source domains and unseen target domains. In this paper, we propose a novel label distribution learning (LDL) method to address generalizable multi-source person Re-ID task ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e</i> ., there are multiple available source domains, and the testing domain is unseen during training), which aims to explore the relation of different classes and mitigate the domain-shift across different domains so as to improve the discrimination of the model and learn the domain-invariant feature, simultaneously. Specifically, during the training process, we produce the label distribution via the online manner to mine the relation information of different classes, thus it is beneficial for extracting the discriminative feature. Besides, for the label distribution of each class, we further revise it to give more and equal attention to the other domains that the class does not belong to, which can effectively reduce the domain gap across different domains and obtain the domain-invariant feature. Furthermore, we also give the theoretical analysis to demonstrate that the proposed method can effectively deal with the domain-shift issue. Extensive experiments on multiple benchmark datasets validate the effectiveness of the proposed method and show that the proposed method can outperform the state-of-the-art methods. Besides, further analysis also reveals the superiority of the proposed method.

Conditional Contrastive Domain Generalization for Fault Diagnosis

Data-driven fault diagnosis plays a key role in stability and reliability of operations in modern industries. Recently, deep learning has achieved remarkable performance in fault classification tasks. However, in reality, the model can be deployed under highly varying working environments. As a result, the model trained under a certain working environment (i.e., certain distribution) can fail to generalize well on data from different working environments (i.e., different distributions). The naive approach of training a new model for each new working environment would be infeasible in practice. To address this issue, we propose a novel conditional contrastive domain generalization (CCDG) approach for fault diagnosis of rolling machinery, which is able to capture shareable class information and learn environment-independent representation among data collected from different environments (also known as domains). Specifically, our CCDG attempts to maximize the mutual information of similar classes across different domains while minimizing mutual information among different classes, such that it can learn domain-independent class representation that can be transferable to new unseen domains. Our proposed approach significantly outperforms state-of-the-art methods on two real-world fault diagnosis datasets with an average improvement of 7.75&#x0025; and 2.60&#x0025;, respectively. The promising performance of our proposed CCDG on new unseen target domain contributes toward more practical data-driven approaches that can work under challenging real-world environments.

Collaboration Between Multiple Experts for Knowledge Adaptation on Multiple Remote Sensing Sources

Due to the unique characteristics of remote sensing (RS) data, it is challenging to collect richer labeled samples for training the deep learning model compared with the natural image data. To solve this problem, recently, multi-source-single-target (MS²T) scenarios have started receiving significant attention in which the knowledge from multiple sources is integrated to transfer to a target domain with the assumption that label spaces of each source and target domain are the same. However, in real-world applications, it can be challenging to find a source domain that completely includes all classes of the target domain. Therefore, to cover all class information of the target domain, they often na&#x00EF;vely merge multiple sources into a complete single source. However, each source domain typically has a different data distribution; thus, the semantic information of each source domain can be damaged, leading to degrading the classification accuracy of the target domain. To address this problem, we propose a unified framework termed Multi-Expert Collaboration for Knowledge Adaptation (MECKA) from various sources. MECKA includes two main processes: multiple-view generation and collaborative learning. Multiview learning plays an essential role in preserving the unique characteristics of each source domain. In contrast, collaborative learning is responsible for connecting these views that leverage complementary information from each other to perform on an unseen target domain robustly. Experimental results showed that the proposed method achieved the best classification accuracy on remote sensing scene benchmark datasets on both complete and incomplete multisource unsupervised domain adaptation (UDA) tasks compared to benchmark methods.

Exploiting reliable pseudo-labels for unsupervised domain adaptive person re-identification

Person re-identification, getting impressive performance under the single-domain setting, often suffers huge performance drop when deploying to the unseen target domain owing to domain gap. Current research mainly focuses on unsupervised domain adaptation to alleviate the domain gap, and the methods by clustering the target-domain samples have achieved significant results. However, some inaccurate pseudo-labels, i.e., noisy pseudo-labels, may be generated on clustering, which will seriously affect the performance of the model. In order to solve the above problem, we propose a novel unsupervised domain adaptive person re-identification method by exploiting reliable pseudo-labels (RPL) from two aspects, i.e., adaptive dynamic clustering (ADC) and cross-camera similarity evaluation (CCSE). Specifically, firstly, for the methods based on the density-based clustering algorithm, we propose the adaptive dynamic clustering which calculates the clustering radius adaptively and dynamically to obtain more reasonable clustering results in the iterative optimization of the model. Next, for noisy pseudo-labels caused by small inter-class variations under the same camera, we propose the cross-camera similarity evaluation to filter out these noises to further improve the discrimination of the model. Extensive experiments on three publicly available large-scale datasets show that the proposed method can achieve state-of-the-art performance on unsupervised domain adaptation person re-identification.

Cross-domain object detection using unsupervised image translation

Unsupervised domain adaptation for object detection addresses the adaption of detectors trained in a source domain to work accurately in an unseen target domain. Recently, methods approaching the alignment of the intermediate features proven to be promising, achieving state-of-the-art results. However, these methods are laborious to implement and hard to interpret. Although promising, there is still room for improvements to close the performance gap toward the upper-bound (when training with the target data). In this work, we propose a method to generate an artificial dataset in the target domain to train an object detector. We employed two unsupervised image translators (CycleGAN and an AdaIN-based model) using only annotated data from the source domain and non-annotated data from the target domain. Our key contributions are the proposal of a less complex yet more effective method that also has an improved interpretability. Results on real-world scenarios for autonomous driving show significant improvements, outperforming state-of-the-art methods in most cases, further closing the gap toward the upper-bound.

Generalizable model-agnostic semantic segmentation via target-specific normalization

Semantic segmentation in a supervised learning manner has achieved significant progress in recent years. However, its performance usually drops dramatically due to the data-distribution discrepancy between seen and unseen domains when we directly deploy the trained model to segment the images of unseen (or new coming) domains. To this end, we propose a novel domain generalization framework for the generalizable semantic segmentation task, which enhances the generalization ability of the model from two different views, including the training paradigm and the test strategy. Concretely, we exploit the model-agnostic learning to simulate the domain shift problem, which deals with the domain generalization from the training scheme perspective. Besides, considering the data-distribution discrepancy between seen source and unseen target domains, we develop the target-specific normalization scheme to enhance the generalization ability. Furthermore, when images come one by one in the test stage, we design the image-based memory bank (Image Bank in short) with style-based selection policy to select similar images to obtain more accurate statistics of normalization. Extensive experiments highlight that the proposed method produces state-of-the-art performance for the domain generalization of semantic segmentation on multiple benchmark segmentation datasets, i.e., Cityscapes, Mapillary.

Dual Distribution Alignment Network for Generalizable Person Re-Identification

Domain generalization (DG) offers a preferable real-world setting for Person Re-Identification (Re-ID), which trains a model using multiple source domain datasets and expects it to perform well in an unseen target domain without any model updating. Unfortunately, most DG approaches are designed explicitly for classification tasks, which fundamentally differs from the retrieval task Re-ID. Moreover, existing applications of DG in Re-ID cannot correctly handle the massive variation among Re-ID datasets. In this paper, we identify two fundamental challenges in DG for Person Re-ID: domain-wise variations and identity-wise similarities. To this end, we propose an end-to-end Dual Distribution Alignment Network (DDAN) to learn domain-invariant features with dual-level constraints: the domain-wise adversarial feature learning and the identity-wise similarity enhancement. These constraints effectively reduce the domain-shift among multiple source domains further while agreeing to real-world scenarios. We evaluate our method in a large-scale DG Re-ID benchmark and compare it with various cutting-edge DG approaches. Quantitative results show that DDAN achieves state-of-the-art performance.

ROBUST WHITE MATTER HYPERINTENSITY SEGMENTATION ON UNSEEN DOMAIN.

Typical machine learning frameworks heavily rely on an underlying assumption that training and test data follow the same distribution. In medical imaging which increasingly begun acquiring datasets from multiple sites or scanners, this identical distribution assumption often fails to hold due to systematic variability induced by site or scanner dependent factors. Therefore, we cannot simply expect a model trained on a given dataset to consistently work well, or generalize, on a dataset from another distribution. In this work, we address this problem, investigating the application of machine learning models to unseen medical imaging data. Specifically, we consider the challenging case of Domain Generalization (DG) where we train a model without any knowledge about the testing distribution. That is, we train on samples from a set of distributions (sources) and test on samples from a new, unseen distribution (target). We focus on the task of white matter hyperintensity (WMH) prediction using the multi-site WMH Segmentation Challenge dataset and our local in-house dataset. We identify how two mechanically distinct DG approaches, namely domain adversarial learning and mix-up, have theoretical synergy. Then, we show drastic improvements of WMH prediction on an unseen target domain.

Transfer Learning in Magnetic Resonance Brain Imaging: A Systematic Review.

(1) Background: Transfer learning refers to machine learning techniques that focus on acquiring knowledge from related tasks to improve generalization in the tasks of interest. In magnetic resonance imaging (MRI), transfer learning is important for developing strategies that address the variation in MR images from different imaging protocols or scanners. Additionally, transfer learning is beneficial for reutilizing machine learning models that were trained to solve different (but related) tasks to the task of interest. The aim of this review is to identify research directions, gaps in knowledge, applications, and widely used strategies among the transfer learning approaches applied in MR brain imaging; (2) Methods: We performed a systematic literature search for articles that applied transfer learning to MR brain imaging tasks. We screened 433 studies for their relevance, and we categorized and extracted relevant information, including task type, application, availability of labels, and machine learning methods. Furthermore, we closely examined brain MRI-specific transfer learning approaches and other methods that tackled issues relevant to medical imaging, including privacy, unseen target domains, and unlabeled data; (3) Results: We found 129 articles that applied transfer learning to MR brain imaging tasks. The most frequent applications were dementia-related classification tasks and brain tumor segmentation. The majority of articles utilized transfer learning techniques based on convolutional neural networks (CNNs). Only a few approaches utilized clearly brain MRI-specific methodology, and considered privacy issues, unseen target domains, or unlabeled data. We proposed a new categorization to group specific, widely-used approaches such as pretraining and fine-tuning CNNs; (4) Discussion: There is increasing interest in transfer learning for brain MRI. Well-known public datasets have clearly contributed to the popularity of Alzheimer’s diagnostics/prognostics and tumor segmentation as applications. Likewise, the availability of pretrained CNNs has promoted their utilization. Finally, the majority of the surveyed studies did not examine in detail the interpretation of their strategies after applying transfer learning, and did not compare their approach with other transfer learning approaches.

Multi-Path and Group-Loss-Based Network for Speech Emotion Recognition in Multi-Domain Datasets.

Speech emotion recognition (SER) is a natural method of recognizing individual emotions in everyday life. To distribute SER models to real-world applications, some key challenges must be overcome, such as the lack of datasets tagged with emotion labels and the weak generalization of the SER model for an unseen target domain. This study proposes a multi-path and group-loss-based network (MPGLN) for SER to support multi-domain adaptation. The proposed model includes a bidirectional long short-term memory-based temporal feature generator and a transferred feature extractor from the pre-trained VGG-like audio classification model (VGGish), and it learns simultaneously based on multiple losses according to the association of emotion labels in the discrete and dimensional models. For the evaluation of the MPGLN SER as applied to multi-cultural domain datasets, the Korean Emotional Speech Database (KESD), including KESDy18 and KESDy19, is constructed, and the English-speaking Interactive Emotional Dyadic Motion Capture database (IEMOCAP) is used. The evaluation of multi-domain adaptation and domain generalization showed 3.7% and 3.5% improvements, respectively, of the F1 score when comparing the performance of MPGLN SER with a baseline SER model that uses a temporal feature generator. We show that the MPGLN SER efficiently supports multi-domain adaptation and reinforces model generalization.

Cluster adaptation networks for unsupervised domain adaptation

Domain adaptation is an important technology for transferring source domain knowledge to new, unseen target domains. Recently, domain adaptation models are applied to learn domain invariant representations by minimizing distribution distance or adversarial training in the feature space. However, existing adversarial domain adaptation methods fail to preserve the data structure in the feature space. In this paper, we propose a novel domain adaptation method called Cluster adaptation Networks (CAN). CAN decreases the domain shift by aligning the category centers of source representations and the cluster centers of target representations in the feature space, which preserves the class-level structure and facilitates the classification of the target domain. Experiments on Digits, Office-Home and ImageCLEF-DA datasets validate the effectiveness of the structure preservation in our model.

Adapting Off-the-Shelf Source Segmenter for Target Medical Image Segmentation.

Unsupervised domain adaptation (UDA) aims to transfer knowledge learned from a labeled source domain to an unlabeled and unseen target domain, which is usually trained on data from both domains. Access to the source domain data at the adaptation stage, however, is often limited, due to data storage or privacy issues. To alleviate this, in this work, we target source free UDA for segmentation, and propose to adapt an "off-the-shelf" segmentation model pre-trained in the source domain to the target domain, with an adaptive batch-wise normalization statistics adaptation framework. Specifically, the domain-specific low-order batch statistics, i.e., mean and variance, are gradually adapted with an exponential momentum decay scheme, while the consistency of domain shareable high-order batch statistics, i.e., scaling and shifting parameters, is explicitly enforced by our optimization objective. The transferability of each channel is adaptively measured first from which to balance the contribution of each channel. Moreover, the proposed source free UDA framework is orthogonal to unsupervised learning methods, e.g., self-entropy minimization, which can thus be simply added on top of our framework. Extensive experiments on the BraTS 2018 database show that our source free UDA framework outperformed existing source-relaxed UDA methods for the cross-subtype UDA segmentation task and yielded comparable results for the cross-modality UDA segmentation task, compared with a supervised UDA methods with the source data.