Cross-domain Adaptation Research Articles

Transferable adversarial masked self-distillation for unsupervised domain adaptation

Unsupervised domain adaptation (UDA) aims to transfer knowledge from a labeled source domain to a related unlabeled target domain. Most existing works focus on minimizing the domain discrepancy to learn global domain-invariant representation using CNN-based architecture while ignoring both transferable and discriminative local representation, e.g, pixel-level and patch-level representation. In this paper, we propose the Transferable Adversarial Masked Self-distillation based on Vision Transformer architecture to enhance the transferability of UDA, named TAMS. Specifically, TAMS jointly optimizes three objectives to learn both task-specific class-level global representation and domain-specific local representation. First, we introduce adversarial masked self-distillation objective to distill representation from a full image to the representation predicted from a masked image, which aims to learn task-specific global class-level representation. Second, we introduce masked image modeling objectives to learn local pixel-level representation. Third, we introduce an adversarial weighted cross-domain adaptation objective to capture discriminative potentials of patch tokens, which aims to learn both transferable and discriminative domain-specific patch-level representation. Extensive studies on four benchmarks and the experimental results show that our proposed method can achieve remarkable improvements compared to previous state-of-the-art UDA methods.

Cross-Domain and Disentangled Face Manipulation With 3D Guidance.

Face image manipulation via three-dimensional guidance has been widely applied in various interactive scenarios due to its semantically-meaningful understanding and user-friendly controllability. However, existing 3D-morphable-model-based manipulation methods are not directly applicable to out-of-domain faces, such as non-photorealistic paintings, cartoon portraits, or even animals, mainly due to the formidable difficulties in building the model for each specific face domain. To overcome this challenge, we propose, as far as we know, the first method to manipulate faces in arbitrary domains using human 3DMM. This is achieved through two major steps: 1) disentangled mapping from 3DMM parameters to the latent space embedding of a pre-trained StyleGAN2 [1] that guarantees disentangled and precise controls for each semantic attribute; and 2) cross-domain adaptation that bridges domain discrepancies and makes human 3DMM applicable to out-of-domain faces by enforcing a consistent latent space embedding. Experiments and comparisons demonstrate the superiority of our high-quality semantic manipulation method on a variety of face domains with all major 3D facial attributes controllable - pose, expression, shape, albedo, and illumination. Moreover, we develop an intuitive editing interface to support user-friendly control and instant feedback. Our project page is https://cassiepython.github.io/cddfm3d/index.html.

Domain-Invariant Feature Learning for Domain Adaptation

Unsupervised domain adaptation (UDA) mainly explores how to learn domain-invariant features from the source domain when the target domain label is unknown. To learn domain-invariant features requires aligning the distribution of samples from two domains in the feature space, which can be achieved by minimizing the maximum mean discrepancy (MMD) of samples from the two domains. However, there is still no effective way to find the best parameter values of MMD. Such a problem is addressed in the MMD with deep kernels (MMD-D), whose optimal parameters can be obtained through training. This study proposes a method of domain-invariant feature learning for UDA, whose architecture, named MMDDCDA, comprises an MMD-D module and a cross domain adaptation (CDA) module. MMDDCDA performs alternating training similar to adversarial training to alternately boost the power of the two modules. To our knowledge, this is the first UDA method that performs such alternating training on a UDA architecture using MMD with deep kernels. Experimental validation showed that the proposed method yields state-of-the-art results among UDA methods using other MMD variants and some UDA benchmarks.

Deep Robust Low Rank Correlation With Unifying Clustering Structure for Cross Domain Adaptation

Cross domain adaptation aims to improve the performance of the target domain model by making full use of information rich source domain samples. However, as information becomes richer, the noise also increases. In order to improve the reliability of cross domain adaptation, we propose a novel method based on deep robust low rank correlation. Borrowed from the traditional idea of Canonical Correlation Analysis (CCA), we developed a robust correlation model to maximize the correlation between source and target domains. Also, the low-rank characteristics of cross domain data can effectively reduce the negative influence of noisy data. Furthermore, in order that the cross-domain data can share a unifying clustering structure, we introduced a common Laplacian affinity structure. Then the learned features can be smoothed and aligned to the unifying structure. In this way, we obtain a deep robust low rank correlation model with the help of the unifying clustering structure, which can effectively reduce the influence of noise and improve the performance of cross domain adaptation. Experimental results on three data sets including Office-31, ImageCLEF-DA and Office-Home show that our model significantly outperforms state-of-the-art cross domain adaptation methods.

Unsupervised style-guided cross-domain adaptation for few-shot stylized face translation

Unsupervised style-guided cross-domain adaptation for few-shot stylized face translation

A machine learning approach for cross-domain plant identification using herbarium specimens

The preservation of plant specimens in herbaria has been carried out for centuries in efforts to study and confirm plant taxa. With the increasing collection of herbaria made available digitally, it is practical to use herbarium specimens for the automation of plant identification. They are also substantially more accessible and less expensive to obtain compared to field images. In fact, in remote and inaccessible habitats, field images of rare plant species are still immensely lacking. As a result, rare plant species identification is challenging due to the deficiency of training data. To address this problem, we investigate a cross-domain adaptation approach that allows knowledge transfer from a model learned from herbarium specimens to field images. We propose a model called Herbarium–Field Triplet Loss Network (HFTL network) to learn the mapping between herbarium and field domains. Specifically, the model is trained to maximize the embedding distance of different plant species and minimize the embedding distance of the same plant species given herbarium–field pairs. This paper presents the implementation and performance of the HFTL network to assess the herbarium–field similarity of plants. It corresponds to the cross-domain plant identification challenge in PlantCLEF 2020 and PlantCLEF 2021. Despite the lack of field images, our results show that the network can generalize and identify rare species. Our proposed HFTL network achieved a mean reciprocal rank score of 0.108 and 0.158 on the test set related to the species with few training field photographs in PlantCLEF 2020 and PlantCLEF 2021, respectively.

Cross-domain intelligent fault diagnosis of rolling bearing based on distance metric transfer learning

Rolling bearings are present ubiquitously in mechanical equipment, timely fault diagnosis has great significance in guaranteeing the safety of mechanical operation. In real world industrial applications, the distribution of training dataset (source domain) and testing dataset (target domain) is often different and varies with operating environment, which may lead to performance degradation. In this study, a cross-domain fault diagnosis of rolling bearing method based on distance metric transfer learning (DMTL) and wavelet packet decomposition (WPD) is proposed. The Mahalanobis distance is adopted for learning the intrinsic similarity or dissimilarity between instances and learned by simultaneously maximizing the intra-class distances and minimizing the inter-class distances for target domain. The features of source domain and target domain are first extracted from original vibration signals by WPD which is a powerful tool in dealing with non-stationary signals and can provide meticulous analysis. Then, the DMTL model is adopted to eliminate the error propagation across different components, which can weaken the weight of low-quality instances and enhance the weight of high-quality samples. Finally, the k-nearest neighbor (KNN) classifier is applied to accomplish the cross-domain intelligent fault-type classification. The superiority and effectiveness of the proposed fault diagnosis model is validated by two diagnosis cases. The experimental results demonstrated that the proposed method performs better than other compared methods in recognizing various fault types and has the capability in handling the complex cross-domain adaptation scenarios.

Real-time multi-modal semantic fusion on unmanned aerial vehicles with label propagation for cross-domain adaptation

Unmanned aerial vehicles (UAVs) equipped with multiple complementary sensors have tremendous potential for fast autonomous or remote-controlled semantic scene analysis, e.g., for disaster examination. Here, we propose a UAV system for real-time semantic inference and fusion of multiple sensor modalities. Semantic segmentation of LiDAR scans and RGB images, as well as object detection on RGB and thermal images, run online onboard the UAV computer using lightweight CNN architectures and embedded inference accelerators. We follow a late fusion approach where semantic information from multiple sensor modalities augments 3D point clouds and image segmentation masks while also generating an allocentric semantic map. Label propagation on the semantic map allows for sensor-specific adaptation with cross-modality and cross-domain supervision.Our system provides augmented semantic images and point clouds with ≈9Hz. We evaluate the integrated system in real-world experiments in an urban environment and at a disaster test site.

Sentiment lexicon for cross-domain adaptation with multi-domain dataset in Indian languages enhanced with BERT classification model

Many websites are attempting to offer a platform for users or customers to leave their reviews and comments about the products or services in their native languages. The cross-domain adaptation (CDA) analyses sentiment across domains. The sentiment lexicon falls short resulting in issues like feature mismatch, sparsity, polarity mismatch and polysemy. In this research, an augmented sentiment dictionary is developed in our native regional language (Tamil) that intends to construct the contextual links between terms in multi-domain datasets to reduce problems like polarity mismatch, feature mismatch, and polysemy. Data from the source domain and target domain both labeled and unlabeled are used in the proposed dictionary. To be more specific, the initial dictionary uses normalised pointwise mutual information (nPMI) to derive contextual weight, whereas the final dictionary uses the value of terms across all reviews to compute the accurate rank score. Here, a deep learning model called BERT is used for sentiment classification. For cross-domain adaptation, a modified multi-layer fuzzy-based convolutional neural network (M-FCNN) is deployed. This work aims to build a single dictionary using large number of vocabularies for classifying the reviews in Tamil for several target domains. This extendible dictionary enhances the accuracy of CDA greatly when compared to existing baseline techniques and easily handles a large number of terms in different domains.

Manifold embedded joint geometrical and statistical alignment for visual domain adaptation

Supervised learning algorithms like KNN assume that the training and the testing data come from the same source, and hence, have the same distribution. However, in real life, such instances are rare. Domain adaptation (DA) aims to model a classifier that is invariant to the different features of the training data. Domain invariant features are utilized to reduce distribution differences, ensuring an iterative improvement on the predicted target labels. Existing DA approaches either perform manifold subspace learning or align cross-domain distributions. At the same time, there exists a couple of imperative limitations: (1) degenerated feature transformation, wherein the seemingly domain invariant features are distorted during transformation (2) formulation of a robust objective function that must consider critical objectives in a unified framework to minimize deviations geometrically and statistically. To the best of our knowledge, none of the existing DA approaches address these two limitations simultaneously. Therefore, we propose a novel domain adaptation framework, called Manifold Embedded Joint Geometrical and Statistical Alignment (MEJGSA) for visual domain adaptation to address these limitations. MEJGSA first learns manifold features, and then formulates a robust objective function to reduce divergence between domains geometrically and statistically. We also consider more reliable classifiers to generate pseudo-labels to preserve objective functions such as conditional distributions and target domain discriminant information. Extensive experiments on two widely used cross-domain adaptation datasets demonstrate that MEJGSA exhibits significant improvements in classification accuracy as compared to cutting-edge shallow and deep DA approaches. The MATLAB code for our proposed method will be publicly available at 11https://github.com/rakesh1000/MEJGSA.

Cross-domain adaptation network based on attention mechanism for tool wear prediction

Cross-domain adaptation network based on attention mechanism for tool wear prediction

ACDC: Online unsupervised cross-domain adaptation

We consider the problem of online unsupervised cross-domain adaptation, where two independent but related data streams with different feature spaces – a fully labeled source stream and an unlabeled target stream – are learned together. Unique characteristics and challenges such as covariate shift, asynchronous concept drifts, and contrasting data throughput arise. We propose ACDC, an adversarial unsupervised domain adaptation framework that handles multiple data streams with a complete self-evolving neural network structure that reacts to these defiances. ACDC encapsulates three modules into a single model: A denoising autoencoder that extracts features, an adversarial module that performs domain conversion, and an estimator that learns the source stream and predicts the target stream. ACDC is a flexible and expandable framework with little hyper-parameter tunability. Our experimental results under the prequential test-then-train protocol indicate an improvement in target accuracy over the baseline methods, achieving more than a 10% increase in some cases.

Real-Time End-to-End Speech Emotion Recognition with Cross-Domain Adaptation

Language resources are the main factor in speech-emotion-recognition (SER)-based deep learning models. Thai is a low-resource language that has a smaller data size than high-resource languages such as German. This paper describes the framework of using a pretrained-model-based front-end and back-end network to adapt feature spaces from the speech recognition domain to the speech emotion classification domain. It consists of two parts: a speech recognition front-end network and a speech emotion recognition back-end network. For speech recognition, Wav2Vec2 is the state-of-the-art for high-resource languages, while XLSR is used for low-resource languages. Wav2Vec2 and XLSR have proposed generalized end-to-end learning for speech understanding based on the speech recognition domain as feature space representations from feature encoding. This is one reason why our front-end network was selected as Wav2Vec2 and XLSR for the pretrained model. The pre-trained Wav2Vec2 and XLSR are used for front-end networks and fine-tuned for specific languages using the Common Voice 7.0 dataset. Then, feature vectors of the front-end network are input for back-end networks; this includes convolution time reduction (CTR) and linear mean encoding transformation (LMET). Experiments using two different datasets show that our proposed framework can outperform the baselines in terms of unweighted and weighted accuracies.

Gradual adaption with memory mechanism for image-based 3D model retrieval

With the development of 3D modeling technology and its wide application in different fields, the number of 3D models increases rapidly, making 3D model retrieval a hot topic in current research. Compared with other 3D model retrieval methods, 2D image-based unsupervised 3D model retrieval takes the 2D images which have rich labels and are easy to obtain as the queries, and also takes into account the difficulties of labeling 3D models. 2D image-based unsupervised 3D model retrieval is a retrieval task involving cross-domain adaptation problem, which main challenge is the excessive domain gap. In this paper, we propose a cross-domain 3D model retrieval method of memory mechanism based on disentangled feature learning. The disentangled feature learning enables to disentangle the twisted original features into the isolated domain-invariant features and domain-specific features, where the former is to be aligned to narrow the domain gap. On this basis, the memory mechanism selects feature vectors from class memory modules constructed by class representative features of the opposite domain for every sample, which are used to update the domain-invariant features with gradient weight. The memory mechanism can gradually improve the adaptability of the model to the very different two domains. Experiments are conducted on the public datasets MI3DOR and MI3DOR-2 to verify the feasibility and the superiority of the proposed method. Especially on MI3DOR-2 dataset, our method outperforms the current state-of-the-art methods with gains of 7.71% for the strictest retrieval metric NN. • An end-to-end unsupervised 2D image-based 3D model retrieval framework. • Transfering knowledge from labeled 2D images to unlabeled 3D models. • Domain-invariant features are disentangled from the original features. • Memory module enhances domain-invariant features by representative features. • Experiments on MI3DOR and MI3DOR-2 verified the superiority of the method.

Manifold discrimination partial adversarial domain adaptation

Most traditional domain adaptations promote knowledge transfer with the assumption that source domain and target domain have the same label space. In a big data environment, we usually obtain the source dataset with a larger label space than the target dataset. Under this background, partial domain adaptation (PDA) comes into being naturally. In PDA, when the target domain label space is a subset of the source domain label space, cross-domain adaptation can be achieved by matching shared class samples in cross-domain. Many researchers make good progress in this field and propose different kinds of PDA methods. But in the existing PDA methods, the samples of the unknown class are still used for distribution alignment, and this makes the model suffer from the risk of negative transfer. In order to solve these problems, we propose manifold discrimination partial adversarial domain adaptation (MDPDA) to align the relevant classes across two domains and obtain effective domain adaptation. MDPDA combines manifold learning and adversarial learning. In MDPDA, we design the manifold discrimination module to map high-dimensional data into low-dimensional space and learn low-dimensional data information of source domain and target domain. We construct discriminative source domain feature space for model discrimination in MDPDA. MDPDA distinguishes and confuses the two domains through manifold learning and similarity structure, and generates discriminant feature space to promote the auxiliary network to better recognize the source domain samples. The comprehensive experiments on several benchmark datasets evaluate the superior performance of the proposed method compared to the state-of-the-art methods.

Quick Learning Mechanism With Cross-Domain Adaptation for Intelligent Fault Diagnosis

The fault diagnostic model trained for a laboratory case machine fails to perform well on the industrial machines running under variable operating conditions. For every new operating condition of such machines, a new diagnostic model has to be trained which is a time-consuming and uneconomical process. Therefore, we propose a quick learning mechanism that can transform the existing diagnostic model into a new model suitable for industrial machines operating in different conditions. The proposed method uses the Net2Net transformation followed by a fine-tuning to cancel/minimize the maximum mean discrepancy between the new data and the previous one. The fine-tuning of the model requires a very less amount of labeled target samples and very few iterations of training. Therefore, the proposed method is capable of learning the new target data pattern quickly. The effectiveness of the proposed fault diagnosis method has been demonstrated on the Case Western Reserve University dataset, Intelligent Maintenance Systems bearing dataset, and Paderborn university dataset under the wide variations of the operating conditions. It has been validated that the diagnostic model trained on artificially damaged fault datasets can be used to quickly train another model for a real damage dataset.

Unsupervised adversarial domain adaptation based on interpolation image for fish detection in aquaculture

Deep-learning-based object detection has brought great convenience to modern intelligent aquaculture. However, its good performance mainly depends on a huge number of labeled training data and the assumption that samples are independent and identically distributed. But in real intensive aquaculture, the acquisition of labeled samples in detection tasks is very time-consuming and laborious. At the same time, due to the influence of light, density, and background complexity, the domain shift caused by different data distribution is inevitable. To solve these problems, we introduce the domain adaptive object detection into aquaculture field for the first time to improve the cross-domain robustness of fish detection. To this end, the interpolation-based unsupervised adversarial domain adaptive fish detection model is proposed in this paper. The proposed model combines the detection network Faster RCNN and three adaptive modules to realize the cross-domain detection of fish. First, the style generation network is used to generate interpolation samples to fill the large distribution gap between the two aquaculture scenes. Then the alignment of local and global features is realized through different levels of adversarial training. Moreover, the focal loss is applied to the global adversarial loss to achieve better global feature alignment. This strategy increases the weight of hard-to-classify samples and enables the feature extraction network to extract more domain-invariant features. We conducted cross-domain adaptation experiments by collecting images from different aquaculture scenes. Compared with the original Faster RCNN and domain adaptation model DA-Faster, the proposed method can not only save the cost of manual annotation, but also effectively improve the detection performance of unlabeled target domain by leveraging the knowledge of the source domain.

Cross-Domain Adaptation Using Domain Interpolation for Rotating Machinery Fault Diagnosis

In recent years, deep neural networks have been successfully used for machine failure diagnosis. However, changes in data distribution related to variations in diverse working environments adversely impact the troubleshooting performance. Consequently, cross-domain adaptation, which minimizes inconsistencies in data distribution, is an important issue. We propose an adversarial domain adaptation-based interpolation method that minimizes domain discrepancy by mixing the data and creating a continuous space between different distributed domains using two regularizations. Data interpolation algorithms generate data that interpolate regions through mutual and coherent reconstructions between labeled and target data. This ensures data transformation diversity and creates a continuous latent space. The generated data achieve the two goals of domain invariance and improved class classification performance, through categorical and domain-based regularization. Extensive experiments upon public and real-world equipment datasets demonstrate that the proposed approach achieves an approximately 10–15% improvement in classification performance when compared to conventional methods at various levels of domain portability and data complexity. Experimental results show that the proposed method is an effective solution that can be applied to actual industrial sites.

Deep Domain Adaptation for Pavement Crack Detection

Deep learning-based pavement cracks detection methods often require large-scale labels with detailed crack location information to learn accurate predictions. In practice, however, crack locations are very difficult to be manually annotated due to various visual patterns of pavement crack. In this paper, we propose a Deep Domain Adaptation-based Crack Detection Network (DDACDN), which learns domain invariant features by taking advantage of the source domain knowledge to predict the multi-category crack location information in the target domain, where only image-level labels are available. Specifically, DDACDN first extracts crack features from both the source and target domain by a two-branch weights-shared backbone network. And in an effort to achieve the cross-domain adaptation, an intermediate domain is constructed by aggregating the three-scale features from the feature space of each domain to adapt the crack features from the source domain to the target domain. Finally, the network involves the knowledge of both domains and is trained to recognize and localize pavement cracks. To facilitate accurate training and validation for domain adaptation, we use two challenging pavement crack datasets CQU-BPDD and RDD2020. Furthermore, we construct a new large-scale Bituminous Pavement Multi-label Disease Dataset named CQU-BPMDD, which contains 38994 high-resolution pavement disease images to further evaluate the robustness of our model. Extensive experiments demonstrate that DDACDN outperforms state-of-the-art pavement crack detection methods in predicting the crack location on the target domain.

Curriculum-Style Local-to-Global Adaptation for Cross-Domain Remote Sensing Image Segmentation

Although domain adaptation has been extensively studied in natural image-based segmentation tasks, the research on cross-domain segmentation for very-high-resolution (VHR) remote sensing images (RSIs) still remains underexplored. The VHR RSI-based cross-domain segmentation mainly faces two critical challenges: 1) large area land covers with many diverse object categories bring severe local patch-level data distribution deviations, thus yielding different adaptation difficulties for different local patches and 2) different VHR sensor types or dynamically changing modes cause the VHR images to go through intensive data distribution differences even for the same geographical location, resulting in different global feature-level domain gaps. To address these challenges, we propose a curriculum-style local-to-global cross-domain adaptation framework for the segmentation of VHR RSIs. The proposed curriculum-style adaptation performs the adaptation process in an easy-to-hard way according to the adaptation difficulties that can be obtained using an entropy-based score for each patch of the target domain and, thus, well aligns the local patches in a domain image. The proposed local-to-global adaptation performs the feature alignment process from the locally semantic to globally structural feature discrepancies and consists of a semantic-level domain classifier and an entropy-level domain classifier that can reduce the above cross-domain feature discrepancies. Extensive experiments have been conducted in various cross-domain scenarios, including geographic location variations and imaging mode variations, and the experimental results demonstrate that the proposed method can significantly boost the domain adaptability of segmentation networks for VHR RSIs.