Unsupervised Domain Adaptation Methods Research Articles

Unsupervised Domain Adaptation via Stacked Convolutional Autoencoder

Unsupervised domain adaptation involves knowledge transfer from a labeled source to unlabeled target domains to assist target learning tasks. A critical aspect of unsupervised domain adaptation is the learning of more transferable and distinct feature representations from different domains. Although previous investigations, using, for example, CNN-based and auto-encoder-based methods, have produced remarkable results in domain adaptation, there are still two main problems that occur with these methods. The first is a training problem for deep neural networks; some optimization methods are ineffective when applied to unsupervised deep networks for domain adaptation tasks. The second problem that arises is that redundancy of image data results in performance degradation in feature learning for domain adaptation. To address these problems, in this paper, we propose an unsupervised domain adaptation method with a stacked convolutional sparse autoencoder, which is based on performing layer projection from the original data to obtain higher-level representations for unsupervised domain adaptation. More specifically, in a convolutional neural network, lower layers generate more discriminative features whose kernels are learned via a sparse autoencoder. A reconstruction independent component analysis optimization algorithm was introduced to perform individual component analysis on the input data. Experiments undertaken demonstrated superior classification performance of up to 89.3% in terms of accuracy compared to several state-of-the-art domain adaptation methods, such as SSRLDA and TLMRA.

Dynamic classifier approximation for unsupervised domain adaptation

In recent years, domain adaptation (DA) method has been proposed to solve the problem of domain shift between the training set and test set. However, most feature-based unsupervised domain adaptation methods have a weakness of strict requirement on the transformation matrix and they are not integrated with classifier. Therefore, this paper proposes a novel dynamic classifier approximation (DCA) method for unsupervised domain adaptation. Specifically, the proposed DCA, which combines classifier method to relax the requirements of transformation matrix, learns domain invariant features from structure and data level. 1) At the structure level, the low-rank representation, sparse representation and manifold learning are combined to preserve the structure information of data in the original feature space. 2) At the data level, distribution alignments are adopted to minimize the distribution difference of the source and target domains in the common subspace, which is helpful to reduce the negative impact caused by domain shift. In addition, for learning the common subspace more flexibly, a dynamic classifier is introduced to reduce the strict requirements on projection matrix. This paper carries out a large number of experiments on four data sets, and the experimental results show that the proposed method is superior to many of the latest domain adaptation methods, which can indicate its effectiveness.

A cross-domain fruit classification method based on lightweight attention networks and unsupervised domain adaptation

AbstractImage-based fruit classification offers many useful applications in industrial production and daily life, such as self-checkout in the supermarket, automatic fruit sorting and dietary guidance. However, fruit classification task will have different data distributions due to different application scenarios. One feasible solution to solve this problem is to use domain adaptation that adapts knowledge from the original training data (source domain) to the new testing data (target domain). In this paper, we propose a novel deep learning-based unsupervised domain adaptation method for cross-domain fruit classification. A hybrid attention module is proposed and added to MobileNet V3 to construct the HAM-MobileNet that can suppress the impact of complex backgrounds and extract more discriminative features. A hybrid loss function combining subdomain alignment and implicit distribution metrics is used to reduce domain discrepancy during model training and improve model classification performance. Two fruit classification datasets covering several domains are established to simulate common industrial and daily life application scenarios. We validate the proposed method on our constructed grape classification dataset and general fruit classification dataset. The experimental results show that the proposed method achieves an average accuracy of 95.0% and 93.2% on the two datasets, respectively. The classification model after domain adaptation can well overcome the domain discrepancy brought by different fruit classification scenarios. Meanwhile, the proposed datasets and method can serve as a benchmark for future cross-domain fruit classification research.

A generative adversarial network-based unsupervised domain adaptation method for magnetic resonance image segmentation

Intelligent medical image segmentation methods have been rapidly developed and applied, while a significant challenge is domain shift. That is, the segmentation performance degrades due to distribution differences between the source domain and the target domain. This paper proposed an unsupervised end-to-end domain adaptation medical image segmentation method based on the generative adversarial network (GAN). A network training and adjustment model was designed, including segmentation and discriminant networks. In the segmentation network, the residual module was used as the basic module to increase feature reusability and reduce model optimization difficulty. Further, it learned cross-domain features at the image feature level with the help of the discriminant network and a combination of segmentation loss with adversarial loss. The discriminant network took the convolutional neural network and used the labels from the source domain, to distinguish whether the segmentation result of the generated network is from the source domain or the target domain. The whole training process was unsupervised. The proposed method was tested with experiments on a public dataset of knee magnetic resonance (MR) images and the clinical dataset from our cooperative hospital. With our method, the mean Dice similarity coefficient (DSC) of segmentation results increased by 2.52% and 6.10% to the classical feature level and image level domain adaptive method. The proposed method effectively improves the domain adaptive ability of the segmentation method, significantly improves the segmentation accuracy of the tibia and femur, and can better solve the domain transfer problem in MR image segmentation.

Unsupervised domain adaptation using fuzzy rules and stochastic hierarchical convolutional neural networks

Unsupervised domain adaptation (UDA) describes a set of techniques for using previously acquired knowledge from labeled original data to support task completion in comparable but unlabeled target data. Existing UDA methods often use two classifiers to detect misaligned local areas between the original and prey vocations, resulting in poor implementation. To address this issue, we propose a fuzzy rules and stochastic classifier-based domain adaptation framework called SH-CNN+SMTEOA. Initially, the cross-domain mixed sampling approach is used to test the original and prey data. After that, the Principal Component Analysis is used to extract the characteristics, and fuzzy criteria are used to choose the suitable characteristics. Finally, we introduce the Stochastic Hierarchical Convolutional Neural Network for classification and the Selective Multi-Threshold Entropy Optimization Algorithm for judging a target instance’s dependability based on its predictive multi-threshold values. Investigations on UDA benchmark datasets reveal that the proposed method outperforms other methods in classification.

A Domain-Adversarial Multi-Graph Convolutional Network for Unsupervised Domain Adaptation Rolling Bearing Fault Diagnosis

The transfer learning method, based on unsupervised domain adaptation (UDA), has been broadly utilized in research on fault diagnosis under variable working conditions with certain results. However, traditional UDA methods pay more attention to extracting information for the class labels and domain labels of data, ignoring the influence of data structure information on the extracted features. Therefore, we propose a domain-adversarial multi-graph convolutional network (DAMGCN) for UDA. A multi-graph convolutional network (MGCN), integrating three graph convolutional layers (multi-receptive field graph convolutional (MRFConv) layer, local extreme value convolutional (LEConv) layer, and graph attention convolutional (GATConv) layer) was used to mine data structure information. The domain discriminators and classifiers were utilized to model domain labels and class labels, respectively, and align the data structure differences through the correlation alignment (CORAL) index. The classification and feature extraction ability of the DAMGCN was significantly enhanced compared with other UDA algorithms by two example validation results, which can effectively achieve rolling bearing cross-domain fault diagnosis.

Unsupervised domain adaptation using modified cycle generative adversarial network for aerial image classification

The unsupervised domain adaptation (UDA) of aerial image classification is of great significance in the traffic management and public safety applications. Due to the differences in the distribution of source and target domain datasets from different sensors and cities, and the acquisition difficulty and high annotation cost of target domain datasets, existing domain adaptive methods have difficulty obtaining high accuracy and stability. In this work, a modified cycle generative adversarial network (CycleGAN) (MGAN) based on the multiscale residual block (MSRB) and multipooling coordinate attention (MPCA) modules is proposed to generate fake images, which is the most critical stage of the UDA method. In the MGAN, the MSRB module can extract the multiscale information of the image, and the MPCA module can fuse the position and channel information in the image. These two modules better extract the features of edges, regions and semantic classes, which makes the generated fake images better retain the information of the source domain and possess the style of the target domain. Otherwise, the Deeplabv3+ model is used to obtain a pretrained and final segmentation model in the first and third stages. In the experiment, the proposed method is validated on the aerial images of the Potsdam and Vaihingen datasets in the International Society for Photogrammetry and Remote Sensing (ISPRS) two-dimensional semantic labeling benchmark and improves overall accuracy from 52% to 60% on the segmentation of the target domain, compared with the existing optimal method; it also achieves an accuracy of 77% for buildings. The results demonstrate that the proposed method can achieve a higher accuracy and stability.

Unsupervised Adversarial Domain Adaptation for Agricultural Land Extraction of Remote Sensing Images

Agricultural land extraction is an essential technical means to promote sustainable agricultural development and modernization research. Existing supervised algorithms rely on many finely annotated remote-sensing images, which is both time-consuming and expensive. One way to reduce the annotation cost approach is to migrate models trained on existing annotated data (source domain) to unannotated data (target domain). However, model generalization capability is often unsatisfactory due to the limit of the domain gap. In this work, we use an unsupervised adversarial domain adaptation method to train a neural network to close the gap between the source and target domains for unsupervised agricultural land extraction. The overall approach consists of two phases: inter-domain and intra-domain adaptation. In the inter-domain adaptation, we use a generative adversarial network (GAN) to reduce the inter-domain gap between the source domain (labeled dataset) and the target domain (unlabeled dataset). The transformer with robust long-range dependency modeling acts as the backbone of the generator. In addition, the multi-scale feature fusion (MSFF) module is designed in the generator to accommodate remote sensing datasets with different spatial resolutions. Further, we use an entropy-based approach to divide the target domain. The target domain is divided into two subdomains, easy split images and hard split images. By training against each other between the two subdomains, we reduce the intra-domain gap. Experiments results on the “DeepGlobe → LoveDA”, “GID → LoveDA” and “DeepGlobe → GID” unsupervised agricultural land extraction tasks demonstrate the effectiveness of our method and its superiority to other unsupervised domain adaptation techniques.

CNNs/ViTs-CNNs/ViTs: Mutual distillation for unsupervised domain adaptation

Unsupervised Domain Adaptation (UDA) is a popular machine learning technique to reduce the distribution discrepancy among domains. In previous UDA methods, only convolutional neural networks (CNNs) or vision transformers (ViTs) are used as the backbone. Therefore, these methods are subjected to inherent characteristics of single-CNNs or single-ViTs. In fact, CNNs are more biased towards local textures, while ViTs are inclined to learn the shape information of images. In this paper, we consider that the model combining the above two bias properties is closer to the human visual neural system and achieves more robust performance. A novel mutual distillation method for UDA without any assistant networks is proposed. Mutual distillation between two backbones with complementary properties (i.e. CNNs & ViTs) can promote each other, leading to better domain knowledge transfer. Additionally, traditional domain-mixup approaches can only mix limited cross-domain information through linear interpolation. To encourage more cross-domain information interaction between the two backbones and bridge the domain gap, we propose a patch-mixup method, building a train of mixed intermediate domains composed of augmented patches. Meanwhile, a cross-domain semantic alignment loss is proposed to align the semantic information of these domains. Extensive experiments show that our method achieves state-of-the-art results on several standard UDA benchmarks, such as Office-31, Office-Home, ImageCLEF-DA and VisDA-2017.

Unsupervised Domain Adaptation for Gesture Identification Against Electrode Shift

Surface electromyogram (sEMG)-based hand gesture recognition, which interprets commands given by humans through sEMG signals, performs well in many studies. However, its recognition accuracy drops dramatically due to electrode shift since the distributions of motion classes are changed. Although calibrating the system with newly collected samples after electrode shift maintains the accuracy, collecting labeled samples is inconvenient and time-consuming since the procedure is rigid. However, the calibration may not work properly without label, especially when the change is significant. This study proposes a user friendly and convenient calibration method for hand gesture recognition by an unsupervised domain adaptation method, which only obtains the unlabeled samples of preselected benchmark classes from users in calibration. The change of benchmark classes is captured by unlabeled samples by a clustering method. The other classes are estimated based on the benchmark classes by regression models. As a result, the information of all classes is used to calibrate the system. Linear discriminant analysis is used to demonstrate our model. A dataset with ten subjects is collected to verify the performance empirically. Experimental results confirm that our method utilizes the unlabeled benchmark class samples in calibration and achieves 75.55% average accuracy. Our method is more robust to electrode shift and improves around 8.5% accuracy consistently on all subjects compared with the methods without calibration or label information in calibration. Although the accuracy of our method is slightly less than the ones using label calibration samples, our calibration data collection is more convenient and less complicated.

Dynamic Distribution Alignment With Dual-Subspace Mapping for Cross-Subject Driver Mental State Detection

For the detection of electroencephalogram-based driving mental states, it is important to utilize transfer learning to overcome individual and period differences. However, there are two challenges in existing unsupervised domain adaptation methods: 1) they ignore the geometric divergence of the source and target domains with single subspace mapping and 2) they usually employ a fixed weight distribution to align the marginal and conditional probability distributions. In this article, we propose a dynamic distribution alignment with dual-subspace mapping (DDADSM) method for cross-subject driver mental state detection. Initially, DDADSM explores two optimally aligned subspaces for the source and target domains, which can significantly reduce the geometric shifting. Subsequently, the dynamic probability distribution alignment method is introduced to acquire the adaptive weight between the marginal and conditional distributions, which adapts to domains with wide variations. Through our experiments based on the driving mental state detection task, DDADSM demonstrated superior performance compared with state-of-the-art models.

Unsupervised Domain Adaptation Through Dynamically Aligning Both the Feature and Label Spaces

In unsupervised domain adaptation (UDA), a target-domain model is trained by the supervised knowledge from a source domain. Although UDA has recently received much attention, most existing UDA methods have ignored the alignment in label space while mostly concentrating on alignment of feature space. Even worse, they have payed less attention to the dynamic relationship between domain alignment and discrimination, leading to degenerated performance. In this work, we propose a new kind of UDA through aligning in both the feature and label spaces (DAFL), in which a dynamic weight is designed and deployed between domain alignment and discrimination enhancement according to their conditions. Specifically, the cross-domain distribution divergence is reduced by the weighted class-level feature space alignment as well as the compacted and discriminative label space alignment. Furthermore, the balancing weight between adaptation alignment and discrimination enhancement is dynamically adjusted to regularize the adversarial domain adaptation. Then, the generalization ability of the DAFL model is enhanced by adding discrepant classification with theoretical analysis. Finally, extensive experiments validate effectiveness and superiority of the proposed approach.

Unsupervised domain adaptation based on the predictive uncertainty of models

Unsupervised domain adaptation (UDA) aims to improve the prediction performance in the target domain under distribution shifts from the source domain. The key principle of UDA is to minimize the divergence between the source and the target domains. To follow this principle, many methods employ a domain discriminator to match the feature distributions. Some recent methods evaluate the discrepancy between two predictions on target samples to detect those that deviate from the source distribution. However, their performance is limited because they either match the marginal distributions or measure the divergence conservatively. In this paper, we present a novel UDA method that learns domain-invariant features that minimize the domain divergence. We propose model uncertainty as a measure of the domain divergence. Our UDA method based on model uncertainty (MUDA) adopts a Bayesian framework and provides an efficient way to evaluate model uncertainty by means of Monte Carlo dropout sampling. Experiment results on image recognition tasks show that our method is superior to existing state-of-the-art methods. We also extend MUDA to multi-source domain adaptation problems.

Divergence-Agnostic Unsupervised Domain Adaptation by Adversarial Attacks.

Conventional machine learning algorithms suffer the problem that the model trained on existing data fails to generalize well to the data sampled from other distributions. To tackle this issue, unsupervised domain adaptation (UDA) transfers the knowledge learned from a well-labeled source domain to a different but related target domain where labeled data is unavailable. The majority of existing UDA methods assume that data from the source domain and the target domain are available and complete during training. Thus, the divergence between the two domains can be formulated and minimized. In this paper, we consider a more practical yet challenging UDA setting where either the source domain data or the target domain data are unknown. Conventional UDA methods would fail this setting since the domain divergence is agnostic due to the absence of the source data or the target data. Technically, we investigate UDA from a novel view-adversarial attack-and tackle the divergence-agnostic adaptive learning problem in a unified framework. Specifically, we first report the motivation of our approach by investigating the inherent relationship between UDA and adversarial attacks. Then we elaborately design adversarial examples to attack the training model and harness these adversarial examples. We argue that the generalization ability of the model would be significantly improved if it can defend against our attack, so as to improve the performance on the target domain. Theoretically, we analyze the generalization bound for our method based on domain adaptation theories. Extensive experimental results on multiple UDA benchmarks under conventional, source-absent and target-absent UDA settings verify that our method is able to achieve a favorable performance compared with previous ones. Notably, this work extends the scope of both domain adaptation and adversarial attack, and expected to inspire more ideas in the community.

Spatial graph convolutional neural network via structured subdomain adaptation and domain adversarial learning for bearing fault diagnosis

Unsupervised domain adaptation (UDA) has shown remarkable results in fault diagnosis under changing working conditions in recent years. However, most UDA methods do not consider the geometric structure of the data. Furthermore, the global domain adaptation technique is commonly applied, which ignores the relation between subdomains. This paper addresses mentioned challenges by presenting the novel deep subdomain adaptation graph convolution neural network (DSAGCN), which has two key characteristics: First, a graph convolution neural network (GCNN) is employed to model the structure of data. Second, adversarial domain adaptation and local maximum mean discrepancy (LMMD) methods are applied concurrently to align the subdomain’s distribution and reduce structure discrepancy between relevant subdomains and global domains. CWRU, PU, and JNU bearing datasets are used to validate the DSAGCN method’s superiority between comparison models. The experimental results demonstrate the significance of aligning structured subdomains along with domain adaptation methods to obtain an accurate data-driven model.

Cross-domain structure learning for visual data recognition

Unsupervised domain adaptation methods are used to train an effect model by utilizing available knowledge from a labeled source domain for solving tasks in an unlabeled target domain. The most difficult challenge is determining methods to reduce distribution discrepancies and extract the largest number of domain-invariant features between the source and target domains to improve model performance. With the aim of minimizing the domain shift and maximizing domain-invariant feature extraction, we propose a cross-domain structure learning (CDSL) method for visual data recognition, which incorporates global distribution alignment and local discriminative structure preservation to capture the common, underlying features between domains. Specifically, we design a simple but effective classwise structure learning strategy with a specific compactness hierarchy to promote intraclass knowledge transfer and reduce the risk of negative transfer between domains. We also extend CDSL to different kinds of kernelization to address complex situations in the real world. Extensive experiments on several visual data benchmarks demonstrate the effectiveness of our proposed method.

Adversarial style discrepancy minimization for unsupervised domain adaptation

Mainstream unsupervised domain adaptation (UDA) methods align feature distributions across different domains via adversarial learning. However, most of them focus on global distribution alignment, ignoring the fine-grained domain discrepancy. Besides, they generally require auxiliary models, bringing extra computation costs. To tackle these issues, this study proposes an UDA method that differentiates individual samples without the help of extra models. To this end, we introduce a novel discrepancy metric, termed style discrepancy, to distinguish different target samples. We also propose a paradigm for adversarial style discrepancy minimization (ASDM). Specifically, we fix the parameters of the feature extractor and maximize style discrepancy to update the classifier, which helps detect more hard samples. Adversely, we fix the parameters of the classifier and minimize the style discrepancy to update the feature extractor, pushing those hard samples near the support of the source distribution. Such adversary helps to progressively detect and adapt more hard samples, leading to fine-grained domain adaptation. Experiments on different UDA tasks validate the effectiveness of ASDM. Overall, without any extra models, ASDM reaches a 46.9% mIoU in the GTA5 to Cityscapes benchmark and an 84.7% accuracy in the VisDA-2017 benchmark, outperforming many existing adversarial-learning-based methods.

Enhanced Sample Self-Revised Network for Cross-Dataset Facial Expression Recognition

Recently, cross-dataset facial expression recognition (FER) has obtained wide attention from researchers. Thanks to the emergence of large-scale facial expression datasets, cross-dataset FER has made great progress. Nevertheless, facial images in large-scale datasets with low quality, subjective annotation, severe occlusion, and rare subject identity can lead to the existence of outlier samples in facial expression datasets. These outlier samples are usually far from the clustering center of the dataset in the feature space, thus resulting in considerable differences in feature distribution, which severely restricts the performance of most cross-dataset facial expression recognition methods. To eliminate the influence of outlier samples on cross-dataset FER, we propose the enhanced sample self-revised network (ESSRN) with a novel outlier-handling mechanism, whose aim is first to seek these outlier samples and then suppress them in dealing with cross-dataset FER. To evaluate the proposed ESSRN, we conduct extensive cross-dataset experiments across RAF-DB, JAFFE, CK+, and FER2013 datasets. Experimental results demonstrate that the proposed outlier-handling mechanism can reduce the negative impact of outlier samples on cross-dataset FER effectively and our ESSRN outperforms classic deep unsupervised domain adaptation (UDA) methods and the recent state-of-the-art cross-dataset FER results.

Cardiac segmentation on late gadolinium enhancement MRI: A benchmark study from multi-sequence cardiac MR segmentation challenge.

Accurate computing, analysis and modeling of the ventricles and myocardium from medical images are important, especially in the diagnosis and treatment management for patients suffering from myocardial infarction (MI). Late gadolinium enhancement (LGE) cardiac magnetic resonance (CMR) provides an important protocol to visualize MI. However, compared with the other sequences LGE CMR images with gold standard labels are particularly limited. This paper presents the selective results from the Multi-Sequence Cardiac MR (MS-CMR) Segmentation challenge, in conjunction with MICCAI 2019. The challenge offered a data set of paired MS-CMR images, including auxiliary CMR sequences as well as LGE CMR, from 45 patients who underwent cardiomyopathy. It was aimed to develop new algorithms, as well as benchmark existing ones for LGE CMR segmentation focusing on myocardial wall of the left ventricle and blood cavity of the two ventricles. In addition, the paired MS-CMR images could enable algorithms to combine the complementary information from the other sequences for the ventricle segmentation of LGE CMR. Nine representative works were selected for evaluation and comparisons, among which three methods are unsupervised domain adaptation (UDA) methods and the other six are supervised. The results showed that the average performance of the nine methods was comparable to the inter-observer variations. Particularly, the top-ranking algorithms from both the supervised and UDA methods could generate reliable and robust segmentation results. The success of these methods was mainly attributed to the inclusion of the auxiliary sequences from the MS-CMR images, which provide important label information for the training of deep neural networks. The challenge continues as an ongoing resource, and the gold standard segmentation as well as the MS-CMR images of both the training and test data are available upon registration via its homepage (www.sdspeople.fudan.edu.cn/zhuangxiahai/0/mscmrseg/).

DAN-PD: Domain adaptive network with parallel decoder for polyp segmentation.

Endoscopy is essential for polyp diagnosis and prevention of colorectal cancer. Many deep learning methods have been proposed to perform automatic semantic segmentation of polyps in endoscopic images. However, labeled training images are always scarce, and the styles of endoscopic images from different medical centers vary greatly. The annotation of medical images requires much effort, and how to make more efficient utilization of the existing labeled data is becoming an increasingly critical issue. Considering the characteristics of polyp segmentation tasks and the need for generalization, we proposed a novel method named DAN-PD based on the Vision Transformer. Moreover, we devised the Teacher Parallel Encoder (TPE) and the Domain-Aware Parallel Decoder (DAPD) for the model. Our design innovatively introduces Unsupervised Domain Adaptation (UDA) methods and adversarial learning strategies to the polyp segmentation task. We conducted four transfer learning experiments with three public polyp image datasets to examine the model's performance. The results shows that our proposed method is ahead of other methods in all experiments and reaches the state-of-the-art level.