Domain Adaptation For Image Classification Research Articles

A Copula Based Unsupervised Domain Adaptation for Image Classification

In this paper, we present an unsupervised domain adaptation algorithm for image classification using principal component analysis (PCA) and Gaussian copula function alignment. The motivation of the proposed algorithm stems from the idea of CORAL algorithm which extracts domain invariant features by aligning the correlation structure between a source and a target domain. However, it suffers from the fact that highly skewed marginal distributions happen to distort the correlation structure so that it may cause a negative transfer. Therefore we utilize a copula function that enables us to analyze separately the dependency structure and the marginals by Sklar’s theorem. In particular, we propose to align the Gaussian copula correlations in the copula feature spaces instead of aligning the correlation matrices in the original space. Considering the extremely skewed distribution of SURF image features in Office-Caltech10 data set we apply PCA first in order to extract some skewness-mitigated principal features and then derive copula features to align for domain adaptation by using CORAL idea with Gaussian copula correlation matrices. The proposed method showed a good classification accuracy when applied to image classification problem in an unsupervised domain adaptation setting.

Open Set Domain Adaptation for Image Classification With Multiple Unknown Labels Using Unsupervised Clustering in a Target Domain

Open Set Domain Adaptation for Image Classification With Multiple Unknown Labels Using Unsupervised Clustering in a Target Domain

Unsupervised Domain Adaptation for Image Classification and Object Detection Using Guided Transfer Learning Approach and JS Divergence.

Unsupervised domain adaptation (UDA) is a transfer learning technique utilized in deep learning. UDA aims to reduce the distribution gap between labeled source and unlabeled target domains by adapting a model through fine-tuning. Typically, UDA approaches assume the same categories in both domains. The effectiveness of transfer learning depends on the degree of similarity between the domains, which determines an efficient fine-tuning strategy. Furthermore, domain-specific tasks generally perform well when the feature distributions of the domains are similar. However, utilizing a trained source model directly in the target domain may not generalize effectively due to domain shift. Domain shift can be caused by intra-class variations, camera sensor variations, background variations, and geographical changes. To address these issues, we design an efficient unsupervised domain adaptation network for image classification and object detection that can learn transferable feature representations and reduce the domain shift problem in a unified network. We propose the guided transfer learning approach to select the layers for fine-tuning the model, which enhances feature transferability and utilizes the JS-Divergence to minimize the domain discrepancy between the domains. We evaluate our proposed approaches using multiple benchmark datasets. Our domain adaptive image classification approach achieves 93.2% accuracy on the Office-31 dataset and 75.3% accuracy on the Office-Home dataset. In addition, our domain adaptive object detection approach achieves 51.1% mAP on the Foggy Cityscapes dataset and 72.7% mAP on the Indian Vehicle dataset. We conduct extensive experiments and ablation studies to demonstrate the effectiveness and efficiency of our work. Experimental results also show that our work significantly outperforms the existing methods.

Unsupervised Domain Adaptation for Image Classification Using Non-Euclidean Triplet Loss

In recent years, computer vision tasks have increasingly used deep learning techniques. In some tasks, however, due to insufficient data, the model is not properly trained, leading to a decrease in generalizability. When trained on a dataset and tested on another similar dataset, the model predicts near-random results. This paper presents an unsupervised multi-source domain adaptation that improves transfer learning and increases generalizability. In the proposed method, a new module infers the source of the input data based on its extracted features. By making the features extractor compete against this objective, the learned feature representation generalizes better across the sources. As a result, representations similar to those from different sources are learned. That is, the extracted representation is generic and independent of any particular domain. In the training stage, a non-Euclidean triplet loss function is also utilized. Similar representations for samples belonging to the same class can be learned more effectively using the proposed loss function. We demonstrate how the developed framework may be applied to enhance accuracy and outperform the outcomes of already effective transfer learning methodologies. We demonstrate how the proposed strategy performs particularly well when dealing with various dataset domains or when there are insufficient data.

Domain-specific feature elimination: multi-source domain adaptation for image classification

Domain-specific feature elimination: multi-source domain adaptation for image classification

Universal multi-Source domain adaptation for image classification

Unsupervised domain adaptation (DA) enables intelligent models to learn transferable knowledge from a labeled source domain and adapt to a similar but unlabeled target domain. Studies showed that knowledge could be transferred from one source domain to another unknown target domain, called Universal DA (UDA). However, there is often more than one source domain in the real-world application to be exploited for DA. In this paper, we formally propose a more general domain adaptation setting for image classification, universal multi-source DA (UMDA), where the label sets of multiple source domains can be different, and the label set of the target domain is completely unknown. The main challenge in UMDA is to identify the common label set among each source and target domain and keep the model scalable as the number of source domains increases. In the face of this challenge, we propose a universal multi-source adaptation network (UMAN) to solve the DA problem without increasing the complexity of the model in various UMDA settings. In UMAN, the reliability of each known class belonging to the common label set is estimated via a novel pseudo-margin vector and its weighted form, which helps adversarial training better align the distributions of multiple source domains and target domain. Moreover, the theoretical guarantee for UMAN is also provided. Massive experimental results show that existing UDA and multi-source DA (MDA) methods cannot be directly deployed to UMDA, and the proposed UMAN achieves the state-of-the-art performance in various UMDA settings.

Improve conditional adversarial domain adaptation using self‐training

Domain adaptation for image classification is one of the most fundamental transfer learning tasks and a promising solution to overcome the annotation burden. Existing deep adversarial domain adaptation approaches imply minimax optimization algorithms, matching the global features across domains. However, the information conveyed in unlabelled target samples is not fully exploited. Here, adversarial learning and self-training are unified in an objective function, where the neural network parameters and the pseudo-labels of target samples are jointly optimized. The model's predictions on unlabelled samples are leveraged to pseudo-label target samples. The training procedure consists of two alternating steps. The first one is to train the network, while the second is to generate pseudo-labels, and the loop continues. The proposed method achieves mean accuracy improvements of 2% on Office-31, 0.7% on ImageCLEF-DA, 1.8% on Office-Home, and 1.2% on Digits than the baseline, which is superior to most state-of-the-art approaches.

Graph-structure constraint and Schatten p-norm-based unsupervised domain adaptation for image classification

Unsupervised domain adaptation, which aims to classify a target domain correctly only using a labeled source domain, has achieved promising performance yet remains a challenging problem. Most traditional methods focus on exploiting either geometric or statistical characteristics to reduce domain shifts. To take advantage of both sides, in this paper, we propose a unified framework incorporating both the geometric and statistical characteristics by adopting the non-convex Schatten p-norm and graph Laplacian constraints to preserve global and local structure information and constructing marginal and conditional distribution minimization terms to reduce the distribution shifts. Moreover, a classification error term on the source domain is embedded into the objective function to increase the discriminability. The proposed method has been evaluated on six datasets and the experimental results demonstrate the superiority of the proposed method over several state-of-the-art methods. The MATLAB code of our method will be publicly available at https://github.com/HeyouChang/unsupervised-domain-adaptation .

Multi-source domain adaptation for image classification

In recent years, domain adaptation and transfer learning are known as promising techniques with admirable performance to deal with problems with distribution difference between the training (source domain) and test (target domain) data. In this paper, a novel unsupervised multi-source transductive transfer learning approach, referred to as multi-source domain adaptation for image classification (MDA), is proposed, to transfer knowledge across the selected samples of multiple-source domains and samples of target domain into a shared low-dimensional subspace with maximum decision regions. MDA extends maximum mean discrepancy criteria across multiple-source domains to find an optimal projection subspace and constructs embedded condensed domain-invariant clusters. Furthermore, MDA minimizes empirical risk and maximizes the rate of consistency between manifold and prediction function via learning an optimal classification. Extensive evaluations on two types of visual benchmark datasets under different difficulties illustrate that MDA significantly outperforms other baseline and state-of-the-art methods in both multiple- and single-source tasks. Our source code is available at https://github.com/jtahmores/MDA .

Sample-oriented Domain Adaptation for Image Classification

Sample-oriented Domain Adaptation for Image Classification

Improved Techniques for Adversarial Discriminative Domain Adaptation.

Adversarial discriminative domain adaptation (ADDA) is an efficient framework for unsupervised domain adaptation in image classification, where the source and target domains are assumed to have the same classes, but no labels are available for the target domain. While ADDA has already achieved better training efficiency and competitive accuracy on image classification in comparison to other adversarial based methods, we investigate whether we can improve its performance with a new framework and new loss formulations. Following the framework of semi-supervised GANs, we first extend the discriminator output over the source classes, in order to model the joint distribution over domain and task. We thus leverage on the distribution over the source encoder posteriors (which is fixed during adversarial training) and propose maximum mean discrepancy (MMD) and reconstruction-based loss functions for aligning the target encoder distribution to the source domain. We compare and provide a comprehensive analysis of how our framework and loss formulations extend over simple multi-class extensions of ADDA and other discriminative variants of semi-supervised GANs. In addition, we introduce various forms of regularization for stabilizing training, including treating the discriminator as a denoising autoencoder and regularizing the target encoder with source examples to reduce overfitting under a contraction mapping (i.e., when the target per-class distributions are contracting during alignment with the source). Finally, we validate our framework on standard datasets like MNIST, USPS, SVHN, MNIST-M and Office-31. We additionally examine how the proposed framework benefits recognition problems based on sensing modalities that lack training data. This is realized by introducing and evaluating on a neuromorphic vision sensing (NVS) sign language recognition dataset, where the source domain constitutes emulated neuromorphic spike events converted from conventional pixel-based video and the target domain is experimental (real) spike events from an NVS camera. Our results on all datasets show that our proposal is both simple and efficient, as it competes or outperforms the state-of-the-art in unsupervised domain adaptation, such as DIFA and MCDDA, whilst offering lower complexity than other recent adversarial methods.

Triplet Loss Network for Unsupervised Domain Adaptation

Domain adaptation is a sub-field of transfer learning that aims at bridging the dissimilarity gap between different domains by transferring and re-using the knowledge obtained in the source domain to the target domain. Many methods have been proposed to resolve this problem, using techniques such as generative adversarial networks (GAN), but the complexity of such methods makes it hard to use them in different problems, as fine-tuning such networks is usually a time-consuming task. In this paper, we propose a method for unsupervised domain adaptation that is both simple and effective. Our model (referred to as TripNet) harnesses the idea of a discriminator and Linear Discriminant Analysis (LDA) to push the encoder to generate domain-invariant features that are category-informative. At the same time, pseudo-labelling is used for the target data to train the classifier and to bring the same classes from both domains together. We evaluate TripNet against several existing, state-of-the-art methods on three image classification tasks: Digit classification (MNIST, SVHN, and USPC datasets), object recognition (Office31 dataset), and traffic sign recognition (GTSRB and Synthetic Signs datasets). Our experimental results demonstrate that (i) TripNet beats almost all existing methods (having a similar simple model like it) on all of these tasks; and (ii) for models that are significantly more complex (or hard to train) than TripNet, it even beats their performance in some cases. Hence, the results confirm the effectiveness of using TripNet for unsupervised domain adaptation in image classification.