Cross-domain Image Classification Research Articles

A Multi-domain and Multi-modal Representation Disentangler for Cross-Domain Image Manipulation and Classification.

Learning interpretable data representation has been an active research topic in deep learning and computer vision. While representation disentanglement is an effective technique for addressing this task, existing works cannot easily handle the problems in which manipulating and recognizing data across multiple domains are desirable. In this paper, we present a unified network architecture of Multi-domain and Multi-modal Representation Disentangler (M2RD), with the goal of learning domain-invariant content representation with the associated domain-specific representation observed. By advancing adversarial learning and disentanglement techniques, the proposed model is able to perform continuous image manipulation across data domains with multiple modalities. More importantly, the resulting domain-invariant feature representation can be applied for unsupervised domain adaptation. Finally, our quantitative and qualitative results would confirm the effectiveness and robustness of the proposed model over state-of-the-art methods on the above tasks.

Multi-representation adaptation network for cross-domain image classification

In image classification, it is often expensive and time-consuming to acquire sufficient labels. To solve this problem, domain adaptation often provides an attractive option given a large amount of labeled data from a similar nature but different domains. Existing approaches mainly align the distributions of representations extracted by a single structure and the representations may only contain partial information, e.g., only contain part of the saturation, brightness, and hue information. Along this line, we propose Multi-Representation Adaptation which can dramatically improve the classification accuracy for cross-domain image classification and specially aims to align the distributions of multiple representations extracted by a hybrid structure named Inception Adaptation Module (IAM). Based on this, we present Multi-Representation Adaptation Network (MRAN) to accomplish the cross-domain image classification task via multi-representation alignment which can capture the information from different aspects. In addition, we extend Maximum Mean Discrepancy (MMD) to compute the adaptation loss. Our approach can be easily implemented by extending most feed-forward models with IAM, and the network can be trained efficiently via back-propagation. Experiments conducted on three benchmark image datasets demonstrate the effectiveness of MRAN.

Robotic pick-and-place of novel objects in clutter with multi-affordance grasping and cross-domain image matching

This article presents a robotic pick-and-place system that is capable of grasping and recognizing both known and novel objects in cluttered environments. The key new feature of the system is that it handles a wide range of object categories without needing any task-specific training data for novel objects. To achieve this, it first uses an object-agnostic grasping framework to map from visual observations to actions: inferring dense pixel-wise probability maps of the affordances for four different grasping primitive actions. It then executes the action with the highest affordance and recognizes picked objects with a cross-domain image classification framework that matches observed images to product images. Since product images are readily available for a wide range of objects (e.g., from the web), the system works out-of-the-box for novel objects without requiring any additional data collection or re-training. Exhaustive experimental results demonstrate that our multi-affordance grasping achieves high success rates for a wide variety of objects in clutter, and our recognition algorithm achieves high accuracy for both known and novel grasped objects. The approach was part of the MIT–Princeton Team system that took first place in the stowing task at the 2017 Amazon Robotics Challenge. All code, datasets, and pre-trained models are available online at http://arc.cs.princeton.edu/

Learning to Generalize: Meta-Learning for Domain Generalization

Domain shift refers to the well known problem that a model trained in one source domain performs poorly when appliedto a target domain with different statistics. Domain Generalization (DG) techniques attempt to alleviate this issue by producing models which by design generalize well to novel testing domains. We propose a novel meta-learning method for domain generalization. Rather than designing a specific model that is robust to domain shift as in most previous DG work, we propose a model agnostic training procedure for DG. Our algorithm simulates train/test domain shift during training by synthesizing virtual testing domains within each mini-batch. The meta-optimization objective requires that steps to improve training domain performance should also improve testing domain performance. This meta-learning procedure trains models with good generalization ability to novel domains. We evaluate our method and achieve state of the art results on a recent cross-domain image classification benchmark, as well demonstrating its potential on two classic reinforcement learning tasks.

Robust domain adaptation image classification via sparse and low rank representation

Domain adaptation image classification addresses the problem of adapting the image distribution of the source domain to the target domain for an effective learning task, where the classification objective is intended but the data distributions are different. However, corrupted data (e.g. noise and outliers, which exist universally in real-world domains) can cause significant deterioration of the practical performance of existing methods in cross-domain image classification. This motivates us to propose a robust domain adaptation image classification method with sparse and low rank representation. Specifically, we first obtain an optimal Domain Adaptation Sparse and Low Rank Representation (DASLRR) for all the data from both domains by incorporating a distribution adaptation regularization term, which is expected to minimize the distribution discrepancy between the source and target domain, into the existing low rank and sparse representation objective function. Formulating an optimization problem that combines the objective function of the sparse and low rank representation, constrained by distribution adaptation and local consistency, we propose an algorithm that alternates between obtaining an effective dictionary, while preserving the DASLRR to make the new representations robust to the distribution difference. Based on the obtained DASLRR, we then provide a flexible semi-supervised learning framework, which can propagate the labels of labeled data from both domains to unlabeled data from In-Sample as well as Out-of-Sample datasets by simultaneously learning a prediction label matrix and a classifier model. The proposed method can capture the global mixture of the clustering structure (by the sparseness and low rankness) and the locally consistent structure (by the local graph regularization) as well as the distribution difference (by the distribution adaptation) of the domains data. Hence, the proposed method is robust for accurately classifying cross-domain images that may be corrupted by noise or outliers. Extensive experiments demonstrate the effectiveness of our method on several types of images and video datasets.

Supervised transfer kernel sparse coding for image classification

When there are a few labeled images, the classifier trained performs poorly even we use sparse coding technique to process image features. So we utilize other data from related domains as source data to help classification tasks. In this paper, we propose a Supervised Transfer Kernel Sparse Coding (STKSC) algorithm to construct discriminative sparse representations for cross domain image classification tasks. Specifically, we map source and target data into a high dimensional feature space by using kernel trick, hence capturing the nonlinear image features. In order to make the sparse representations robust to the domain mismatch, we incorporate the Maximum Mean Discrepancy (MMD) criterion into the objective function of kernel sparse coding. We also use label information to learn more discriminative sparse representations. Furthermore, we provide a unified framework to learn the dictionary and the discriminative sparse representations, which can be further used for classification. The experiment results validate that our method outperforms many state-of-art methods.

Unsupervised domain adaptation via representation learning and adaptive classifier learning

The existing learning methods usually assume that training data and test data follow the same distribution, while this is not always true. Thus, in many cases the performance of these methods on the test data will be severely degraded. In this paper, we study the problem of unsupervised domain adaptation, where no labeled data in the target domain is available. The proposed method first finds a new representation for both the source and the target domain and then learns a prediction function for the classifier by optimizing an objective function which simultaneously tries to minimize the loss function on the source domain while also maximizes the consistency of manifold (which is based on the nature of data) with the prediction function. We run the proposed method on four types of cross-domain image classification problems and show that the proposed method significantly outperforms several state-of-the-art domain adaptation methods.