Source Domain Instances Research Articles

Kernel Extreme Learning Machine with Discriminative Transfer Feature and Instance Selection for Unsupervised Domain Adaptation

The goal of domain adaptation (DA) is to develop a robust decision model on the source domain effectively generalize to the target domain data. State-of-the-art domain adaptation methods typically focus on finding an optimal inter-domain invariant feature representation or helpful instances from the source domain. In this paper, we propose a kernel extreme learning machine with discriminative transfer features and instance selection (KELM-DTF-IS) for unsupervised domain adaptation tasks, which consists of two steps: discriminative transfer feature extraction and classification with instance selection. At the feature extraction stage, we extend cross domain mean approximation(CDMA) by incorporating a penalty term and develop discriminative cross domain mean approximation (d-CDMA) to optimize the category separability between instances. Subsequently, d-CDMA is integrated into kernel ELM-AutoEncoder(KELM-AE) for extracting inter-domain invariant features. During the classification process, our approach uses CDMA metrics to compute a weights to each source instances based on their impact in reducing distribution differences between domains. Instances with a greater effect receive higher weights and vice versa. These weights are then used to distinguish and select source domain instances before incorporating them into weight KELM for proposing an adaptive classifier. Finally, we apply our approach to conduct classification experiments on publicly available domain adaptation datasets, and the results demonstrate its superiority over KELM and numerous other domain adaptation approaches.

MMT: Cross Domain Few-Shot Learning via Meta-Memory Transfer.

Few-shot learning aims to recognize novel categories solely relying on a few labeled samples, with existing few-shot methods primarily focusing on the categories sampled from the same distribution. Nevertheless, this assumption cannot always be ensured, and the actual domain shift problem significantly reduces the performance of few-shot learning. To remedy this problem, we investigate an interesting and challenging cross-domain few-shot learning task, where the training and testing tasks employ different domains. Specifically, we propose a Meta-Memory scheme to bridge the domain gap between source and target domains, leveraging style-memory and content-memory components. The former stores intra-domain style information from source domain instances and provides a richer feature distribution. The latter stores semantic information through exploration of knowledge of different categories. Under the contrastive learning strategy, our model effectively alleviates the cross-domain problem in few-shot learning. Extensive experiments demonstrate that our proposed method achieves state-of-the-art performance on cross-domain few-shot semantic segmentation tasks on the COCO-20 i, PASCAL-5 i, FSS-1000, and SUIM datasets and positively affects few-shot classification tasks on Meta-Dataset.

Heterogeneous domain adaptation by Features Normalization and Data Topology Preserving

Transfer Learning (TL) algorithms are effective methods for utilizing the source domain knowledge to improve classifier learning in the target domain. These algorithms use labeled source domain instances to reinforce learning in the new target domain. The challenging model of these algorithms, i.e. heterogeneous domain adaptation, is characterized by the source and target domains, which have different features spaces, different data distributions and labels. Two effective factors for improving the performance of TL algorithms are reducing the difference in feature space and distribution between domains. Recently, some TL methods have focused on reducing the difference in distribution and some on reducing the difference in feature space between domains, and a few number of methods have considered the two issues, simultaneously. However, these methods usually use complex computational structures, such as deep neural networks and optimization methods, to adapt the feature space and domain distribution. Another important factor for increasing the efficiency of TL algorithms is preserving topology of data during transfer between domains that have not been studied in the existing TL algorithms. Simultaneous use of these three factors in a single framework improve the performance of TL algorithms. To this end, this paper proposes a novel method for solving heterogeneous domain adaptation problems based on Feature Normalization and Data Topology Preserving (FN-DTP). FN-DTP employs the feature normalization technique in the source and target domains. Hence, the feature spaces of the source and target domains closes together and reduces the difference in the distribution of domain data, while simultaneously preserve topology of data during transfer between domains. This method, without using complex computational structures, employs the mentioned three factors to improve the performance of TL algorithms in a unified framework. Experimental evaluation by using the Office-Caltech and PIE benchmark datasets demonstrates the effectiveness and efficiency of the proposed method compared with the state-of-the-art method in improving learning in semi-supervised classifications.

Unsupervised Domain Adaptive 3-D Detection With Data Adaption From LiDAR Point Cloud

Existing unsupervised domain adaptive (UDA) 3D detection methods only address the domain gap caused by the prior size of 3D bounding boxes between different datasets, which ignore the difference in the distribution of point clouds. To address this challenge, we propose an unsupervised domain adaptive 3D detection by data adaption, which trains the model by transferring the source domain instances into the target domain scenes by adaptive point distribution. First, an instance transferring method is proposed for selecting and transferring suitable instances from the source domain into the target domain scene; Second, we propose an adaptive downsampling method to adjust the point cloud distribution of the transferred instances to approximate the points distribution of the target domain. Finally, our method trains the randomly initialized detector with the pseudo-instances in the target domain. To the best of our knowledge, we first address the UDA problem of the 3D detectors from the perspective of data. Extensive experiments on several popular datasets show that the proposed method outperforms the existing state-of-the-art methods by a large margin. Further experiments also show our approach is detector-agnostic and achieves consistent and significant gains on all types of 3D detectors.

Unsupervised Domain Adaptation Method Based on Discriminant Sample Selection

In order to solve the problem that low classification accuracy caused by the different distribution of training set and test set, an unsupervised domain adaptation method based on discriminant sample selection (DSS) is proposed. DSS projects the samples of different domains onto a same subspace to reduce the distribution discrepancy between the source domain and the target domain, and weights the source domain instances to make the samples more discriminant. Different from the previous method based on the probability density estimation of samples, DSS tries to obtain the sample weights by solving a quadratic programming problem, which avoids the distribution estimation of samples and can be applied to any fields without suffering from the dimensional trouble caused by high-dimensional density estimation. Finally, DSS congregates the same classes by minimizing the intra-class distance. Experimental results show that the proposed method improves the classification accuracy and robustness.

Few-Shot Text and Image Classification via Analogical Transfer Learning

Learning from very few samples is a challenge for machine learning tasks, such as text and image classification. Performance of such task can be enhanced via transfer of helpful knowledge from related domains, which is referred to as transfer learning. In previous transfer learning works, instance transfer learning algorithms mostly focus on selecting the source domain instances similar to the target domain instances for transfer. However, the selected instances usually do not directly contribute to the learning performance in the target domain. Hypothesis transfer learning algorithms focus on the model/parameter level transfer. They treat the source hypotheses as well-trained and transfer their knowledge in terms of parameters to learn the target hypothesis. Such algorithms directly optimize the target hypothesis by the observable performance improvements. However, they fail to consider the problem that instances that contribute to the source hypotheses may be harmful for the target hypothesis, as instance transfer learning analyzed. To relieve the aforementioned problems, we propose a novel transfer learning algorithm, which follows an analogical strategy. Particularly, the proposed algorithm first learns a revised source hypothesis with only instances contributing to the target hypothesis. Then, the proposed algorithm transfers both the revised source hypothesis and the target hypothesis (only trained with a few samples) to learn an analogical hypothesis. We denote our algorithm as Analogical Transfer Learning. Extensive experiments on one synthetic dataset and three real-world benchmark datasets demonstrate the superior performance of the proposed algorithm.

Unsupervised Domain Adaptation Using Exemplar-SVMs with Adaptation Regularization

Domain adaptation has recently attracted attention for visual recognition. It assumes that source and target domain data are drawn from the same feature space but different margin distributions and its motivation is to utilize the source domain instances to assist in training a robust classifier for target domain tasks. Previous studies always focus on reducing the distribution mismatch across domains. However, in many real-world applications, there also exist problems of sample selection bias among instances in a domain; this would reduce the generalization performance of learners. To address this issue, we propose a novel model named Domain Adaptation Exemplar Support Vector Machines (DAESVMs) based on exemplar support vector machines (exemplar-SVMs). Our approach aims to address the problems of sample selection bias and domain adaptation simultaneously. Comparing with usual domain adaptation problems, we go a step further in slacking the assumption of i.i.d. First, we formulate the DAESVMs training classifiers with reducing Maximum Mean Discrepancy (MMD) among domains by mapping data into a latent space and preserving properties of original data, and then, we integrate classifiers to make a prediction for target domain instances. Our experiments were conducted on Office and Caltech10 datasets and verify the effectiveness of the model we proposed.

Minimising disparity in distribution for unsupervised domain adaptation by preserving the local spatial arrangement of data

Domain adaptation is used for machine learning tasks, when the distribution of the training (obtained from source domain) set differs from that of the testing (referred as target domain) set. In the work presented in this study, the problem of unsupervised domain adaptation is solved using a novel optimisation function to minimise the global and local discrepancies between the transformed source and the target domains. The dissimilarity in data distributions is the major contributor to the global discrepancy between the two domains. The authors propose two techniques to preserve the local structural information of source domain: (i) identify closest pair of instances in source domain and minimise the distances between these pairs of instances after transformation; (ii) preserve the naturally occurring clusters present in source domain during transformation. This cost function and constraints yield a non-linear optimisation problem, used to estimate the weight matrix. An iterative framework solves the optimisation problem, providing a sub-optimal solution. Next, using orthogonality constraint, an optimisation task is formulated in the Stiefel manifold. Performance analysis using real-world datasets show that the proposed methods perform better than a few recently published state-of-the-art methods.