A MultiKernel Domain Adaptation Method for Unsupervised Transfer Learning on Cross-Source and Cross-Region Remote Sensing Data Classification

Source-Free Domain Adaptation (SFDA) aims to address the challenge of effectively transferring a source domain model to a target domain when the target domain data is unlabeled and the source domain data is unavailable. Traditional Unsupervised Domain Adaptation (UDA) methods rely on simultaneous access to both source and target domain data. However, in many practical scenarios, such as medical data privacy protection or resource-constrained devices, direct access to source domain data is not feasible. SFDA leverages only a pre-trained source domain model and unlabeled target domain data to update the model, avoiding the direct use of source domain data and thereby meeting privacy and security requirements. This paper provides a systematic classification and review of SFDA research methods, categorizing them into three main types: data-related methods, model-related methods, and loss-related methods. Data-related methods replace missing source data by extracting data or feature augmentation information from pre-trained models; model-related methods reduce domain discrepancies by optimizing feature representations or utilizing information in the feature space; and loss-related methods enhance the model's generalization ability through specific loss functions. This paper aims to offer a clear research roadmap for researchers in the field by systematically classifying and analyzing existing SFDA methods, facilitating the selection of appropriate methods or the development of new strategies to address specific problems.

The focus of Source-free Unsupervised Domain Adaptation (SFUDA) is to effectively transfer a well-trained model from the source domain to an unlabelled target domain. During the target domain adaptation, the source domain data is no longer accessible. Prevalent methodologies attempt to synchronize the data distributions between the source and target domains, utilizing pseudo-labels to impart categorical information, which has made some progress in improving the model’s performance. However, performance impairments persist due to the introduction of learning bias from the source model and the impact of noisy pseudo-labels generated for the target domain. In this research, we reveal that the central cause for feature misalignment during domain transition is the learning bias, which is generated by the discrepancy of information between source and target domain data. The source domain data may contain distinguishable features that do not appear on the target domain, which causes the pre-trained source model to fail to work during domain adaptation. To overcome the information discrepancy, we propose a Prototypical Feature Compensation (PFC) Network. The network extracts representative feature maps of the source domain. Then use them to minimize the discrepancy information in the target domain feature maps. This mechanism facilitates feature alignment across different domains, allowing the model to generate more accurate categorical data through pseudo-labelling. The experimental results and ablation studies demonstrate exceptional performance on three SFUDA datasets and provide evidence of the proposed PFC method’s ability to adjust the feature distribution of both source and target domain data, ensuring their overlap in the latent space.

Domain adaptation (DA) is the process in which labeled training samples available from one domain is used to improve the performance of statistical tasks performed on test samples drawn from a different domain. The domain from which the training samples are obtained is termed as the source domain, and the counterpart consisting of the test samples is termed as the target domain. Few unlabeled training samples are also taken from the target domain in order to approximate its distribution. In this paper, we propose a new method of unsupervised DA, where a set of domain invariant sub-spaces are estimated using the geometrical and statistical properties of the source and target domains. This is a modification of the work done by Gopalan et al. [2], where the geodesic path from the principal components of the source to that of the target is considered in the Grassmann manifold, and the intermediary points are sampled to represent the incremental change in the geometric properties of the data in source and target domains. Instead of the geodesic path, we consider an alternate path of shortest length between the principal components of source and target, with the property that the intermediary sample points on the path form domain invariant sub-spaces using the concept of Maximum Mean Discrepancy (MMD) [3]. Thus we model the change in the geometric properties of data in both the domains sequentially, in a manner such that the distributions of projected data from both the domains always remain similar along the path. The entire formulation is done in the kernel space which makes it more robust to non-linear transformations. Let X and Y be the source and target domains having nX and nY number of instances respectively. If Φ(.) is a universal kernel function, then in kernel space the source and target domains are Φ(X) ∈ RnX×d and Φ(Y ) ∈ RnX×d respectively. Let KXX and KYY be the kernel gram matrices of Φ(X) and Φ(Y ) respectively. Let D = [X ;Y ] denote the combined source and target domain data, and the corresponding data in kernel space is given as Φ(D). The kernel gram matrix formed using D is given by

In industrial scenarios, the source domain (SD) data typically encompasses condition monitoring (CM) data from all machines within a workshop or factory setting, while the target domain (TD) data may only include CM data from one or a small number of machines. The intelligent diagnostic method based on partial domain adaptation (PDA) represents a powerful tool for aligning features between SD and TD data within partial categories. However, existing PDA techniques can only align either the marginal or conditional distributions between SD and TD data within the shared label space, but not both simultaneously. To overcome this limitation, our study introduces a dual structural consistent PDA network. This network leverages the vision transformer as its foundation, ensuring effective extraction of distinguishable features from both SD and TD data. A weight balance mechanism is integrated into the partial adversarial training process, facilitating marginal distribution alignment between SD and TD data within the shared label space. Additionally, a knowledge distillation based approach is employed for conditional distribution alignment across the two structural consistent networks, ensuring consistency in predictions for TD data. The effectiveness of our proposed method is demonstrated through its application on two sets of experimental faulty data, confirming its ability to provide a feature distribution that is not affected by domain changes but is discriminative for different classes when dealing with PDA tasks.

Structure preservation and distribution alignment in discriminative transfer subspace learning

Reducing the effect of sample bias for small data sets with double‐weighted support vector transfer regression

Tabular data is a widely used data form in many fields such as product marketing. In some cases, the domain shift between source and target domain of tabular data may occur with the changing of collection conditions such as time. The extant methods on tabular data mainly consist of neural-network-based methods and tree-based methods. They both meet challenges induced by domain shift on tabular data. First, neural-network-based methods are lack of effective mechanism to extract the features of tabular data and the performance may not be higher than tree-based models. Second, tree-based methods are lack of effective feature representations to model the associations between source domain and target domain. To improve the performance of tree-based methods for domain shift, a novel pseudo-label based domain adaptation method is proposed for the tree-based method called Xgboost. The proposed method consists of pseudo-label generation and selection strategies. The pseudo-label generation strategy can control the effects of pseudo-labels on Xgboost in a more flexible way by setting proper values of pseudo-labels. The pseudo-label selection strategy can select the pseudo-labels with high confidences under a consistency condition based on the outputs of Xgboost. The quality of pseudo-labels for the data in target domain is improved and so does the performance of Xgboost trained by the data in both source domain and target domain. In the experiment, several UCI datasets and 5G terminal datasets are used to show that the proposed methods can effectively improve the performance of Xgboost.

Domain adaptation techniques have been developed to handle data from multiple sources or domains. Most existing domain adaptation models assume that source and target domains are homogeneous, i.e., they have the same feature space. Nevertheless, many real world applications often deal with data from heterogeneous domains that come from completely different feature spaces. In our remote sensing application, data in source domain (from an active spaceborne Lidar sensor CALIOP onboard CALIPSO satellite) contain 25 attributes, while data in target domain (from a passive spectroradiometer sensor VIIRS onboard Suomi-NPP satellite) contain 20 different attributes. CALIOP has better representation capability and sensitivity to aerosol types and cloud phase, while VIIRS has wide swaths and better spatial coverage but has inherent weakness in differentiating atmospheric objects on different vertical levels. To address this mismatch of features across the domains/sensors, we propose a novel end-to-end deep domain adaptation with domain mapping and correlation alignment (DAMA) to align the heterogeneous source and target domains in active and passive satellite remote sensing data. It can learn domain invariant representation from source and target domains by transferring knowledge across these domains, and achieve additional performance improvement by incorporating weak label information into the model (DAMA-WL). Our experiments on a collocated CALIOP and VIIRS dataset show that DAMA and DAMA-WL can achieve higher classification accuracy in predicting cloud types.

In order to solve the core problems of modal heterogeneity, semantic gap and model performance degradation in multi-modal data applications, this paper proposes a framework of Dynamic Domain-aware Multimodal Contrastive Learning (DD MCL), aiming at constructing a general multi-modal representation model and realizing efficient cross-domain migration. The framework is optimized through three stages: pre-training, self-adaptation and joint optimization. In the pre-training stage, Modality Attention Weight Network (MAWN) is introduced to dynamically fuse modal features, and the modes are initially aligned with standard comparison loss. In the adaptive phase, Domain-Aware Contrastive Loss (DACL) is activated for unlabeled target domain data, balancing the contrastive learning intensity between the source and target domains through dynamic weight coefficients to alleviate distribution differences between domains; In the joint optimization stage, a Memory Replay Mechanism (MRM) is introduced to construct a source domain sample "memory bank" to avoid catastrophic forgetting of the model when adapting to the target domain. At the same time, the weighted sum of source domain contrastive loss, target domain contrastive loss, and memory replay loss is used as the overall loss function to optimize the model. In the experiment, MS-COCO is used as the source domain data set, cartoon illustration data set CIBR and medical image report data set MIMIC-CXR are used as the target domain data set, ViT-B/16 and RoBERTa-base are used as encoders, and Recall@1, Recall@5 and Recall@10 of cross-modal retrieval tasks are used as evaluation indexes. The results show that the Recall@1 index of image-to-text (I2T) and text-to-image (T2I) retrieval tasks of DDMCL on CIBR and MIMIC-CXR reaches 45.7 and 43.9, respectively, which is significantly better than the baseline models such as CLIP and ALBEF. Ablation experiments verify that the core components of MAWN, DACL and MRM contribute significantly to the performance, and the model performance is the best when the domain sensing weight β is in the range of 0.6-0.8. The research shows that DDMCL can effectively realize cross-modal semantic fusion and efficient cross-domain migration without target domain labeling, and improve the generalization ability and application efficiency of the model in complex and data-scarce scenarios.

Image semantic segmentation methods based on convolutional neural networks rely on supervised learning with labels, and their performance often drops significantly when applied to unlabelled datasets from different sources. The domain adaptation methods can reduce the inconsistency of feature distribution between the unlabelled target domain data used for testing and the labelled source domain data used for training, thus improve the segmentation performance and have more practical applications. However, in the field of remote sensing image processing, if the spatial resolutions of the source domain and the target domain are different and this problem is not to be solved, the performance of the transferred model will be affected. In this paper, we propose a bidirectional semantic segmentation method based on super-resolution and domain adaption (BSSM-SRDA), which is suitable for the transfer learning task of a semantic segmentation model from a low-resolution source domain data to a high-resolution target domain data. BSSM-SRDA mainly consists of three parts: a shared feature extraction network; a super-resolution image translation module, which incorporates a super-resolution approach to reduce spatial resolution differences and visual style differences of the two domains; a domain-adaptive semantic segmentation module, which combines an adversarial domain adaptation approach to reduce differences at the output level. At the same time, we design a new bidirectional self-supervised learning algorithm for BSSM-SRDA that facilitates mutually beneficial learning of the super-resolution image translation module and the domain-adaptive semantic segmentation module. The experiments demonstrate the superiority of the proposed method over other state-of-the-art methods on two remote-sensing image datasets, with mIoU improvements of 2.5% and 3.2%, respectively. Code: https://github.com/mageliang/BSSM-SRDA.git

The unsupervised domain adaptive classification task can learn domain-invariant features between the unlabeled target domain data and the labeled source domain data, thereby improving the classification performance of the classifier in the target domain. However, privacy protection and memory-constrained often make it difficult to obtain labeled source domain samples, which will bring bottlenecks to the existing domain adaptation tasks. To this end, we propose a novel source free domain adaptive classification model, that is, without any source domain data, a classifier with good performance in the target domain can be obtained only by using the source domain pre-trained classifier and the target domain data. The method first proposes a novel conditional information generative adversarial module based on combined discriminators. Through the confrontation between combined discriminators and the generator, the middle domain with pseudo-labels is generated to solve the problem of missing source domain. Then when training the new classifier in the domain adaptation module, we add a distillation loss mechanism to deal with the lack of source domain data supervision, thereby minimizing the difference between the old classifier response and the new classifier response to ensure that the network output retains the source domain information. We conducted experiments on three groups of 10 data sets, which proved that our method can effectively solve the problem of source free domain adaptive classification and effectively improve the classification accuracy of the model in each domain.

Domain adaptation (DA) tackles the issue of distribution shift by learning a model from a source domain that generalizes to a target domain. However, most existing DA methods are designed for scenarios where the source and target domain data lie within the same feature space, which limits their applicability in real-world situations. Recently, heterogeneous DA (HeDA) methods have been introduced to address the challenges posed by heterogeneous feature space between source and target domains. Despite their successes, current HeDA techniques fall short when there is a mismatch in both feature and label spaces. To address this, this paper explores a new DA scenario called open-set HeDA (OSHeDA). In OSHeDA, the model must not only handle heterogeneity in feature space but also identify samples belonging to novel classes. To tackle this challenge, we first develop a novel theoretical framework that constructs learning bounds for prediction error on target domain. Guided by this framework, we propose a new DA method called Representation Learning for OSHeDA (RL-OSHeDA). This method is designed to simultaneously transfer knowledge between heterogeneous data sources and identify novel classes. Experiments across text, image, and clinical data demonstrate the effectiveness of our algorithm. Model implementation is available at https://github.com/pth1993/OSHeDA.

The variousness as well as the inaccessibility of defect characteristics make the defect detection of monocrystalline silicon solar cells more challenging. To address these problems, a novel domain adaptive target detection algorithm based on pseudo-label learning, which is an efficient and feasible weakly supervised learning method, is proposed in this paper. Firstly, in the early stage of the model, the loss function is improved to solve the problem that the model is not easy to converge due to a large number of false labels in the pseudo labels. Then, to classify the sample space better, entropy regularization is applied to the sample boundary data, and the unlabels of the target domain images are labeled with pseudo labels. Finally, the source domain data and the target domain data are used for training together, and the generalization performance of the model obtained is greatly improved. The results show that in solar cell image detection, the accuracy of the domain adaptive method based on pseudo-label learning can reach over 90%, which is better than the target detection accuracy of using only the source domain dataset.

Domain adaptation (DA) has been widely investigated as a framework to alleviate the laborious task of data annotation for image segmentation. Most DA investigations operate under the unsupervised domain adaptation (UDA) setting, where the modeler has access to a large cohort of source domain labeled data and target domain data with no annotations. UDA techniques exhibit poor performance when the domain gap, i.e., the distribution overlap between the data in source and target domain is large. We hypothesize that the DA performance gap can be improved with the availability of a small subset of labeled target domain data. In this paper, we systematically investigate the impact of varying amounts of labeled target domain data on the performance gap for DA. We specifically focus on the problem of segmenting eye-regions from eye images collected using two different head mounted display systems. Source domain is comprised of 12,759 eye images with annotations and target domain is comprised of 4,629 images with varying amounts of annotations. Experiments are performed to compare the impact on DA performance gap under three schemes: unsupervised (UDA), supervised (SDA) and semi-supervised (SSDA) domain adaptation. We evaluate these schemes by measuring the mean intersection-over-union (mIoU) metric. Using only 200 samples of labeled target data under SDA and SSDA schemes, we show an improvement in mIoU of 5.4% and 6.6% respectively, over mIoU of 81.7% under UDA. By using all available labeled target data, models trained under SSDA achieve a competitive mIoU score of 89.8%. Overall, we conclude that availability of a small subset of target domain data with annotations can substantially improve DA performance.

Recent domain adaptation work tends to obtain a uniformed representation in an adversarial manner through joint learning of the domain discriminator and feature generator. However, this domain adversarial approach could render sub-optimal performances due to two potential reasons: First, it might fail to consider the task at hand when matching the distributions between the domains. Second, it generally treats the source and target domain data in the same way. In our opinion, the source domain data which serves the feature adaption purpose should be supplementary, whereas the target domain data mainly needs to consider the task-specific classifier. Motivated by this, we propose a dual adversarial network for domain adaptation, where two adversarial learning processes are conducted iteratively, in correspondence with the feature adaptation and the classification task respectively. The efficacy of the proposed method is first demonstrated on Visual Domain Adaptation Challenge (VisDA) 2017 challenge, and then on two newly proposed Ground/Satellite-to-Aerial Scene adaptation tasks. For the proposed tasks, the data for the same scene is collected not only by the traditional camera on the ground, but also by satellite from the out space and unmanned aerial vehicle (UAV) at the high-altitude. Since the semantic gap between the ground/satellite scene and the aerial scene is much larger than that between ground scenes, the newly proposed tasks are more challenging than traditional domain adaptation tasks. The datasets/codes can be found at https://github.com/jianzhelin/DuAN.