Heterogeneous Domain Adaptation Research Articles

Weighted Heterogeneous Domain Adaptation for GIS Insulation Fault Diagnosis Based on SF6 Decomposition Products

Weighted Heterogeneous Domain Adaptation for GIS Insulation Fault Diagnosis Based on SF₆ Decomposition Products

Heterogeneous domain adaptation via incremental discriminative knowledge consistency

Heterogeneous domain adaptation is a challenging problem in transfer learning since samples from the source and target domains reside in different feature spaces with different feature dimensions. The key problem is how to minimize some gaps (e.g., data distribution mismatch) presented in two heterogeneous domains and produce highly discriminative representations for the target domain. In this paper, we attempt to address these challenges with the proposed incremental discriminative knowledge consistency (IDKC) method, which integrates cross-domain mapping, distribution matching, discriminative knowledge preservation, and domain-specific geometry structure consistency into a unified learning model. Specifically, we attempt to learn a domain-specific projection to project original samples into a common subspace in which the marginal distribution is well aligned and the discriminative knowledge consistency is preserved by leveraging the labeled samples from both domains. Moreover, domain-specific structure consistency is enforced to preserve the data manifold from the original space to the common feature space in each domain. Meanwhile, we further apply pseudo labeling to unlabeled target samples based on the feature correlation and retain pseudo labels with high feature correlation coefficients for the next iterative learning. Our pseudo-labeling strategy expands the number of labeled target samples in each category and thus enforces class-discriminative knowledge consistency to produce more discriminative feature representations for the target domain. Extensive experiments on several standard benchmarks for object recognition, cross-language text classification, and digit classification tasks verify the effectiveness of our method.

Unsupervised heterogeneous domain adaptation for EEG classification

Objective. Domain adaptation has been recognized as a potent solution to the challenge of limited training data for electroencephalography (EEG) classification tasks. Existing studies primarily focus on homogeneous environments, however, the heterogeneous properties of EEG data arising from device diversity cannot be overlooked. This motivates the development of heterogeneous domain adaptation methods that can fully exploit the knowledge from an auxiliary heterogeneous domain for EEG classification. Approach. In this article, we propose a novel model named informative representation fusion (IRF) to tackle the problem of unsupervised heterogeneous domain adaptation in the context of EEG data. In IRF, we consider different perspectives of data, i.e. independent identically distributed (iid) and non-iid, to learn different representations. Specifically, from the non-iid perspective, IRF models high-order correlations among data by hypergraphs and develops hypergraph encoders to obtain data representations of each domain. From the non-iid perspective, by applying multi-layer perceptron networks to the source and target domain data, we achieve another type of representation for both domains. Subsequently, an attention mechanism is used to fuse these two types of representations to yield informative features. To learn transferable representations, the maximum mean discrepancy is utilized to align the distributions of the source and target domains based on the fused features. Main results. Experimental results on several real-world datasets demonstrate the effectiveness of the proposed model. Significance. This article handles an EEG classification situation where the source and target EEG data lie in different spaces, and what’s more, under an unsupervised learning setting. This situation is practical in the real world but barely studied in the literature. The proposed model achieves high classification accuracy, and this study is important for the commercial applications of EEG-based BCIs.

Partial label learning with heterogeneous domain adaptation

Partial label learning (PLL) seeks a classification model with partially labeled (PL) instances. Each PL instance is attached with a candidate label set, with only one being ground-truth. A fundamental assumption of the previous PLL works is that there are sufficient PL instances in the learning process. Nevertheless, this assumption may not always be valid. In real-world scenarios, there may be only a few PL instances available for training. To this end, we introduce a new PLL method with heterogeneous domain adaptation (PLL-HDA). PLL-HDA leverages the knowledge from the source domain to induce a PLL classifier of the target domain, which contains only a few PL instances for training. Firstly, PLL-HDA introduces a common subspace to measure the instances from the source and target domains, where the instances from these domains are described by heterogeneous features of distinct dimensions. Secondly, we augment the features of the common subspace by merging the original features from both domains. Thirdly, a large-margin-based PLL learning system is established on the augmented features. It leads to a simplified dual form, which can be directly resolved by off-the-shelf support vector machine solvers. Lastly, a heuristic framework is proposed to resolve the PLL-HDA classification model. In this framework, the tasks of learning the classifier on the augmented features and identifying the ground-truth labels of target domain are conducted alternately. Extensive experiments on both the controlled PLL and real-world PLL datasets illustrate the superiority of PLL-HDA over the existing PLL methods. Our code is available at: https://github.com/Sux86.

A Co-Training Framework for Heterogeneous Heuristic Domain Adaptation.

The purpose of this article is to address unsupervised domain adaptation (UDA) where a labeled source domain and an unlabeled target domain are given. Recent advanced UDA methods attempt to remove domain-specific properties by separating domain-specific information from domain-invariant representations, which heavily rely on the designed neural network structures. Meanwhile, they do not consider class discriminate representations when learning domain-invariant representations. To this end, this article proposes a co-training framework for heterogeneous heuristic domain adaptation (CO-HHDA) to address the above issues. First, a heterogeneous heuristic network is introduced to model domain-specific characters. It allows structures of heuristic network to be different between domains to avoid underfitting or overfitting. Specially, we initialize a small structure that is shared between domains and increase a subnetwork for the domain which preserves rich specific information. Second, we propose a co-training scheme to train two classifiers, a source classifier and a target classifier, to enhance class discriminate representations. The two classifiers are designed based on domain-invariant representations, where the source classifier learns from the labeled source data, and the target classifier is trained from the generated target pseudolabeled data. The two classifiers teach each other in the training process with high-quality pseudolabeled data. Meanwhile, an adaptive threshold is presented to select reliable pseudolabels in each classifier. Empirical results on three commonly used benchmark datasets demonstrate that the proposed CO-HHDA outperforms the state-of-the-art domain adaptation methods.

Semi-supervised heterogeneous domain adaptation for few-sample credit risk classification

Credit risk classification is a crucial area in machine learning-enhanced financial decision support systems, and numerous studies have achieved significant progress. However, data in the modern financial landscape is inherently complex, characterized by a lack of labeled data, cross-domain heterogeneity, and class imbalance. These three major problems hinder the achievement of credit risk classification. Therefore, we propose a semi-supervised heterogeneous domain adaptation method and an imbalanced data augmentation method to overcome these challenges with a single neural network-based model. Experimental results obtained from four representative benchmark datasets confirm the superior performance of the proposed method.

Heterogeneous domain adaptation by class centroid matching and local discriminative structure preservation

Heterogeneous domain adaptation by class centroid matching and local discriminative structure preservation

Robust multiple subspaces transfer for heterogeneous domain adaptation

Heterogeneous domain adaptation (HDA) aims to execute knowledge transfer from a source domain to a heterogeneous target domain. Previous works typically inject knowledge from the source and target domain into a common subspace. However, this may lead to the ineffectiveness of knowledge transfer due to the existence of heterogeneity. To overcome this drawback, in this paper, we propose a robust multiple subspaces transfer method for heterogeneous domain adaptation. Specifically, knowledge of two domains is projected into a union of multiple subspaces via a self-expressive model, in which joint distribution alignment and dynamic Laplacian regularization on self-repressive coefficients are included in the loss for characterizing transferability. Moreover, we provide a comprehensive analysis of stability, complexity, generalization, and convergence guarantee for the proposed method. Experiments on benchmark vision and Language datasets verify effectiveness of the proposed approach for heterogeneous domain adaptation.

Optimal heterogeneous domain adaptation for text classification in transfer learning

The prime challenge of unsupervised symmetric heterogeneous cross-domain adaptation is to train the source domain and apply the trained knowledge to the target domain. Most of the existing algorithms for unsupervised transfer learning create the subspace of the source domain features and target domain features for training purposes. It is an extensive computational process as most of the techniques require labeled source data. Many techniques also suffer from the loss of originality of features in both domains. This paper aims to consider the feature vectors of both the source and target domain for training the data based on the similarity of exemplar (feature) vectors of different instances, known as Instance Similarity Feature (ISF). The use of vectorization method for the similarity of features is proposed in this paper. The exemplar vectors are chosen randomly for the target datasets. Hence, to acquire relevant factual data in the knowledge base for training in our research, we worked to increase the domain separation error between source and target instances. To avoid the instability caused due to poor exemplar vector selection, the K-means clustering approach is followed after feature similarity, known as K-means Instance Similarity Feature (KISF). Many existing transfer learning techniques are based on the original feature set, which can cause degeneracy, hence affecting Accuracy. In order to vanquish the limitations of existing approaches, we have introduced novel optimal models with KISF and Ant Lion Optimizer (KISFA), KISF with Particle Swarm Optimization (KISFP) and KISF with Biogeography Based Optimization (KISFB). High-dimensionality can impact efficacy of the model, hence, feature selection with nature-based optimizer namely: Ant Lion Optimizer, Particle Swarm Optimization and Biogeography-Based Optimization are applied. We measure the performance of the proposed models by using Support Vector Machine, Logistic Regression, Random Forest, Naive Baye’s, K-Nearest Neighbor and Decision Tree as classifiers, and Accuracy and F1-score as fitness functions. Extensive experiments are performed on four datasets with 50 iterations. The proposed model is compared with eleven other techniques and our technique outperforms all other techniques in average Accuracy. The validation is performed on the dataset using 10-fold cross-validation. The statistical test was performed using ANOVA, proving that our technique is significantly better than other techniques.

Multitask Particle Swarm Optimization With Heterogeneous Domain Adaptation

Multitask Particle Swarm Optimization With Heterogeneous Domain Adaptation

Heterogeneous Domain Adaptation Framework for Logging Lithofacies Identification

Summary Reservoir lithofacies type is an important indicator of reservoir quality and oiliness, and understanding lithofacies type can help geologists and engineers make informed decisions about exploration and development activities. The use of well-log data to establish machine learning models for lithofacies identification has gained popularity; however, the assumption that data are independent identical distribution followed by these models is often unrealistic. Additionally, there is a possible incompatibility between the training and test data in terms of feature space dimensions. We propose the heterogeneous domain adaptation framework for logging lithofacies identification (HDAFLI) to address these problems. The framework comprises three main contributions: (i) The denoising autoencoder feature mapping (DAFM) module is adopted to resolve the incompatibility issue in feature space between training and test data. The connection between training and test data can be effectively established to improve the performance and generalization ability. (ii) The transferability and discriminative joint probability distribution adaptive (TDJPDA) module addresses the issue of data distribution differences. It improves the transferability of training and test data by minimizing the maximum mean difference (MMD) of the joint probabilities of the source and target domains and enhances their discriminative ability by maximizing the joint probability MMD of different lithofacies categories. (iii) Bayesian optimization is used to optimize hyperparameters in the light gradient boosting machine (LightGBM) model for high computational efficiency in determining the best accuracy. We selected well-logging data from eight wells in the Pearl River Mouth Basin of the South China Sea to design four tasks and compared HDAFLI with various baseline machine learning algorithms and baseline domain adaptive algorithms. The results show that HDAFLI has the highest average accuracy among the four tasks. It is 19.76% and 8.94% higher than the best-performing baseline machine learning algorithm and baseline domain adaptive method among the comparison algorithms, respectively. For HDAFLI, we also conducted ablation experiments, time cost and convergence performance analysis, parameter sensitivity experiments, and feature visualization experiments. The results of ablation experiments show that the three modules of HDAFLI all play an active role, working together to achieve the best results. In addition, HDAFLI has a reasonable time cost, can become stable after several iterations, and has good convergence performance. The results of parameter sensitivity experiments confirm that the accuracy of HDAFLI does not change significantly with changes in hyperparameters, which is robust. The results of feature visualization experiments show that the data of the training set and the test set are concentrated together to a certain extent, which indicates that HDAFLI has completed the task of data distribution alignment very well. The findings of this study can help for a better understanding of how to address the challenge of reservoir lithofacies identification through a heterogeneous domain adaptation framework. By solving the problem of feature space incompatibility and data distribution difference between training data and test data, the application of HDAFLI provides geologists and engineers with more accurate lithofacies classification tools. This study has practical application value for reservoir quality assessment, oiliness prediction, and exploration and development decision-making.

Species-agnostic transfer learning for cross-species transcriptomics data integration without gene orthology.

Novel hypotheses in biomedical research are often developed or validated in model organisms such as mice and zebrafish and thus play a crucial role. However, due to biological differences between species, translating these findings into human applications remains challenging. Moreover, commonly used orthologous gene information is often incomplete and entails a significant information loss during gene-id conversion. To address these issues, we present a novel methodology for species-agnostic transfer learning with heterogeneous domain adaptation. We extended the cross-domain structure-preserving projection toward out-of-sample prediction. Our approach not only allows knowledge integration and translation across various species without relying on gene orthology but also identifies similar GO among the most influential genes composing the latent space for integration. Subsequently, during the alignment of latent spaces, each composed of species-specific genes, it is possible to identify functional annotations of genes missing from public orthology databases. We evaluated our approach with four different single-cell sequencing datasets focusing on cell-type prediction and compared it against related machine-learning approaches. In summary, the developed model outperforms related methods working without prior knowledge when predicting unseen cell types based on other species' data. The results demonstrate that our novel approach allows knowledge transfer beyond species barriers without the dependency on known gene orthology but utilizing the entire gene sets.

Semantic Correlation Transfer for Heterogeneous Domain Adaptation.

Heterogeneous domain adaptation (HDA) is expected to achieve effective knowledge transfer from a label-rich source domain to a heterogeneous target domain with scarce labeled data. Most prior HDA methods strive to align the cross-domain feature distributions by learning domain invariant representations without considering the intrinsic semantic correlations among categories, which inevitably results in the suboptimal adaptation performance across domains. Therefore, to address this issue, we propose a novel semantic correlation transfer (SCT) method for HDA, which not only matches the marginal and conditional distributions between domains to mitigate the large domain discrepancy, but also transfers the category correlation knowledge underlying the source domain to target by maximizing the pairwise class similarity across source and target. Technically, the domainwise and classwise centroids (prototypes) are first computed and aligned according to the feature embeddings. Then, based on the derived classwise prototypes, we leverage the cosine similarity of each two classes in both domains to transfer the supervised source semantic correlation knowledge among different categories to target effectively. As a result, the feature transferability and category discriminability can be simultaneously improved during the adaptation process. Comprehensive experiments and ablation studies on standard HDA tasks, such as text-to-image, image-to-image, and text-to-text, have demonstrated the superiority of our proposed SCT against several state-of-the-art HDA methods.

Heterogeneous Domain Adaptation With Generalized Similarity and Dissimilarity Regularization.

Heterogeneous domain adaptation (HDA) aims to address the transfer learning problems where the source domain and target domain are represented by heterogeneous features. The existing HDA methods based on matrix factorization have been proven to learn transferable features effectively. However, these methods only preserve the original neighbor structure of samples in each domain and do not use the label information to explore the similarity and separability between samples. This would not eliminate the cross-domain bias of samples and may mix cross-domain samples of different classes in the common subspace, misleading the discriminative feature learning of target samples. To tackle the aforementioned problems, we propose a novel matrix factorization-based HDA method called HDA with generalized similarity and dissimilarity regularization (HGSDR). Specifically, we propose a similarity regularizer by establishing the cross-domain Laplacian graph with label information to explore the similarity between cross-domain samples from the identical class. And we propose a dissimilarity regularizer based on the inner product strategy to expand the separability of cross-domain labeled samples from different classes. For unlabeled target samples, we keep their neighbor relationship to preserve the similarity and separability between them in the original space. Hence, the generalized similarity and dissimilarity regularization is built by integrating the above regularizers to facilitate cross-domain samples to form discriminative class distributions. HGSDR can more efficiently match the distributions of the two domains both from the global and sample viewpoints, thereby learning discriminative features for target samples. Extensive experiments on the benchmark datasets demonstrate the superiority of the proposed method against several state-of-the-art methods.

Trans-Diff: Heterogeneous Domain Adaptation for Remote Sensing Segmentation With Transfer Diffusion

Trans-Diff: Heterogeneous Domain Adaptation for Remote Sensing Segmentation With Transfer Diffusion

Correlation-Guided Distribution and Geometry Alignments for Heterogeneous Domain Adaptation

Correlation-Guided Distribution and Geometry Alignments for Heterogeneous Domain Adaptation

Heterogeneous Domain Adaptation for Multistream Classification on Cyber Threat Data

Under a newly introduced setting of multistream classification, two data streams are involved, which are referred to as source and target streams. The source stream continuously generates data instances from a certain domain with labels, while the target stream does the same task without labels from another domain. Existing approaches assume that domains for both data streams are identical, which is not quite true, since data streams from different sources may contain distinct features. Indeed, they may even have different numbers of features. Furthermore, obtaining labels for every instance in a data stream is often expensive and time-consuming. Therefore, it has become an important topic to explore if classes of labeled instances from other related streams are helpful to predict the classes of unlabeled instances in a different stream. Note that domains of source and target streams may have distinct feature spaces and data distributions. Our objective is to predict class labels of data instances in the target stream by using the classifiers trained by the source stream. We propose a framework of multistream classification by using projected data from a common latent feature space, which is embedded from both source and target domains. This framework is also crucial for enterprise system defenders to detect cross-platform attacks, such as Advanced Persistent Threats (APTs). Empirical valuation and analysis on both real-world and synthetic datasets are performed to validate the effectiveness of our proposed algorithm, comparing to state-of-the-art techniques. Experimental results show that our approach significantly outperforms other existing approaches.

Graph embedding-based heterogeneous domain adaptation with domain-invariant feature learning and distributional order preserving

Heterogeneous domain adaptation (HDA) methods leverage prior knowledge from the source domain to train models for the target domain and address the differences in their feature spaces. However, incorrect alignment of categories and distribution structure disruption may be caused by unlabeled target samples during the domain alignment process for most existing methods, resulting in negative transfer. Additionally, the previous works rarely focus on the robustness and interpretability of the model. To address these issues, we propose a novel Graph embedding-based Heterogeneous domain-Invariant feature learning and Distributional order preserving framework (GHID). Specifically, a bidirectional robust cross-domain alignment graph embedding structure is proposed to globally align two domains, which learns the domain-invariant and discriminative features simultaneously. In addition, the interpretability of the proposed graph structures is demonstrated through two theoretical analyses, which can elucidate the correlation between important samples from a global perspective in heterogeneous domain alignment scenarios. Then, a heterogeneous discriminative distributional order preserving graph embedding structure is designed to preserve the original distribution relationship of each domain to prevent negative transfer. Moreover, the dynamic centroid strategy is incorporated into the graph structures to improve the robustness of the model. Comprehensive experimental results on four benchmarks demonstrate that the proposed method outperforms other state-of-the-art approaches in effectiveness.

Heterogeneous domain adaptation method for tomato leaf disease classification base on CycleGAN

Despite the significant improvements in the detection and diagnosis of plant diseases at an early stage facilitated by deep learning technology, there are challenges associated with the generalization performance of deep learning models. These problems from the differences between in-field and in-lab data, as well as the heterogeneity of training and prediction data features. In the case of tomato leaf diseases, the PlantVillage dataset is widely used and has already demonstrated accuracy of more than 99%. However, using trained model based on this dataset to predict in-field data results in low accuracy due to domain differences and heterogeneous features. In this paper, we propose a domain adaptation method based on CycleGAN to solve this problem, followed by a preprocessing technique that utilizes both the OpenCV module and a segmentation model based on U-Net for the best generalization performance. The classification accuracy is evaluated by applying the DenseNet121 model trained on the PlantVillage dataset to the images generated by CycleGAN. Our results demonstrate, with an F1-score of 95.6%, that our domain adaptation method between the two domains is effective in mitigating the effect of domain shift.

Learning from streaming data with unsupervised heterogeneous domain adaptation

Learning from streaming data with unsupervised heterogeneous domain adaptation