Semi-supervised Domain Adaptation Research Articles

Information filtering and interpolating for semi-supervised graph domain adaptation

Graph domain adaptation, which falls under the umbrella of graph transfer learning, involves transferring knowledge from a labeled source graph to improve prediction accuracy on an unlabeled target graph, where both graphs have identical label spaces but exhibit distribution discrepancies due to temporal data shifts or distinct data collection methods. This adaptation is complicated by the challenges of graph-specific domain discrepancies and cross-graph label scarcity. This paper proposes a semi-supervised Graph domain adaptation method via Information Filtering and Interpolating (GIFI). Specifically, GIFI utilizes a parameterized graph reduction module and a variational information bottleneck to adequately filter out irrelevant information from the source and target graphs to eliminate distribution discrepancy. GIFI also introduces an interpolation-enhanced pseudo-labeling strategy for cross-graph semi-supervised learning, which can mitigate model over-fitting on domain-specific features and limited labeled nodes, thus improving the model’s adaptation and discriminative capability. Experimental results on various graph domain adaptation benchmarks demonstrate GIFI’s superior performance over state-of-the-art methods. Our code is available at https://github.com/joe817/GIFI.

Multicentric intelligent cardiotocography signal interpretation using deep semi-supervised domain adaptation via minimax entropy and domain invariance

Background and ObjectiveObstetricians use Cardiotocography (CTG), which is the continuous recording of fetal heart rate and uterine contraction, to assess fetal health status. Deep learning models for intelligent fetal monitoring trained on extensively labeled and identically distributed CTG records have achieved excellent performance. However, creation of these training sets requires excessive time and specialist labor for the collection and annotation of CTG signals. Previous research has demonstrated that multicenter studies can improve model performance. However, models trained on cross-domain data may not generalize well to target domains due to variance in distribution among datasets. Hence, this paper conducted a multicenter study with Deep Semi-Supervised Domain Adaptation (DSSDA) for intelligent interpretation of antenatal CTG signals. This approach helps to align cross-domain distribution and transfer knowledge from a label-rich source domain to a label-scarce target domain. MethodsWe proposed a DSSDA framework that integrated Minimax Entropy and Domain Invariance (DSSDA-MMEDI) to reduce inter-domain gaps and thus achieve domain invariance. The networks were developed using GoogLeNet to extract features from CTG signals, with fully connected, softmax layers for classification. We designed a Dynamic Gradient-driven strategy based on Mutual Information (DGMI) to unify the losses from Minimax Entropy (MME), Domain Invariance (DI), and supervised cross-entropy during iterative learning. ResultsWe validated our DSSDA model on two datasets collected from collaborating healthcare institutions and mobile terminals as the source and target domains, which contained 16,355 and 3,351 CTG signals, respectively. Compared to the results achieved with deep learning networks without DSSDA, DSSDA-MMEDI significantly improved sensitivity and F1-score by over 6%. DSSDA-MMEDI also outperformed other state-of-the-art DSSDA approaches for CTG signal interpretation. Ablation studies were performed to determine the unique contribution of each component in our DSSDA mechanism. ConclusionsThe proposed DSSDA-MMEDI is feasible and effective for alignment of cross-domain data and automated interpretation of multicentric antenatal CTG signals with minimal annotation cost.

Transferable Time-Series Forecasting Under Causal Conditional Shift.

This paper focuses on the problem of semi-supervised domain adaptation for time-series forecasting, which is underexplored in literature, despite being often encountered in practice. Existing methods on time-series domain adaptation mainly follow the paradigm designed for static data, which cannot handle domain-specific complex conditional dependencies raised by data offset, time lags, and variant data distributions. In order to address these challenges, we analyze variational conditional dependencies in time-series data and find that the causal structures are usually stable among domains, and further raise the causal conditional shift assumption. Enlightened by this assumption, we consider the causal generation process for time-series data and propose an end-to-end model for the semi-supervised domain adaptation problem on time-series forecasting. Our method can not only discover the Granger-Causal structures among cross-domain data but also address the cross-domain time-series forecasting problem with accurate and interpretable predicted results. We further theoretically analyze the superiority of the proposed method, where the generalization error on the target domain is bounded by the empirical risks and by the discrepancy between the causal structures from different domains. Experimental results on both synthetic and real data demonstrate the effectiveness of our method for the semi-supervised domain adaptation method on time-series forecasting.

Multilevel Attention Network with Semi-supervised Domain Adaptation for Drug-Target Prediction

Prediction of drug-target interactions (DTIs) is a crucial step in drug discovery, and deep learning methods have shown great promise on various DTI datasets. However, existing approaches still face several challenges, including limited labeled data, hidden bias issue, and a lack of generalization ability to out-of-domain data. These challenges hinder the model's capacity to learn truly informative interaction features, leading to shortcut learning and inferior predictive performance on novel drug-target pairs. To address these issues, we propose MlanDTI, a semi-supervised domain adaptive multilevel attention network (Mlan) for DTI prediction. We utilize two pre-trained BERT models to acquire bidirectional representations enriched with information from unlabeled data. Then, we introduce a multilevel attention mechanism, enabling the model to learn domain-invariant DTIs at different hierarchical levels. Moreover, we present a simple yet effective semi-supervised pseudo-labeling method to further enhance our model's predictive ability in cross-domain scenarios. Experiments on four datasets show that MlanDTI achieves state-of-the-art performances over other methods under intra-domain settings and outperforms all other approaches under cross-domain settings. The source code is available at https://github.com/CMACH508/MlanDTI.

Enhancing Semi-supervised Domain Adaptation via Effective Target Labeling

Existing semi-supervised domain adaptation (SSDA) models have exhibited impressive performance on the target domain by effectively utilizing few labeled target samples per class (e.g., 3 samples per class). To guarantee an equal number of labeled target samples for each class, however, they require domain experts to manually recognize a considerable amount of the unlabeled target data. Moreover, as the target samples are not equally informative for shaping the decision boundaries of the learning models, it is crucial to select the most informative target samples for labeling, which is, however, impossible for human selectors. As a remedy, we propose an EFfective Target Labeling (EFTL) framework that harnesses active learning and pseudo-labeling strategies to automatically select some informative target samples to annotate. Concretely, we introduce a novel sample query strategy, called non-maximal degree node suppression (NDNS), that iteratively performs maximal degree node query and non-maximal degree node removal to select representative and diverse target samples for labeling. To learn target-specific characteristics, we propose a novel pseudo-labeling strategy that attempts to label low-confidence target samples accurately via clustering consistency (CC), and then inject information of the model uncertainty into our query process. CC enhances the utilization of the annotation budget and increases the number of “labeled” target samples while requiring no additional manual effort. Our proposed EFTL framework can be easily coupled with existing SSDA models, showing significant improvements on three benchmarks

Semi-supervised domain adaptation incorporating three-way decision for multi-view echocardiographic sequence segmentation

Multi-view echocardiographic sequence segmentation is essential for the diagnosis of cardiac diseases in clinical practice. However, the variation in cardiac structures in different views and the lack of manual annotations make it challenging to establish a generalized segmentation model. In this paper, we propose a Bidirectional semi-supervised domain adaptation (BSDA) method based on the three-way decision to learn a generalized segmentation model for different views. Specifically, the two-branch structure of BSDA regards echocardiographic sequence data of different views as different domains. The source-pretrained model first roughly predicted the segmentation results for the target domain. Then, BSDA provides the segmentation model with reliable probabilistic supervision and feature-based pseudo-labels to make secondary decisions. Besides, the proposed stage-wise training strategy can better cope with the varied appearance of the cardiac structures in echocardiographic sequence. We evaluate our BSDA on three publicly available datasets, corroborating the superiority of BSDA to segment cardiac structures of multi-view echocardiographic sequences.

Semi-supervised domain adaptation on graphs with contrastive learning and minimax entropy

Label scarcity in a graph is frequently encountered in real-world applications due to the high cost of data labeling. To this end, semi-supervised domain adaptation (SSDA) on graphs aims to leverage the knowledge of a labeled source graph to aid in node classification on a target graph with limited labels. SSDA tasks need to overcome the domain gap between the source and target graphs. However, to date, this challenging research problem has yet to be formally considered by the existing approaches designed for cross-graph node classification. This paper proposes a novel method called SemiGCL to tackle the graph Semi-supervised domain adaptation with Graph Contrastive Learning and minimax entropy training. SemiGCL generates informative node representations by contrasting the representations learned from a graph’s local and global views. Additionally, SemiGCL is adversarially optimized with the entropy loss of unlabeled target nodes to reduce domain divergence. Experimental results on benchmark datasets demonstrate that SemiGCL outperforms the state-of-the-art baselines on the SSDA tasks.

Unsupervised and Semi-Supervised Robust Spherical Space Domain Adaptation.

Adversarial domain adaptation has been an effective approach for learning domain-invariant features by adversarial training. In this paper, we propose a novel adversarial domain adaptation approach defined in the spherical feature space, in which we define spherical classifier for label prediction and spherical domain discriminator for discriminating domain labels. In the spherical feature space, we develop a spherical robust pseudo-label loss to utilize pseudo-labels robustly, which weights the importance of the estimated labels of target domain data by the posterior probability of correct labeling, modeled by the Gaussian-uniform mixture model in the spherical space. Our proposed approach can be generally applied to both unsupervised and semi-supervised domain adaptation settings. In particular, to tackle the semi-supervised domain adaptation setting where a few labeled target domain data are available for training, we propose a novel reweighted adversarial training strategy for effectively reducing the intra-domain discrepancy within the target domain. We also present theoretical analysis for the proposed method based on the domain adaptation theory. Extensive experiments are conducted on multiple benchmarks for object recognition, digit recognition, and face recognition. The results show that our method either surpasses or is competitive compared with the recent methods for both unsupervised and semi-supervised domain adaptation. Ablation studies also confirm the effectiveness of the spherical classifier, spherical discriminator, spherical robust pseudo-label loss, and reweighted adversarial training strategy.

MGFKD: A semi-supervised multi-source domain adaptation algorithm for cross-subject EEG emotion recognition

Currently, most models rarely consider the negative transfer problem in the research field of cross-subject EEG emotion recognition. To solve this problem, this paper proposes a semi-supervised domain adaptive algorithm based on few labeled samples of target subject, which called multi-domain geodesic flow kernel dynamic distribution alignment (MGFKD). It consists of three modules: 1) GFK common feature extractor: projects the feature distribution of source and target subjects to the Grassmann manifold space, and obtains the latent common features of the two feature distributions through GFK method. 2) Source domain selector: obtains pseudo-labels of the target subject through weak classifier, finds "golden source subjects" by using few known labels of target subjects. 3) Label corrector: uses a dynamic distribution balance strategy to correct the pseudo-labels of the target subject. We conducted comparison experiments on the SEED and SEED-IV datasets, and the results show that MGFKD outperforms unsupervised and semi-supervised domain adaptation algorithms, achieving an average accuracy of 87.51±7.68% and 68.79±8.25% on the SEED and SEED-IV datasets with only one labeled sample per video for target subject. Especially when the number of source domains is set as 6 and the number of known labels is set as 5, the accuracy increase to 90.20±7.57% and 69.99±7.38%, respectively. The above results prove that our proposed algorithm can efficiently and quickly improve the cross-subject EEG emotion classification performance. Since it only need a small number of labeled samples of new subjects, making it has strong application value in future EEG-based emotion recognition applications.

M-Net: a novel unsupervised domain adaptation framework based on multi-kernel maximum mean discrepancy for fault diagnosis of rotating machinery

AbstractThe intelligent fault diagnosis model has made a significant development, whose high-precision results rely on a large amount of labeled data. However, in the actual industrial environment, it is very difficult to obtain a large amount of labeled data. It will make it difficult for the fault diagnosis model to converge with limited labeled industrial data. To address this paradox, we propose a novel unsupervised domain adaptation framework (M-Net) for fault diagnosis of rotating machinery, which only requires unlabeled industrial data. The M-Net will be pretrained using the labeled data, which can be accessed through the labs. In this stage, we propose a multi-scale feature extractor that can extract and fuse multi-scale features. This operation will generalize the features further. Then, we will align the distribution of the labeled data and unlabeled industrial data using the generator model based on multi-kernel maximum mean discrepancy. This will reduce the distribution distance between the labeled data and the unlabeled industrial data. For now, the unsupervised domain adaptation problem has shifted to a semi-supervised domain adaptation problem. The results, obtained through experimental comparison, demonstrate that the M-Net can achieve an accuracy of over 99.99% with labeled data and a maximum transfer accuracy of over 99% with unlabeled industrial data.

Dual Dynamic Consistency Regularization for Semi-Supervised Domain Adaptation

Dual Dynamic Consistency Regularization for Semi-Supervised Domain Adaptation

Semi-Supervised Domain Adaptation for Major Depressive Disorder Detection

Semi-Supervised Domain Adaptation for Major Depressive Disorder Detection

Investigating Generalizability of Speech-based Suicidal Ideation Detection Using Mobile Phones

Speech-based diaries from mobile phones can capture paralinguistic patterns that help detect mental illness symptoms such as suicidal ideation. However, previous studies have primarily evaluated machine learning models on a single dataset, making their performance unknown under distribution shifts. In this paper, we investigate the generalizability of speech-based suicidal ideation detection using mobile phones through cross-dataset experiments using four datasets with N=786 individuals experiencing major depressive disorder, auditory verbal hallucinations, persecutory thoughts, and students with suicidal thoughts. Our results show that machine and deep learning methods generalize poorly in many cases. Thus, we evaluate unsupervised domain adaptation (UDA) and semi-supervised domain adaptation (SSDA) to mitigate performance decreases owing to distribution shifts. While SSDA approaches showed superior performance, they are often ineffective, requiring large target datasets with limited labels for adversarial and contrastive training. Therefore, we propose sinusoidal similarity sub-sampling (S3), a method that selects optimal source subsets for the target domain by computing pair-wise scores using sinusoids. Compared to prior approaches, S3 does not use labeled target data or transform features. Fine-tuning using S3 improves the cross-dataset performance of deep models across the datasets, thus having implications in ubiquitous technology, mental health, and machine learning.

Semi-Supervised Domain Adaptation for Individual Identification from Electrocardiogram Signals

When acquiring electrocardiogram (ECG) signals, the placement of electrode patches is crucial for acquiring electrocardiographic signals. Constant displacement positions are essential for ensuring the consistency of the ECG signal when used for individual identification. However, achieving constant placement of ECG electrode patches in every trial for data acquisition is challenging. This is because different individuals may attach patches, and even when the same person attaches them, it may be difficult to specify the exact position. Therefore, gathering ECG data from various locations is necessary. However, this process requires a substantial amount of labor and time, owing to the requirement for multiple attempts. Nonetheless, persisting with these efforts enables the endurance of some ECG differences. To reduce labor and time, we propose a semi-supervised domain adaptation for individual identification using ECG signals. The method operates with a full set of original ECG signals and a small set of ECG signals from different placements to account for the differences between the signals in the generative adversarial network (CycleGAN). Specifically, to train the CycleGAN, the ECG signals were transformed into time–frequency representations, and the trained generator was used to generate ECG signals to expand the small set of ECG signals from different placements. Subsequently, both the original and generated signals were used to train the classifier for individual identification. This scenario can also be applied to the classification of ECG signals from different sensors. The PTB-ECG dataset was used for this experiment. We found that the proposed method showed higher accuracy than when only the original ECG signals were used for the training classifier.

GDmicro: classifying host disease status with GCN and deep adaptation network based on the human gut microbiome data.

With advances in metagenomic sequencing technologies, there are accumulating studies revealing the associations between the human gut microbiome and some human diseases. These associations shed light on using gut microbiome data to distinguish case and control samples of a specific disease, which is also called host disease status classification. Importantly, using learning-based models to distinguish the disease and control samples is expected to identify important biomarkers more accurately than abundance-based statistical analysis. However, available tools have not fully addressed two challenges associated with this task: limited labeled microbiome data and decreased accuracy in cross-studies. The confounding factors, such as the diet, technical biases in sample collection/sequencing across different studies/cohorts often jeopardize the generalization of the learning model. To address these challenges, we develop a new tool GDmicro, which combines semi-supervised learning and domain adaptation to achieve a more generalized model using limited labeled samples. We evaluated GDmicro on human gut microbiome data from 11 cohorts covering 5 different diseases. The results show that GDmicro has better performance and robustness than state-of-the-art tools. In particular, it improves the AUC from 0.783 to 0.949 in identifying inflammatory bowel disease. Furthermore, GDmicro can identify potential biomarkers with greater accuracy than abundance-based statistical analysis methods. It also reveals the contribution of these biomarkers to the host's disease status. https://github.com/liaoherui/GDmicro.

Privacy-preserving multi-source semi-supervised domain adaptation for seizure prediction

Privacy-preserving multi-source semi-supervised domain adaptation for seizure prediction

MADAv2: Advanced Multi-Anchor Based Active Domain Adaptation Segmentation.

Unsupervised domain adaption has been widely adopted in tasks with scarce annotated data. Unfortunately, mapping the target-domain distribution to the source-domain unconditionally may distort the essential structural information of the target-domain data, leading to inferior performance. To address this issue, we firstly propose to introduce active sample selection to assist domain adaptation regarding the semantic segmentation task. By innovatively adopting multiple anchors instead of a single centroid, both source and target domains can be better characterized as multimodal distributions, in which way more complementary and informative samples are selected from the target domain. With only a little workload to manually annotate these active samples, the distortion of the target-domain distribution can be effectively alleviated, achieving a large performance gain. In addition, a powerful semi-supervised domain adaptation strategy is proposed to alleviate the long-tail distribution problem and further improve the segmentation performance. Extensive experiments are conducted on public datasets, and the results demonstrate that the proposed approach outperforms state-of-the-art methods by large margins and achieves similar performance to the fully-supervised upperbound, i.e., 71.4% mIoU on GTA5 and 71.8% mIoU on SYNTHIA. The effectiveness of each component is also verified by thorough ablation studies. Code is available at https://github.com/munanning/MADAv2.

Inter-domain mixup for semi-supervised domain adaptation

Semi-supervised domain adaptation (SSDA) aims to bridge source and target domain distributions, with a small number of target labels available, achieving better classification performance than unsupervised domain adaptation (UDA). However, existing SSDA work fails to make full use of label information from both source and target domains for feature alignment across domains, resulting in label mismatch in the label space during model testing. This paper presents a novel SSDA approach, Inter-domain Mixup with Neighborhood Expansion (IDMNE), to tackle this issue. Firstly, we introduce a cross-domain feature alignment strategy, Inter-domain Mixup, that incorporates label information into model adaptation. Specifically, we employ sample-level and manifold-level data mixing to generate compatible training samples. These newly established samples, combined with reliable and actual label information, display diversity and compatibility across domains, while such extra supervision thus facilitates cross-domain feature alignment and mitigates label mismatch. Additionally, we utilize Neighborhood Expansion to leverage high-confidence pseudo-labeled samples in the target domain, diversifying the label information of the target domain and thereby further increasing the performance of the adaptation model. Accordingly, the proposed approach outperforms existing state-of-the-art methods, achieving significant accuracy improvements on popular SSDA benchmarks, including DomainNet, Office-Home, and Office-31.

Class-imbalanced Unsupervised and Semi-Supervised Domain Adaptation for Histopathology Images.

In dealing with the lack of sufficient annotated data and in contrast to supervised learning, unsupervised, self-supervised, and semi-supervised domain adaptation methods are promising approaches, enabling us to transfer knowledge from rich labeled source domains to different (but related) unlabeled target domains, reducing distribution discrepancy between the source and target domains. However, most existing domain adaptation methods do not consider the imbalanced nature of the real-world data, affecting their performance in practice. We propose to overcome this limitation by proposing a novel domain adaptation approach that includes two modifications to the existing models. Firstly, we leverage the focal loss function in response to class-imbalanced labeled data in the source domain. Secondly, we introduce a novel co-training approach to involve pseudo-labeled target data points in the training process. Experiments show that the proposed model can be effective in transferring knowledge from source to target domain. As an example, we use the classification of prostate cancer images into low-cancerous and high-cancerous regions.

Cross-Domain Multi-Prototypes with Contradictory Structure Learning for Semi-Supervised Domain Adaptation Segmentation of Remote Sensing Images

Recently, unsupervised domain adaptation (UDA) segmentation of remote sensing images (RSIs) has attracted a lot of attention. However, the performance of such methods still lags far behind that of their supervised counterparts. To this end, this paper focuses on a more practical yet under-investigated problem, semi-supervised domain adaptation (SSDA) segmentation of RSIs, to effectively improve the segmentation results of targeted RSIs with a few labeled samples. First, differently from the existing single-prototype mode, a novel cross-domain multi-prototype constraint is proposed, to deal with large inter-domain discrepancies and intra-domain variations. Specifically, each class is represented as a set of prototypes, so that multiple sets of prototypes corresponding to different classes can better model complex inter-class differences, while different prototypes within the same class can better describe the rich intra-class relations. Meanwhile, the multi-prototypes are calculated and updated jointly using source and target samples, which can effectively promote the utilization and fusion of the feature information in different domains. Second, a contradictory structure learning mechanism is designed to further improve the domain alignment, with an enveloping form. Third, self-supervised learning is adopted, to increase the number of target samples involved in prototype updating and domain adaptation training. Extensive experiments verified the effectiveness of the proposed method for two aspects: (1) Compared with the existing SSDA methods, the proposed method could effectively improve the segmentation performance by at least 7.38%, 4.80%, and 2.33% on the Vaihingen, Potsdam, and Urban datasets, respectively; (2) with only five labeled target samples available, the proposed method could significantly narrow the gap with its supervised counterparts, which was reduced to at least 4.04%, 6.04%, and 2.41% for the three RSIs.