Domain Adaptation Research Articles

A few-shot based phase-batch multi-layer domain adaptation pattern recognition method

Deep transfer learning has been widely applied in the field of intelligent fault diagnosis. However, existing deep transfer learning-based diagnostic methods struggle to train reliable diagnostic models when there is a lack of data and significant distribution differences between the two domains. To address this issue, a few-shot based phase-batch multi-layer domain adaptation pattern recognition method is proposed. This method simultaneously measures the feature distribution differences of both the fully connected layers and the classification layers, thus better correcting the data domain bias. Additionally, a phase-batch training strategy and pseudo-label learning are employed to improve the convergence speed and stability of the training process. The proposed method is validated on two public datasets, Jiang Nan and Paderborn University, as well as a dataset obtained through independent experiments. It is compared with traditional feature-based transfer learning methods, the results show that the proposed method achieves higher diagnostic accuracy, faster convergence, and greater stability. Furthermore, its superior diagnostic performance in the few-shot scenario is demonstrated through experiments on a self-collected dataset.

Development and analysis of medical instruction-tuning for Japanese large language models

In the ongoing wave of impact driven by large language models (LLMs) like ChatGPT, the adaptation of LLMs to the medical domain has emerged as a crucial research frontier. Since mainstream LLMs tend to be designed for general-purpose applications, constructing a medical LLM through domain adaptation is a huge challenge. While instruction-tuning, particularly based on low-rank adaptation (LoRA), has become a frequently employed strategy to fine-tune LLMs recently, its precise roles in domain adaptation remain unknown. Here, we investigated how LoRA-based instruction-tuning improves the performance of Japanese medical question-answering tasks by employing a multifaceted evaluation of multiple-choice questions, including scoring based on “Exact match” and “Gestalt distance” in addition to the conventional accuracy. Our findings suggest that LoRA-based instruction-tuning can partially incorporate domain-specific knowledge into LLMs, with larger models demonstrating more pronounced effects. Furthermore, our results underscore the potential of adapting English-centric models for Japanese applications in domain adaptation, while also highlighting the persisting limitations of Japanese-centric models. This initiative represents a pioneering effort in enabling medical institutions to fine-tune and operate models without relying on external services.

Transferable mapping shift network for unsupervised domain adaptation using in vibration signal fault diagnosis under variable conditions

Transferable mapping shift network for unsupervised domain adaptation using in vibration signal fault diagnosis under variable conditions

Virtual reality games for cognitive rehabilitation of older adults: a review of adaptive games, domains and techniques

In recent decades, the senior adults population worldwide has increased, as well as the medical conditions related to aging, such as cognitive decline. Virtual reality (VR) games are a valuable addition to conventional cognitive rehabilitation as they increase engagement to the therapy through customization, socialization, immersion, and feedback. This review, performed according to PRISMA protocol, addresses the following questions: How VR games have been used for cognitive rehabilitation?, What cognitive domains have been addressed by VR games and in which populations have these games been used?, Which features have been considered to improve engagement in VR games for cognitive rehabilitation?, How is the difficulty adjustment of exercises carried out in VR games for cognitive rehabilitation?. We found 25 scientific works related to these questions, 92% of them treating one cognitive domain at a time, despite the fact that the related literature recognizes the value of training multiple domains simultaneously. Our review indicates that, despite the existence of serious VR games for working memory training, such as those described in Flak et al. (Front Psychol 10:807, 2019. https://doi.org/10.3389/fpsyg.2019.00807), to our knowledge, there are no applications that simultaneously address multiple cognitive domains and incorporate dynamic difficulty adjustment, which are important to ensure ecological validity of therapy and therapy adherence, respectively. In addition, we found that games themselves could be used to monitor the user’s progression. It is also important to determine the impact of multiplayer interactions in the game, test difficulty adjustment approaches that use physiological variables, and define difficulty-skill relationships aligned with the user’s preferences. This paper concludes that the main barriers to implement dynamic difficulty adjustment in VR games for cognitive rehabilitation are: (i) the absence of metrics to estimate when the game offers to the players a challenge adapted their skills, and (ii) the lack of a conceptual framework that integrates relevant theories such as state of flow, cognitive load, cognitive rehabilitation, and feedback systems.

Reusability report: Uncovering associations in biomedical bipartite networks via a bilinear attention network with domain adaptation

Reusability report: Uncovering associations in biomedical bipartite networks via a bilinear attention network with domain adaptation

SFDA-CD: A Source-Free Unsupervised Domain Adaptation for VHR Image Change Detection

Deep models may have disappointing performance in real applications due to the domain shifts in data distributions between the source and target domain. Although a few unsupervised domain adaptation methods have been proposed to make the pre-train models effective on target domain datasets, constraints like data privacy, security, and transmission limits restrict access to VHR remote sensing images, making existing unsupervised domain adaptation methods almost ineffective in specific change detection areas. Therefore, we propose a source-free unsupervised domain adaptation change detection structure to complete specific change detection tasks, using only the pre-trained source model and unlabelled target data. The GAN-based source generation component is designed to generate synthetic source data, which, to some extent, reflects the distribution of the source domain. Moreover, these data can be utilised in model knowledge transfer. The model adaptation component facilitates knowledge transfer between models by minimising the differences between deep features, using AAM (Attention Adaptation Module) to extract the difference between high-level features, meanwhile we proposed ISM (Intra-domain Self-supervised Module) to train target model in a self-supervised strategy in order to improve the knowledge adaptation. Our SFDA-CD framework demonstrates superior accuracy over existing unsupervised domain adaptation change detection methods, which has 0.6% cIoU and 1.5% F1 score up in cross-regional tasks and 1.4% cIoU and 1.9% F1 score up in cross-scenario tasks, proving that it can effectively reduce the domain shift between the source and target domains even without access to source data. Additionally, it can facilitate knowledge transfer from the source model to the target model.

A universal fault diagnosis framework for marine machinery based on domain adaptation

In the digital and automated era, the significance of machine health monitoring has been amplified, especially for marine machinery, which plays a pivotal role in the sustainable advancement of modern industries. Two salient challenges currently hampering fault diagnostics are spotlighted: firstly, the variability in mechanical operating conditions gives rise to inconsistent data distributions, thereby prompting domain shift. Secondly, unobserved in training datasets, testing scenarios may unveil unpredictable, unknown fault modes, culminating in class discrepancies. To effectively tackle these intertwined challenges, we introduce the Two-Stage Universal Domain Adaptation (TSUDA) — a novel fault diagnosis methodology. TSUDA adeptly differentiates new fault types from labeled known counterparts by leveraging a composite transferability metric. Furthermore, it deploys a parameter-adaptive unsupervised algorithm to discern the count of emerging fault types. Rigorous experiments on datasets from marine diesel engines, emblematic of marine machinery, demonstrated TSUDA's superiority over conventional domain adaptation techniques in diverse diagnostic challenges, registering accuracy rates surpassing 90%. While the effectiveness of TSUDA is authenticated within marine machinery contexts in this investigation, its architecture suggests potential applicability in a broader spectrum of mechanical fault diagnostic scenarios.

Cross-domain knowledge collaboration for blending-target domain adaptation

Unsupervised domain adaptation (UDA) avoids expensive data annotation for the unlabeled target domains by fully utilizing the knowledge of existing source domain. In practice, the target data are usually highly heterogeneous that mix multiple latent domains. And sometimes the source data involve the user private information which is forbidden to access directly. To this end, this paper tackles the blending-target data under the source-available setting, and for the first time under the source-free setting. Specifically, we devise a novel Cross-domain Knowledge Collaboration (CdKC) framework, which mainly includes a prediction network and an adaptation network. The complementarity of two networks are exploited to explore the intrinsic structure in targets. Specifically, CdKC is capable of learning domain-invariant space, and disentangling domain-specific feature simultaneously to boost the UDA performance greatly. A total of 12 tasks are conducted on three visual datasets to verify the superior performance of CdKC by comparing with the state-of-the-art models designed under 4 different UDA settings. The experiments demonstrate that the accuracy of CdKC model still exceeds D-CGCT 0.4% on Office dataset and 1.2% on Office–Home dataset although D-CGCT can access source data and domain labels of targets that CdKC cannot do. It verifies the effectiveness of CdKC although under much looser source and target domain restrictions.

An adversarial transfer learning method based on domain distribution prediction for aero-engine fault diagnosis

The utilization of transfer learning enables effective realization of the transferability of aero-engine fault diagnosis models across disparate states. However, the gradual degradation of engine performance and the complex and variable flight states lead to continuous changes in data distribution. The common transfer learning methods employ discrete divisions of data domain distributions, which are insufficient to cope with the continuous changes of the domain. To accomplish transfer learning towards a continuous and multi-dimensional target domain, we propose a continuous domain distribution adversarial network (CDDAN) based on deep domain adversarial network. The method defines the number of effective cycles directly associated with engine degradation as a continuous domain index. When the domain index is extended to multiple dimensions, a hybrid gaussian model is introduced to represent distribution prediction within multi-dimensional continuous domains. Subsequently, we validate the feasibility and superiority of our method using datasets generated based on twin-spool turbofan engines. In comparison to other transfer learning methods, the proposed method achieves superior effect in continuous domain adaptation. The experimental results demonstrate that this method exhibits adaptability to failures in the diagnosis method caused by performance degradation and changes in flight state of the aero engine, while remaining independent of target domain data annotation. This advantage becomes particularly apparent in scenarios with limited target domain data and multi-state fault diagnosis.

CAT-DTI: cross-attention and Transformer network with domain adaptation for drug-target interaction prediction

Accurate and efficient prediction of drug-target interaction (DTI) is critical to advance drug development and reduce the cost of drug discovery. Recently, the employment of deep learning methods has enhanced DTI prediction precision and efficacy, but it still encounters several challenges. The first challenge lies in the efficient learning of drug and protein feature representations alongside their interaction features to enhance DTI prediction. Another important challenge is to improve the generalization capability of the DTI model within real-world scenarios. To address these challenges, we propose CAT-DTI, a model based on cross-attention and Transformer, possessing domain adaptation capability. CAT-DTI effectively captures the drug-target interactions while adapting to out-of-distribution data. Specifically, we use a convolution neural network combined with a Transformer to encode the distance relationship between amino acids within protein sequences and employ a cross-attention module to capture the drug-target interaction features. Generalization to new DTI prediction scenarios is achieved by leveraging a conditional domain adversarial network, aligning DTI representations under diverse distributions. Experimental results within in-domain and cross-domain scenarios demonstrate that CAT-DTI model overall improves DTI prediction performance compared with previous methods.

Optimal heterogeneous domain adaptation for text classification in transfer learning

The prime challenge of unsupervised symmetric heterogeneous cross-domain adaptation is to train the source domain and apply the trained knowledge to the target domain. Most of the existing algorithms for unsupervised transfer learning create the subspace of the source domain features and target domain features for training purposes. It is an extensive computational process as most of the techniques require labeled source data. Many techniques also suffer from the loss of originality of features in both domains. This paper aims to consider the feature vectors of both the source and target domain for training the data based on the similarity of exemplar (feature) vectors of different instances, known as Instance Similarity Feature (ISF). The use of vectorization method for the similarity of features is proposed in this paper. The exemplar vectors are chosen randomly for the target datasets. Hence, to acquire relevant factual data in the knowledge base for training in our research, we worked to increase the domain separation error between source and target instances. To avoid the instability caused due to poor exemplar vector selection, the K-means clustering approach is followed after feature similarity, known as K-means Instance Similarity Feature (KISF). Many existing transfer learning techniques are based on the original feature set, which can cause degeneracy, hence affecting Accuracy. In order to vanquish the limitations of existing approaches, we have introduced novel optimal models with KISF and Ant Lion Optimizer (KISFA), KISF with Particle Swarm Optimization (KISFP) and KISF with Biogeography Based Optimization (KISFB). High-dimensionality can impact efficacy of the model, hence, feature selection with nature-based optimizer namely: Ant Lion Optimizer, Particle Swarm Optimization and Biogeography-Based Optimization are applied. We measure the performance of the proposed models by using Support Vector Machine, Logistic Regression, Random Forest, Naive Baye’s, K-Nearest Neighbor and Decision Tree as classifiers, and Accuracy and F1-score as fitness functions. Extensive experiments are performed on four datasets with 50 iterations. The proposed model is compared with eleven other techniques and our technique outperforms all other techniques in average Accuracy. The validation is performed on the dataset using 10-fold cross-validation. The statistical test was performed using ANOVA, proving that our technique is significantly better than other techniques.

DDAGCN: an unsupervised cross-domain identification method for tie rod bolt loosening in a rod-fastening rotor system under different working conditions

The cross-domain diagnosis of tie rod bolt loosening is essential for guaranteeing the healthy operation of rod-fastening rotor (RFR) systems. The unsupervised domain adaptation (UDA) method effectively alleviates the impact of domain discrepancy and has been applied for cross-domain diagnosis. Traditional UDA methods mainly focus on the marginal and conditional distributions with fixed weights to adapt the domain distribution discrepancy. However, the fixed distribution combination cannot satisfy the requirement of feature domain alignment under different working conditions, and the relative importance of the two distributions cannot be evaluated quantitatively. This paper proposes an improved dynamic distribution adaptive graph convolutional network (DDAGCN) for the cross-domain diagnosis of tie rod bolt loosening under different working conditions. This method can quantitatively evaluate the relative significance of each distribution in representing the distribution discrepancy. First, it combines the convolutional neural network and the graph convolutional network to extract the features in the graph structure by using the connection relationship between nodes, and realizes the full extraction of neighbourhood information of nodes. Then, the dynamic distribution adaptive alignment strategy is introduced to construct the dynamic linear combination of marginal and conditional distributions, so as to measure the distribution discrepancy between domains. Meanwhile, the domain adversarial module is combined to further reduce the domain gap and finally realize feature alignment. The extracted domain invariant features can effectively enhance the generalization ability and fault identification ability of the model. The case of the public bearing dataset verifies that the effectiveness and generalization ability of the proposed method for cross-domain fault diagnosis under different working conditions is superior to other compared methods. In addition, the identification ability of the proposed method for the degree of tie rod bolt loosening is verified by the self-made bolt loosening dataset of the RFR system.

Domain-adaptation method between acoustic-response data using different insert earphones.

Classifying acoustic responses captured through earphones offers valuable insights into nearby environments, such as whether the earphones are in or out of the ear. However, the performances of classification algorithms often suffer when applied to other devices due to domain mismatches. This study proposes a domain-adaptation method tailored for acoustic-response data from two distinct insert earphone models. The method trains a domain-adaptation function using a pair of datasets obtained from a set of acoustic loads, yielding a domain-adapted dataset suitable for training classification algorithms in a target domain. The effectiveness of this approach is validated through assessments of domain adaptation quality and resulting performance enhancements in the classification algorithm tasked with discerning whether an earphone is positioned inside or outside the ear. Importantly, our method requires significantly fewer measurements than the original dataset, reducing data collection time while providing a suitable training dataset for the target domain. Additionally, the method's reusability across future devices streamlines data collection time and efforts for the future devices.

Domain Adaptation and Generalization of Functional Medical Data: A Systematic Survey of Brain Data

In spite of the excellent capabilities of machine learning algorithms, their performance deteriorates when the distribution of test data differs from the distribution of training data. In medical data research, this problem is exacerbated by its connection to human health, expensive equipment, and meticulous setups. Consequently, achieving domain generalizations (DG) and domain adaptations (DA) under distribution shifts is an essential step in the analysis of medical data. As the first systematic review of DG and DA on functional brain signals, the paper discusses and categorizes various methods, tasks, and datasets in this field. Moreover, it discusses relevant directions for future research.

Battery state of health estimation across electrochemistry and working conditions based on domain adaptation

Accurately assessing the health status of lithium-ion batteries is essential to ensure their safe and efficient application in electric vehicles and energy storage systems. Although various methods have been proposed to achieve battery state of health (SOH) estimation, most of them are only applicable to certain battery types or operating conditions. To address this issue, a novel method is proposed in this study, which leverages data-driven techniques and domain adaptation to cater to different battery electrochemistry and operating conditions. First, the evolution of battery aging is investigated and the incremental capacity sequence with ample aging information is extracted to indicate battery health. Then, the convolutional neural network and bidirectional long-short term memory network are combined to capture the nonlinear relationship between the input sequence and battery SOH. Next, a domain adaptation (DA) based on adversarial training is employed to enhance model adaptability by realizing domain-invariant features extraction. Furthermore, data augmentation is utilized to address data imbalance caused by significant lifespan disparity among different batteries. Finally, datasets with different battery types and aging conditions are used to verify the proposed method. The average estimation error of battery SOH can be controlled within 4.77%, with over 30% reduction contributed by the DA.

Multi-Source Domain Adaptation for Medical Image Segmentation.

Unsupervised domain adaptation(UDA) aims to mitigate the performance drop of models tested on the target domain, due to the domain shift from the target to sources. Most UDA segmentation methods focus on the scenario of solely single source domain. However, in practical situations data with gold standard could be available from multiple sources (domains), and the multi-source training data could provide more information for knowledge transfer. How to utilize them to achieve better domain adaptation yet remains to be further explored. This work investigates multi-source UDA and proposes a new framework for medical image segmentation. Firstly, we employ a multi-level adversarial learning scheme to adapt features at different levels between each of the source domains and the target, to improve the segmentation performance. Then, we propose a multi-model consistency loss to transfer the learned multi-source knowledge to the target domain simultaneously. Finally, we validated the proposed framework on two applications, i.e., multi-modality cardiac segmentation and cross-modality liver segmentation. The results showed our method delivered promising performance and compared favorably to state-of-the-art approaches.

Domain adaptive learning for multi realm sentiment classification on big data.

Machine learning techniques that rely on textual features or sentiment lexicons can lead to erroneous sentiment analysis. These techniques are especially vulnerable to domain-related difficulties, especially when dealing in Big data. In addition, labeling is time-consuming and supervised machine learning algorithms often lack labeled data. Transfer learning can help save time and obtain high performance with fewer datasets in this field. To cope this, we used a transfer learning-based Multi-Domain Sentiment Classification (MDSC) technique. We are able to identify the sentiment polarity of text in a target domain that is unlabeled by looking at reviews in a labelled source domain. This research aims to evaluate the impact of domain adaptation and measure the extent to which transfer learning enhances sentiment analysis outcomes. We employed transfer learning models BERT, RoBERTa, ELECTRA, and ULMFiT to improve the performance in sentiment analysis. We analyzed sentiment through various transformer models and compared the performance of LSTM and CNN. The experiments are carried on five publicly available sentiment analysis datasets, namely Hotel Reviews (HR), Movie Reviews (MR), Sentiment140 Tweets (ST), Citation Sentiment Corpus (CSC), and Bioinformatics Citation Corpus (BCC), to adapt multi-target domains. The performance of numerous models employing transfer learning from diverse datasets demonstrating how various factors influence the outputs.

Tissue Segmentation of Thick-Slice Fetal Brain MR Scans With Guidance From High-Quality Isotropic Volumes.

Accurate tissue segmentation of thick-slice fetal brain magnetic resonance (MR) scans is crucial for both reconstruction of isotropic brain MR volumes and the quantification of fetal brain development. However, this task is challenging due to the use of thick-slice scans in clinically-acquired fetal brain data. To address this issue, we propose to leverage high-quality isotropic fetal brain MR volumes (and also their corresponding annotations) as guidance for segmentation of thick-slice scans. Due to existence of significant domain gap between high-quality isotropic volume (i.e., source data) and thick-slice scans (i.e., target data), we employ a domain adaptation technique to achieve the associated knowledge transfer (from high-quality "source" volumes to thick-slice "target" scans). Specifically, we first register the available high-quality isotropic fetal brain MR volumes across different gestational weeks to construct longitudinally-complete source data. To capture domain-invariant information, we then perform Fourier decomposition to extract image content and style codes. Finally, we propose a novel Cycle-Consistent Domain Adaptation Network (C 2DA-Net) to efficiently transfer the knowledge learned from high-quality isotropic volumes for accurate tissue segmentation of thick-slice scans. Our C 2DA-Net can fully utilize a small set of annotated isotropic volumes to guide tissue segmentation on unannotated thick-slice scans. Extensive experiments on a large-scale dataset of 372 clinically acquired thick-slice MR scans demonstrate that our C 2DA-Net achieves much better performance than cutting-edge methods quantitatively and qualitatively.

Reinforced fuzzy domain adaptation: Revolutionizing data-unaccessible rotating machinery fault diagnosis across multiple domains

Existing domain adaptation methods typically require access to raw data from both the source and target domains, increasing the risk of data leakage. In light of this, a fuzzy domain adaptation-based rotation machinery fault diagnosis method that does not require access to data is proposed in this paper. Specifically, during the source domain training phase, all source models are simultaneously trained by contributing model parameters. The developed joint training strategy significantly enhances the multi-domain performance of each private source model. Simultaneously, a model incorporating a fusion fuzzy strategy is constructed to acquire fuzzy rules for the source domain samples and anchor points corresponding to each fault mode. During the target domain adaptation phase, firstly, under the setting of frozen fuzzy rules, the source feature extractor is retrained using target data. Subsequently, an anchor point alignment strategy is introduced to transfer diagnostic knowledge between domains. Finally, a filtering strategy is devised to obtain high-confidence pseudo-labels for the target domain, coupled with a self-supervised learning strategy for the final optimization of the target model. Experimental results demonstrate significant achievements of the proposed method across twelve multi-source cross-domain fault diagnosis cases, with an average diagnostic accuracy of 98.88%. Encouragingly, the proposed method maintains outstanding performance even without accessing source data, outperforming several other methods.

Shallow Inception Domain Adaptation Network for EEG-Based Motor Imagery Classification

Electroencephalography (EEG) data across multiple individuals have a high variance. Directly using the data to train a deep learning model usually degrades the performance. To address this issue, we propose a shallow Inception domain adaptation framework to extract informative deep features from data of multiple subjects for accurate motor imagery (MI) recognition. To our best knowledge, the Inception architecture in deep learning is combined with domain adaptation (DA) scheme for the first time for the MI classification task. The approach contains two compact Inception blocks that decode temporal features in different scales. In addition, we jointly optimize a novel combined loss function to reduce both marginal and class conditional discrepancies caused by the multi-modal structure of EEG signals. The DA-based loss enables Inception blocks to take full advantage of their learning abilities to capture discriminative patterns of MI data from multiple subjects instead of relying on the target user only. To demonstrate the effectiveness of our approach, we conduct substantial experiments on two wellknown datasets, BCI competition IV-2a and IV-2b. Results show that our model achieves better performance than state-of-theart strategies. The proposed model is able to extract informative features from high-variant EEG data collected from different individuals and achieves accurate MI classifications.