Domain Shift Problem Research Articles

A domain generalization network for imbalanced machinery fault diagnosis

Traditional models for Imbalanced Fault Diagnosis (IFD) face challenges in practical applications due to domain shifts caused by varying working conditions and machinery. Domain Generalization (DG) models provide an advantage over traditional approaches by learning class-discriminative and domain-invariant feature representations, allowing them to generalize to unseen target data. However, the scarcity of fault samples relative to healthy ones limits their application in real-world industrial scenarios. In this paper, we propose a Domain Mixed-Enhanced Domain Generalization Network (DEMDGN) that enhances IFD performance by utilizing mixup-based data augmentation and domain-based discrepancy metrics to align feature distributions across multiple heterogeneous source domains. By creating domain-invariant features, DEMDGN allows robust fault diagnosis under varying conditions. Extensive experiments on one marine machinery dataset and two bearing datasets demonstrate that the proposed method effectively addresses class imbalance and domain shift problems, achieving superior diagnostic performance.

An effective unsupervised domain adaptation for in-field potato disease recognition

Accurate disease recognition through computer vision is crucial for the intelligent management of potato production. Popular data-driven classification methods face challenges including limited labelled data and poor model portability. Unsupervised Domain Adaptation (UDA) addresses these challenges with a novel learning strategy. However, the complex field environment introduces a significant domain shift problem due to varying conditions. Existing UDA methods usually concentrate on aligning global data distribution and employ a single structure for disease feature extraction, thereby limiting their efficacy in true field environment. To tackle this challenge of potato disease recognition, the Multi-Representation Adaptive Network (MRSAN) based on subdomain alignment is presented. MRSAN effectively aligns feature distributions across diverse data by minimising distribution differences among relevant subdomains. Simultaneously, the multi-representation extraction method captures finer details from various perspectives in the disease images. The combination of these two approaches efficiently mitigates the adverse effects caused by various interference factors in field environment. Based on the acquisition conditions of light variation and disease progression, two field potato disease image datasets are created, containing five and six kinds of potato leaf disease, respectively. Extensive transfer experiments are conducted on the two datasets. MRSAN achieves average classification accuracies of 87.03% and 80.06% on the datasets for the corresponding transfer tasks, outperforming the other compared methods. This not only validates the effectiveness of MRSAN but also demonstrates its robust ability to generalise across changes in regard to light variation and disease progression.

Instance-wise multi-view visual fusion for zero-shot learning

Zero-shot learning (ZSL) has become increasing popular in computer vision due to its ability to recognize categories unobserved in the training data. So far, most existing ZSL approaches adopt visual representations that are either derived from pretrained networks or learned using an end-to-end architecture. However, a single group of visual representations can hardly capture all features hidden in the images, yielding incomplete visual information. In numerous real-life scenarios, multi-view visual representations are often accessible which describe the instances more comprehensively and are potential for better learning performance. In this paper, we introduce an instance-wise multi-view visual fusion (IMVF) for zero-shot learning (ZSL) model. In accordance with the consensus principle, a multi-view visual-semantic mapping is created by minimizing the disparities of seen-class semantic projections on different views. Meanwhile, a straightforward linear constraint is performed on each seen-class instance to adhere to the complementary principle so that the cross-view information exchange is well motivated. In order to mitigate the domain shift problem, the predicted unseen-class semantic projections are further refined through a multi-view manifold alignment under the consensus principle. Our proposed IMVFZSL is compared with the State-of-the-Art ZSL methods on AwA2, CUB and SUN datasets. Exciting experimental results validate the effectiveness of the IMVF mechanism. To the best of our understanding, this is an initial attempt to fuse multi-view visual representations in ZSL, which will stimulate valuable contemplation in this field.

Unpaired Image Translation to Mitigate Domain Shift in Liquid Argon Time Projection Chamber Detector Responses

Abstract Deep learning algorithms often are developed and trained on a training dataset and deployed on test datasets. Any systematic difference between the training and a test dataset may severely degrade the final algorithm performance on the test dataset -- what is known as the domain shift problem. This issue is prevalent in many scientific domains where algorithms are trained on simulated data but applied to real-world datasets. Typically, the domain shift problem is solved through various domain adaptation methods. However, these methods are often tailored to a specific downstream task and may not easily generalize to different tasks. This work explores the feasibility of using an alternative way to solve the domain shift problem that is not specific to any downstream algorithm. The proposed approach relies on modern Unpaired Image-to-Image translation techniques, designed to find translations between different image domains in a fully unsupervised fashion. In this study, the approach is applied to a domain shift problem commonly encountered in Liquid Argon Time Projection Chamber detector research when seeking a way to translate samples between two differently distributed LArTPC detector datasets deterministically. This translation allows for mapping real-world data into the simulated data domain where the downstream algorithms can be run with much less domain-shift-related performance degradation. Conversely, using the translation from the simulated data in a real-world domain can increase the realism of the simulated dataset and reduce the magnitude of any systematic uncertainties. We adapted several popular UI2I translation algorithms to work on scientific data and demonstrated the viability of these techniques for solving the domain shift problem with LArTPC detector data. To facilitate further development of domain adaptation techniques for scientific datasets, the "Simple Liquid-Argon Track Samples" (SLATS) dataset used in this study also is published.

Learn Then Adapt: A Novel Test-Time Adaptation Method for Cross-Domain Fault Diagnosis of Rolling Bearings

Cross-domain fault diagnosis enhances the generalization capability of diagnostic models across different operating conditions and machines. Current studies tackle the domain shift problem by adapting the model during training with data from the target domain or multiple source domains. However, a more realistic and less explored scenario is automatically adapting a trained (developed) model at test time (deployment period) using limited normal-condition data. To bridge this research gap, we propose a novel test-time adaptation framework to rapidly and effectively adapt the trained model, which only requires mini-batch test data (normal condition). Specifically, we first transform input signals to informative signal embedding and mitigate its noise with a reconstruction loss. Then, we decompose the signal embedding to the domain-related healthy component and the domain-invariant faulty component to better leverage the normal-condition data. Finally, we adapt the model by re-identifying the normal signals of the target domain during the test stage. Extensive experiments verify the effectiveness of our method, demonstrating performance improvements across public and private datasets.

Multisource partial domain adaptation for bearing fault diagnosis

Abstract Domain adaptation has been widely used in fault diagnosis and dealt with data distribution discrepancies, but the labels in source and target domains are usually assumed to be identical. The complexity of the working conditions, in reality, leads to the fact that the labels in target domains are often a subset of the labels in source domains. This special case is called the partial domain problem. However, most of the existing proposed methods for solving partial domain problems are limited to single-source-domain scenarios and fail to effectively integrate multisource knowledge. Hence, this study proposes a new approach of multisource domain obfuscation-subdomain alignment (MSDO-SA) for partial domain adaptation fault diagnosis in multisource domains. Through domain obfuscation, the multisource domains are converted into a single source domain. The subdomain alignment aims at improving the generalization ability and relevance of the model, and effectively alleviates the domain shift problem. Finally, multiple partial domain fault diagnosis tasks using the CWRU dataset validate the effectiveness, robustness, and superiority of the proposed method.

ConvNext Mitosis Identification—You Only Look Once (CNMI-YOLO): Domain Adaptive and Robust Mitosis Identification in Digital Pathology

In digital pathology, accurate mitosis detection in histopathological images is critical for cancer diagnosis and prognosis. However, this remains challenging due to the inherent variability in cell morphology and the domain shift problem. This study introduces ConvNext Mitosis Identification-You Only Look Once (CNMI-YOLO), a new 2-stage deep learning method that uses the YOLOv7 architecture for cell detection and the ConvNeXt architecture for cell classification. The goal is to improve the identification of mitosis in different types of cancers. We utilized the Mitosis Domain Generalization Challenge 2022 data set in the experiments to ensure the model’s robustness and success across various scanners, species, and cancer types. The CNMI-YOLO model demonstrates superior performance in accurately detecting mitotic cells, significantly outperforming existing models in terms of precision, recall, and F1 score. The CNMI-YOLO model achieved an F1 score of 0.795 on the Mitosis Domain Generalization Challenge 2022 and demonstrated robust generalization with F1 scores of 0.783 and 0.759 on the external melanoma and sarcoma test sets, respectively. Additionally, the study included ablation studies to evaluate various object detection and classification models, such as Faster-RCNN and Swin Transformer. Furthermore, we assessed the model’s robustness performance on unseen data, confirming its ability to generalize and its potential for real-world use in digital pathology, using soft tissue sarcoma and melanoma samples not included in the training data set.

Mitigating domain shift in online process monitoring for material extrusion additive manufacturing via transfer learning

The additive manufacturing (AM) method has experienced rapid growth in recent decades. However, its application in end-use products is constrained by printing defects. Therefore, online process evaluation and optimization are crucial for producing parts that meet quality standards. Data-driven methods have been extensively employed for in-situ process evaluation and control in various AM processes to establish correlations among process parameters, process information, and part quality. However, a critical weakness of these methods is their poor generalization, primarily due to two factors: the diverse nature of printing itself and the significant influence of environmental factors. This study proposes a framework for in-situ process evaluation and control of AM using transfer learning to address the domain shift problems. It highlights a significant domain shift issue in AM monitoring, especially when using deep learning models. A comprehensive monitoring system for material extrusion additive manufacturing is constructed to collect data from different domains, emphasizing the notable differences in data characteristics across these domains. Lastly, transfer learning methods are tested to tackle the domain shift issue in AM, enhancing the robustness and reliability of the monitoring system.

Haze generation and feature fusion network aiming at real-world single image dehazing

Abstract Because most dehazing methods use synthesized haze images for training, they may perform well on synthesized datasets. However, when these methods are applied to real-world scenes, their performance may significantly decrease due to domain shift. Therefore, we propose a dehazing network for real-world hazy scenes. This network includes a haze generation network that can utilize the hazy information of real haze images to generate images that are closer to real hazy scenes, generating training pairs to address the domain shift problem. The network also includes a dehazing network that integrates feature fusion attention mechanisms, which can achieve better dehazing performance.

Multi-scale task-aware structure graph modeling for few-shot image recognition

The Few-shot image recognition attempts to recognize images from a novel class with only a limited number of labeled training images, which is a few-shot learning (FSL) task. FSL is very challenging. Limited labeled training samples cannot adequately represent the distribution of classes, and the base and novel classes in the training and testing stages do not intersect and have different distributions, leading to a domain shift problem in generalizing the learned model to the novel class dataset. In this paper, we propose multi-scale task-aware structure graph modeling for few-shot image recognition. We train a meta-filter learner to generate task-aware local structure filters for each scale and adaptively capture the local structures at each scale. Moreover, we introduce a novel multi-scale graph attention network (MGAT) module to model the multi-scale local structures of an image, fully exploring the correlations between different local structures at all scales of the image. Finally, we leverage the attention mechanism of graph attention network to achieve information aggregation and propagation, aiming to obtain more representative and discriminative local structure features that integrate both local and global information. We conducted comprehensive experiments on four benchmark datasets widely adopted in FSL tasks. The experimental results demonstrate that the MTSGM obtains state-of-the-art performance, which validates the effectiveness of MTSGM.

A cross-scenario and cross-subject domain adaptation method for driving fatigue detection

Objective. The scarcity of electroencephalogram (EEG) data, coupled with individual and scenario variations, leads to considerable challenges in real-world EEG-based driver fatigue detection. We propose a domain adaptation method that utilizes EEG data collected from a laboratory to supplement real-world EEG data and constructs a cross-scenario and cross-subject driver fatigue detection model for real-world scenarios. Approach. First, we collect EEG data from subjects participating in a driving experiment conducted in both laboratory and real-world scenarios. To address the issue of data scarcity, we build a real-world fatigued driving detection model by integrating the real-world data with the laboratory data. Then, we propose a method named cross-scenario and cross-subject domain adaptation (CS2DA), which aims to eliminate the domain shift problem caused by individual variances and scenario differences. Adversarial learning is adopted to extract the common features observed across different subjects within the same scenario. The multikernel maximum mean discrepancy (MK-MMD) method is applied to further minimize scenario differences. Additionally, we propose a conditional MK-MMD constraint to better utilize label information. Finally, we use seven rules to fuse the predicted labels. Main results. We evaluate the CS2DA method through extensive experiments conducted on the two EEG datasets created in this work: the SEED-VLA and the SEED-VRW datasets. Different domain adaptation methods are used to construct a real-world fatigued driving detection model using data from laboratory and real-world scenarios, as well as a combination of both. Our findings show that the proposed CS2DA method outperforms the existing traditional and adversarial learning-based domain adaptation approaches. We also find that combining data from both laboratory and real-world scenarios improves the performance of the model. Significance. This study contributes two EEG-based fatigue driving datasets and demonstrates that the proposed CS2DA method can effectively enhance the performance of a real-world fatigued driving detection model.

Interoperability Analysis of Tomato Fruit Detection Models for Images Taken at Different Facilities, Cultivation Methods, and Times of the Day

This study investigated the interoperability of a tomato fruit detection model trained using nighttime images from two greenhouses. The goal was to evaluate the performance of the models in different environmets, including different facilities, cultivation methods, and imaging times. An innovative imaging approach is introduced to eliminate the background, highlight the target plants, and test the adaptability of the model under diverse conditions. The results demonstrate that the tomato fruit detection accuracy improves when the domain of the training dataset contains the test environment. The quantitative results showed high interoperability, achieving an average accuracy (AP50) of 0.973 in the same greenhouse and a stable performance of 0.962 in another greenhouse. The imaging approach controlled the lighting conditions, effectively eliminating the domain-shift problem. However, training on a dataset with low diversity or inferring plant appearance images but not on the training dataset decreased the average accuracy to approximately 0.80, revealing the need for new approaches to overcome fruit occlusion. Importantly, these findings have practical implications for the application of automated tomato fruit set monitoring systems in greenhouses to enhance agricultural efficiency and productivity.

Entropy and distance-guided super self-ensembling for optic disc and cup segmentation.

Segmenting the optic disc (OD) and optic cup (OC) is crucial to accurately detect changes in glaucoma progression in the elderly. Recently, various convolutional neural networks have emerged to deal with OD and OC segmentation. Due to the domain shift problem, achieving high-accuracy segmentation of OD and OC from different domain datasets remains highly challenging. Unsupervised domain adaptation has taken extensive focus as a way to address this problem. In this work, we propose a novel unsupervised domain adaptation method, called entropy and distance-guided super self-ensembling (EDSS), to enhance the segmentation performance of OD and OC. EDSS is comprised of two self-ensembling models, and the Gaussian noise is added to the weights of the whole network. Firstly, we design a super self-ensembling (SSE) framework, which can combine two self-ensembling to learn more discriminative information about images. Secondly, we propose a novel exponential moving average with Gaussian noise (G-EMA) to enhance the robustness of the self-ensembling framework. Thirdly, we propose an effective multi-information fusion strategy (MFS) to guide and improve the domain adaptation process. We evaluate the proposed EDSS on two public fundus image datasets RIGA+ and REFUGE. Large amounts of experimental results demonstrate that the proposed EDSS outperforms state-of-the-art segmentation methods with unsupervised domain adaptation, e.g., the Dicemean score on three test sub-datasets of RIGA+ are 0.8442, 0.8772 and 0.9006, respectively, and the Dicemean score on the REFUGE dataset is 0.9154.

BrainDAS: Structure-aware domain adaptation network for multi-site brain network analysis

In the medical field, datasets are mostly integrated across sites due to difficult data acquisition and insufficient data at a single site. The domain shift problem caused by the heterogeneous distribution among multi-site data makes autism spectrum disorder (ASD) hard to identify. Recently, domain adaptation has received considerable attention as a promising solution. However, domain adaptation on graph data like brain networks has not been fully studied. It faces two major challenges: (1) complex graph structure; and (2) multiple source domains. To overcome the issues, we propose an end-to-end structure-aware domain adaptation framework for brain network analysis (BrainDAS) using resting-state functional magnetic resonance imaging (rs-fMRI). The proposed approach contains two stages: supervision-guided multi-site graph domain adaptation with dynamic kernel generation and graph classification with attention-based graph pooling. We evaluate our BrainDAS on a public dataset provided by Autism Brain Imaging Data Exchange (ABIDE) which includes 871 subjects from 17 different sites, surpassing state-of-the-art algorithms in several different evaluation settings. Furthermore, our promising results demonstrate the interpretability and generalization of the proposed method. Our code is available at https://github.com/songruoxian/BrainDAS.

CrossHAR: Generalizing Cross-dataset Human Activity Recognition via Hierarchical Self-Supervised Pretraining

The increasing availability of low-cost wearable devices and smartphones has significantly advanced the field of sensor-based human activity recognition (HAR), attracting considerable research interest. One of the major challenges in HAR is the domain shift problem in cross-dataset activity recognition, which occurs due to variations in users, device types, and sensor placements between the source dataset and the target dataset. Although domain adaptation methods have shown promise, they typically require access to the target dataset during the training process, which might not be practical in some scenarios. To address these issues, we introduce CrossHAR, a new HAR model designed to improve model performance on unseen target datasets. CrossHAR involves three main steps: (i) CrossHAR explores the sensor data generation principle to diversify the data distribution and augment the raw sensor data. (ii) CrossHAR then employs a hierarchical self-supervised pretraining approach with the augmented data to develop a generalizable representation. (iii) Finally, CrossHAR fine-tunes the pretrained model with a small set of labeled data in the source dataset, enhancing its performance in cross-dataset HAR. Our extensive experiments across multiple real-world HAR datasets demonstrate that CrossHAR outperforms current state-of-the-art methods by 10.83% in accuracy, demonstrating its effectiveness in generalizing to unseen target datasets.

Standardization of ultrasound images across various centers: M2O-DiffGAN bridging the gaps among unpaired multi-domain ultrasound images

Domain shift problem is commonplace for ultrasound image analysis due to difference imaging setting and diverse medical centers, which lead to poor generalizability of deep learning-based methods. Multi-Source Domain Transformation (MSDT) provides a promising way to tackle the performance degeneration caused by the domain shift, which is more practical and challenging compared to conventional single-source transformation tasks. An effective unsupervised domain combination strategy is highly required to handle multiple domains without annotations. Fidelity and quality of generated images are also important to ensure the accuracy of computer-aided diagnosis. However, existing MSDT approaches underperform in above two areas. In this paper, an efficient domain transformation model named M2O-DiffGAN is introduced to achieve a unified mapping from multiple unlabeled source domains to the target domain. A cycle-consistent “many-to-one” adversarial learning architecture is introduced to model various unlabeled domains jointly. A condition adversarial diffusion process is employed to generate images with high-fidelity, combining an adversarial projector to capture reverse transition probabilities over large step sizes for accelerating sampling. Considering the limited perceptual information of ultrasound images, an ultrasound-specific content loss helps to capture more perceptual features for synthesizing high-quality ultrasound images. Massive comparisons on six clinical datasets covering thyroid, carotid and breast demonstrate the superiority of the M2O-DiffGAN in the performance of bridging the domain gaps and enlarging the generalization of downstream analysis methods compared to state-of-the-art algorithms. It improves the mean MI, Bhattacharyya Coefficient, dice and IoU assessments by 0.390, 0.120, 0.245 and 0.250, presenting promising clinical applications.

Self-supervised adversarial adaptation network for breast cancer detection

Breast cancer is the most commonly diagnosed cancer worldwide, and early detection is essential for reducing mortality rates. Digital mammography is currently the best standard for early detection, as it can assist physicians in treating the disease. However, inaccurate diagnoses from mammography are common and can lead to patients undergoing unnecessary tests and treatments. To address this challenge, deep-learning techniques have shown promising results in improving the accuracy and reliability of breast cancer detection. However, existing methods face two primary challenges: the lack of the annotated data, and the inability to adapt to new data domains. In this paper, we propose SelfAdaptNet to address these issues. Specifically, SelfAdaptNet employs self-supervised learning techniques, such as Bootstrap Your Own Latent (BYOL) and Simple Framework for Learning of Visual Representations (SimCLR), to tackle the problem of limited annotated data. Additionally, the adversarial technique is used to address the problem of domain shift. By successfully reducing domain disparities, this strategy enhances the model’s adaptability and robustness across a variety of clinical scenarios. Overall, our contributions offer a more effective and flexible approach for early breast cancer detection, and experimental results demonstrate that SelfAdaptNet can produce promising results as compared with other methods.

Multi-source self-supervised domain adaptation network for VRLA battery anomaly detection of data center under non-ideal conditions

As the key component of uninterruptible power supply, the anomaly detection of valve-regulated lead-acid (VRLA) batteries is the reliable guarantee for the operation of internet data centers. Although deep neural networks (DNNs) have been investigated in battery anomaly detection, the key problem of feature learning under non-ideal conditions, i.e., data scarcity and class-imbalance, is not addressed very well. In this article, a novel DNN called multi-source self-supervised domain adaptation network (MSSSDAN) is proposed for VRLA battery anomaly detection of data center under non-ideal conditions. Firstly, multi-dimensional collaborative self-supervised learning (MDCSSL) is proposed to address the class-imbalanced problem by performing auxiliary feature learning task considering the temporal dependency of different channels. Secondly, a novel transfer learning (TL) framework, multi-source adaptive transfer learning (MSATL) is developed to solve the domain shift problem by adaptively minimizing the distribution discrepancy of different domains. Finally, the outperformance of MSSSDAN is verified on the VRLA battery dataset collected from Tencent data center. The testing results demonstrate that MSSSDAN can effectively solve the problem of battery anomaly detection under non-ideal conditions.

Meta-learning based infrared ship object detection model for generalization to unknown domains

Infrared images exhibit considerable variations in probability distributions, stemming from the utilization of distinct infrared sensors and the influence of diverse environmental conditions. The variations pose great challenges for deep learning models to detect ship objects and adapt to unseen maritime environments. To address the domain shift problem, we propose an end-to-end infrared ship object detection model based on meta-learning neural network to improve domain adaptation for target domain where data is not available at training phase. Different from existing domain generalization methods, the novelty of our model lies in the effective exploitation of meta-learning and domain adaptation, ensuring that the extracted domain-independent features are meaningful and domain-invariant at the semantic level. Firstly, a double gradient-based meta-learning algorithm is designed to solve the common optimal descent direction between different domains through two gradient updates in the inner and outer loops. The algorithm enables extraction of domain-invariant features from the pseudo-source and pseudo-target domain data. Secondly, a domain discriminator with dynamic-weighted gradient reversal layer (DWGRL) is designed to accurately classify domain-invariant features and provide additional global supervision information. Finally, a multi-scale feature aggregation method is proposed to improve the extraction of multi-scale domain-invariant features. It can effectively fuse local features at different scales and global features of targets. Extensive experimental results conducted in real nighttime water surface scenes demonstrate that the proposed model achieves very high detection accuracy on target domain data, even no target domain data was used during the training phase. Compared to the existing methods, our method not only improves the detection accuracy of infrared ships by 18%, but also exhibits the smallest standard deviation with a value of 0.93, indicating its superior generalization performance.

Decentralized federated domain generalization with cluster alignment for fault diagnosis

Fault diagnosis is important for maintaining safety in industrial scenarios. Due to the complex operating conditions, there is usually a domain shift between training (source) data and testing (target) data. Recent years have witnessed the emergence of numerous transfer learning methods dealing with the domain shift. However, existing methods often fail to deal with the situation where target data are unavailable. Moreover, the majority of these methods require aggregating data distributed across various users (nodes) for model training, raising privacy concerns. Despite existing federated learning methods can protect privacy, they mostly rely on a central server, which may lead to a single point of failure. To address the above issues, we propose a fully decentralized Federated Domain Generalization with Cluster Alignment (FDG-CA), which deals with the domain shift problem without accessing target data and eliminates the need of a central server while protecting privacy. During the training phase, the proposed FDG-CA learns domain-invariant representations by aligning clusters statistics of different source nodes through information exchange. Subsequently, during the testing phase, we propose an ensemble strategy based on a learner filter and a voting scheme to get the prediction results. Experiments demonstrate that our proposed method is superior to existing methods, achieving higher accuracy while addressing privacy concerns.