Domain Shift Research Articles

Multiscale Conditional Adversarial Networks based domain-adaptive method for rotating machinery fault diagnosis under variable working conditions

Deep learning has been increasingly used in health management and maintenance decision-making for rotating machinery. However, some challenges must be addressed to make this technology more effective. For example, the collected data is assumed to follow the same feature distribution, and sufficient labeled training data are available. Unfortunately, domain shifts occur inevitably in real-world scenarios due to different working conditions, and acquiring sufficient labeled samples is time-consuming and expensive in complex environments. This study proposes a novel domain adaptive framework called deep Multiscale Conditional Adversarial Networks (MCAN) for machinery fault diagnosis to address these shortcomings. The MCAN model comprises two key components. Constructed by a novel multiscale module with an attention mechanism, the first component is a shared feature generator that captures rich features at different internal perceptual scales, and the attention mechanism determines the weights assigned to each scale, enhancing the model’s dynamic adjustment and self-adaptation capabilities. The second component consists of two domain classifiers based on Bidirectional Long Short-Term Memory (BiLSTM) leveraging spatiotemporal features at various levels to achieve domain adaptation in the output space. The deep domain classifier also captures the cross-covariance dependencies between feature representations and classifier predictions, thereby improving the predictions’ discriminability. The proposed method has been evaluated using two publicly available fault diagnosis datasets and one condition monitoring experiment. The results of cross-domain transfer tasks demonstrated that the proposed method outperformed several state-of-the-art methods in terms of transferability and stability. This result is a significant step forward in deep learning for health management and maintenance decision-making for rotating machinery, and it has the potential to revolutionize its future application.

RF-ULM: Ultrasound Localization Microscopy Learned From Radio-Frequency Wavefronts.

In Ultrasound Localization Microscopy (ULM), achieving high-resolution images relies on the precise localization of contrast agent particles across a series of beamformed frames. However, our study uncovers an enormous potential: The process of delay-and-sum beamforming leads to an irreversible reduction of Radio-Frequency (RF) channel data, while its implications for localization remain largely unexplored. The rich contextual information embedded within RF wavefronts, including their hyperbolic shape and phase, offers great promise for guiding Deep Neural Networks (DNNs) in challenging localization scenarios. To fully exploit this data, we propose to directly localize scatterers in RF channel data. Our approach involves a custom super-resolution DNN using learned feature channel shuffling, non-maximum suppression, and a semi-global convolutional block for reliable and accurate wavefront localization. Additionally, we introduce a geometric point transformation that facilitates seamless mapping to the B-mode coordinate space. To understand the impact of beamforming on ULM, we validate the effectiveness of our method by conducting an extensive comparison with State-Of-The-Art (SOTA) techniques. We present the inaugural in vivo results from a wavefront-localizing DNN, highlighting its real-world practicality. Our findings show that RF-ULM bridges the domain shift between synthetic and real datasets, offering a considerable advantage in terms of precision and complexity. To enable the broader research community to benefit from our findings, our code and the associated SOTA methods are made available at https://github.com/hahnec/rf-ulm.

Dual-Spatial Domain Generalization for Fundus Lesion Segmentation in Unseen Manufacturer's OCT Images.

Optical Coherence Tomography (OCT) images can provide non-invasive visualization of fundus lesions; however, scanners from different OCT manufacturers largely vary from each other, which often leads to model deterioration to unseen OCT scanners due to domain shift. To produce the T-styles of the potential target domain, an Orthogonal Style Space Reparameterization (OSSR) method is proposed to apply orthogonal constraints in the latent orthogonal style space to the sampled marginal styles. To leverage the high-level features of multi-source domains and potential T-styles in the graph semantic space, a Graph Adversarial Network (GAN) is constructed to align the generated samples with the source domain samples. To align features with the same label based on the semantic feature in the graph semantic space, Graph Semantic Alignment (GSA) is performed to focus on the shape and the morphological differences between the lesions and their surrounding regions. Comprehensive experiments have been performed on two OCT image datasets. Compared to state-of-the-art methods, the proposed method can achieve better segmentation. The proposed fundus lesion segmentation method can be trained with labeled OCT images from multiple manufacturers' scanners and be tested on an unseen manufacturer's scanner with better domain generalization. The proposed method can be used in routine clinical occasions when an unseen manufacturer's OCT image is available for a patient.

A Chebyshev Confidence Guided Source-Free Domain Adaptation Framework for Medical Image Segmentation.

Source-free domain adaptation (SFDA) aims to adapt models trained on a labeled source domain to an unlabeled target domain without access to source data. In medical imaging scenarios, the practical significance of SFDA methods has been emphasized due to data heterogeneity and privacy concerns. Recent state-of-the-art SFDA methods primarily rely on self-training based on pseudo-labels (PLs). Unfortunately, the accuracy of PLs may deteriorate due to domain shift, thus limiting the effectiveness of the adaptation process. To address this issue, we propose a Chebyshev confidence guided SFDA framework to accurately assess the reliability of PLs and generate self-improving PLs for self-training. The Chebyshev confidence is estimated by calculating the probability lower bound of PL confidence, given the prediction and the corresponding uncertainty. Leveraging the Chebyshev confidence, we introduce two confidence-guided denoising methods: direct denoising and prototypical denoising. Additionally, we propose a novel teacher-student joint training scheme (TJTS) that incorporates a confidence weighting module to iteratively improve PLs' accuracy. The TJTS, in collaboration with the denoising methods, effectively prevents the propagation of noise and enhances the accuracy of PLs. Extensive experiments in diverse domain scenarios validate the effectiveness of our proposed framework and establish its superiority over state-of-the-art SFDA methods. Our paper contributes to the field of SFDA by providing a novel approach for precisely estimating the reliability of PLs and a framework for obtaining high-quality PLs, resulting in improved adaptation performance.

ConvNext Mitosis Identification—You Only Look Once (CNMI-YOLO): Domain Adaptive and Robust Mitosis Identification in Digital Pathology

In digital pathology, accurate mitosis detection in histopathological images is critical for cancer diagnosis and prognosis. However, this remains challenging due to the inherent variability in cell morphology and the domain shift problem. This study introduces ConvNext Mitosis Identification-You Only Look Once (CNMI-YOLO), a new 2-stage deep learning method that uses the YOLOv7 architecture for cell detection and the ConvNeXt architecture for cell classification. The goal is to improve the identification of mitosis in different types of cancers. We utilized the Mitosis Domain Generalization Challenge 2022 data set in the experiments to ensure the model’s robustness and success across various scanners, species, and cancer types. The CNMI-YOLO model demonstrates superior performance in accurately detecting mitotic cells, significantly outperforming existing models in terms of precision, recall, and F1 score. The CNMI-YOLO model achieved an F1 score of 0.795 on the Mitosis Domain Generalization Challenge 2022 and demonstrated robust generalization with F1 scores of 0.783 and 0.759 on the external melanoma and sarcoma test sets, respectively. Additionally, the study included ablation studies to evaluate various object detection and classification models, such as Faster-RCNN and Swin Transformer. Furthermore, we assessed the model’s robustness performance on unseen data, confirming its ability to generalize and its potential for real-world use in digital pathology, using soft tissue sarcoma and melanoma samples not included in the training data set.

Cross-Domain Adaptation Techniques for Robust Plant Disease Detection: A DANN-CORAL Hybrid Approach

Plant disease detection with deep learning models has shown promising results, but these models often struggle with generalizing across diverse agricultural environments due to domain shifts in imaging conditions. This paper presents a novel hybrid approach focusing on cross-domain adaptation techniques to address the challenge of domain shift. Our proposed method combines the Domain-Adversarial Neural Network (DANN) with Correlation Alignment (CORAL) to mitigate domain shifts between datasets. The DANN framework enforces domain-invariant feature learning through adversarial training. Using the PlantVillage Dataset, with controlled environment images, and the New Plant Village Dataset, with varied conditions, the model is first trained on PlantVillage and then adapted to New Plant Village using the CORAL loss to support the second-order statistics. In case of domain shift experiments with various datasets, DANN-CORAL achieved accuracies 91.39%, precision 93.36%, recall 88.9% and F1-scores 91.05% indicating the robustness and generalizability of our model is better than the other baseline models. This approach enhances model robustness and adaptability, providing insights into combining adversarial and statistical alignment for cross-domain adaptation in agricultural imaging.

A Dataset for Detection and Segmentation of Underwater Marine Debris in Shallow Waters

Robust object detection is crucial for automating underwater marine debris collection. While supervised deep learning achieves state-of-the-art performance in discriminative tasks, replicating this success on underwater data is challenging. The generalization of these methods suffers due to a lack of available annotated data considering different sources of variation in the unstructured underwater environment and imaging conditions. In this paper, we present the Seaclear Marine Debris Dataset, the first publicly available shallow-water marine debris dataset annotated for instance segmentation/object detection. The dataset contains 8610 images collected using ROVs at multiple locations and with different cameras, annotated for 40 object categories, encompassing not only litter but also observed animals, plants, and robot parts. As part of the technical validation, we provide baseline results for object detection using Faster RCNN and YOLOv6 models. Furthermore, we demonstrate the non-triviality of generalizing the trained model performance to unseen sites and cameras due to domain shift. This underscores the value of the presented dataset in further developing robust models for underwater debris detection.

Piston aero-engine fault cross-domain diagnosis based on unpaired generative transfer learning

As artificial intelligence (AI) and machine learning continue to advance, they have become key players in signal analysis and pattern recognition. However, challenges such as domain shift and a lack of labeled data hinder the application of machine learning to new, unfamiliar signals, particularly in the development of intelligent diagnostic methods for mechanical faults. In this study, we address these challenges by framing them as the extreme class imbalance problem. We introduce a novel generative transfer learning framework that leverages cycle-consistent adversarial networks with hard constraints (CycleGAN-HCs). This framework is designed to generate unpaired mechanical fault signals, which are then used to train classifiers that can operate across different domains. To enhance feature extraction capabilities for time series data, we developed a new generator model called U-Attention. Additionally, we refined the training loss function to better capture domain-specific and classification-specific features in mechanical vibration signals. The proposed method successfully facilitates feature transfer and data augmentation. Its effectiveness and reliability have been demonstrated through four cross-domain fault diagnosis tasks involving piston aero-engines. Comparative verification shows that the generative transfer learning diagnostic framework outperforms traditional methods, offering superior performance and better generalization in complex mechanical fault diagnosis scenarios.

Forecasting battery degradation trajectory under domain shift with domain generalization

Rechargeable batteries play a pivotal role in the carbon-neutral green environment by electrifying transportation and mitigating the intermittency of renewable energies. Forecasting the degradation of batteries is crucial for the optimal usage of batteries, while predicting battery degradation is not trivial due to diverse working conditions and complex failure mechanisms. To address this challenge, we develop a deep learning model that treats differences in operating conditions as domain shifts and utilizes meta-learning-based and task-driven domain generalization techniques to attack the domain shifts. The model effectiveness is demonstrated on three datasets comprising 203 cells of various operating conditions and chemistries, with improvements in prediction accuracy ranging from 18.1% to 30.0% (23.8% on average). Moreover, the model has gained some generalization capability via learning the correlation between domain gaps in the model and the degradation modes behind various operating conditions. Collectively, our work not only showcases the promise of the high-reliability data-driven model for diverse conditions and chemistries by exploiting domain generalization, but also spotlights the potential interplay between artificial intelligence and domain knowledge.

Cross-Domain Object Detection through Consistent and Contrastive Teacher with Fourier Transform

The teacher–student framework has been employed in unsupervised domain adaptation, which transfers knowledge learned from a labeled source domain to an unlabeled target domain. However, this framework suffers from two serious challenges: the domain gap, causing performance degradation, and noisy teacher pseudo-labels, which tend to mislead students. In this paper, we propose a Consistent and Contrastive Teacher with Fourier Transform (CCTF) method to address these challenges for high-performance cross-domain object detection. To mitigate the negative impact of domain shifts, we use the Fourier transform to exchange the low-frequency components of the source and target domain images, replacing the source domain inputs with the transformed image, thereby reducing domain gaps. In addition, we encourage the localization and classification branches of the teacher to make consistent predictions to minimize the noise in the generated pseudo-labels. Finally, contrastive learning is employed to resist the impact of residual noise in pseudo-labels. After extensive experiments, we show that our method achieves the best performance. For example, our model outperforms previous methods by 3.0% on FoggyCityscapes.

Adversarial Training Based Domain Adaptation of Skin Cancer Images.

Skin lesion datasets used in the research are highly imbalanced; Generative Adversarial Networks can generate synthetic skin lesion images to solve the class imbalance problem, but it can result in bias and domain shift. Domain shifts in skin lesion datasets can also occur if different instruments or imaging resolutions are used to capture skin lesion images. The deep learning models may not perform well in the presence of bias and domain shift in skin lesion datasets. This work presents a domain adaptation algorithm-based methodology for mitigating the effects of domain shift and bias in skin lesion datasets. Six experiments were performed using two different domain adaptation architectures. The domain adversarial neural network with two gradient reversal layers and VGG13 as a feature extractor achieved the highest accuracy and F1 score of 0.7567 and 0.75, respectively, representing an 18.47% improvement in accuracy over the baseline model.

Mask-Shift-Inference: A novel paradigm for domain generalization

Domain Generalization (DG) focuses on the Out-Of-Distribution (OOD) generalization, which is able to learn a robust model that generalizes the knowledge acquired from the source domain to the unseen target domain. However, due to the existence of the domain shift, domain-invariant representation learning is challenging. Guided by fine-grained knowledge, we propose a novel paradigm Mask-Shift-Inference (MSI) for DG based on the architecture of Convolutional Neural Networks (CNN). Different from relying on a series of constraints and assumptions for model optimization, this paradigm novelly shifts the focus to feature channels in the latent space for domain-invariant representation learning. We put forward a two-branch working mode of a main module and multiple domain-specific sub-modules. The latter can only achieve good prediction performance in its own specific domain but poor predictions in other source domains, which provides the main module with the fine-grained knowledge guidance and contributes to the improvement of the cognitive ability of MSI. Firstly, during the forward propagation of the main module, the proposed MSI accurately discards unstable channels based on spurious classifications varying across domains, which have domain-specific prediction limitations and are not conducive to generalization. In this process, a progressive scheme is adopted to adaptively increase the masking ratio according to the training progress to further reduce the risk of overfitting. Subsequently, our paradigm enters the compatible shifting stage before the formal prediction. Based on maximizing semantic retention, we implement the domain style matching and shifting through the simple transformation through Fourier transform, which can explicitly and safely shift the target domain back to the source domain whose style is closest to it, requiring no additional model updates and reducing the domain gap. Eventually, the paradigm MSI enters the formal inference stage. The updated target domain is predicted in the main module trained in the previous stage with the benefit of familiar knowledge from the nearest source domain masking scheme. Our paradigm is logically progressive, which can intuitively exclude the confounding influence of domain-specific spurious information along with mitigating domain shifts and implicitly perform semantically invariant representation learning, achieving robust OOD generalization. Extensive experimental results on PACS, VLCS, Office-Home and DomainNet datasets verify the superiority and effectiveness of the proposed method.

Transfer learning with generative models for object detection on limited datasets

Abstract The availability of data is limited in some fields, especially for object detection tasks, where it is necessary to have correctly labeled bounding boxes around each object. A notable example of such data scarcity is found in the domain of marine biology, where it is useful to develop methods to automatically detect submarine species for environmental monitoring. To address this data limitation, the state-of-the-art machine learning strategies employ two main approaches. The first involves pretraining models on existing datasets before generalizing to the specific domain of interest. The second strategy is to create synthetic datasets specifically tailored to the target domain using methods like copy-paste techniques or ad-hoc simulators. The first strategy often faces a significant domain shift, while the second demands custom solutions crafted for the specific task. In response to these challenges, here we propose a transfer learning framework that is valid for a generic scenario. In this framework, generated images help to improve the performances of an object detector in a few-real data regime. This is achieved through a diffusion-based generative model that was pretrained on large generic datasets. With respect to the state-of-the-art, we find that it is not necessary to fine tune the generative model on the specific domain of interest. We believe that this is an important advance because it mitigates the labor-intensive task of manual labeling the images in object detection tasks. We validate our approach focusing on fishes in an underwater environment, and on the more common domain of cars in an urban setting. Our method achieves detection performance comparable to models trained on thousands of images, using only a few hundreds of input data. Our results pave the way for new generative AI-based protocols for machine learning applications in various domains, for instance ranging from geophysics to biology and medicine.

Cross-Modality Medical Image Segmentation via Enhanced Feature Alignment and Cross Pseudo Supervision Learning.

Given the diversity of medical images, traditional image segmentation models face the issue of domain shift. Unsupervised domain adaptation (UDA) methods have emerged as a pivotal strategy for cross modality analysis. These methods typically utilize generative adversarial networks (GANs) for both image-level and feature-level domain adaptation through the transformation and reconstruction of images, assuming the features between domains are well-aligned. However, this assumption falters with significant gaps between different medical image modalities, such as MRI and CT. These gaps hinder the effective training of segmentation networks with cross-modality images and can lead to misleading training guidance and instability. To address these challenges, this paper introduces a novel approach comprising a cross-modality feature alignment sub-network and a cross pseudo supervised dual-stream segmentation sub-network. These components work together to bridge domain discrepancies more effectively and ensure a stable training environment. The feature alignment sub-network is designed for the bidirectional alignment of features between the source and target domains, incorporating a self-attention module to aid in learning structurally consistent and relevant information. The segmentation sub-network leverages an enhanced cross-pseudo-supervised loss to harmonize the output of the two segmentation networks, assessing pseudo-distances between domains to improve the pseudo-label quality and thus enhancing the overall learning efficiency of the framework. This method's success is demonstrated by notable advancements in segmentation precision across target domains for abdomen and brain tasks.

Domain generalization for semantic segmentation: a survey

Deep neural networks (DNNs) have proven explicit contributions in making autonomous driving cars and related tasks such as semantic segmentation, motion tracking, object detection, sensor fusion, and planning. However, in challenging situations, DNNs are not generalizable because of the inherent domain shift due to the nature of training under the i.i.d. assumption. The goal of semantic segmentation is to preserve information from a given image into multiple meaningful categories for visual understanding. Particularly for semantic segmentation, pixel-wise annotation is extremely costly and not always feasible. Domain generalization for semantic segmentation aims to learn pixel-level semantic labels from multiple source domains and generalize to predict pixel-level semantic labels on multiple unseen target domains. In this survey, for the first time, we present a comprehensive review of DG for semantic segmentation. we present a comprehensive summary of recent works related to domain generalization in semantic segmentation, which establishes the importance of generalizing to new environments of segmentation models. Although domain adaptation has gained more attention in segmentation tasks than domain generalization, it is still worth unveiling new trends that are adopted from domain generalization methods in semantic segmentation. We cover most of the recent and dominant DG methods in the context of semantic segmentation and also provide some other related applications. We conclude this survey by highlighting the future directions in this area.

Progressive Decision Boundary Shifting for Unsupervised Domain Adaptation.

Unsupervised domain adaptation (UDA) is attracting more attention from researchers for boosting the task-specific generalization on target domain. It focuses on addressing the domain shift between the labeled source domain and the unlabeled target domain. Recent biclassifier-based UDA models perform category-level alignment to reduce domain shift, and meanwhile, self-training is used for improving the discriminability of target instances. However, the error accumulation problem of instances with high semantic uncertainty may cause discriminability degradation and category-level misalignment. To solve this issue, we design the progressive decision boundary shifting algorithm, where stable category information of target instances is explored for learning a discriminability structure on target domain. Specifically, we first model the semantic uncertainty of instances by progressively shifting decision boundaries of category. Then, we introduce the uncertainty decoupling in a contrastive manner, where the discriminative information is learned from the source domain for instance with low semantic uncertainty. Furthermore, we minimize the predictive entropy of instances with high semantic uncertainty to reduce their prediction confidence. Extensive experiments on three popular datasets show that our model outperforms the current state-of-the-art (SOTA) UDA methods.

Leveraging Visual Language Model and Generative Diffusion Model for Zero-Shot SAR Target Recognition

Simulated data play an important role in SAR target recognition, particularly under zero-shot learning (ZSL) conditions caused by the lack of training samples. The traditional SAR simulation method is based on manually constructing target 3D models for electromagnetic simulation, which is costly and limited by the target’s prior knowledge base. Also, the unavoidable discrepancy between simulated SAR and measured SAR makes the traditional simulation method more limited for target recognition. This paper proposes an innovative SAR simulation method based on a visual language model and generative diffusion model by extracting target semantic information from optical remote sensing images and transforming it into a 3D model for SAR simulation to address the challenge of SAR target recognition under ZSL conditions. Additionally, to reduce the domain shift between the simulated domain and the measured domain, we propose a domain adaptation method based on dynamic weight domain loss and classification loss. The effectiveness of semantic information-based 3D models has been validated on the MSTAR dataset and the feasibility of the proposed framework has been validated on the self-built civilian vehicle dataset. The experimental results demonstrate that the first proposed SAR simulation method based on a visual language model and generative diffusion model can effectively improve target recognition performance under ZSL conditions.

Deep learning for autosegmentation for radiotherapy treatment planning: State-of-the-art and novel perspectives.

The rapid development of artificial intelligence (AI) has gained importance, with many tools already entering our daily lives. The medical field of radiation oncology is also subject to this development, with AI entering all steps of the patient journey. In this review article, we summarize contemporary AI techniques and explore the clinical applications of AI-based automated segmentation models in radiotherapy planning, focusing on delineation of organs at risk (OARs), the gross tumor volume (GTV), and the clinical target volume (CTV). Emphasizing the need for precise and individualized plans, we review various commercial and freeware segmentation tools and also state-of-the-art approaches. Through our own findings and based on the literature, we demonstrate improved efficiency and consistency as well as time savings in different clinical scenarios. Despite challenges in clinical implementation such as domain shifts, the potential benefits for personalized treatment planning are substantial. The integration of mathematical tumor growth models and AI-based tumor detection further enhances the possibilities for refining target volumes. As advancements continue, the prospect of one-stop-shop segmentation and radiotherapy planning represents an exciting frontier in radiotherapy, potentially enabling fast treatment with enhanced precision and individualization.

Implicit Sharpness-Aware Minimization for Domain Generalization

Domain generalization (DG) aims to learn knowledge from multiple related domains to achieve a robust generalization performance in unseen target domains, which is an effective approach to mitigate domain shift in remote sensing image classification. Although the sharpness-aware minimization (SAM) method enhances DG capability and improves remote sensing image classification performance by promoting the convergence of the loss minimum to a flatter loss surface, the perturbation loss (maximum loss within the neighborhood of a local minimum) of SAM fails to accurately measure the true sharpness of the loss landscape. Furthermore, its variants often overlook gradient conflicts, thereby limiting further improvement in DG performance. In this paper, we introduce implicit sharpness-aware minimization (ISAM), a novel method that addresses the deficiencies of SAM and mitigates gradient conflicts. Specifically, we demonstrate that the discrepancy in training loss during gradient ascent or descent serves as an equivalent measure of the dominant eigenvalue of the Hessian matrix. This discrepancy provides a reliable measure for sharpness. ISAM effectively reduces sharpness and mitigates potential conflicts between gradients by implicitly minimizing the discrepancy between training losses while ensuring a sufficiently low minimum through minimizing perturbation loss. Extensive experiments and analyses demonstrate that ISAM significantly enhances the model’s generalization ability on remote sensing and DG datasets, outperforming existing state-of-the-art methods.

From asylums to deinstitutionalization and after: an analytic review.

After a period of institution-based mental health care, in which the asylum system was the way in which the mental patients were treated, deinstitutionalization brought a set of significant changes and transformations in the conceptualization of mental illness and, by extension, the traditional therapeutic settings in which those in most need were assisted. However, this shift in the psychiatric domain was not only accompanied by valued achievements, but also by difficulties and challenges, as has been evidenced today. The aim of this paper is thus to examine the pros and cons of the closure of asylums, and the subsequent implementation of deinstitutionalization over the 60 years or so of such important transformations in the field of psychiatry. In considering this question, I examine in detail recent works of literature based on scholarly knowledge. In addition, I identify various issues involved, as well as ways of confronting these so as to attempt to overcome the difficulties they present. As I show here, the changes in the treatment and care of the mentally ill after asylum and deinstitutionalization brought a new air of hope to patients and their families, but also had undesirable effects. The paper also considers how mental health professionals today try to solve these effects on behalf of patients and society as a whole.