Domain Adaptation Approach Research Articles

WASPSYN: A Challenge for Domain Adaptive Synapse Detection in Microwasp Brain Connectomes.

The size of image volumes in connectomics studies now reaches terabyte and often petabyte scales with a great diversity of appearance due to different sample preparation procedures. However, manual annotation of neuronal structures (e.g., synapses) in these huge image volumes is time-consuming, leading to limited labeled training data often smaller than 0.001% of the large-scale image volumes in application. Methods that can utilize in-domain labeled data and generalize to out-of-domain unlabeled data are in urgent need. Although many domain adaptation approaches are proposed to address such issues in the natural image domain, few of them have been evaluated on connectomics data due to a lack of domain adaptation benchmarks. Therefore, to enable developments of domain adaptive synapse detection methods for large-scale connectomics applications, we annotated 14 image volumes from a biologically diverse set of Megaphragma viggianii brain regions originating from three different whole-brain datasets and organized the WASPSYN challenge at ISBI 2023. The annotations include coordinates of pre-synapses and post-synapses in the 3D space, together with their one-to-many connectivity information. This paper describes the dataset, the tasks, the proposed baseline, the evaluation method, and the results of the challenge. Limitations of the challenge and the impact on neuroscience research are also discussed. The challenge is and will continue to be available at https://codalab.lisn.upsaclay.fr/competitions/9169. Successful algorithms that emerge from our challenge may potentially revolutionize real-world connectomics research and further the cause that aims to unravel the complexity of brain structure and function.

Continual Unsupervised Domain Adaptation in Data-Constrained Environments

Domain Adaptation (DA) techniques aim to overcome the domain shift between the source domain used for training and the target domain where testing takes place. However, current DA methods assume that the entire target domain is available during adaptation, which may not hold in practice. We introduce a new, data-constrained DA paradigm where unlabeled target samples are received in batches and adaptation is performed continually. We propose a novel source-free method for continual unsupervised domain adaptation that utilizes a buffer for selective replay of previously seen samples. In our continual DA framework, we selectively mix samples from incoming batches with data stored in a buffer using buffer management strategies and use the combination to incrementally update our model. We evaluate and compare the classification performance of the continual DA approach with state-of-the-art (SOTA) DA methods based on the entire target domain. Our results on three popular DA datasets demonstrate the benefits of our method when operating in data constrained environments. We further extend our experiments to adapting over multiple target domains and our method performs favorably with the SOTA methods.

Self-Supervised Autoregressive Domain Adaptation for Time Series Data.

Unsupervised domain adaptation (UDA) has successfully addressed the domain shift problem for visual applications. Yet, these approaches may have limited performance for time series data due to the following reasons. First, they mainly rely on the large-scale dataset (i.e., ImageNet) for source pretraining, which is not applicable for time series data. Second, they ignore the temporal dimension on the feature space of the source and target domains during the domain alignment step. Finally, most of the prior UDA methods can only align the global features without considering the fine-grained class distribution of the target domain. To address these limitations, we propose a SeLf-supervised AutoRegressive Domain Adaptation (SLARDA) framework. In particular, we first design a self-supervised (SL) learning module that uses forecasting as an auxiliary task to improve the transferability of source features. Second, we propose a novel autoregressive domain adaptation technique that incorporates temporal dependence of both source and target features during domain alignment. Finally, we develop an ensemble teacher model to align class-wise distribution in the target domain via a confident pseudo labeling approach. Extensive experiments have been conducted on three real-world time series applications with 30 cross-domain scenarios. The results demonstrate that our proposed SLARDA method significantly outperforms the state-of-the-art approaches for time series domain adaptation. Our source code is available at: https://github.com/mohamedr002/SLARDA.

A Virtual-Label-Based Hierarchical Domain Adaptation Method for Time-Series Classification.

Unsupervised domain adaptation (UDA) is becoming a prominent solution for the domain-shift problem in many time-series classification tasks. With sequence properties, time-series data contain both local and sequential features, and the domain shift exists in both features. However, conventional UDA methods usually cannot distinguish those two features but mix them into one variable for direct alignment, which harms the performance. To address this problem, we propose a novel virtual-label-based hierarchical domain adaptation (VLH-DA) approach for time-series classification. Specifically, we first slice the original time-series data and introduce virtual labels to represent the type of each slice (called local patterns). With the help of virtual labels, we decompose the end-to-end (i.e., signal to time-series label) time-series task into two parts, i.e., signal sequence to local pattern sequence and local pattern sequence to time-series label. By decomposing the complex time-series UDA task into two simpler subtasks, the local features and sequential features can be aligned separately, making it easier to mitigate distribution discrepancies. Experiments on four public time-series datasets demonstrate that our VLH-DA outperforms all state-of-the-art (SOTA) methods.

Increasing the Robustness of Deep Learning Models for Object Segmentation: A Framework for Blending Automatically Annotated Real and Synthetic Data.

Recent problems in robotics can sometimes only be tackled using machine learning technologies, particularly those that utilize deep learning (DL) with transfer learning. Transfer learning takes advantage of pretrained models, which are later fine-tuned using smaller task-specific datasets. The fine-tuned models must be robust against changes in environmental factors such as illumination since, often, there is no guarantee for them to be constant. Although synthetic data for pretraining has been shown to enhance DL model generalization, there is limited research on its application for fine-tuning. One limiting factor is that the generation and annotation of synthetic datasets can be cumbersome and impractical for the purpose of fine-tuning. To address this issue, we propose two methods for automatically generating annotated image datasets for object segmentation, one for real-world and another for synthetic images. We also introduce a novel domain adaptation approach called filling the reality gap (FTRG), which can blend elements from real-world and synthetic scenes in a single image to achieve domain adaptation. We demonstrate through experimentation on a representative robot application that FTRG outperforms other domain adaptation techniques, such as domain randomization or photorealistic synthetic images, in creating robust models. Furthermore, we evaluate the benefits of using synthetic data for fine-tuning in transfer learning and continual learning with experience replay using our proposed methods and FTRG. Our findings indicate that fine-tuning with synthetic data can produce superior results compared to solely using real-world data.

Learning to Discover Knowledge: A Weakly-Supervised Partial Domain Adaptation Approach.

Domain adaptation has shown appealing performance by leveraging knowledge from a source domain with rich annotations. However, for a specific target task, it is cumbersome to collect related and high-quality source domains. In real-world scenarios, large-scale datasets corrupted with noisy labels are easy to collect, stimulating a great demand for automatic recognition in a generalized setting, i.e., weakly-supervised partial domain adaptation (WS-PDA), which transfers a classifier from a large source domain with noises in labels to a small unlabeled target domain. As such, the key issues of WS-PDA are: 1) how to sufficiently discover the knowledge from the noisy labeled source domain and the unlabeled target domain, and 2) how to successfully adapt the knowledge across domains. In this paper, we propose a simple yet effective domain adaptation approach, termed as self-paced transfer classifier learning (SP-TCL), to address the above issues, which could be regarded as a well-performing baseline for several generalized domain adaptation tasks. The proposed model is established upon the self-paced learning scheme, seeking a preferable classifier for the target domain. Specifically, SP-TCL learns to discover faithful knowledge via a carefully designed prudent loss function and simultaneously adapts the learned knowledge to the target domain by iteratively excluding source examples from training under the self-paced fashion. Extensive evaluations on several benchmark datasets demonstrate that SP-TCL significantly outperforms state-of-the-art approaches on several generalized domain adaptation tasks. Code is available at https://github.com/mc-lan/SP-TCL.

Deep Domain Adaptation Approach Using an Improved Parallel Residual Network for Cross‐Domain Bearing Fault Diagnosis

Recently, bearing fault diagnosis based on transfer learning (TL) has been a hot topic, which has attracted widespread interest due to its ability to adapt bearing fault datasets with different feature distributions. However, existing research suffer from low diagnosis efficiency and poor generalization capabilities. Therefore, an improved parallel residual network‐domain adaptation (IPRN‐DA) method for bearing fault diagnosis is proposed in this paper, which is to address these challenges. Firstly, a parallel residual block (PRB) is designed to extract critical features that can fully characterize the original signals without significantly increasing the model parameters and more attention is paid to them. Secondly, a hybrid attention mechanism (HAM) is constructed to adaptively integrate channel and spatial features to enhance fault feature information. Finally, multikernel maximum mean discrepancy (MK‐MMD) is employed to measure the distribution difference between the source and target domains in advanced feature extraction and predicted label spaces, implementing high‐precision bearing transfer diagnosis. Rolling bearing datasets from Case Western Reserve University (CWRU) and Jiangnan University (JNU) are used to validate the effectiveness of the presented method. Experimental results illustrate that the algorithm can extract domain‐invariant features for different cross‐domain diagnosis tasks and thus improve fault diagnosis accuracy.

Rolling Element Bearing Degradation Prediction Using Dynamic Model and an Improved Adversarial Domain Adaptation Approach

Rolling Element Bearing Degradation Prediction Using Dynamic Model and an Improved Adversarial Domain Adaptation Approach

MLFA: Towards Realistic Test Time Adaptive Object Detection by Multi-level Feature Alignment.

Object detection methods have achieved remarkable performances when the training and testing data satisfy the assumption of i.i.d. However, the training and testing data may be collected from different domains, and the gap between the domains can significantly degrade the detectors. Test Time Adaptive Object Detection (TTA-OD) is a novel online approach that aims to adapt detectors quickly and make predictions during the testing procedure. TTA-OD is more realistic than the existing unsupervised domain adaptation and source-free unsupervised domain adaptation approaches. For example, self-driving cars need to improve their perception of new environments in the TTA-OD paradigm during driving. To address this, we propose a multi-level feature alignment (MLFA) method for TTA-OD, which is able to adapt the model online based on the steaming target domain data. For a more straightforward adaptation, we select informative foreground and background features from image feature maps and capture their distributions using probabilistic models. Our approach includes: i) global-level feature alignment to align all informative feature distributions, thereby encouraging detectors to extract domain-invariant features, and ii) cluster-level feature alignment to match feature distributions for each category cluster across different domains. Through the multi-level alignment, we can prompt detectors to extract domain-invariant features, as well as align the category-specific components of image features from distinct domains. We conduct extensive experiments to verify the effectiveness of our proposed method. Our code is accessible at https://github.com/yaboliudotug/MLFA.

StAC-DA: Structure aware cross-modality domain adaptation framework with image and feature-level adaptation for medical image segmentation.

Convolutional neural networks (CNNs) have achieved state-of-the-art results in various medical image segmentation tasks. However, CNNs often assume that the source and target dataset follow the same probability distribution and when this assumption is not satisfied their performance degrades significantly. This poses a limitation in medical image analysis, where including information from different imaging modalities can bring large clinical benefits. In this work, we present an unsupervised Structure Aware Cross-modality Domain Adaptation (StAC-DA) framework for medical image segmentation. StAC-DA implements an image- and feature-level adaptation in a sequential two-step approach. The first step performs an image-level alignment, where images from the source domain are translated to the target domain in pixel space by implementing a CycleGAN-based model. The latter model includes a structure-aware network that preserves the shape of the anatomical structure during translation. The second step consists of a feature-level alignment. A U-Net network with deep supervision is trained with the transformed source domain images and target domain images in an adversarial manner to produce probable segmentations for the target domain. The framework is evaluated on bidirectional cardiac substructure segmentation. StAC-DA outperforms leading unsupervised domain adaptation approaches, being ranked first in the segmentation of the ascending aorta when adapting from Magnetic Resonance Imaging (MRI) to Computed Tomography (CT) domain and from CT to MRI domain. The presented framework overcomes the limitations posed by differing distributions in training and testing datasets. Moreover, the experimental results highlight its potential to improve the accuracy of medical image segmentation across diverse imaging modalities.

Adversarial Domain Adaptation Approach for Axial Piston Pump Fault Diagnosis Under Small Sample Condition Based on Measured and Simulated Signals

Adversarial Domain Adaptation Approach for Axial Piston Pump Fault Diagnosis Under Small Sample Condition Based on Measured and Simulated Signals

A dual-weight domain adversarial network for partial domain fault diagnosis of feedwater heater system

Domain adaptation (DA) approaches have received significant attention in industrial cross-domain fault diagnosis. However, the scarcity of sufficient labeled fault data, particularly under varying loading conditions and harsh operational environments, can give rise to distinct label spaces between two domains, thereby impeding the application of DA-based diagnosis methods. In this paper, we propose a novel dual-weight domain adversarial network (DWDAN) for diagnosing partial domain faults of feedwater heater system in a large-scale power unit, where the target label space is a subset of the source domain. Firstly, domain adversarial network with an instance-based feature learning strategy is constructed to capture domain-invariant and class-discriminative features hidden in raw process data, thereby enhancing feature extraction and generalization abilities of fault diagnosis. Furthermore, a dual-stage reweighted induction module is designed to quantify the contribution of samples from both class-level and sample-level for selective adaptation. This module can automatically eliminate outlier fault categories in the source domain and facilitates alignment of feature distributions for shared fault categories. Comprehensive experiments conducted on the feedwater heater system of a 600 MW coal-fired generating unit demonstrate the outstanding performance of DWDAN.

Weakly supervised end-to-end domain adaptation for person re-identification

Unsupervised domain adaptation (UDA) approaches for person re-identification (Re-ID) heavily rely on accurate pedestrian location, which is often unavailable in real-world scenarios. With respect to this situation, a two-step approach is commonly employed. However, this approach involves a complex process of disentangling UDA person Re-ID and pedestrian location, which consumes significant time and resources. This paper introduces a novel approach called Weakly Supervised End-to-End Domain Adaptation for Person Re-ID (WS-PreID) to address this problem. WS-PreID is designed to perform weakly supervised end-to-end person Re-ID by integrating pedestrian location and UDA person Re-ID in a unified manner. Our framework is a weakly supervised end-to-end network (WS-net) that operates solely on pedestrian bounding boxes. This approach mitigates the impact of background noise on the overall performance. Additionally, WS-PreID introduces a pseudo-label generation strategy with feature invariance (PLFI) to calculate pseudo-labels for person Re-ID training. PLFI utilizes both current features and comprehensive features to improve the precision of the assigned pseudo-labels. To evaluate the effectiveness of WS-PreID, we conducted comparative evaluations using the PRW and CUHK-SYSU datasets. The experimental results validate that WS-PreID achieves an end-to-end solution and significantly improves the performance of person Re-ID.

Optimal domain adaptive object detection with self-training and adversarial-based approach for construction site monitoring

In practice, object detection models used for construction site monitoring exhibit performance degradation owing to different monitoring settings and dynamic construction environments. Collecting and labeling images from new construction sites is necessary to mitigate these challenges; however, this requires significant resources. This paper presents an optimal domain-adaptive object detection model that employs an unsupervised domain adaptation approach to address the performance degradation challenges. Experiments were conducted at two construction sites to validate and identify the most effective domain-adaptive object detection model for construction sites. Several domain-adaptive object detection models have been developed, and our experimental results demonstrate that a hybrid approach, combining self-training and adversarial-based approaches improves the mean average precision by 12.8 and 7.1 over that of the baseline model at each construction site. These findings underscore the potential of domain adaptation-based methods to train object detection models in the construction domain, offering improved performance and reduced labeling efforts at specific target construction sites.

WCAL: Weighted and center-aware adaptation learning for partial domain adaptation

Partial domain adaptation, which aims to transfer knowledge from a source domain with rich labels to a unlabeled target domain where target class space is a subspace of source class space, is a challenging task in pattern recognition. Previous partial domain adaptation approaches tend to immerse in filtering out anomaly categories by weighting and the importance of ignoring the transferability of generated features. In the light of this, this article proposes a novel partial domain adaptation method, dubbed Weighted and Center-aware Adaptation Learning (WCAL). Specifically, WCAL presents a weighted adversarial learning module learns a category classifier to filter out the outlier categories from source domain. Also, it seeks a domain discriminator for cross-domain to further address the negative transfer. Then, the Center-aware adaptation learning module minimizes the distribution discrepancy across domains, which makes the features more transferability for the adaptation model. Extensive experiments on popular domain adaptation datasets testify that the proposed WCAL approach exceeds state-of-the-art baselines significantly with a large margin, in terms of average classification result, for example, 3.36% and 1.71% on Office-Home and Office-Caltech, respectively.

A novel generalized source-free domain adaptation approach for cross-domain industrial fault diagnosis

Domain adaptation has been widely applied in data-driven fault diagnosis tasks to address the domain shift problem between source and target data. However, conventional domain adaptation methods require both domains to be known, which is not always feasible due to privacy concerns and big data transmission. To overcome this limitation, a dedicated method called source-free domain adaptation (SFDA) has been developed to ensure reliable performance without relying on source data during target model adaptation. SFDA can achieve accurate classification tasks under domain shift problems and source data-free scenarios. We propose a generalized source model with manifold Mixup data augmentation and label smoothing techniques to avoid overfitting during the source model training. Based on this model, a novel self-training framework is proposed to implement the domain adaptation task and achieve accurate prediction performance. The experimental results from three real-world datasets demonstrate the effectiveness of the proposed approach.

Towards Minimizing the LiDAR Sim-to-Real Domain Shift: Object-Level Local Domain Adaptation for 3D Point Clouds of Autonomous Vehicles.

Perception algorithms for autonomous vehicles demand large, labeled datasets. Real-world data acquisition and annotation costs are high, making synthetic data from simulation a cost-effective option. However, training on one source domain and testing on a target domain can cause a domain shift attributed to local structure differences, resulting in a decrease in the model's performance. We propose a novel domain adaptation approach to address this challenge and to minimize the domain shift between simulated and real-world LiDAR data. Our approach adapts 3D point clouds on the object level by learning the local characteristics of the target domain. A key feature involves downsampling to ensure domain invariance of the input data. The network comprises a state-of-the-art point completion network combined with a discriminator to guide training in an adversarial manner. We quantify the reduction in domain shift by training object detectors with the source, target, and adapted datasets. Our method successfully reduces the sim-to-real domain shift in a distribution-aligned dataset by almost 50%, from 8.63% to 4.36% 3D average precision. It is trained exclusively using target data, making it scalable and applicable to adapt point clouds from any source domain.

Unsupervised domain adaptation for brain structure segmentation via mutual information maximization alignment

Brain structure segmentation in Magnetic Resonance image (MRI) plays an important role in the diagnosis of various neuropsychiatric diseases. Recently, the high cost of manual annotation has motivated research in medical image analysis to focus on unsupervised learning methods. As a promising solution, unsupervised domain adaptation (UDA) has received increasing attention in various medical image segmentation tasks. In this work, we propose a novel UDA framework for brain structure segmentation from MRI volumes. Specifically, we design a self-training-based network architecture to learn cross-domain representations, which is achieved by transferring essential knowledge from a well-trained cumbersome teacher network to a non-trained compact student network using distillation learning technology. The pseudo-labels generated by the pre-trained student model are used to supervise the domain adaptation process of the teacher model. Furthermore, we develop a mutual information maximization alignment module to learn domain-invariant features and cross-domain segmentation knowledge, in which contrastive learning techniques are utilized to maximize the correlation between samples. Finally, we build a unified distillation framework by integrating mutual information distillation and output map distillation, correcting pseudo-labels during the self-training stage, and continuously refining the final predictions of compact networks. Extensive experiments conducted on two datasets with vastly different styles and settings demonstrate that our method is effective in improving segmentation performance on unlabeled target images, and outperforms the state-of-the-art domain adaptation approaches by a large margin. The source code is available at https://github.com/huqian999/UDA-MIMA.

Utilization of deep learning tools to map and monitor biological soil crusts

Biological soil crusts (biocrusts) form a layer of only one to few centimeters depth on the soil surface and occur mostly in hot and cold deserts. Biocrusts have a major impact on different processes in these ecosystems, like carbon and nitrogen cycling, biodiversity preservation, erosion protection and soil dust emission reduction, but also react highly sensitive upon climate alterations and land use intensification. Therefore, monitoring tools are required to keep track of the changes of these specialized communities in an altering environment. In the current study, we applied a semantic image segmentation approach, using neural networks. One main problem to be solved was, that the training data and target data, on which the model is applied, are often recorded with different camera devices. This leads to different statistical properties of the image data, like different scale, resolution, brightness etc., which could significantly affect the model's performance. To solve this problem, we propose a new domain adaption method using a joint energy-based approach. To test a semantic segmentation approach in general, we utilized biocrust imagery taken in Utah (United States of America) and two sub datasets from the National Park Gesäuse (Austria). Here, we achieved highly reliable results with an overall classification accuracy of 85.9% for the USA data and 88.6% and 91.4%, respectively, for the two sub datasets of the National Park Gesäuse. To test our joint energy-based domain adaption approach, we used the two sub datasets from the National Park Gesäuse, which were recorded with different camera devices. With this newly established approach, we improved the accuracy of our segmentation on the unlabeled sub dataset from 70.4% to 75.3%. The results suggest that joint energy-based modelling is a well-suited domain adaption method for semantic segmentation that could be applied to face various deep learning and image-based biomonitoring challenges.

Zeroth- and first-order difference discrimination for unsupervised domain adaptation

Unsupervised domain adaptation transfers empirical knowledge from a label-rich source domain to a fully unlabeled target domain with a different distribution. A core idea of many existing approaches is to reduce the distribution divergence between domains. However, they focused only on part of the discrimination, which can be categorized into optimizing the following four objectives: reducing the intraclass distance between domains, enlarging the interclass distances between domains, reducing the intraclass distances within domains, and enlarging the interclass distances within domains. Moreover, because few methods consider multiple types of objectives, the consistency of data representations produced by different types of objectives has not yet been studied. In this paper, to address the above issues, we propose a zeroth- and first-order difference discrimination (ZFOD) approach for unsupervised domain adaptation. It first optimizes the above four objectives simultaneously. To improve the discrimination consistency of the data across the two domains, we propose a first-order difference constraint to align the interclass differences across domains. Because the proposed method needs pseudolabels for the target domain, we adopt a recent pseudolabel generation method to alleviate the negative impact of imprecise pseudolabels. We conducted an extensive comparison with nine representative conventional methods and seven remarkable deep learning-based methods on four benchmark datasets. Experimental results demonstrate that the proposed method, as a conventional approach, not only significantly outperforms the nine conventional comparison methods but is also competitive with the seven deep learning-based comparison methods. In particular, our method achieves an accuracy of 93.4% on the Office+Caltech10 dataset, which outperforms the other comparison methods. An ablation study further demonstrates the effectiveness of the proposed constraint in aligning the objectives.