Deep Domain Adaptation Research Articles

Deep domain adaptation network for transfer learning of state of charge estimation among batteries

Machine learning-based methods have been extensively explored for state of charge (SoC) estimation of lithium-ion batteries. However, for transfer learning among different batteries, most methods focus on the fine tuning of network weights while the domain feature adaptation from source set to target set is not explicitly explored. Here, a Deep Domain Adaptation Network (DDAN) is proposed for SoC estimation with transfer purpose among different batteries. The proposed method incorporates domain adversarial mechanism and maximum mean discrepancy (MMD) to get domain invariant features between source and target datasets. It can be used for both supervised and unsupervised learning depending on the availability of target labels. Experimental results on different batteries show the effectiveness of the proposed method with reduced error to over half compared with the direct prediction baseline, which can be attributed to the intrinsic domain adaptation mechanism between source and target batteries. Our proposed method outperforms the state of the art methods and can be generalized to other batteries.

Combining generative modelling and semi-supervised domain adaptation for whole heart cardiovascular magnetic resonance angiography segmentation

BackgroundQuantification of three-dimensional (3D) cardiac anatomy is important for the evaluation of cardiovascular diseases. Changes in anatomy are indicative of remodeling processes as the heart tissue adapts to disease. Although robust segmentation methods exist for computed tomography angiography (CTA), few methods exist for whole-heart cardiovascular magnetic resonance angiograms (CMRA) which are more challenging due to variable contrast, lower signal to noise ratio and a limited amount of labeled data. MethodsTwo state-of-the-art unsupervised generative deep learning domain adaptation architectures, generative adversarial networks and variational auto-encoders, were applied to 3D whole heart segmentation of both conventional (n = 20) and high-resolution (n = 45) CMRA (target) images, given segmented CTA (source) images for training. An additional supervised loss function was implemented to improve performance given 10%, 20% and 30% segmented CMRA cases. A fully supervised nn-UNet trained on the given CMRA segmentations was used as the benchmark. ResultsThe addition of a small number of segmented CMRA training cases substantially improved performance in both generative architectures in both standard and high-resolution datasets. Compared with the nn-UNet benchmark, the generative methods showed substantially better performance in the case of limited labelled cases. On the standard CMRA dataset, an average 12% (adversarial method) and 10% (variational method) improvement in Dice score was obtained. ConclusionsUnsupervised domain-adaptation methods for CMRA segmentation can be boosted by the addition of a small number of supervised target training cases. When only few labelled cases are available, semi-supervised generative modelling is superior to supervised methods.

Deep learning-based open set domain adaptation for plant disease recognition across multiple greenhouse scenarios

Deep learning-based open set domain adaptation for plant disease recognition across multiple greenhouse scenarios

Prototype-guided multi-scale domain adaptation for Alzheimer's disease detection

Alzheimer's disease (AD) is the most common form of dementia and there is no effective treatment currently. Using artificial intelligence technology to assist the diagnosis and intervention as early as possible is of great significance to delay the development of AD. Structural Magnetic Resonance Imaging (sMRI) has shown great practical values on computer-aided AD diagnosis. Affected by data from different sources or acquisition domains in realistic scenarios, MRI data often suffer from domain shift problem. In this paper, we propose a deep Prototype-Guided Multi-Scale Domain Adaptation (PMDA) framework to handle MRI data with domain shift problem, and realize automatic auxiliary diagnosis of AD, Mild Cognitive Impairment (MCI) and Cognitively Normal (CN). PMDA is composed of three modules: (1) MRI multi-scale feature extraction module combines the advantages of 3D convolution and self-attention to effectively extract multi-scale features in high-dimensional space, (2) Prototype Maximum Density Divergence (Pro-MDD) module adopts prototype learning to constrain the feature outlier samples in a mini-batch when MDD is used to align source domain and target domain, and (3) Adversarial Domain Adaptation module is applied to achieve global feature alignment of the source domain and target domain and co-training two distinctive discriminators to mitigate the over-fitting issue. Experiments have been performed on 3T and 1.5T sMRI with domain shift in ADNI dataset. The experimental results demonstrated that the proposed framework PMDA outperforms supervised learning methods and several state-of-the-art domain adaptation methods and achieves a superior accuracy of 92.11%, 76.01% and 82.37% on AD vs. CN, AD vs. MCI, and MCI vs. CN tasks, respectively.

Image‐based taxonomic classification of bulk insect biodiversity samples using deep learning and domain adaptation

AbstractComplex bulk samples of insects from biodiversity surveys present a challenge for taxonomic identification, which could be overcome by high‐throughput imaging combined with machine learning for rapid classification of specimens. These procedures require that taxonomic labels from an existing source data set are used for model training and prediction of an unknown target sample. However, such transfer learning may be problematic for the study of new samples not previously encountered in an image set, for example, from unexplored ecosystems, and require methods of domain adaptation that reduce the differences in the feature distribution of the source and target domains (training and test sets). We assessed the efficiency of domain adaptation for family‐level classification of bulk samples of Coleoptera, as a critical first step in the characterization of biodiversity samples. Neural network models trained with images from a global database of Coleoptera were applied to a biodiversity sample from understudied forests in Cyprus as the target. Within‐dataset classification accuracy reached 98% and depended on the number and quality of training images, and on dataset complexity. The accuracy of between‐datasets predictions (across disparate source–target pairs that do not share any species or genera) was at most 82% and depended greatly on the standardization of the imaging procedure. An algorithm for domain adaptation, domain adversarial training of neural networks (DANN), significantly improved the prediction performance of models trained by non‐standardized, low‐quality images. Our findings demonstrate that existing databases can be used to train models and successfully classify images from unexplored biota, but the imaging conditions and classification algorithms need careful consideration.

Deep Domain Adaptation Using Cascaded Learning Networks and Metric Learning

How can we bridge efficiently distribution difference between the source and target domains in an isomorphic latent feature space by using metric learning. This study introduces metric learning on manifolds which combine a cascaded learning network and a metric learning model to form a Unified Domain Adaptation Model. Our approach is based on formulating a transfer from the source to the target as a geometric mean metric learning problem on manifolds. The solution of symmetric positive definite covariance matrices not only reduces the statistical differences between the source and target domains, but also the underlying geometry of the source and target domains using diffusions on the underlying source and target manifolds. By retaining both the nonlinear structure of the Riemannian geometry of the open cone of symmetric positive definite matrices and cascaded learning networks, we improve the state-of-the-art results on the Amazon (A), Caltech256 (C), DSLR (D), Webcam (W) and VisDA benchmark datasets by knowledge transfer, while achieving comparable performances to competing methods on domain adaptation modeling.

A novel deep domain adaptation method for automated detection of sleep apnea/hypopnea events

Objective. Sleep apnea–hypopnea syndrome (SAHS) is a common sleep-related respiratory disorder that is generally assessed for severity using polysomnography (PSG); however, the diversity of sampling devices and patients makes this not only costly but may also degrade the performance of the algorithms. Approach. This paper proposes a novel deep domain adaptation module which uses a long short-term memory–convolutional neural network embedded with the channel attention mechanism to achieve autonomous extraction of high-quality features. Meanwhile, a domain adaptation module was built to achieve domain-invariant feature extraction for reducing the differences in data distribution caused by different devices and other factors. In addition, during the training process, the algorithm used the last second label as the label of the PSG segment, so that second-by-second evaluation of respiratory events could be achieved. Main results. The algorithm applied the two datasets provided by PhysioNet as the source and target domains. The accuracy, sensitivity and specificity of the algorithm on the source domain were 86.46%, 86.11% and 93.17%, respectively, and on the target domain were 83.63%, 82.52%, 91.62%, respectively. The proposed algorithm showed strong generalization ability and the classification results were comparable to the current advanced methods. Besides, the apnea–hypopnea index values estimated by the proposed algorithm showed a high correlation with the manual scoring values on both domains. Significance. The proposed algorithm can effectively perform SAHS detection and evaluation with certain generalization.

Deep domain adaptation, pseudo-labeling, and shallow network for accurate and fast gait prediction of unlabeled datasets.

Developing personalized gait phase prediction models is difficult because acquiring accurate gait phases requires expensive experiments. This problem can be addressed via semi-supervised domain adaptation (DA), which minimizes the discrepancy between the source and target subject features. However, classical DA models have a trade-off between accuracy and inference speed. Whereas deep DA models provide accurate prediction results with a slow inference speed, shallow DA models produce less accurate results with a fast inference speed. To achieve both high accuracy and fast inference, a dual-stage DA framework is proposed in this study. The first stage uses a deep network for precise DA. Then, a pseudo-gait-phase label of the target subject is obtained using the first-stage model. In the second stage, a shallow but fast network is trained using the pseudo-label. Because computation for DA is not conducted in the second stage, an accurate prediction can be accomplished even with the shallow network. Test results show that the proposed DA framework reduces the prediction error by 1.04% compared with a shallow DA model while maintaining its fast inference speed. The proposed DA framework can be used to provide fast personalized gait prediction models for real-time control systems such as wearable robots.

Instance Weighting-Based Partial Domain Adaptation for Intelligent Fault Diagnosis of Rotating Machinery

The cross-domain fault diagnosis problem based on deep domain adaptation (deep DA) has gained great attention in recent years. However, a required but not easily satisfied assumption in many researches that the label space of the source domain and the target domain should be identical limits their applications in practice. In industrial reality, the label space of the target domain may be only a subset of that of the source domain, which is defined as partial DA diagnosis scenario. Focusing on this scenario, this paper proposes a novel diagnosis method named instance weighted mean maximum discrepancy (IWMMD). A new weighting mechanism inspired by instance discrimination is designed to realize DA on shared label space between domains. Also, discrimination structure enhancement for both domains is introduced to encourage better classification ability and safer domain alignment. The effectiveness of IWMMD is verified by two datasets. In the gear dataset, the diagnosis accuracy is 89.65% with a 5.11% improvement. In the bearing dataset, the diagnosis accuracy is 96.28% with a 4.46% improvement. The results and analysis show that the proposed method can reduce the negative transfer effects caused by outlier class samples in source domain and learn a more separable discrimination structure, which is effective in both no-partial and partial diagnosis scenarios and time-efficient.

SAR Ship Detection Based on Deep Domain Adaptation with Limited Samples

Ship detection in synthetic aperture radar (SAR) images have important applications in military and civil fields. Nonetheless, SAR images usually suffer from the disadvantage of having limited labeled samples. This is undoubtedly a challenge for the deep learning based methods, which require a large number of samples for training. To solve this problem, many researchers have applied the domain adaptation (DA) methods for the SAR ship detection tasks, which use optical images as the source domain and transform optical images to SAR domain directly. However, there is large difference between SAR and optical images. Hence, the deep learning detector may not be able to learn the target features accurately when the transformed images are used for training. In order to solve above problem, a novel ship detection method with limited labeled samples is proposed, whose training and test process is mainly implemented in the optical domain. In the training stage, optical images are used to pre-train the model. Here, the transformed optical images from SAR images are used to further fine-tune the pre-trained model. Accordingly, in the test stage, SAR images are transformed to optical domain and input in the model to obtain the detection results. Experimental results based on the real dataset demonstrate the effectiveness of the proposed method for SAR ship detection.

Self-Supervised Plant Phenotyping by Combining Domain Adaptation with 3D Plant Model Simulations: Application to Wheat Leaf Counting at Seedling Stage.

The number of leaves at a given time is important to characterize plant growth and development. In this work, we developed a high-throughput method to count the number of leaves by detecting leaf tips in RGB images. The digital plant phenotyping platform was used to simulate a large and diverse dataset of RGB images and corresponding leaf tip labels of wheat plants at seedling stages (150,000 images with over 2 million labels). The realism of the images was then improved using domain adaptation methods before training deep learning models. The results demonstrate the efficiency of the proposed method evaluated on a diverse test dataset, collecting measurements from 5 countries obtained under different environments, growth stages, and lighting conditions with different cameras (450 images with over 2,162 labels). Among the 6 combinations of deep learning models and domain adaptation techniques, the Faster-RCNN model with cycle-consistent generative adversarial network adaptation technique provided the best performance (R2 = 0.94, root mean square error = 8.7). Complementary studies show that it is essential to simulate images with sufficient realism (background, leaf texture, and lighting conditions) before applying domain adaptation techniques. Furthermore, the spatial resolution should be better than 0.6 mm per pixel to identify leaf tips. The method is claimed to be self-supervised since no manual labeling is required for model training. The self-supervised phenotyping approach developed here offers great potential for addressing a wide range of plant phenotyping problems. The trained networks are available at https://github.com/YinglunLi/Wheat-leaf-tip-detection.

Domain Adapted Multitask Learning for Segmenting Amoeboid Cells in Microscopy.

The method proposed in this paper is a robust combination of multi-task learning and unsupervised domain adaptation for segmenting amoeboid cells in microscopy. A highlight of this work is the manner in which the model's hyperparameters are estimated. The detriments of ad-hoc parameter estimation are well known, but this issue remains largely unaddressed in the context of CNN-based segmentation. Using a novel min-max formulation of the segmentation cost function our proposed method analytically estimates the model's hyperparameters, while simultaneously learning the CNN weights during training. This end-to-end framework provides a consolidated mechanism to harness the potential of multi-task learning to isolate and segment clustered cells from low contrast brightfield images, and it simultaneously leverages deep domain adaptation to segment fluorescent cells without explicit pixel-level re- annotation of the data. Experimental validations on multi-cellular images strongly suggest the effectiveness of the proposed technique, and our quantitative results show at least 15% and 10% improvement in cell segmentation on brightfield and fluorescence images respectively compared to contemporary supervised segmentation methods.

A cross-domain fruit classification method based on lightweight attention networks and unsupervised domain adaptation

AbstractImage-based fruit classification offers many useful applications in industrial production and daily life, such as self-checkout in the supermarket, automatic fruit sorting and dietary guidance. However, fruit classification task will have different data distributions due to different application scenarios. One feasible solution to solve this problem is to use domain adaptation that adapts knowledge from the original training data (source domain) to the new testing data (target domain). In this paper, we propose a novel deep learning-based unsupervised domain adaptation method for cross-domain fruit classification. A hybrid attention module is proposed and added to MobileNet V3 to construct the HAM-MobileNet that can suppress the impact of complex backgrounds and extract more discriminative features. A hybrid loss function combining subdomain alignment and implicit distribution metrics is used to reduce domain discrepancy during model training and improve model classification performance. Two fruit classification datasets covering several domains are established to simulate common industrial and daily life application scenarios. We validate the proposed method on our constructed grape classification dataset and general fruit classification dataset. The experimental results show that the proposed method achieves an average accuracy of 95.0% and 93.2% on the two datasets, respectively. The classification model after domain adaptation can well overcome the domain discrepancy brought by different fruit classification scenarios. Meanwhile, the proposed datasets and method can serve as a benchmark for future cross-domain fruit classification research.

Video surveillance using deep transfer learning and deep domain adaptation: Towards better generalization

Recently, developing automated video surveillance systems (VSSs) has become crucial to ensure the security and safety of the population, especially during events involving large crowds, such as sporting events. While artificial intelligence (AI) smooths the path of computers to think like humans, machine learning (ML) and deep learning (DL) pave the way more, even by adding training and learning components. DL algorithms require data labeling and high-performance computers to effectively analyze and understand surveillance data recorded from fixed or mobile cameras installed in indoor or outdoor environments. However, they might not perform as expected, take much time in training, or not have enough input data to generalize well. To that end, deep transfer learning (DTL) and deep domain adaptation (DDA) have recently been proposed as promising solutions to alleviate these issues. Typically, they can (i) ease the training process, (ii) improve the generalizability of ML and DL models, and (iii) overcome data scarcity problems by transferring knowledge from one domain to another or from one task to another. Although the increasing number of articles proposed to develop DTL- and DDA-based VSSs, a thorough review that summarizes and criticizes the state-of-the-art is still missing. To that end, this paper introduces, to the best of the authors’ knowledge, the first overview of existing DTL- and DDA-based video surveillance to (i) shed light on their benefits, (ii) discuss their challenges, and (iii) highlight their future perspectives.

Terahertz transfer characterization for composite delamination under variable conditions based on deep adversarial domain adaptation

For the reliability assessment of composite materials, the data-driven terahertz nondestructive evaluation (THz NDE) technique has emerged attractive potentials for automatic and intelligent identification of composite damages. Most previous data-drive methods depend on the sufficient labeled damage data and work under the strict assumption that the training data (source domain) and the testing data (target domain) must conform to similar distributions. However, in the actual THz NDE scenario, due to different testing conditions such as environmental changes, equipment instability and system noise etc., the collected THz signals may exhibit different variations in both of the amplitude and phase, which will result in the domain shift between the source domain and target domain and thus highly degrade the robustness and generalization performance of data-driven methods. A large training dataset can alleviate this issue, however the collection and labeling of the dataset composed of all possible damage scenarios is cumbersome and inaccessible for the practical industrial applications. Herein, an intelligent THz 3D characterization system based on the deep adversarial domain adaptation (DADA) strategy is proposed to automatically locate and image the hidden delamination defects in the composite under different operation conditions. Specifically, the unsupervised CNN-DADA model is established to address the domain shift problem between different THz datasets by adversarial learning. A series of experiments verify the superior generalization performance of the CNN-DADA model on different THz datasets. Compared with the traditional deep learning models, our proposed method can fulfill the automatic localization and imaging of delamination defects with high accuracy and resolution even for the THz datasets with significant distribution discrepancies, which will further facilitate the deployment of the data-driven THz intelligent characterization in practical industrial scenarios.

Improving Synthetic to Realistic Semantic Segmentation With Parallel Generative Ensembles for Autonomous Urban Driving

Semantic segmentation is paramount for autonomous vehicles to have a deeper understanding of the surrounding traffic environment and enhance safety. Deep neural networks (DNNs) have achieved remarkable performances in semantic segmentation. However, training such a DNN requires a large amount of labeled data at the pixel level. In practice, it is a labor-intensive task to manually annotate dense pixel-level labels. To tackle the problem associated with a small amount of labeled data, deep domain adaptation (DDA) methods have recently been developed to examine the use of synthetic driving scenes so as to significantly reduce the manual annotation cost. Despite remarkable advances, these methods, unfortunately, suffer from the generalizability problem that fails to provide a holistic representation of the mapping from the source image domain to the target image domain. In this article, we, therefore, develop a novel ensembled DDA to train models with different upsampling strategies, discrepancy, and segmentation loss functions. The models are, therefore, complementary with each other to achieve better generalization in the target image domain. Such a design does not only improves the adapted semantic segmentation performance but also strengthens the model reliability and robustness. Extensive experimental results demonstrate the superiorities of our approach over several state-of-the-art methods.

Cross-domain vulnerability detection using graph embedding and domain adaptation

Vulnerability detection is an effective means to maintain cyberspace security. Machine learning methods have risen much attention in software security due to their advantage of accuracy and automation. However, current researches mainly focus on in-domain vulnerability detection where the training data and test data belong to the same domain. Due to application scenarios, coding habits, and other factors, vulnerabilities in different software projects may obey different probability distributions. This discrepancy compromises the performance of machine learning methods when they are applied to a brand-new project. To address this cold start problem, we propose a cross-domain vulnerability detection framework using graph embedding and deep domain adaption (VulGDA). It works in a variety of cross-domain fashions, including the Zero-Shot fashion that no labeled data in the target domain is available for training. VulGDA is decomposed to graph embedding and domain adaptation. At the graph embedding stage, we transform the samples in source code into graph representations where elements are directly concatenated according to their syntactic and semantic relationships. Then, we aggregate information from neighbors and edges defined in the graph into real-valued vectors. By graph embedding, VulGDA extracts comprehensive vulnerability features and compromises the challenge of long-term dependency. Aiming at the discrepancy between training data and test data, domain adaption is used to train a feature generator. This feature generator maps the graph embedding to a “deep” feature that is discriminative for vulnerability detection, and invariant to the shift between domains. We perform a systematic experiment to validate the effectiveness of VulGDA. The results show that combining graph embedding and deep domain adaptation promotes VulGDA's performance in cross-domain vulnerability detection. Compared with the state-of-the-art methods, our method has better performance under the cold start condition.

Deep adversarial domain adaptation for hyperspectral calibration model transfer among plant phenotyping systems

Hyperspectral imaging has widely been used for plant phenotyping applications. Calibration transfer is crucial for eliminating the prediction drift when a model developed in one plant phenotyping system needs to be deployed in another system. In this study two approaches (Domain Adversarial Neural Network - DANN and Adversarial Discriminative Domain Adaptation – ADDA) for transferring a deep learning model across hyperspectral imaging-based plant phenotyping systems were presented. The proposed techniques employ adversarial learning to minimise the discrepancy between the spectral data of two imaging facilities in the feature space. The calibration transfer analysis was examined by collecting the hyperspectral images of maize plants in two phenotyping facilities equipped with different sensors covering the visible near-infrared (VNIR) region under different lighting conditions. The capability of the suggested methods to transfer a relative water content (RWC) prediction model was investigated. The results showed that the prediction R 2 improved from 0.603 to 0.849 and 0.843 and RMSE reduced from 23.52% to 14.15% and 12.73% for DANN and ADDA, respectively. The study showed that both approaches improved the performance of the transferred model compared to a similar deep learning model without adversarial transfer. Visualisation of feature space showed that both methods lead to similar source and target feature distributions, thus indicating that a prediction model developed using these common features can successfully be applied to both imaging facilities. The main benefit of the DANN and ADDA is that they do not need the standard samples to be imaged in two phenotyping systems or the collection of labelled data to carry out the transfer. The adversarial learning-based calibration transfer can promote the sharing of prediction models across different phenotyping systems. • Differences across phenotyping facilities hinders the application of exiting models. • Calibration transfer between different phenotyping facilities is critically important. • Adversarial domain adaptation based methods were proposed for calibration transfer. • Proposed methods resulted in RWC model shared between two facilities. • Proposed methods can help to eliminate the need of full recalibration.

Improving the Robustness and Adaptability of sEMG-Based Pattern Recognition Using Deep Domain Adaptation.

The pattern recognition (PR) based on surface electromyography (sEMG) could improve the quality of daily life of amputees. However, the lack of robustness and adaptability hinders its practical application. To realize the long-term reliability and user adaptability simultaneously, a novel multi-task dual-stream supervised domain adaptation (MDSDA) network based on convolutional neural network (CNN) was proposed. A long-term multi-subject sEMG signal acquisition was conducted to validate the performance of MDSDA, recruiting 12 able-bodied subjects. A total of thirty gestures were used for the acquisition, including one set of static gestures and two sets of dynamic gestures. The long-term multi-subject sEMG dataset is publicly available at the website. Four train-test estimations were designed to evaluate the robustness and adaptability of MDSDA. The results showed that MDSDA outperformed CNN and fune-tuning. Furthermore, we studied the divisibility between static and dynamic gestures that performed similar actions. The outcomes demonstrated that there existed high separability between them. This may be helpful to reduce the signal collection burden. Experimental results proved MDSDA has the potential to provide a robust and generalized PR system for the clinic applications.

Adversarial Entropy Optimization for Unsupervised Domain Adaptation.

Domain adaptation is proposed to deal with the challenging problem where the probability distribution of the training source is different from the testing target. Recently, adversarial learning has become the dominating technique for domain adaptation. Usually, adversarial domain adaptation methods simultaneously train a feature learner and a domain discriminator to learn domain-invariant features. Accordingly, how to effectively train the domain-adversarial model to learn domain-invariant features becomes a challenge in the community. To this end, we propose in this article a novel domain adaptation scheme named adversarial entropy optimization (AEO) to address the challenge. Specifically, we minimize the entropy when samples are from the independent distributions of source domain or target domain to improve the discriminability of the model. At the same time, we maximize the entropy when features are from the combined distribution of source domain and target domain so that the domain discriminator can be confused and the transferability of representations can be promoted. This minimax regime is well matched with the core idea of adversarial learning, empowering our model with transferability as well as discriminability for domain adaptation tasks. Also, AEO is flexible and compatible with different deep networks and domain adaptation frameworks. Experiments on five data sets show that our method can achieve state-of-the-art performance across diverse domain adaptation tasks.