VAE deep learning model with domain adaptation, transfer learning and harmonization for diagnostic classification from multi-site neuroimaging data

The Internet of Things has a wide range of applications in the medical field. Due to the heterogeneity of medical data generated by different hospitals, it is very important to analyze and integrate data from different institutions. Functional magnetic resonance imaging (fMRI) is widely used in clinical medicine and cognitive neuroscience, while resting‐state fMRI (rs‐fMRI) can help reveal functional biomarkers of neurological disorders for computer‐assisted clinical diagnosis and prognosis. Recently, how to retrieve similar images or case histories from large‐scale medical image repositories acquired from multiple sites has attracted widespread attention in the field of intelligent diagnosis of diseases. Although using multisite data effectively helps increase the sample size, it also inevitably introduces the problem of data heterogeneity across sites. To address this problem, we propose a multisite fMRI retrieval (MSFR) method that uses a deep hashing approach and an optimal transport‐based domain adaptation strategy to mitigate multisite data heterogeneity for accurate fMRI search. Specifically, for a given target domain site and multiple source domain sites, our approach uses a deep neural network to map the source and target domain data into the latent feature space and minimize their Wasserstein distance to reduce their distribution differences. We then use the source domain data to learn high‐quality hash code through a global similarity metric, thereby improving the performance of cross‐site fMRI retrieval. We evaluated our method on the publicly available Autism Brain Imaging Data Exchange (ABIDE) dataset. Experimental results show the effectiveness of our method in resting‐state fMRI retrieval.

State-of-the-art deep learning models have demonstrated success in classifying facial expressions of adults by relying on large datasets of labeled images. Unfortunately, there is a scarcity of labeled images of child expressions. Deep learning models trained on adult data do not generalize well on child data due to the domain shift caused by morphological differences in their faces. Recent deep domain adaptation approaches align the data distribution of a target domain with the source domain using a few target domain samples. We propose that the domain adaptation may be improved by incorporating steps of deep transfer learning, such as initialization with pre-trained source weights and freezing early layers of the model. The knowledge of a few labeled examples from the child data (target domain) is incorporated into the adult data distribution (source domain) using a contrastive semantic alignment (CSA) loss. This work combines deep transfer learning and domain adaptation approaches to generate seven expression labels (‘happy’, ‘sad’, ‘anger’, ‘fear’, ‘surprise’, ‘disgust’, plus ‘neutral’) for facial images of children in reference to the source domain, adult facial expressions, using 10 or fewer samples per expression. Our hybrid approach outperforms the transfer learning model by 12% on mean accuracy using only 10 samples per expression class.

Autism spectrum disorder (ASD) is a neurodevelopmental disorder that is characterized by a wide range of symptoms. Identifying biomarkers for accurate diagnosis is crucial for early intervention of ASD. While multi-site data increase sample size and statistical power, they suffer from inter-site heterogeneity. To address this issue, we propose a multi-site adaption framework via low-rank representation decomposition (maLRR) for ASD identification based on functional MRI (fMRI). The main idea is to determine a common low-rank representation for data from the multiple sites, aiming to reduce differences in data distributions. Treating one site as a target domain and the remaining sites as source domains, data from these domains are transformed (i.e., adapted) to a common space using low-rank representation. To reduce data heterogeneity between the target and source domains, data from the source domains are linearly represented in the common space by those from the target domain. We evaluated the proposed method on both synthetic and real multi-site fMRI data for ASD identification. The results suggest that our method yields superior performance over several state-of-the-art domain adaptation methods.

Optical network failure management (ONFM) is a promising application of machine learning (ML) to optical networking. Typical ML-based ONFM approaches exploit historical monitored data, retrieved in a specific domain (e.g., a link or a network), to train supervised ML models and learn failure characteristics (a signature) that will be helpful upon future failure occurrence in that domain. Unfortunately, in operational networks, data availability often constitutes a practical limitation to the deployment of ML-based ONFM solutions, due to scarce availability of labeled data comprehensively modeling all possible failure types. One could purposely inject failures to collect training data, but this is time consuming and not desirable by operators. A possible solution is transfer learning (TL), i.e., training ML models on a source domain (SD), e.g., a laboratory testbed, and then deploying trained models on a target domain (TD), e.g., an operator network, possibly fine-tuning the learned models by re-training with few TD data. Moreover, in those cases when TL re-training is not successful (e.g., due to the intrinsic difference in SD and TD), another solution is domain adaptation, which consists of combining unlabeled SD and TD data before model training. We investigate domain adaptation and TL for failure detection and failure-cause identification across different lightpaths leveraging real optical SNR data. We find that for the considered scenarios, up to 20% points of accuracy increase can be obtained with domain adaptation for failure detection, while for failure-cause identification, only combining domain adaptation with model re-training provides significant benefit, reaching 4%–5% points of accuracy increase in the considered cases.

Recently, domain adaptation based on deep models has been a promising way to deal with the domains with scarce labeled data, which is a critical problem for deep learning models. Domain adaptation propagates the knowledge from a source domain with rich information to the target domain. In reality, the source and target domains are mostly unbalanced in that the source domain is more resource-rich and thus has more reliable knowledge than the target domain. However, existing deep domain adaptation approaches often pre-assume the source and target domains balanced and equally, leading to a medium solution between the source and target domains, which is not optimal for the unbalanced domain adaptation. In this paper, we propose a novel Deep Asymmetric Transfer Network (DATN) to address the problem of unbalanced domain adaptation. Specifically, our model will learn a transfer function from the target domain to the source domain and meanwhile adapting the source domain classifier with more discriminative power to the target domain. By doing this, the deep model is able to adaptively put more emphasis on the resource-rich source domain. To alleviate the scarcity problem of supervised data, we further propose an unsupervised transfer method to propagate the knowledge from a lot of unsupervised data by minimizing the distribution discrepancy over the unlabeled data of two domains. The experiments on two real-world datasets demonstrate that DATN attains a substantial gain over state-of-the-art methods.

Over the past few years, there has been extensive exploration of machine learning (ML), especially deep learning (DL), for crop yield prediction, resulting in impressive levels of accuracy. However, such models are highly dependent on training samples with ground truth labels (i.e., crop yield records), which are not available in some regions. Additionally, due to the existence of domain shifts between different spatial regions, DL models trained within one region (i.e., source domain) tend to have poor performance when directly applied to other regions (i.e., target domain). Unsupervised domain adaptation (UDA) has become a promising strategy to improve the transferability of DL models by aligning the feature distributions in the source domain and the target domain. Despite the success, existing UDA models generally assume an identical label space across different domains. This assumption can be invalid in crop yield prediction scenarios, as crop yields can vary significantly in heterogeneous regions. Due to the mismatch between label spaces, negative transfer may occur if the entire source and target domains are forced to align. To address this issue, we proposed a novel partial domain adversarial neural network (PDANN), which relaxes the assumption of fully, equally shared label spaces across domains by downweighing the outlier source samples. Specifically, during model training, the PDANN weighs each labeled source sample based on the likelihood of its yield value given the expected target yield distribution. Instead of aligning the target domain to the entire source domain, the PDANN model downweighs the outlier source samples and performs partial weighted alignment of the target domain to the source domain. As a result, the negative transfer caused by source samples in the outlier label space would be alleviated. In this study, we assessed the model’s performance on predicting yields for two main commodities in the U.S., including corn and soybean, using the U.S. corn belt as the study region. The counties under study were divided into two distinct ecological zones and alternatively used as the source and target domains. Feature variables, including time-series vegetation indices (VIs) and sequential meteorological variables, were collected and aggregated at the county level. Next, the PDANN model was trained with the extracted features and corresponding crop yield records from the U.S. Department of Agriculture (USDA). Finally, the trained model was evaluated for three testing years from 2019 to 2021. The experimental results showed that the developed PDANN model had achieved a mean coefficient of determination (R2) of 0.70 and 0.67, respectively, in predicting corn and soybean yields, outperforming three other ML and UDA models by a large margin from 6% to 46%. As the first study performing partial domain adaptation for crop yield prediction, this research demonstrates a novel solution for addressing negative transfer and improving DL models’ transferability on crop yield prediction.

The performance of the supervised learning algorithms such as k-nearest neighbor (k-NN) depends on the labeled data. For some applications (Target Domain), obtaining such labeled data is very expensive and labor-intensive. In a real-world scenario, the possibility of some other related application (Source Domain) is always accompanied by sufficiently labeled data. However, there is a distribution discrepancy between the source domain and the target domain application data as the background of collecting both the domains data is different. Therefore, source domain application with sufficient labeled data cannot be directly utilized for training the target domain classifier. Domain Adaptation (DA) or Transfer learning (TL) provides a way to transfer knowledge from source domain application to target domain application. Existing DA methods may not perform well when there is a much discrepancy between the source and the target domain data, and the data is non-linear separable. Therefore, in this paper, we provide a Kernelized Unified Framework for Domain Adaptation (KUFDA) that minimizes the discrepancy between both the domains on linear or non-linear data-sets and aligns them both geometrically and statistically. The substantial experiments verify that the proposed framework outperforms state-of-the-art Domain Adaptation and the primitive methods (Non- Domain Adaptation) on real-world Office-Caltech and PIE Face data-sets. Our proposed approach (KUFDA) achieved mean accuracies of 86.83% and 74.42% for all possible tasks of Office-Caltech with VGG-Net features and PIE Face data-sets.

Recent progress in machine learning has been mainly due to the availability of large amounts of annotated data used for training complex models with deep architectures. Annotating this training data becomes burdensome and creates a major bottleneck in maintaining machine-learning databases. Moreover, these trained models fail to generalize to new categories or new varieties of the same categories. This is because new categories or new varieties have data distribution different from the training data distribution. To tackle these problems, this thesis proposes to develop a family of transfer-learning techniques that can deal with different training (source) and testing (target) distributions with the assumption that the availability of annotated data is limited in the testing domain. This is done by using the auxiliary data-abundant source domain from which useful knowledge is transferred that can be applied to data-scarce target domain. This transferable knowledge serves as a prior that biases target-domain predictions and prevents the target-domain model from overfitting. Specifically, we explore structural priors that encode relational knowledge between different data entities, which provides more informative bias than traditional priors. The choice of the structural prior depends on the information availability and the similarity between the two domains. Depending on the domain similarity and the information availability, we divide the transfer learning problem into four major categories and propose different structural priors to solve each of these sub-problems. This thesis first focuses on the unsupervised-domain-adaptation problem, where we propose to minimize domain discrepancy by transforming labeled source-domain data to be close to unlabeled target-domain data. For this problem, the categories remain the same across the two domains and hence we assume that the structural relationship between the source-domain samples is carried over to the target domain. Thus, graph or hyper-graph is constructed as the structural prior from both domains and a graph/hyper-graph matching formulation is used to transform samples in the source domain to be closer to samples in the target domain. An efficient optimization scheme is then proposed to tackle the time and memory inefficiencies associated with the matching problem. The few-shot learning problem is studied next, where we propose to transfer knowledge from source-domain categories containing abundantly labeled data to novel categories in the target domain that contains only few labeled data. The knowledge transfer biases the novel category predictions and prevents the model from overfitting. The knowledge is encoded using a neural-network-based prior that transforms a data sample to its corresponding class prototype. This neural network is trained from the source-domain data and applied to the target-domain data, where it transforms the few-shot samples to the novel-class prototypes for better recognition performance. The few-shot learning problem is then extended to the situation, where we do not have access to the source-domain data but only have access to the source-domain class prototypes. In this limited information setting, parametric neural-network-based priors would overfit to the source-class prototypes and hence we seek a non-parametric-based prior using manifolds. A piecewise linear manifold is used as a structural prior to fit the source-domain-class prototypes. This structure is extended to the target domain, where the novel-class prototypes are found by projecting the few-shot samples onto the manifold. Finally, the zero-shot learning problem is addressed, which is an extreme case of the few-shot learning problem where we do not have any labeled data in the target domain. However, we have high-level information for both the source and target domain categories in the form of semantic descriptors. We learn the relation between the sample space and the semantic space, using a regularized neural network so that classification of the novel categories can be carried out in a common representation space. This same neural network is then used in the target domain to relate the two spaces. In case we want to generate data for the novel categories in the target domain, we can use a constrained generative adversarial network instead of a traditional neural network. Thus, we use structural priors like graphs, neural networks and manifolds to relate various data entities like samples, prototypes and semantics for these different transfer learning sub-problems. We explore additional post-processing steps like pseudo-labeling, domain adaptation and calibration and enforce algorithmic and architectural constraints to further improve recognition performance. Experimental results on standard transfer learning image recognition datasets produced competitive results with respect to previous work. Further experimentation and analyses of these methods provided better understanding of machine learning as well.

With the rapid development of deep learning technology, semantic segmentation methods have been widely used in remote sensing data. A pretrained semantic segmentation model usually cannot perform well when the testing images (target domain) have an obvious difference from the training data set (source domain), while a large enough labeled data set is almost impossible to be acquired for each scenario. Unsupervised domain adaptation (DA) techniques aim to transfer knowledge learned from the source domain to a totally unlabeled target domain. By reducing the domain shift, DA methods have shown the ability to improve the classification accuracy for the target domain. Hence, in this letter, we propose an unsupervised adversarial DA network that converts deep features into 2-D feature curves and reduces the discrepancy between curves from the source domain and curves from the target domain based on a conditional generative adversarial networks (cGANs) model. Our proposed DA network is able to improve the semantic labeling accuracy when we apply a pretrained semantic segmentation model to the target domain. To test the effectiveness of the proposed method, experiments are conducted on the International Society for Photogrammetry and Remote Sensing (ISPRS) 2-D Semantic Labeling data set. Results show that our proposed network is able to stably improve overall accuracy not only when the source and target domains are from the same city but with different building styles but also when the source and target domains are from different cities and acquired by different sensors. By comparing with a few state-of-the-art DA methods, we demonstrate that our proposed method achieves the best cross-domain semantic segmentation performance.

In many applications, a face recognition model learned on a source domain but applied to a novel target domain degenerates even significantly due to the mismatch between the two domains. Aiming at learning a better face recognition model for the target domain, this paper proposes a simple but effective domain adaptation approach that transfers the supervision knowledge from a labeled source domain to the unlabeled target domain. Our basic idea is to convert the source domain images to target domain (termed as targetize the source domain hereinafter), and at the same time keep its supervision information. For this purpose, each source domain image is simply represented as a linear combination of sparse target domain neighbors in the image space, with the combination coefficients however learnt in a common subspace. The principle behind this strategy is that, the common knowledge is only favorable for accurate cross-domain reconstruction, but for the classification in the target domain, the specific knowledge of the target domain is also essential and thus should be mostly preserved (through targetization in the image space in this work). To discover the common knowledge, specifically, a common subspace is learnt, in which the structures of both domains are preserved and meanwhile the disparity of source and target domains is reduced. The proposed method is extensively evaluated under three face recognition scenarios, i.e., domain adaptation across view angle, domain adaptation across ethnicity and domain adaptation across imaging condition. The experimental results illustrate the superiority of our method over those competitive ones.

Universal multi-Source domain adaptation for image classification

Transfer learning and domain adaptation are promising solutions to solve the problem that the training set (source domain) and the test set (target domain) follow different distributions. In this paper, we investigate the unsupervised domain adaptation in which the target samples are unlabeled whereas the source domain is fully labeled. We find distinct transformation matrices to transfer both the source and the target domains into the disjointed subspaces where the distribution of each target sample in the transformed space is similar to the source samples. Moreover, the marginal and conditional probability disparities are minimized across the transformed source and target domains via a non-parametric criterion, i.e., maximum mean discrepancy. Therefore, different classes in the source domain are discriminated using the between-class maximization and within-class minimization. In addition, the local information of the source and target data including geometrical structures of the data are preserved via sample labels. The performance of the proposed method is verified using various visual benchmarks experiments. The average accuracy of our proposed method on three standard benchmarks is 70.63%. We compared our method against other state-of-the-art domain adaptation methods where the results prove that it outperforms other domain adaptation methods with 22.9% improvement.

Deep neural networks can learn powerful representations from massive amounts of labeled data; however, their performance is unsatisfactory in the case of large samples and small labels. Transfer learning can bridge between a source domain with rich sample data and a target domain with only a few or zero labeled samples and, thus, complete the transfer of knowledge by aligning the distribution between domains through methods, such as domain adaptation. Previous domain adaptation methods mostly align the features in the feature space of all categories on a global scale. Recently, the method of locally aligning the sub-categories by introducing label information achieved better results. Based on this, we present a deep fuzzy domain adaptation (DFDA) that assigns different weights to samples of the same category in the source and target domains, which enhances the domain adaptive capabilities. Our experiments demonstrate that DFDA can achieve remarkable results on standard domain adaptation datasets.

Domain adaptation (DA) has been a fundamental building block for Transfer Learning (TL) which assumes that source and target domain share the same label space. A more general and realistic setting is that the label space of target domain is a subset of the source domain, as termed by Partial domain adaptation (PDA). Previous methods typically match the whole source domain to target domain, which causes negative transfer due to the source-negative classes in source domain that does not exist in target domain. In this paper, a novel Discriminative Partial Domain Adversarial Network (DPDAN) is developed. We first propose to use hard binary weighting to differentiate the source-positive and source-negative samples in the source domain. The source-positive samples are those with labels shared by two domains, while the rest in the source domain are treated as source-negative samples. Based on the above binary relabeling strategy, our algorithm maximizes the distribution divergence between source-negative samples and all the others (source-positive and target samples), meanwhile minimizes domain shift between source-positive samples and target domain to obtain discriminative domain-invariant features. We empirically verify DPDAN can effectively reduce the negative transfer caused by source-negative classes, and also theoretically show it decreases negative transfer caused by domain shift. Experiments on four benchmark domain adaptation datasets show DPDAN consistently outperforms state-of-the-art methods.KeywordsPartial domain adaptationAdversarial learningDiscriminative learning

Domain adaptation has demonstrated success in classification of multi-center autism spectrum disorder (ASD). However, current domain adaptation methods primarily focus on classifying data in a single target domain with the assistance of one or multiple source domains, lacking the capability to address the clinical scenario of identifying ASD in multiple target domains. In response to this limitation, we propose a Trustworthy Curriculum Learning Guided Multi-Target Domain Adaptation (TCL-MTDA) network for identifying ASD in multiple target domains. To effectively handle varying degrees of data shift in multiple target domains, we propose a trustworthy curriculum learning procedure based on the Dempster-Shafer (D-S) Theory of Evidence. Additionally, a domain-contrastive adaptation method is integrated into the TCL-MTDA process to align data distributions between source and target domains, facilitating the learning of domain-invariant features. The proposed TCL-MTDA method is evaluated on 437 subjects (including 220 ASD patients and 217 NCs) from the Autism Brain Imaging Data Exchange (ABIDE). Experimental results validate the effectiveness of our proposed method in multi-target ASD classification, achieving an average accuracy of 71.46% (95% CI: 68.85% - 74.06%) across four target domains, significantly outperforming most baseline methods (p<0.05).