Cross-domain Adaptation Research Articles

Rotation-Invariant and Relation-Aware Cross-Domain Adaptation Object Detection Network for Optical Remote Sensing Images

In recent years, object detection has shown excellent results on a large number of annotated data, but when there is a discrepancy between the annotated data and the real test data, the performance of the trained object detection model is often degraded when it is directly transferred to the real test dataset. Compared with natural images, remote sensing images have great differences in appearance and quality. Traditional methods need to re-label all image data before interpretation, which will consume a lot of manpower and time. Therefore, it is of practical significance to study the Cross-Domain Adaptation Object Detection (CDAOD) of remote sensing images. To solve the above problems, our paper proposes a Rotation-Invariant and Relation-Aware (RIRA) CDAOD network. We trained the network at the image-level and the prototype-level based on a relation aware graph to align the feature distribution and added the rotation-invariant regularizer to deal with the rotation diversity. The Faster R-CNN network was adopted as the backbone framework of the network. We conducted experiments on two typical remote sensing building detection datasets, and set three domain adaptation scenarios: WHU 2012 → WHU 2016, Inria (Chicago) → Inria (Austin), and WHU 2012 → Inria (Austin). The results show that our method can effectively improve the detection effect in the target domain, and outperform competing methods by obtaining optimal results in all three scenarios.

Temporal convolution-based transferable cross-domain adaptation approach for remaining useful life estimation under variable failure behaviors

Many data-driven models normally assume that the training and test data are independent and identically distributed to predict the remaining useful life (RUL) of industrial machines. However, different failure models caused by variable failure behaviors may lead to a domain shift. Meanwhile, conventional methods lack comprehensive attention to temporal information, resulting in a limitation. To handle the aforementioned challenges, a transferable cross-domain approach for RUL estimation is proposed. The hidden features are extracted adaptively by a temporal convolution network in which residual self-attention is able to fully consider the contextual degradation information. Furthermore, a new cross-domain adaption architecture with the contrastive loss and multi-kernel maximum mean discrepancy is designed to learn the domain invariant features. The effectiveness and superiority of the proposed method are proved by the case study on IEEE PHM challenge 2012 bearing dataset and the comparison with other methods.

Bayesian Learning of LF-MMI Trained Time Delay Neural Networks for Speech Recognition

Discriminative training techniques define state-of-the-art performance for automatic speech recognition systems. However, they are inherently prone to overfitting, leading to poor generalization performance when using limited training data. In order to address this issue, this paper presents a full Bayesian framework to account for model uncertainty in sequence discriminative training of factored TDNN acoustic models. Several Bayesian learning based TDNN variant systems are proposed to model the uncertainty over weight parameters and choices of hidden activation functions, or the hidden layer outputs. Efficient variational inference approaches using as few as one single parameter sample ensure their computational cost in both training and evaluation time comparable to that of the baseline TDNN systems. Statistically significant word error rate (WER) reductions of 0.4%-1.8% absolute (5%-11% relative) were obtained over a state-of-the-art 900 h speed perturbed Switchboard corpus trained baseline LF-MMI factored TDNN system using multiple regularization methods including F-smoothing, L2 norm penalty, natural gradient, model averaging and dropout, in addition to i-Vector plus learning hidden unit contribution (LHUC) based speaker adaptation and RNNLM rescoring. The efficacy of the proposed Bayesian techniques is further demonstrated in a comparison against the state-of-the-art performance obtained on the same task using the most recent hybrid and end-to-end systems reported in the literature. Consistent performance improvements were also obtained on a 450-h HKUST conversational Mandarin telephone speech recognition task. On a third cross domain adaptation task requiring rapidly porting a 1000-h LibriSpeech data trained system to a small DementiaBank elderly speech corpus, the proposed Bayesian TDNN LF-MMI systems outperformed the baseline system using direct weight fine-tuning by up to 2.5% absolute WER reduction.

Incremental Cross-Domain Adaptation for Robust Retinopathy Screening via Bayesian Deep Learning

Retinopathy represents a group of retinal diseases that, if not treated timely, can cause severe visual impairments or even blindness. Many researchers have developed autonomous systems to recognize retinopathy via fundus and optical coherence tomography (OCT) imagery. However, most of these frameworks employ conventional transfer learning and fine-tuning approaches, requiring a decent amount of well-annotated training data to produce accurate diagnostic performance. This paper presents a novel incremental cross-domain adaptation instrument that allows any deep classification model to progressively learn abnormal retinal pathologies in OCT and fundus imagery via few-shot training. Furthermore, unlike its competitors, the proposed instrument is driven via a Bayesian multi-objective function that not only enforces the candidate classification network to retain its prior learned knowledge during incremental training but also ensures that the network understands the structural and semantic relationships between previously learned pathologies and newly added disease categories to effectively recognize them at the inference stage. The proposed framework, evaluated on six public datasets acquired with three different scanners to screen thirteen retinal pathologies, outperforms the state-of-the-art competitors by achieving an overall accuracy and F1 score of 0.9826 and 0.9846, respectively.

Hybrid-COVID: a novel hybrid 2D/3D CNN based on cross-domain adaptation approach for COVID-19 screening from chest X-ray images.

The novel Coronavirus disease (COVID-19), which first appeared at the end of December 2019, continues to spread rapidly in most countries of the world. Respiratory infections occur primarily in the majority of patients treated with COVID-19. In light of the growing number of COVID-19 cases, the need for diagnostic tools to identify COVID-19 infection at early stages is of vital importance. For decades, chest X-ray (CXR) technologies have proven their ability to accurately detect respiratory diseases. More recently, with the availability of COVID-19 CXR scans, deep learning algorithms have played a critical role in the healthcare arena by allowing radiologists to recognize COVID-19 patients from their CXR images. However, the majority of screening methods for COVID-19 reported in recent studies are based on 2D convolutional neural networks (CNNs). Although 3D CNNs are capable of capturing contextual information compared to their 2D counterparts, their use is limited due to their increased computational cost (i.e. requires much extra memory and much more computing power). In this study, a transfer learning-based hybrid 2D/3D CNN architecture for COVID-19 screening using CXRs has been developed. The proposed architecture consists of the incorporation of a pre-trained deep model (VGG16) and a shallow 3D CNN, combined with a depth-wise separable convolution layer and a spatial pyramid pooling module (SPP). Specifically, the depth-wise separable convolution helps to preserve the useful features while reducing the computational burden of the model. The SPP module is designed to extract multi-level representations from intermediate ones. Experimental results show that the proposed framework can achieve reasonable performances when evaluated on a collected dataset (3 classes to be predicted: COVID-19, Pneumonia, and Normal). Notably, it achieved a sensitivity of 98.33%, a specificity of 98.68% and an overall accuracy of 96.91%

Improving the performance of drifted/shifted electronic nose systems by cross-domain transfer using common transfer samples

Sensor drift/shift is a challenging and high-profile issue in the field of sensors and measurements. Because of the time variability and unpredictable properties of drift/shift, traditional compensation methods are costly and laborious. Considering the distribution alignments of different domains at both the feature and decision level, a unified double-level drift/shift compensation framework is presented to address sensor drift/shift. The framework consists of cross-domain feature subspace transfer learning coupled with a cross-domain adaptation extreme learning machine (CFST-AELM), which learns a low-dimensional discriminant subspace and a robust classifier using small numbers of labeled samples selected from the target domain. In general, the unified framework has the following advantages: (1) implementing anti-drift/shift at both the feature and decision level by cross-domain transfer learning which takes full advantage of transfer samples; (2) the subspace learning is a joint distribution adaptation algorithm and can be easily realized by eigenvalue decomposition; (3) more discriminative information of the source and target datasets remain after linear mapping; and (4) there is enhancement of the generalization performance of the model by adding the prediction error of transfer samples as a constraint of extreme learning machines (ELM) classification. The results suggest that compared with several state-of-the-art strategies, the CFST-AELM framework has superior performance in terms of classification accuracy.

Blockchain-Enabled Cross-Domain Object Detection for Autonomous Driving: A Model Sharing Approach

Object detection for autonomous driving is a huge challenge in the cross-domain adaptation scenario, especially for the time- and resource-consuming task. Distributed deep learning (DDL) has demonstrated a considerably good balance between efficiency and computation complexity. However, the reliability of DDL is low. Moreover, the cost of training data and model is not priced well. In this article, a novel blockchain-enabled model sharing approach is proposed to improve the performance of object detection with cross-domain adaptation for autonomous driving systems. Based on the blockchain and mobile-edge computing (MEC) technology, a domain-adaptive you-only-look-once (YOLOv2) model is trained across nodes, which can reduce significantly the domain discrepancy for different object categories. Furthermore, smart contracts are developed to perform data storage and model sharing tasks efficiently. The reliability of model sharing is ensured with blockchain consensus. We evaluate the proposed method under public data sets. The simulation results demonstrate that the efficiency and reliability of the proposed approach are better than the reference model.

Tracklet Self-Supervised Learning for Unsupervised Person Re-Identification

Existing unsupervised person re-identification (re-id) methods mainly focus on cross-domain adaptation or one-shot learning. Although they are more scalable than the supervised learning counterparts, relying on a relevant labelled source domain or one labelled tracklet per person initialisation still restricts their scalability in real-world deployments. To alleviate these problems, some recent studies develop unsupervised tracklet association and bottom-up image clustering methods, but they still rely on explicit camera annotation or merely utilise suboptimal global clustering. In this work, we formulate a novel tracklet self-supervised learning (TSSL) method, which is capable of capitalising directly from abundant unlabelled tracklet data, to optimise a feature embedding space for both video and image unsupervised re-id. This is achieved by designing a comprehensive unsupervised learning objective that accounts for tracklet frame coherence, tracklet neighbourhood compactness, and tracklet cluster structure in a unified formulation. As a pure unsupervised learning re-id model, TSSL is end-to-end trainable at the absence of source data annotation, person identity labels, and camera prior knowledge. Extensive experiments demonstrate the superiority of TSSL over a wide variety of the state-of-the-art alternative methods on four large-scale person re-id benchmarks, including Market-1501, DukeMTMC-ReID, MARS and DukeMTMC-VideoReID.

Deep learning-based cross-domain adaptation for gearbox fault diagnosis under variable speed conditions

Existing intelligent gearbox fault diagnosis approaches have two shortcomings: (a) their performance is mostly confined to manual handcrafted features, and (b) they follow a general assumption that the distribution of the data in the source domain (labeled data on which the model is trained) is similar to the target domain (unlabeled data on which the model is tested), which might not be the case in real-world applications. Substantial human expertise and domain knowledge is required for manual feature extraction, and moreover, deploying the same model for a target domain whose distribution is different from the source domain would lead to poor generalization. Since deep learning methods can automatically learn high dimensional feature representations from raw measurement data, this paper proposes a novel deep learning-based domain adaptation (DA) method for gearbox fault diagnosis under significant speed variations. A deep convolutional neural network is used as the main architecture. The paper proposes to minimize the summation of cross-entropy loss (between the labeled source domain data) and maximum mean discrepancy loss (between the labeled source and unlabeled target datasets) simultaneously to adapt the source domain model to be applied in the target domain. The proposed deep learning DA approach is evaluated using experimental data from a gearbox under variable speeds and multiple health conditions. An appropriate benchmarking with both traditional machine learning methods and other DA methods demonstrate the superiority of the proposed method.

Cross-Domain Deep Visual Feature Generation for Mandarin Audio–Visual Speech Recognition

There has been a long term interest in using visual information to improve automatic speech recognition (ASR) system performance. Both audio and visual information are required in conventional audio visual speech recognition (AVSR) systems. This limits their wider applications when visual modality is not present. To this end, one possible solution is to use acoustic-to-visual (A2V) inversion techniques to generate visual features. Previous research in this direction used synthetic acoustic-articulatory parallel data in inversion model training. The acoustic mismatch between the audio-visual (AV) parallel data and target data was not considered. In addition, the target language to apply these technologies has been focused on English. In this article, a real 3D Audio-Visual Mandarin Continuous Speech (3DAV-MCS) corpus was used to train deep neural network based A2V inversion models. Cross-domain adaptation of the inversion models allows suitable visual features to be generated from acoustic data of mismatched domains. The proposed cross-domain deep visual feature generation techniques were evaluated on two state-of-the-art Mandarin speech recognition tasks: DAPRA GALE broadcast transcription and BOLT conversational telephone speech recognition. The AVSR systems constructed using the cross-domain generated visual features consistently outperformed the baseline convolutional neural network (CNN) ASR systems by up to 3.3% absolute (9.1% relative) character error rate (CER) reductions after both speaker adaptive training and sequence discriminative training were performed.

Style Transfer for CNC Machine Input Data Preprocessing

Advances in deep neural networks have led to impressive results in recent years. The new technologies such as cross-domain adaptation, reinforcement learning and generative adversarial networks have shown a real promise for industrial and real-life applications. In this paper, the results of the experimental research on designing, training and implementation of the preprocessing algorithm for the computer numerical control machine input were presented. The algorithm of neural network transfer of artistic style has demonstrated wide possibilities in the field of generating graphic content. This paper demonstrates the possibility of using a generating neural network for the synthesis of stylized images that can be used as input images for a computer numerical control machine. Thus, the proposed algorithm is pre-processing the input image. The design feature of the laser engraver does not allow styling using an arbitrary style image, so dotted or linearized binary images are used as a style. The proposed preprocessing algorithm allows synthesizing binary images reproduced by a laser engraver. At the same time, image generation is performed in one forward pass of the generating neural network.

Longitudinal Examination of Emotional Functioning in Older Adults After Spousal Bereavement

This study examined trajectories of emotional functioning in three domains (depressive symptoms, emotional, and social loneliness) for individuals who experienced spousal bereavement and investigated cross-domain adaptation. We hypothesized that emotional difficulties after bereavement would be more detectable in emotional loneliness than depressive symptoms or social loneliness. Using latent class growth analysis, we modeled changes in depressive symptoms, emotional loneliness, and social loneliness from 12 years pre- to 12 years post-bereavement on data from 686 older adults to identify trajectories indicating adaptive and maladaptive functioning in each domain. Most participants reported depressive symptoms below the clinically relevant threshold by showing a resilient (15.5%) or a slightly elevated (53.5%) trajectory post-bereavement. One third (31%) reported clinically relevant depressive symptoms. More than half of the sample reported emotional loneliness post-bereavement, varying form prolonged (17%), increasing and prolonged (28.3%), and chronically high (8.9%) levels. Remaining participants displayed resilience (13.5%) or recovery (32.3%). Social loneliness showed four trajectories: very low and resilient (43.3%), low and resilient (27.5%), increasing (20.2%), and chronically high (9%) levels. One third of participants maintained adaptive, whereas 12% displayed maladaptive, functioning across all domains post-bereavement. An increase in emotional loneliness was the most commonly observed change after spousal bereavement. This highlights the central role of emotional loneliness in depression after bereavement.

A framework for semi-supervised metric transfer learning on manifolds

A common assumption of statistical learning theory is that the training and testing data are drawn from the same distribution. However, in many real-world applications, this assumption does not hold true. Hence, a realistic strategy, Cross Domain Adaptation (DA) or Transfer Learning (TA), can be used to employ previously labelled source domain data to boost the task in the new target domain. Previously, Cross Domain Adaptation methods have been focused on re-weighting the instances or aligning the cross-domain distributions. However, these methods have two significant challenges: (1) There is no proper consideration of the unlabelled data of target task as in the real-world, an abundant amount of unlabelled data is available, (2) The use of normal Euclidean distance function fails to capture the appropriate similarity or dissimilarity between samples. To deal with this issue, we have proposed a Semi-Supervised Metric Transfer Learning framework called SSMT that reduces the distribution between domains both statistically and geometrically by learning the instance weights, while a regularized distance metric is learned to minimize the within-class co-variance and maximize the between-class co-variance simultaneously for the target domain. Compared with the previous works where Mahalanobis distance metric and instance weights are learned by using the labelled data or in a pipelined framework that leads to a decrease in the performance, our proposed SSMT attempts to learn a regularized distance metric and instance weights by considering unlabelled data in a parallel framework. Experimental evaluation on three cross-domain visual data sets, e.g., PIE Face, Handwriting Digit Recognition on MNIST–USPS and Object Recognition, demonstrates the effectiveness of our designed approach on facilitating the unlabelled target task learning, compared to current state-of-the-art domain adaptation approaches.

Joint Domain Matching and Classification for cross-domain adaptation via ELM

Recent years, domain adaptation has attracted much attention in the community of machine learning. In this paper, we mainly focus on the tasks of Joint Domain Matching and Classification (JDMC) under the framework of extreme learning machine (ELM). Specifically, our JDMC method is formulated by optimizing both the output-adapted transformation and the cross-domain classifier, which allows us to simultaneously (1) align the source domain and target domain in the feature space with correlation alignment, (2) minimize the discrepancy between the source and target domain, measured in terms of both marginal and conditional probability distribution in the mapped feature space, (3) select informative features which behave similarly in both domains for knowledge transfer by imposing ℓ2,1-norm on the output weights of ELM. In this respect, the proposed JDMC integrates the feature matching, feature selection and classifier design in a unified framework. Besides, an efficient alternative optimization strategy is exploited to solve the joint learning model. To evaluate the effectiveness of the proposed method, extensive experiments on several commonly used domain adaptation datasets are presented, the results show that the proposed method significantly outperforms the non-transfer ELM networks and consistently outperforms several state-of-art domain adaptation methods.

Unsupervised domain adaptation with target reconstruction and label confusion in the common subspace

Deep neural networks can learn powerful and discriminative representations from a large number of labeled samples. However, it is typically costly to collect and annotate large-scale datasets, which limits the applications of deep learning in many real-world scenarios. Domain adaptation, as an option to compensate for the lack of labeled data, has attracted much attention in the community of machine learning. Although a mass of methods for domain adaptation has been presented, many of them simply focus on matching the distribution of the source and target feature representations, which may fail to encode useful information about the target domain. In order to learn invariant and discriminative representations for both domains, we propose a Cross-Domain Minimization with Deep Autoencoder method for unsupervised domain adaptation, which simultaneously learns label prediction on the source domain and input reconstruction on the target domain using shared feature representations aligned with correlation alignment in a unified framework. Furthermore, inspired by adversarial training and cluster assumption, a task-specific class label discriminator is incorporated to confuse the predicted target class labels with samples draw from categorical distribution, which can be regarded as entropy minimization regularization. Extensive empirical results demonstrate the superiority of our approach over the state-of-the-art unsupervised adaptation methods on both visual and non-visual cross-domain adaptation tasks.

Domain adaptation with gradient reversal for MC/real data calibration

It is quite common part of the data analysis in High Energy Physics to train a classifier for signal and background separation. In case the signal under investigation is a rare process, the signal sample is simulated and background sample is taken from the real data. Such setting create an unnecessary bias: the classifier might learn not the characteristic of the signal but the characteristic of the imperfect simulation. So the challenge is to train the classifier in such way that it picks up signal/background difference and doesnt overfit to the simulation-specific features. The suggested approach is based on cross-domain adaptation technique using neural networks with gradient reversal. The network architecture is a dense multi-branch structure. One branch is responsible for the signal/background discrimination, the second branch helps to avoid the overfitting on the Monte-Carlo training dataset. The tests showed that this architecture is a robust mechanism for choosing trade-offs between discrimination power and overfitting. So the resulting networks successfully distinguishes the signal from the background, but does not distinguish simulated events from the real ones. Moreover, such architecture could to be easily extended with more branches, and each one could be responsible for specific discrete and continuous domains. For example, the additional third network’s branch could help to reduce the correlation between the classifier predictions and reconstructed mass of the decay, thereby making such approach highly viable for wide variety of physics searches. But such network’s extensions weren’t investigated during this work.

Tumor-aware, Adversarial Domain Adaptation from CT to MRI for Lung Cancer Segmentation.

We present an adversarial domain adaptation based deep learning approach for automatic tumor segmentation from T2-weighted MRI. Our approach is composed of two steps: (i) a tumor-aware unsupervised cross-domain adaptation (CT to MRI), followed by (ii) semi-supervised tumor segmentation using Unet trained with synthesized and limited number of original MRIs. We introduced a novel target specific loss, called tumor-aware loss, for unsupervised cross-domain adaptation that helps to preserve tumors on synthesized MRIs produced from CT images. In comparison, state-of-the art adversarial networks trained without our tumor-aware loss produced MRIs with ill-preserved or missing tumors. All networks were trained using labeled CT images from 377 patients with non-small cell lung cancer obtained from the Cancer Imaging Archive and unlabeled T2w MRIs from a completely unrelated cohort of 6 patients with pre-treatment and 36 on-treatment scans. Next, we combined 6 labeled pre-treatment MRI scans with the synthesized MRIs to boost tumor segmentation accuracy through semi-supervised learning. Semi-supervised training of cycle-GAN produced a segmentation accuracy of 0.66 computed using Dice Score Coefficient (DSC). Our method trained with only synthesized MRIs produced an accuracy of 0.74 while the same method trained in semi-supervised setting produced the best accuracy of 0.80 on test. Our results show that tumor-aware adversarial domain adaptation helps to achieve reasonably accurate cancer segmentation from limited MRI data by leveraging large CT datasets.

Cross domain adaptation by learning partially shared classifiers and weighting source data points in the shared subspaces

Transfer learning is a problem defined over two domains. These two domains share the same feature space and class label space, but have significantly different distributions. One domain has sufficient labels, named as source domain, and the other domain has few labels, named as target do- main. The problem is to learn a effective classifier for the target domain. In this paper, we propose a novel transfer learning method for this problem by learning a partially shared classifier for the target domain, and weighting the source domain data points. We learn some shared subspaces for both the data points of the two domains, and a shared classifier in the shared subspaces. We hope that in the shared subspaces, the distributions of two domain can match each other well, and to match the distributions, we weight the source domain data points with different weighting factors. Moreover, we adapt the shared classifier to each domain by learning different adaptation functions. To learn the subspace transformation matrices, the classifier parameters, and the adaptation parameters, we build a objective function with weighted clas- sification errors, parameter regularization, local reconstruction regularization, and distribution matching. This objective function is minimized by an itera- tive algorithm. Experiments show its effectiveness over benchmark data sets, including travel destination review data set, face expression data set, spam email data set, etc.

A Framework for Classifier Adaptation for Large-Scale Multimedia Data

Machine learning techniques have been used extensively to build models for the analysis and retrieval of multimedia data. The explosion of multimedia data on the Web poses a great challenge to such techniques not simply because of the sheer data volume, but also because of the heterogeneity of the data. With data from a wide variety of domains, models trained from one domain do not generalize well to other domains, while at the same time it is prohibitively expensive to build new models for each and every domain due to the high cost for labeling training examples. In this paper, we tackle the heterogeneity challenge in large-scale multimedia data using cross-domain model adaptation for better performance and reduced human cost. Specifically, we investigate the problem of adapting supervised classifiers trained from one or more source domains to a new classifier for a target domain that has only limited labeled examples. The foundation of our work is a general framework for function-level classifier adaptation based on the regularized loss minimization principle, which adapts a classifier by directly modifying its decision function. Under this framework, one can derive concrete adaptation algorithms by plugging in any loss and regularization functions, among which we elaborate on adaptive support vector machines (a-SVM). We further extend this framework for multiclassifier adaptation, namely adapting multiple existing classifiers into a classifier for the target domain, in a way that the contributions of these existing classifiers are automatically determined. We evaluate the proposed approaches in cross-domain semantic concept detection based on TRECVID corpora. The results show that our approaches outperform existing (adaptation and nonadaptation) methods in terms of accuracy and/or efficiency, and adaptation from multiple classifiers offers further benefits.

Cross–layer cross–domain adaptation of mobile video services

This paper deals with the analysis of user perceived visual quality for mobilemultimedia services. H.264/AVC is selected for low resolution video encoding, and a mobile UMTS network is considered for the access to services. First, from subjective tests results, the combined impact of different servi