Target Domain Data Research Articles

Data efficient indoor thermal comfort prediction using instance based transfer learning method

The precise prediction of indoor thermal sensation is critical to building energy efficiency and satisfaction of occupants. In this field, data-driven models have shown superior performance than Predicted Mean Vote (PMV) model. However, a reliable and accurate model of this kind relies on sufficient and high-quality data from physiological and environmental sensing, which in practice is difficult or expensive to collect. Rather than developing a separated predictive model for certain task, transfer learning method could intelligently use knowledge from similar domains to solve the target problem, and thus the requirement of data resources is reduced. In the last few years, transfer learning has been introduced into the field of thermal comfort prediction. Most of them used the source domain data as a whole for model pre-training, and the samples' suitability judgment was not fully considered. But the general situation in this field is: there are few public and available data resources and their data distributions are hardly consistent with that of specific target task. Aiming at the issue, this study proposes an instance based transfer learning model, which is based on data weighting and reusing for data efficient thermal comfort prediction. We assume that samples from source and target domains obey different distributions but have some common features that can be transferred across domains. The Nearest Neighbor Search (NNS) is used for similarity judgement and selection of source domain data; A Boosting type transfer learning algorithm, i.e., TrAdaBoost, is improved for reliable thermal comfort prediction. In the algorithm, an automatic weighting mechanism could optimally adjust the source training data's weights for performance improvement. By using field experimental data set and a public data set as target and source domains data respectively, the performance of the proposed model is investigated. Results show when target domain data is insufficient, the proposed NNS-iTrAdaBoost method has superior transfer learning ability and could achieve much better performance than traditional data-driven models. In practice, the proposed method is easy for engineering implementation and has great potential for thermal comfort prediction based on small-scale experiments.

An FCN-based transfer-learning method for spatial infrared moving-target recognition

Spatial infrared (IR) moving-target sample data are difficult to collect; thus, they cannot be used for mass-training, and limit development and application of deep learning methods in spatial IR target recognition. This study proposes a transfer-learning method based on fully convolutional network (FCN) for spatial IR moving-target recognition. First, we built a set of semi-physical scale models of spatial IR moving targets and obtained IR radiation characteristics as target-domain data using actual shot data and image inversion. Simulation data of the spatial target were obtained using the IR radiation intensity model, and used as the source-domain data together with the public UCR dataset. We used the dynamic time warping distance to measure the similarity between datasets to select appropriate source-domain datasets. We used an FCN framework as the main classification framework with a large number of source-domain samples for training. Knowledge transfer and model fine-tuning facilitate target-domain learning using only a few samples to improve classification accuracy. The experimental results show that the proposed model achieved an accuracy rate of 78%, which increases by the 20% improvement compared to direct training on the target domain. This study verifies the effectiveness of transfer learning between spatial targets and related domains and provides a reference for application of deep learning in spatial situational awareness.

A domain adaptation approach to damage classification with an application to bridge monitoring

Data-driven machine-learning algorithms generally suffer from a lack of labelled health-state data, mainly those referring to damage conditions. To address such an issue, population-based structural health monitoring seeks to enrich the original dataset by transferring knowledge from a population of monitored structures. Within this context, this paper presents a transfer learning approach, based on domain adaptation, to leverage information from completely-labelled bridge structure data to accurately predict new instances of an unknown target domain. Since intrinsic structural differences may cause distribution shifts, domain adaptation attempts to minimise the distance between the domains and to learn a mapping within a shared feature space. Specifically, the methodology involves the long-term acquisition of natural frequencies from several structural scenarios. Such damage-sensitive features are then aligned via domain adaptation so that a machine-learning algorithm can effectively utilise the labelled source domain data and generalise well to the unlabelled target-domain data. The described procedure is applied to two case studies, including the Z24 and the S101 benchmark bridges and their finite element models, respectively. The results demonstrate the successful exchange of health-state labels to identify the damage class within a population of bridges equipped with SHM systems, showing potential to reduce computational efforts and to deal with scarce or poor data sets in application to bridge network monitoring.

Spatial–Temporal Temperature Forecasting Using Deep-Neural-Network-Based Domain Adaptation

Accurate temperature forecasting is critical for various sectors, yet traditional methods struggle with complex atmospheric dynamics. Deep neural networks (DNNs), especially transformer-based DNNs, offer potential advantages, but face challenges with domain adaptation across different geographical regions. We evaluated the effectiveness of DNN-based domain adaptation for daily maximum temperature forecasting in experimental low-resource settings. We used an attention-based transformer deep learning architecture as the core forecasting framework and used kernel mean matching (KMM) for domain adaptation. Domain adaptation significantly improved forecasting accuracy in most experimental settings, thereby mitigating domain differences between source and target regions. Specifically, we observed that domain adaptation is more effective than exclusively training on a small amount of target-domain training data. This study reinforces the potential of using DNNs for temperature forecasting and underscores the benefits of domain adaptation using KMM. It also highlights the need for caution when using small amounts of target-domain data to avoid overfitting. Future research includes investigating strategies to minimize overfitting and to further probe the effect of various factors on model performance.

A novel classification method for GPR B-scan images based on weak-shot learning

Ground Penetrating Radar (GPR) is extensively employed for underground target detection, and the advancements in this domain have been hastened by the utilization of deep learning methodologies. Despite advancements in deep learning, practical engineering applications often face challenges in meeting the demanding requirements of large-scale data and high data quality. In this study, based on cross-domain transfer learning, we proposed a method for underground target classification of weak labeled GPR data. It combined strongly annotated source domain data with weakly annotated target domain data through adversarial learning to leverage the prior knowledge from existing large datasets and addressed the classification problem of small-sample GPR data. By employing adversarial learning to integrate strongly labeled source domain data with weakly labeled target domain data, we leverage prior knowledge from existing large datasets, thereby enhancing classification accuracy on a small GPR dataset. This approach reduces the requirements for annotation quality and sample quantity in the GPR B-scan dataset of underground targets. Additionally, it exhibits robustness to label noise, making it applicable to real-world GPR data classification tasks. Compared to some well-known networks, the proposed model achieved the best classification performance in multiclass classification problems, with an accuracy of 87.62% and an F1macro score of 87.60% on the test set. These results represent an improvement of 17.14 and 17.20 percentage points, respectively, over the best-performing classical classification networks. Additionally, the proposed model's reliability in cross-domain transfer learning, even with limited data, was substantiated through a comparison with other few-shot learning networks.

Source-free domain adaptation with unrestricted source hypothesis

Domain adaptation aims to bridge the distribution discrepancy across different domains and improve the generalization ability of learning models on the target domain. The existing domain adaptation approaches align the distribution shift via adversarial training on the source and target data. In practice, however, the source data is usually unavailable due to the privacy factor. In this work, we mainly focus on the source-free domain adaptation setting, in which we are only accessible to the model trained on the source data and the unlabeled target data. To this end, we propose the Source-Free Adversarial Domain Adaptation (SFADA) approach to align the distribution of the target domain data in the absence of source domain data. In particular, we develop an effective metric to measure the domain discrepancy by introducing the proxy data of the source domain. To generate the proxy data, our approach retrieves target data which lie over the intersection of the supports of the source and target domains. We also derive the learning bound of the source-free domain adaptation theoretically and show that our proposed SFADA approach is capable of reducing the bound effectively. Additionally, instead of modifying the source model in previous source-free approaches, our SFADA does not require training the source model with specific restrictions (i.e., normalizing the classifier weight) for practice and privacy-related concerns. State-of-the-art results are achieved for different standard domain adaptation benchmarks. The code can be available from https://github.com/tiggers23/SFADA-main.

An Integrated Transfer Learning Method for Power Generation Prediction of Run-Off Small Hydropower in Data-Scarce Areas

historical data scarce and varying patterns of new built run-off small hydropower (RSHP) limits precise power generation prediction. Unforeseen hydropower can induce uneconomic power grid operations. To address this issue, a novel transfer learning method enabling integration of public RSHP knowledge is proposed. First, a RSHP data matching algorithm is proposed to pre-filter similar source domain data and produce a RSHP database matching patterns of target RSHP. This algorithm allows us to improve performance of transfer learning model. Next, public prediction knowledge implicated in the RSHP database is learned towards a CNN-BiLSTM hybrid pre-trained network. Then, the pre-trained network is transferred to the target RSHP prediction models by hyper-parameter fine-tuning algorithm, which reduces divergence between the pre-trained network outputs and the target domain data. As a result, accurate new RSHP prediction models can be generated under the challenge of data lack. At the last, the RSHP prediction models are fed back to the fine-tuning algorithm such that generalizability of the models enables life-long self-renewal. The real-world case demonstrates the superiority of the proposed method in terms of accuracy and data utilization. The average prediction error of the proposed method is 16.27% lower than the best traditional alternative.

Fine-Grained Self-Supervision for Generalizable Semantic Segmentation

Unsupervised domain adaptative semantic segmentation is a powerful solution for the distribution shift problem between the source and target domains. However, such methods need specified target domain data that may be unavailable in actual applications due to excess expensive collection. Generalizable semantic segmentation as a new paradigm appears in recent research, which aims to generalize well on distinct unseen domains only using source domain data. The existing methods focus on learning domain-invariant features by using global distribution alignment strategies, which may lead to a decreased discriminability of the model. To cope with this challenge, we propose a fine-grained self-supervision (FGSS) framework for generalizable semantic segmentation that takes into account both discriminability and generalizability from the perspective of the intra-class relationship. The FGSS framework contains single-view and multi-view versions. In the single-view version, we propose a fine-grained self-supervision strategy to distinguish the sub-parts of the semantic class for better class discriminability. In the multi-view version, we propose a class prototype feature enhancement strategy to generate another view (i.e. another representation of the original representation). Then, we propose a multi-view mutual supervision loss to enforce consistency between different views and further enhance the generalizability of the model. Experimental results on five widely-used datasets, i.e., GTAV, SYNTHIA, BDD100K, Cityscapes, and Mapillary, demonstrate that our FGSS framework achieves superior performance compared to state-of-the-art methods.

Unsupervised Domain-Adaptive Object Detection via Localization Regression Alignment.

Unsupervised domain-adaptive object detection uses labeled source domain data and unlabeled target domain data to alleviate the domain shift and reduce the dependence on the target domain data labels. For object detection, the features responsible for classification and localization are different. However, the existing methods basically only consider classification alignment, which is not conducive to cross-domain localization. To address this issue, in this article, we focus on the alignment of localization regression in domain-adaptive object detection and propose a novel localization regression alignment (LRA) method. The idea is that the domain-adaptive localization regression problem can be transformed into a general domain-adaptive classification problem first, and then adversarial learning is applied to the converted classification problem. Specifically, LRA first discretizes the continuous regression space, and the discrete regression intervals are treated as bins. Then, a novel binwise alignment (BA) strategy is proposed through adversarial learning. BA can further contribute to the overall cross-domain feature alignment for object detection. Extensive experiments are conducted on different detectors in various scenarios, and the state-of-the-art performance is achieved; these results demonstrate the effectiveness of our method. The code will be available at: https://github.com/zqpiao/LRA.

A Trustworthy Curriculum Learning Guided Multi-Target Domain Adaptation Network for Autism Spectrum Disorder Classification.

Domain adaptation has demonstrated success in classification of multi-center autism spectrum disorder (ASD). However, current domain adaptation methods primarily focus on classifying data in a single target domain with the assistance of one or multiple source domains, lacking the capability to address the clinical scenario of identifying ASD in multiple target domains. In response to this limitation, we propose a Trustworthy Curriculum Learning Guided Multi-Target Domain Adaptation (TCL-MTDA) network for identifying ASD in multiple target domains. To effectively handle varying degrees of data shift in multiple target domains, we propose a trustworthy curriculum learning procedure based on the Dempster-Shafer (D-S) Theory of Evidence. Additionally, a domain-contrastive adaptation method is integrated into the TCL-MTDA process to align data distributions between source and target domains, facilitating the learning of domain-invariant features. The proposed TCL-MTDA method is evaluated on 437 subjects (including 220 ASD patients and 217 NCs) from the Autism Brain Imaging Data Exchange (ABIDE). Experimental results validate the effectiveness of our proposed method in multi-target ASD classification, achieving an average accuracy of 71.46% (95% CI: 68.85% - 74.06%) across four target domains, significantly outperforming most baseline methods (p<0.05).

Adaptive Stability Contingency Screening for Operational Planning Based on Domain-Adversarial Graph Neural Network

Compared to contingency screening (CS) techniques that focus on thermal/voltage limit violation, CS that addresses the transient and small-signal stability contingencies in the existing references requires a higher computational cost, limiting its use to off-line contingency analysis. Therefore, it poses the challenge of developing an effective CS scheme for fast recognition of critical contingency, which is one of the major concerns of system operators. To deal with this problem, this paper proposes an adaptive stability CS (SCS) scheme for operational planning. One challenge in developing the adaptive SCS for different cardinal points is data distribution discrepancy resulted from load/topology changes. To align data distribution between different domains (i.e., cardinal points), a domain-adversarial graph neural network (DAGNN) is developed to learn the domain-invariant features, so that the DAGNN model trained on the labeled source domain (e.g., peak cardinal point 2B) data can be applied to the unlabeled target domain (e.g., trough cardinal point 1B) data for SCS. To make the proposed SCS more efficient in dealing with power system data, a graph learning approach combined with graph transformer and graph isomorphism network is used in DAGNN to provide feature representations considering the graph properties of power systems, where nodes and edges refer to buses and transmission lines, respectively. Experiments on IEEE 39 Bus system and IEEE 118 Bus system have verified the effectiveness of the proposed model.

MLFA: Towards Realistic Test Time Adaptive Object Detection by Multi-level Feature Alignment.

Object detection methods have achieved remarkable performances when the training and testing data satisfy the assumption of i.i.d. However, the training and testing data may be collected from different domains, and the gap between the domains can significantly degrade the detectors. Test Time Adaptive Object Detection (TTA-OD) is a novel online approach that aims to adapt detectors quickly and make predictions during the testing procedure. TTA-OD is more realistic than the existing unsupervised domain adaptation and source-free unsupervised domain adaptation approaches. For example, self-driving cars need to improve their perception of new environments in the TTA-OD paradigm during driving. To address this, we propose a multi-level feature alignment (MLFA) method for TTA-OD, which is able to adapt the model online based on the steaming target domain data. For a more straightforward adaptation, we select informative foreground and background features from image feature maps and capture their distributions using probabilistic models. Our approach includes: i) global-level feature alignment to align all informative feature distributions, thereby encouraging detectors to extract domain-invariant features, and ii) cluster-level feature alignment to match feature distributions for each category cluster across different domains. Through the multi-level alignment, we can prompt detectors to extract domain-invariant features, as well as align the category-specific components of image features from distinct domains. We conduct extensive experiments to verify the effectiveness of our proposed method. Our code is accessible at https://github.com/yaboliudotug/MLFA.

COFT-AD: COntrastive Fine-Tuning for Few-Shot Anomaly Detection.

Existing approaches towards anomaly detection (AD) often rely on a substantial amount of anomaly-free data to train representation and density models. However, large anomaly-free datasets may not always be available before the inference stage; in which case an anomaly detection model must be trained with only a handful of normal samples, a.k.a. few-shot anomaly detection (FSAD). In this paper, we propose a novel methodology to address the challenge of FSAD which incorporates two important techniques. Firstly, we employ a model pre-trained on a large source dataset to initialize model weights. Secondly, to ameliorate the covariate shift between source and target domains, we adopt contrastive training to fine-tune on the few-shot target domain data. To learn suitable representations for the downstream AD task, we additionally incorporate cross-instance positive pairs to encourage a tight cluster of the normal samples, and negative pairs for better separation between normal and synthesized negative samples. We evaluate few-shot anomaly detection on 3 controlled AD tasks and 4 real-world AD tasks to demonstrate the effectiveness of the proposed method.

Unsupervised Domain Adaptation for Low-Dose CT Reconstruction via Bayesian Uncertainty Alignment.

Low-dose computed tomography (LDCT) image reconstruction techniques can reduce patient radiation exposure while maintaining acceptable imaging quality. Deep learning (DL) is widely used in this problem, but the performance of testing data (also known as target domain) is often degraded in clinical scenarios due to the variations that were not encountered in training data (also known as source domain). Unsupervised domain adaptation (UDA) of LDCT reconstruction has been proposed to solve this problem through distribution alignment. However, existing UDA methods fail to explore the usage of uncertainty quantification, which is crucial for reliable intelligent medical systems in clinical scenarios with unexpected variations. Moreover, existing direct alignment for different patients would lead to content mismatch issues. To address these issues, we propose to leverage a probabilistic reconstruction framework to conduct a joint discrepancy minimization between source and target domains in both the latent and image spaces. In the latent space, we devise a Bayesian uncertainty alignment to reduce the epistemic gap between the two domains. This approach reduces the uncertainty level of target domain data, making it more likely to render well-reconstructed results on target domains. In the image space, we propose a sharpness-aware distribution alignment (SDA) to achieve a match of second-order information, which can ensure that the reconstructed images from the target domain have similar sharpness to normal-dose CT (NDCT) images from the source domain. Experimental results on two simulated datasets and one clinical low-dose imaging dataset show that our proposed method outperforms other methods in quantitative and visualized performance.

Predicting Clinical Anticancer Drug Response of Patients by using Domain Alignment and Prototypical Learning.

Anticancer drug response prediction is crucial in developing personalized treatment plans for cancer patients. However, High-quality patient anticancer drug response data are scarce and cell line data and patient data have different distributions, models trained solely on cell line data perform poorly. Some existing methods predict anticancer drug response by transferring knowledge from the cell line domain to the patient domain using transfer learning. However, the robustness of these classifiers is affected by anomalies in the cell line data, and they do not utilize the knowledge in the unlabeled target domain data. To this end, we proposed a model called DAPL to predict patient responses to anticancer drugs. The model extracts domain-invariant features from cell lines and patients by constructing multiple VAEs and extracts drug features using GNNs. These features are then combined for prototypical learning to train a classifier, resulting in better predictions of patient anticancer drug response. We used the cell line datasets CCLE and GDSC as source domains and the patient datasets TCGA and PDTC as target domains and conducted experiments. The results indicate that DAPL shows excellent performance in predicting patient anticancer drug response compared to other state-of-the-art methods.

Multi-Source Data Integration for Segmentation of Unannotated MRI Images.

Automatic semantic segmentation of magnetic resonance imaging (MRI) images using deep neural networks greatly assists in evaluating and planning treatments for various clinical applications. However, training these models is conditioned on the availability of abundant annotated data. Even if we annotate enough data, MRI images display considerable variability due to factors such as differences among patients, MRI scanners, and imaging protocols. This variability necessitates retraining neural networks for each specific application domain, which, in turn requires manual annotation by expert radiologists for all new domains. To relax the need for persistent data annotation, we develop a method for unsupervised federated domain adaptation using multiple annotated source domains. Our approach enables the transfer of knowledge from several annotated source domains for use in an unannotated target domain. Initially, we ensure that the target domain data shares similar representations with each source domain in a latent embedding space by minimizing the pair-wise distances between the distributions for the target and the source domains. We then employ an ensemble approach to leverage the knowledge obtained from all domains to build an integrated outcome. We perform experiments on two datasets to demonstrate our method is effective. Our implementation code is publicly available: https://github.com/navapatn/Unsupervised -Federated-Domain-Adaptation-for-Image-Segmentation new.

Multiscale capsule networks with attention mechanisms based on domain-invariant properties for cross-domain lifetime prediction

Remaining useful life prediction (RUL) under variable operating conditions has been a hot research topic in engineering. Still, current unsupervised domain adaptive (UDA) methods all use a single metric (MK-MMD or adversarial mechanism), which somewhat improves the cross-domain RUL prediction performance. However, using MK-MMD or the adversarial mechanism alone to measure the inter-domain differences has a single perspective. At the same time, MK-MMD focuses on measuring the differences in the feature distribution layer and fails to capture the differences in the nonlinear relationship between the source and target domains. In addition, learning high-quality degraded features is also a vital issue in RUL prediction. Based on the above points, this paper proposes an attention mechanism multiscale capsule network cross-domain lifetime prediction method (DI-MCNAM) based on domain invariant properties. First, the attention mechanism multiscale capsule network (MCNAM) extracts the deep degradation features of the weighted data. Second, two modules, multilevel domain adaptation and domain classifier, are integrated to reduce inter-domain differences from different perspectives. The maximum information coefficient (MIC) is introduced in the multilevel domain adaptation module. MIC can learn the nonlinear relationship between source and target domain data over time separately, combined with MK-MMD, to reduce the inter-domain feature differences from different levels. Domain classifiers discriminate the confusing domains and reduce inter-domain differences by minimizing the classification power. The performance of the proposed model is validated using publicly available datasets and compared with existing popular methods, and the final results show that the proposed method has high prediction accuracy.

Targeted transfer learning through distribution barycenter medium for intelligent fault diagnosis of machines with data decentralization

Deep transfer learning-based fault diagnosis of machines is achieved based on the assumption that the source and target domain data could be centralized to assess the distribution discrepancy. In engineering scenarios, however, data centralization is difficult to true due to data privacy and the huge cost of large-volume data transmission and analysis. To achieve transfer fault diagnosis with respect to data decentralization, federated learning comes to reform the transfer fault diagnosis methods, where the intermedia distribution could serve as a medium to indirectly evaluate the cross-domain discrepancy instead of raw data centralization. Existing works have two weaknesses: (1) the conditional distribution discrepancy is mostly ignored by federated adaptation, and (2) there is no effective approach to generate the appreciated distribution medium. For these weaknesses, we propose a federated semi-supervised transfer fault diagnosis method called targeted transfer learning through distribution barycenter medium (TTL-DBM). The TTL-DBM contains a server and two clients respectively in the source and target domain sides. In the server, the auto-encoders are stacked to aggregate the key parameters of data distributions from both domain sides, and further generate the distribution barycenter as intermedia for federated adaptation. In two clients, the adaptation trajectory towards the distribution medium is designed according to the associated labels among the fully-labeled source domain and the one-shot labeled target domain. Through collaboratively training with the server and the clients, the joint distribution discrepancy across domains is reduced in a federated setting. The TTL-DBM is demonstrated in two cases including transfer diagnosis across different machine-used bearings as well as transfer diagnosis of industrial robots. The results show that TTL-DBM could obtain similar cross-domain features through the adaptation via distribution medium, and achieve higher diagnosis accuracy than other federated adaptation methods in the presence of data decentralization.

Photovoltaic power forecasting: A dual-attention gated recurrent unit framework incorporating weather clustering and transfer learning strategy

Accurate forecasting of photovoltaic power is essential in the integration, operation, and scheduling of hybrid energy systems. However, modeling for newly built photovoltaic sites is restricted by insufficient training data and computational burden. In this study, a weather clustering-based photovoltaic power forecasting framework incorporating attention mechanism and transfer learning strategy is proposed. By clustering historical days into multiple weather types, the gated recurrent unit-based encoder-decoders with dual-attention mechanism are designed to predict the photovoltaic power generations. The input attention and temporal attention mechanism are responsible for rebuilding input variables and context vectors of the encoder-decoder structure, respectively. Furthermore, a knowledge-transferring strategy, which focuses on establishing an alignment mapping module between the pre-trained structure and the target domain data, is designed for overcoming insufficient data of newly built sites. The data from the actual photovoltaic system are acquired to validate the proposed framework. The proposed forecasting model presents superior performance than other benchmark models, and the knowledge-transferring strategy not only addresses data shortage but also significantly accelerates the training process. With the introduction of knowledge-transferring, the maximum improvement in forecasting accuracy and training efficiency reaches 67.40% and 59.10%.

An Adversarial Single-Domain Generalization Network for Fault Diagnosis of Wind Turbine Gearboxes

In deep learning-based fault diagnosis of the wind turbine gearbox, a commonly faced challenge is the domain shift caused by differing operational conditions. Traditional domain adaptation methods aim to learn transferable features from the source domain and apply them to the target data. However, such methods still require access to target domain data during the training process, which limits their applicability in real-time fault diagnosis. To address this issue, we introduce an adversarial single-domain generalization network (ASDGN). It relies solely on data from a single length of data acquisition in wind turbine fault diagnosis. This novel approach introduces a more flexible and efficient solution to the field of real-time fault diagnosis for wind turbines.