Domain Transfer Research Articles

Remaining Useful Life Prediction based on Multisource Domain Transfer and Unsupervised Alignment

Transfer learning （TL） enhances remaining useful life (RUL) predictions by addressing data scarcity and operational challenges. Nonetheless, when a significant disparity in degradation data distribution exists between source and target domains, single-source domain TL may lead to misleading or negative transfer. Multisource domain TL partially mitigates these issues but fails to account for substantial discrepancies in feature-label correlations, impairing RUL prediction accuracy. To cope with this problem, we propose a multisource domain unsupervised adaptive learning method powered by a temporal convolutional network. Using a multilinear conditioning strategy, we combine degradation data and subregion labels to construct input characteristics for the domain discriminator. Additionally, we design a feature extractor that produces label-related features invariant across domains, thereby enhancing prediction precision. We evaluate our method using the publicly available C-MAPSS degradation dataset, demonstrating its effectiveness through a case study and ablation experiments.

Theory and algorithm for domain adaptation

Domain adaptation is an important subfield of transfer learning, it has been successfully applied in many applications of machine learning. Recently, significant theoretical and algorithmic advances have been achieved in domain adaptation. The theoretical analyses for domain adaptation are based on VC dimension and Rademacher complexity. There are also some covering number-based results, but most of these bounds are based on the results of Rademacher complexity, indirectly given by the relationship between covering number and Rademacher complexity. In this paper, we propose a theoretical analysis framework for domain adaptation, thus the error bound can be derived directly by covering number, which is an effective method for analyzing the generalization error in statistical learning theory. We derive generalization error bound for domain adaptation with a class of loss functions satisfying the assumptions. We also propose a mixup contrastive adversarial network for domain adaptation by introducing a mixup module for enhancing the alignment of the source and target domains during domain transfer, and a contrastive learning module for solving class-level alignment after domain transfer. Experimental results demonstrate the effectiveness of the proposed algorithm and the property of the theoretical results.

Linear parameter varying μ synthesis robust control of CDC semi‐active suspension system for nonlinearity and uncertainty

AbstractBecause the continuous damping control (CDC) semi‐active suspension system has complex nonlinear and uncertain characteristics that can significantly affect the performance of the system. And in the controller design process, achieving a balance of multiple performance objectives is often difficult. Therefore, this paper designs a linear parameter varying (LPV) μ synthesis robust controller for the vibration suppression and multi‐objective control of a 7‐degree‐of‐freedom (7‐DOF) vehicle suspension system equipped with CDC dampers. Firstly, the nonlinear force model of the solenoid valve CDC damper is developed based on the actual damping variation. Considering the nonlinear forces of the coil springs, the nonlinear model of the 7‐DOF suspension system is expressed as LPV form. Further, the linear fractional transformation (LFT) method is used to analyze and reconstruct the system model for multiple parameter uncertainties of the suspension. The actuator time delays are also simulated through a frequency domain transfer function, which is considered to be an unmodeled dynamic at the input of the system. Finally, an LPV‐μ synthesis robust controller based on nonlinearity and mixed uncertainty is designed to improve car ride comfort and achieve the best relationship between comfort and handling stability. Simulation and experimental results under random disturbance conditions show that the LPV‐μ synthesis controller has better anti‐jamming performance and controllability compared to the H∞ controller and μ synthesis controller.

Image-Based Generative Artificial Intelligence in Radiology: Comprehensive Updates.

Generative artificial intelligence (AI) has been applied to images for image quality enhancement, domain transfer, and augmentation of training data for AI modeling in various medical fields. Image-generative AI can produce large amounts of unannotated imaging data, which facilitates multiple downstream deep-learning tasks. However, their evaluation methods and clinical utility have not been thoroughly reviewed. This article summarizes commonly used generative adversarial networks and diffusion models. In addition, it summarizes their utility in clinical tasks in the field of radiology, such as direct image utilization, lesion detection, segmentation, and diagnosis. This article aims to guide readers regarding radiology practice and research using image-generative AI by 1) reviewing basic theories of image-generative AI, 2) discussing the methods used to evaluate the generated images, 3) outlining the clinical and research utility of generated images, and 4) discussing the issue of hallucinations.

Adversarial contrastive representation training with external knowledge injection for zero-shot stance detection

Zero-shot stance detection (ZSSD) is a task that involves identifying the author’s perspective on specific issues in text, particularly when the target topic has not been encountered during the model training process, to address rapidly evolving topics on social media. This paper introduces a ZSSD framework named KEL-CA. To enable the model to more effectively utilize transferable stance features for representing unseen targets, the framework incorporates a multi-layer contrastive learning and adversarial domain transfer module. Unlike traditional contrastive or adversarial learning, our framework captures both correlations and distinctions between invariant and specific features, as well as between different stance labels, and enhances the generalisation ability and robustness of the features. Subsequently, to address the problem of insufficient information about the target context, we designed a dual external knowledge injection module that uses a large language model (LLM) to extract external knowledge from a Wikipedia-based local knowledge base and a Chain-of-Thought (COT) process to ensure the timeliness and relevance of the knowledge to infer the stances of unseen targets. Experimental results demonstrate that our approach outperforms existing models on two benchmark datasets, thereby validating its efficacy in ZSSD tasks.

An effectual underwater image enhancement framework using adaptive trans-resunet ++ with attention mechanism

ABSTRACT The intricacy of the underwater setting makes it difficult for optical lenses to capture clear underwater photos without haze and colour distortion. Some studies use domain adaptation and transfer learning to address this issue, they aim to reduce the latent mismatch between composition and real-world data, making the space of latent data difficult to read and impractical to control. The background light is a crucial component of the decaying paradigm that directly impacts how well images are enhanced. Thus, to improve the quality of the images over the underwater, new deep-learning techniques are being designed in this paper. Here, the Adaptive Trans-ResUnet++ with Attention Mechanism-based model performs the real-time underwater image enhancement process. In addition, a novel Random Enhanced Artificial Gorilla Troops Optimizer algorithm model is used for optimising the parameters over the given model to further enhance the given model’s performance. A diverse quantitative and qualitative validation is also carried out to learn the enhancement of underwater image quality. The enhanced underwater image may be also useful in the underwater object detection process. Thus, the enhanced images obtained from the developed model are compared with the existing techniques to confirm the efficacy of the suggested underwater image enhancement process.

Multi-source domain transfer network based on subdomain adaptation and minimum class confusion for EEG emotion recognition

Electroencephalogram (EEG) signals, which objectively reflect the state of the brain, are widely favored in emotion recognition research. However, the presence of cross-session and cross-subject variation in EEG signals has hindered the practical implementation of EEG-based emotion recognition technologies. In this article, we propose a multi-source domain transfer method based on subdomain adaptation and minimum class confusion (MS-SAMCC) in response to the addressed issue. First, we introduce the mix-up data augmentation technique to generate augmented samples. Next, we propose a minimum class confusion subdomain adaptation method (MCCSA) as a sub-module of the multi-source domain adaptation module. This approach enables global alignment between each source domain and the target domain, while also achieving alignment among individual subdomains within them. Additionally, we employ minimum class confusion (MCC) as a regularizer for this sub-module. We performed experiments on SEED, SEED IV, and FACED datasets. In the cross-subject experiments, our method achieved mean classification accuracies of 87.14% on SEED, 63.24% on SEED IV, and 42.07% on FACED. In the cross-session experiments, our approach obtained average classification accuracies of 94.20% on SEED and 71.66% on SEED IV. These results demonstrate that the MS-SAMCC approach proposed in this study can effectively address EEG-based emotion recognition tasks.

A multi-source domain feature-decision dual fusion adversarial transfer network for cross-domain anti-noise mechanical fault diagnosis in sustainable city

Rotating machinery forms the critical backbone of infrastructure in a sustainable city, with bearings playing a pivotal role as key mechanical transmission components. Therefore, the health status of these bearings directly influences the safe operation of the infrastructure. Accurate and reliable diagnosis of defects in these components minimizes downtime, reduces maintenance costs, and prevents major accidents, ultimately providing insights in the construction and management of a sustainable city. Typically, in actual industrial scenarios, varying working conditions and various types of machines can result in significant discrepancies in the distribution of sample data. Moreover, the non-negligible noise may degrade the diagnostic performance. Therefore, realizing an accurate and reliable bearing diagnosis considering the cross-domain and noise environment remains a challenge. Leveraging the merits of information fusion and multi-source domain transfer learning, this article proposes a multi-source domain feature-decision dual fusion adversarial transfer network (DFATN) to break through the aforesaid limitations. Initially, an adversarial transfer framework is developed, incorporating novel feature matching evaluation and joint distribution difference losses. This framework is designed to facilitate the learning of feature invariants across domains and to enhance the sharing of domain-specific knowledge, even in noise. Relying on channel-spatial interactive feature fusion, a multi-scale feature extractor (MFE) is constructed to share the interaction and enhance the modeling of complex features in multiple dimensions. Additionally, a fault state-related decision fusion mechanism (SDF) is also implemented to integrate diagnostic information, significantly enhancing the generalization performance and robustness of the proposed network. By employing both public Paderborn University (PU) and laboratory-collected (Lab) datasets, the effectiveness and superiority of the proposed DFATN on bearing fault diagnosis are validated. For cross-working condition tasks, the proposed method realizes impressive performance, with average accuracies of 96.52% and 98.76% for Paderborn University (PU) and laboratory-collected (Lab) datasets, respectively. For cross-machine tasks, the average accuracy is 83.36%, outperforming other latest cross-domain fault diagnosis techniques.

FGTL: Federated Graph Transfer Learning for Node Classification

Unsupervised multi-source domain transfer in federated scenario has become an emerging research direction, which can help unlabeled target domain to obtain the adapted model through source domains under privacy-preserving. However, when local data is graph, the difference of domains (or data heterogeneity) mainly originate from the difference in node attributes and subgraph structures, leading to serious model drift, which is not considered by the existing related algorithms. Currently, there are two challenges in this scenario: (1) The node representations extracted directly through conventional GNNs lack inter-domain generalized and consistent information, making it difficult to apply existing federated learning algorithms. (2) The knowledge of source domains has quality differences, which may lead to negative transfer. To address these issues, we propose a novel two-phase Federated Graph Transfer Learning (FGTL) framework. In the generalization phase, FGTL utilizes local contrastive learning and global context embedding to force node representations to capture the inter-domain generalized and consistent information, lightly alleviating model drift. In the transfer phase, FGTL utilizes consensus knowledge to force the decision bound of classifier to adapt to the target client. In addition, FGTL+ exploits model grouping to make consensus knowledge generation more efficient, further enhancing the scalability of FGTL. Extensive experiments show that FGTL significantly outperforms state-of-the-art related methods, while FGTL+ further enhances privacy protection and reduces both communication and computation overhead.

Deep Learning for Generating Time-of-Flight Camera Artifacts.

Time-of-Flight (ToF) cameras are subject to high levels of noise and errors due to Multi-Path-Interference (MPI). To correct these errors, algorithms and neuronal networks require training data. However, the limited availability of real data has led to the use of physically simulated data, which often involves simplifications and computational constraints. The simulation of such sensors is an essential building block for hardware design and application development. Therefore, the simulation data must capture the major sensor characteristics. This work presents a learning-based approach that leverages high-quality laser scan data to generate realistic ToF camera data. The proposed method employs MCW-Net (Multi-Level Connection and Wide Regional Non-Local Block Network) for domain transfer, transforming laser scan data into the ToF camera domain. Different training variations are explored using a real-world dataset. Additionally, a noise model is introduced to compensate for the lack of noise in the initial step. The effectiveness of the method is evaluated on reference scenes to quantitatively compare to physically simulated data.

Research on digital vibration model of typical components in pipe system

The spatial distribution of pipe system is complex, long span and multi-point elastic support, so vibration analysis of pipe system is required in the pipeline design. This paper takes the pipe system as a typical component, based on the Timoshenko beam model, establishes a dynamic procedure for straight pipe elements that considers the axial flow rate of fluid, the pressure on the pipe wall, the Poisson coupling effect of fluid and pipeline structure, and the axial, transverse and torsional vibration. Through Laplace transform, the frequency simplified dynamic model is established. And the frequency transfer matrix model of straight pipe elements and the boundary constraint matrix of common boundary (such as fixed support) is established. Finally, the frequency domain transfer matrix model of different pipeline elements are set to the overall transfer matrix of the flow pipe system, providing a convenient means for rapid optimization of pipe system vibration.

Improving a shell-tube latent heat thermal energy storage unit for building hot water demand using metal foam inserts at a constant pumping power

The phase transition heat transfer during the melting and solidification processes of phase change materials (PCMs) was modeled in a shell-tube thermal energy storage unit. For the first time, special attention was dedicated to the heat transfer enhancement on the heat transfer fluid (HTF) side by placing metal foam (MF) inserts in the HTF tube. The MF inserts on the HTF side were placed next to the fin bases to produce the maximum heat transfer enhancement. A two-temperature local thermal non-equilibrium model was applied to accurately model the transient heat transfer in the MF domains. To perform a fair heat transfer analysis and consider the hydrodynamic aspects of heat transfer, the pumping power for driving the HTF fluid was kept fixed. The finite element method with automatic time-step control was used to integrate the model equations. Results show that the insertion of the MF in the HTF tube enhances conduction heat transfer but reduces flow for a constant pump power. Using a small amount of MF provides a good flow rate and a convective heat transfer while using a large amount of MF provides good conduction heat transfer but a poor flow rate and convection. A moderate amount of MF inserts results in poor conduction and a poor flow rate, notably increasing the phase transition time. A 5 % MF insert could provide about a 13 % shorter solidification time compared to a 25 % MF insert. Thus, a well-designed and engineered MF insert configuration on the HTF side can reduce the phase transition time and improve the energy storage power.

Transfer graph feature alignment guided multi-source domain adaptation network for machinery fault diagnosis

In recent years, the application of unsupervised multi-source domain adaptation (MSDA) techniques for fault diagnosis has gained significant traction. Current research typically overlooks or fails to effectively capture critical data structure information during feature extraction. Another challenge is optimising the integration of information from multiple source domains to diagnose the target domain while avoiding negative transfer. To address these challenges, this study proposes a transfer graph feature alignment guided multi-source domain adaptation network (MDTGAL). In the proposed method, a transfer graph sample generator module (GSG) is constructed to model the data structure between source and target domains, and multiple graph feature extractors are employed to learn the data structure information from different domain combinations. A regularisation technique is introduced to extract the domain-invariant features by aligning the parameters across multiple independent graph feature extractors. In addition, a weighted soft-voting mechanism based on the polynomial kernel-induced maximum mean difference metric (PK-MMD) is designed to fuse the outputs from multiple classifiers, to comprehensively account for the influence of each source domain. The proposed method was tested on multi-source domain transfer tasks involving various operating conditions of rotating machinery. The experimental results demonstrate that the MDTGAL exhibits superior cross-domain diagnostic performance, outperforming existing mainstream methods. In addition, this study explored the impact of varying numbers of source domains on the diagnostic accuracy of the target domain, providing insights into selecting the correct number of source domains for specific MSDA tasks.

Language Model Adaptation for Legal Ukrainian Domain

Language models in recent decades make a huge step towards solving the tasks that previously could be done only by humans. Development of NLP area is different scopes gives an opportunity to solve domain specific tasks and transfer knowledge from learnt data towards the useful inferences based on that. This article provides the NLP model approach in specific legal domain. Additionally, this article explores performance of pre-training small models and its utilization and checks the scores on fine-tuned task of checking sentence similarities via SBERT. According to this articles it is proven that domain-specific pre-trained models can perform better results than generally trained language model. This article also provides the language model that is adopted to the Ukrainian legal domain.

Leukocyte classification using relative-relationship-guided contrastive learning

Hematologic diseases and blood disorders can be studied through microscopic examination of blood smear images or chemical assays. Many researchers are focused on utilizing deep learning (DL) to identify, quantify, and classify various types of blood cells, which is crucial for addressing theoretical and practical challenges in disease diagnosis and treatment planning. However, there is a significant deficiency in annotations for leukocyte classification, limiting the effectiveness of current DL methods. Contrastive learning (CL) facilitates the extraction of prior knowledge from unlabeled data, reducing the dependency on manual annotations. While CL demonstrates remarkable achievements in classifying natural images, the direct application to leukocyte classification presents certain defects. We observe that standard data augmentation techniques in CL exhibit varying sensitivity to different categories of white blood cell images. Employing a uniform augmentation strategy may lead the network into an optimization trap, acquiring inadequate representations from erroneous sample relationships. To address this issue, we propose a Relative-Relationship-Guided Contrastive Learning Representation (ReCLR) framework for the leukocyte classification. ReCLR mines positive and negative samples based on relative distance knowledge. Specifically, we provide adversarial guidance for the positive sample, restricting the distance between the positive sample and the original sample less than but as close as possible to the distance between the original sample and the farthest negative sample. Then, we utilize the entropy constraint to regulate the relative distance relationship between the negative and original samples. Finally, the guided positive and negative samples are employed for contrastive learning in leukocyte classification. The extensive experiments, including linear evaluation, domain transfer, and fine-tuning, demonstrate the effectiveness of the proposed method. Our ReCLR achieves accuracies of 92.07%, 65.36%, and 92.49% on three real-world leukocyte datasets, respectively, outperforming several state-of-the-art methods. The source code is released at https://github.com/AlchemyEmperor/ReCLR.

Recognition of Molecular Structure of Phosphonium Salts from the Visual Appearance of Material with Deep Learning Can Reveal Subtle Homologs.

Determining molecular structures is foundational in chemistry and biology. The notion of discerning molecular structures simply from the visual appearance of a material remained almost unthinkable until the advent of machine learning. This paper introduces a pioneering approach bridging the visual appearance of materials (both at the micro- and nanostructural levels) with traditional chemical structure analysis methods. Quaternary phosphonium salts are opted as the model compounds, given their significant roles in diverse chemical and medicinal fields and their ability to form homologs with only minute intermolecular variances. This research results in the successful creation of a neural network model capable of recognizing molecular structures from visual electron microscopy images of the material. The performance of the model is evaluated and related to the chemical nature of the studied chemicals. Additionally, unsupervised domain transfer is tested as a method to use the resulting model on optical microscopy images, as well as test models trained on optical images directly. The robustness of the method is further tested using a complex system of phosphonium salt mixtures. To the best of the authors' knowledge, this study offers the first evidence of the feasibility of discerning nearly indistinguishable molecular structures.

Local Modeling by Adapting Source Calibration Models to Analyte Shifted Target Domain Samples Without Reference Values.

Spectral multivariate calibration aims to derive models characterizing mathematical relationships between sample analyte amounts and corresponding spectral responses. These models are effective at predicting target domain sample analyte amounts when target samples are within the analyte and spectral calibration source domain. Models fail when target samples shift (analyte amounts and/or spectra) from the original calibration domain model. A total recalibration solution requires acquisition of new sample reference values and spectra. However, obtaining enough reference values to distinguish the target domain may be challenging or expensive. A simpler approach adapts the original model to the target domain using target sample spectra without analyte reference values (unlabeled). Analytical chemists have developed several machine learning algorithms using unlabeled regression domain adaptation processes. Unfortunately, prediction accuracy declines for these methods depending on how much the target domain analyte distribution has shifted from the calibration distribution, and regression transfer learning methods are instead needed. Regression domain adaptation and transfer learning are often referred to as model updating in analytical chemistry, but regression domain adaptation only applies to spectral shifts. The regression transfer learning method presented in this paper named null augmentation regression constant analyte (NARCA) leverages unlabeled repeat spectra of a single target sample to update an original calibration model to the shifted target domain sample. With sample repeat spectra, the analyte amount can be assumed constant or nearly constant for NARCA and because models are formed for one sample, NARCA operates as a local modeling method. The performance of NARCA as a regression transfer learning method is evaluated using five near-infrared data sets.

Cross-Modal Face Super-Resolution Based on Quasi-Siamese Domain Transfer Fusion Network

In this paper, we propose a Cross-Modal Face Super-Resolution (CMFSR) method to construct high-resolution (HR) facial images from low-resolution (LR) cross-modal facial images captured respectively by disjoint visible light (VIS) and near-infrared (NIR) cameras. Due to the coupling of modality transformation and information fusion, CMFSR is more difficult to obtain HR reconstructed results compared with traditional super-resolution. To solve this problem, a Quasi-Siamese Domain Transfer Fusion Network (QSDTFN) for CMFSR is proposed in this paper, whose two branches transfer two LR face modality to HR face modality by domain transfer respectively. Different from two completely independent branches in the traditional pseudo-siamese network, only the HR-to-LR face transfer processes of the two branches in our quasi-siamese network are independent, while the LR-to-HR face transfer processes are coupled. This coupled module called the Adaptive Weighted Domain Transfer Fusion Module (AWDTFM) disentangles the modality and identity information in the two LR faces, thus achieving modality transformation and identity information fusion simultaneously. In order to strengthen the optimization on the process of CMFSR, this method further introduces the backward QSDTFN to form a higher-level bidirectional structure with the forward QSDTFN, and specifically designs two types of losses: intra-network loss and inter-network loss, to constrain the modality and identity consistencies within one QSDTFN and between two QSDTFNs respectively. The experimental results on the challenging LR cross-modal face datasets demonstrate that the proposed method performs favorably against the state-of-the-art methods.

Multi-source transfer learning method for enhancing the deployment of deep reinforcement learning in multi-zone building HVAC control

Deep reinforcement learning (DRL) control methods have shown great potential for optimal HVAC control, but they require significant time and data to learn effective policies. By employing transfer learning (TL) with pre-trained models, the need to learn the data from scratch is avoided, saving time and resources. However, there are two main critical issues with this approach: the inappropriate selection of the source domain resulting in worse control performance and inefficient utilization of multi-source domain control experience. To address these challenges, a multi-source transfer learning and deep reinforcement learning (MTL-DRL) integrated framework is proposed for efficient HVAC system control. In order to select appropriate source domains, the contribution of various source domains to the target task is quantified first, followed by a comprehensive evaluation of transfer performance based on average energy consumption and average temperature deviation. The well-pretrained DRL parameters from the optimal multi-source transfer set are then sequentially transferred to the target DRL controller. Results from a series of transfer experiments between buildings with different thermal zones and weather conditions indicate that the MTL-DRL framework significantly reduces the training time of HVAC control, with improvements of up to 20% compared to DRL baseline models trained from scratch. Additionally, the MTL-DRL method leads to reductions in average energy consumption ranging from 1.43% to 3.12% and average temperature deviation up to 14.32%. The impact of the source domain transfer sequence on the performance of the DRL-based control method is also discussed. Overall, the proposed framework presents a promising solution for enhancing DRL-based HVAC control methods by reducing training time and energy consumption while maintaining occupants’ comfort.

Inter-participant transfer learning with attention based domain adversarial training for P300 detection

A Brain-computer interface (BCI) system establishes a novel communication channel between the human brain and a computer. Most event related potential-based BCI applications make use of decoding models, which requires training. This training process is often time-consuming and inconvenient for new users. In recent years, deep learning models, especially participant-independent models, have garnered significant attention in the domain of ERP classification. However, individual differences in EEG signals hamper model generalization, as the ERP component and other aspects of the EEG signal vary across participants, even when they are exposed to the same stimuli. This paper proposes a novel One-source domain transfer learning method based Attention Domain Adversarial Neural Network (OADANN) to mitigate data distribution discrepancies for cross-participant classification tasks. We train and validate our proposed model on both a publicly available OpenBMI dataset and a Self-collected dataset, employing a leave one participant out cross validation scheme. Experimental results demonstrate that the proposed OADANN method achieves the highest and most robust classification performance and exhibits significant improvements when compared to baseline methods (CNN, EEGNet, ShallowNet, DeepCovNet) and domain generalization methods (ERM, Mixup, and Groupdro). These findings underscore the efficacy of our proposed method.