Domain-specific Features Research Articles

Information filtering and interpolating for semi-supervised graph domain adaptation

Graph domain adaptation, which falls under the umbrella of graph transfer learning, involves transferring knowledge from a labeled source graph to improve prediction accuracy on an unlabeled target graph, where both graphs have identical label spaces but exhibit distribution discrepancies due to temporal data shifts or distinct data collection methods. This adaptation is complicated by the challenges of graph-specific domain discrepancies and cross-graph label scarcity. This paper proposes a semi-supervised Graph domain adaptation method via Information Filtering and Interpolating (GIFI). Specifically, GIFI utilizes a parameterized graph reduction module and a variational information bottleneck to adequately filter out irrelevant information from the source and target graphs to eliminate distribution discrepancy. GIFI also introduces an interpolation-enhanced pseudo-labeling strategy for cross-graph semi-supervised learning, which can mitigate model over-fitting on domain-specific features and limited labeled nodes, thus improving the model’s adaptation and discriminative capability. Experimental results on various graph domain adaptation benchmarks demonstrate GIFI’s superior performance over state-of-the-art methods. Our code is available at https://github.com/joe817/GIFI.

Cross-domain knowledge collaboration for blending-target domain adaptation

Unsupervised domain adaptation (UDA) avoids expensive data annotation for the unlabeled target domains by fully utilizing the knowledge of existing source domain. In practice, the target data are usually highly heterogeneous that mix multiple latent domains. And sometimes the source data involve the user private information which is forbidden to access directly. To this end, this paper tackles the blending-target data under the source-available setting, and for the first time under the source-free setting. Specifically, we devise a novel Cross-domain Knowledge Collaboration (CdKC) framework, which mainly includes a prediction network and an adaptation network. The complementarity of two networks are exploited to explore the intrinsic structure in targets. Specifically, CdKC is capable of learning domain-invariant space, and disentangling domain-specific feature simultaneously to boost the UDA performance greatly. A total of 12 tasks are conducted on three visual datasets to verify the superior performance of CdKC by comparing with the state-of-the-art models designed under 4 different UDA settings. The experiments demonstrate that the accuracy of CdKC model still exceeds D-CGCT 0.4% on Office dataset and 1.2% on Office–Home dataset although D-CGCT can access source data and domain labels of targets that CdKC cannot do. It verifies the effectiveness of CdKC although under much looser source and target domain restrictions.

Automating Feature Extraction from Entity-Relation Models: Experimental Evaluation of Machine Learning Methods for Relational Learning

With the exponential growth of data, extracting actionable insights becomes resource-intensive. In many organizations, normalized relational databases store a significant portion of this data, where tables are interconnected through some relations. This paper explores relational learning, which involves joining and merging database tables, often normalized in the third normal form. The subsequent processing includes extracting features and utilizing them in machine learning (ML) models. In this paper, we experiment with the propositionalization algorithm (i.e., Wordification) for feature engineering. Next, we compare the algorithms PropDRM and PropStar, which are designed explicitly for multi-relational data mining, to traditional machine learning algorithms. Based on the performed experiments, we concluded that Gradient Boost, compared to PropDRM, achieves similar performance (F1 score, accuracy, and AUC) on multiple datasets. PropStar consistently underperformed on some datasets while being comparable to the other algorithms on others. In summary, the propositionalization algorithm for feature extraction makes it feasible to apply traditional ML algorithms for relational learning directly. In contrast, approaches tailored specifically for relational learning still face challenges in scalability, interpretability, and efficiency. These findings have a practical impact that can help speed up the adoption of machine learning in business contexts where data is stored in relational format without requiring domain-specific feature extraction.

Offloading the computational complexity of transfer learning with generic features.

Deep learning approaches are generally complex, requiring extensive computational resources and having high time complexity. Transfer learning is a state-of-the-art approach to reducing the requirements of high computational resources by using pre-trained models without compromising accuracy and performance. In conventional studies, pre-trained models are trained on datasets from different but similar domains with many domain-specific features. The computational requirements of transfer learning are directly dependent on the number of features that include the domain-specific and the generic features. This article investigates the prospects of reducing the computational requirements of the transfer learning models by discarding domain-specific features from a pre-trained model. The approach is applied to breast cancer detection using the dataset curated breast imaging subset of the digital database for screening mammography and various performance metrics such as precision, accuracy, recall, F1-score, and computational requirements. It is seen that discarding the domain-specific features to a specific limit provides significant performance improvements as well as minimizes the computational requirements in terms of training time (reduced by approx. 12%), processor utilization (reduced approx. 25%), and memory usage (reduced approx. 22%). The proposed transfer learning strategy increases accuracy (approx. 7%) and offloads computational complexity expeditiously.

Geometry-Guided Domain Generalization for Monocular 3D Object Detection

Monocular 3D object detection (M3OD) is important for autonomous driving. However, existing deep learning-based methods easily suffer from performance degradation in real-world scenarios due to the substantial domain gap between training and testing. M3OD's domain gaps are complex, including camera intrinsic parameters, extrinsic parameters, image appearance, etc. Existing works primarily focus on the domain gaps of camera intrinsic parameters, ignoring other key factors. Moreover, at the feature level, conventional domain invariant learning methods generally cause the negative transfer issue, due to the ignorance of dependency between geometry tasks and domains. To tackle these issues, in this paper, we propose MonoGDG, a geometry-guided domain generalization framework for M3OD, which effectively addresses the domain gap at both camera and feature levels. Specifically, MonoGDG consists of two major components. One is geometry-based image reprojection, which mitigates the impact of camera discrepancy by unifying intrinsic parameters, randomizing camera orientations, and unifying the field of view range. The other is geometry-dependent feature disentanglement, which overcomes the negative transfer problems by incorporating domain-shared and domain-specific features. Additionally, we leverage a depth-disentangled domain discriminator and a domain-aware geometry regression attention mechanism to account for the geometry-domain dependency. Extensive experiments on multiple autonomous driving benchmarks demonstrate that our method achieves state-of-the-art performance in domain generalization for M3OD.

Graph Disentangled Contrastive Learning with Personalized Transfer for Cross-Domain Recommendation

Cross-Domain Recommendation (CDR) has been proven to effectively alleviate the data sparsity problem in Recommender System (RS). Recent CDR methods often disentangle user features into domain-invariant and domain-specific features for efficient cross-domain knowledge transfer. Despite showcasing robust performance, three crucial aspects remain unexplored for existing disentangled CDR approaches: i) The significance nuances of the interaction behaviors are ignored in generating disentangled features; ii) The user features are disentangled irrelevant to the individual items to be recommended; iii) The general knowledge transfer overlooks the user's personality when interacting with diverse items. To this end, we propose a Graph Disentangled Contrastive framework for CDR (GDCCDR) with personalized transfer by meta-networks. An adaptive parameter-free filter is proposed to gauge the significance of diverse interactions, thereby facilitating more refined disentangled representations. In sight of the success of Contrastive Learning (CL) in RS, we propose two CL-based constraints for item-aware disentanglement. Proximate CL ensures the coherence of domain-invariant features between domains, while eliminatory CL strives to disentangle features within each domains using mutual information between users and items. Finally, for domain-invariant features, we adopt meta-networks to achieve personalized transfer. Experimental results on four real-world datasets demonstrate the superiority of GDCCDR over state-of-the-art methods.

Advancing Message Board Topic Modeling Through Stack Ensemble Techniques

In the digital era, message boards serve as vital hubs for diverse discussions, knowledge dissemination, and community interaction. However, navigating the vast and varied content on these platforms presents a formidable challenge. This research pioneers the utilization of stack ensemble techniques to revolutionize topic modeling on message board data. Integrating Latent Dirichlet Allocation (LDA), Non-negative Matrix Factorization (NMF), and Latent Semantic Analysis (LSA) within a sophisticated ensemble framework, this study introduces a paradigm shift in extracting nuanced insights. Incorporating domain-specific features, sentiment analysis, and temporal patterns enriches contextual understanding. Rigorous evaluation across diverse message board datasets underscores the ensemble method's unparalleled accuracy, stability, and interpretability, setting a new standard for discourse analysis in online communities. Keywords: Topic Modeling, Latent Dirichlet Allocation, Stack Ensemble Techniques, Natural Language Processing, Message Boards, Ensemble Learning

A self-attention knowledge domain adaptation network for commercial lithium-ion batteries state-of-health estimation under shallow cycles

Accurate state-of-health (SOH) estimation is crucial for the safety, efficiency, and reliability of battery-powered systems. Conventional SOH estimation methods are designed for the full state-of-charge (SOC) range of 0%–100%, assuming uniform distributions. Yet, batteries in real-world applications often operate under partial SOC ranges and experience shallow-cycle conditions, which result in distinct and unlabeled degradation profiles that make SOH estimation more complex. To tackle this issue, we propose a novel method—self-attention knowledge domain adaptation network (SKDAN)—which utilizes a self-attention distillation module combined with a multi-kernel maximum mean discrepancy framework to effectively navigate between these disparate domains. This strategy successfully extracts domain-specific features from charging curves, facilitating the transfer of knowledge from well-labeled full cycles to unlabeled shallow cycles. The effectiveness of the SKDAN method is validated through extensive testing on the CALCE and SNL battery datasets, demonstrating its robust capability to estimate SOH accurately across different SOC ranges, temperatures, and discharge rates. With a root-mean-square error (RMSE) of less than 2%, the SKDAN method outperforms existing transfer learning techniques for various SOC ranges. Additionally, it exhibits exceptional SOH estimation performance when applied to batteries from different manufacturers and operating under varied conditions. This study represents the first instance of accurately estimating battery SOH under shallow-cycle conditions without relying on full-cycle characteristic tests.

Dual-Teacher Feature Distillation: A Transfer Learning Method for Insomniac PSG Staging.

Insomnia is the most common sleep disorder linked with adverse long-term medical and psychiatric outcomes. Automatic sleep staging plays a crucial role in aiding doctors to diagnose insomnia disorder. Only a few studies have been conducted to develop automatic sleep staging methods for insomniacs, and most of them have utilized transfer learning methods, which involve pre-training models on healthy individuals and then fine-tuning them on insomniacs. Unfortunately, significant differences in feature distribution between the two subject groups impede the transfer performance, highlighting the need to effectively integrate the features of healthy subjects and insomniacs. In this paper, we propose a dual-teacher cross-domain knowledge transfer method based on the feature-based knowledge distillation to improve the performance of sleep staging for insomniacs. Specifically, the insomnia teacher directly learns from insomniacs and feeds the corresponding domain-specific features into the student network, while the health domain teacher guide the student network to learn domain-generic features. During the training process, we adopt the OFD (Overhaul of Feature Distillation) method to build the health domain teacher. We conducted the experiments to validate the proposed method, using the Sleep-EDF database as the source domain and the CAP-Database as the target domain. The results demonstrate that our method surpasses advanced techniques, achieving an average sleep staging accuracy of 80.56% on the CAP-Database. Furthermore, our method exhibits promising performance on the private dataset.

Feature Re-Representation and Reliable Pseudo Label Retraining for Cross-Domain Semantic Segmentation.

This paper presents a novel unsupervised domain adaptation method for semantic segmentation. We argue that a good representation of the target-domain data should keep both the knowledge from the source domain and the target-domain-specific information. To obtain the knowledge from the source domain, we first learn a set of bases to characterize the feature distribution of the source domain, then features from both the source and the target domain are re-represented as a weighted summation of the source bases. A discriminator is additionally introduced to make the re-representation responsibilities of both domain features under the same bases indistinguishable. In this way, the domain gap between the source re-representation and target re-representation is minimized, and the re-represented target domain features contain the source domain information. Then we combine the feature re-representation with the original domain-specific feature together for subsequent pixel-wise classification. To further make the re-represented target features semantically meaningful, a Reliable Pseudo Label Retraining (RPLR) strategy is proposed, which utilizes the consistency of the prediction by the networks trained with multi-view source images to select the clean pseudo labels on unlabeled target images for re-training. Extensive experiments demonstrate the competitive performance of our approach for unsupervised domain adaptation on the semantic segmentation benchmarks.

CR-TransR: A Knowledge Graph Embedding Model for Cultural Domain

As a combination of information computing technology and the cultural field, cultural computing is gaining more attention. The knowledge graph is also gradually applied as a particular data structure in the cultural area. Based on the domain knowledge graph data of the Beijing Municipal Social Science Project “Mining and Utilization of Cultural Resources in the Ancient Capital of Beijing,” this article proposes a graph representation learning model CR-TransR that integrates cultural attributes. Through the analysis of the data in the cultural field of the ancient capital of Beijing, a cultural feature dictionary is constructed, and a domain-specific feature matrix is constructed in the form of word vector splicing. The feature matrix is used to constrain the embedding graph model TransR, and then the feature matrix and the TransR model are jointly trained to complete the embedded expression of the knowledge graph. Finally, a comparative experiment is carried out on the Beijing ancient capital cultural knowledge graph dataset and the effects of the classic graph embedding algorithms TransE, TransH, and TransR. At the same time, we try to reproduce the embedding method with the core idea of neighbor node information aggregation as the core idea, and CRTransR are compared. The experimental tasks include link prediction and triplet classification, and the experimental results show that the CRTransR model performs better.

Intelligent fault diagnosis method of rolling bearing based on multi-source domain fast adversarial network

Fault diagnosis of rolling bearings is among the most crucial links in the prognostic and health management of bearings. To solve the problem of single-source domain transfer learning that cannot adapt well to the target domain, a transfer diagnosis method based on multi-source domain fast adversarial network (MSDFAN) is proposed. First, signals from all domains are input into a common subnetwork of fast neural networks to reduce the complexity and network running time of neural networks. Secondly, several adversarial networks are constructed as domain specific feature extractors and then use Higher-order Moment Matching to reduce distribution differences between A and B domains. The two experimental cases of rolling bearing support the effectiveness and superiority of the proposed MSDFAN.

CrossFormer: Cross-guided attention for multi-modal object detection

Object detection is one of the essential tasks in a variety of real-world applications such as autonomous driving and robotics. In a real-world scenario, unfortunately, there are numerous challenges such as illumination changes, adverse weather conditions, and geographical changes, to name a few. To tackle the problem, we propose a novel multi-modal object detection model that is built upon a hierarchical transformer and cross-guidance between different modalities. The proposed hierarchical transformer consists of domain-specific feature extraction networks where intermediate features are connected by the proposed Cross-Guided Attention Module (CGAM) to enrich their representational power. Specifically, in the CGAM, one domain is regarded as a guide and the other is assigned to a base. After that, the cross-modal attention from the guide to the base is applied to the base feature. The CGAM works bidirectionally in parallel by exchanging roles between modalities to refine multi-modal features simultaneously. Experimental results on FLIR-aligned, LLVIP, and KAIST multispectral pedestrian datasets demonstrate that the proposed method is superior to previous multi-modal detection algorithms quantitatively and qualitatively.

MISNet: multi-source information-shared EEG emotion recognition network with two-stream structure.

When constructing machine learning and deep neural networks, the domain shift problem on different subjects complicates the subject independent electroencephalography (EEG) emotion recognition. Most of the existing domain adaptation methods either treat all source domains as equivalent or train source-specific learners directly, misleading the network to acquire unreasonable transfer knowledge and thus resulting in negative transfer. This paper incorporates the individual difference and group commonality of distinct domains and proposes a multi-source information-shared network (MISNet) to enhance the performance of subject independent EEG emotion recognition models. The network stability is enhanced by employing a two-stream training structure with loop iteration strategy to alleviate outlier sources confusing the model. Additionally, we design two auxiliary loss functions for aligning the marginal distributions of domain-specific and domain shared features, and then optimize the convergence process by constraining gradient penalty on these auxiliary loss functions. Furthermore, the pre-training strategy is also proposed to ensure that the initial mapping of shared encoder contains sufficient emotional information. We evaluate the proposed MISNet to ascertain the impact of several hyper-parameters on the domain adaptation capability of network. The ablation experiments are conducted on two publically accessible datasets SEED and SEED-IV to assess the effectiveness of each loss function. The experimental results demonstrate that by disentangling private and shared emotional characteristics from differential entropy features of EEG signals, the proposed MISNet can gain robust subject independent performance and strong domain adaptability.

Deep Multi-task Learning for Animal Chest Circumference Estimation from Monocular Images

The applications of deep learning algorithms with images to various scenarios have attracted significant research attention. However, application scenarios in animal breeding managements are still limited. In this paper we propose a new deep learning framework to estimate the chest circumference of domestic animals from images. This parameter is a key metric for breeding and monitoring the quality of animal in animal husbandry. We design a set of feature extraction methods based on a multi-task learning framework to address the challenging issues in the main estimation task. The multiple tasks in our proposed framework include object segmentation, keypoint estimation, and depth estimation of cow from monocular images. The domain-specific features extracted from these tasks improve upon our main estimation task. In addition, we also attempt to reduce unnecessary computations during the framework design to reduce the cost of subsequent practical implementation of the developed system. Our proposed framework is tested on our own collected dataset to evaluate its performance.

An Ensemble Strategy with Gradient Conflict for Multi-Domain Neural Machine Translation

Multi-domain neural machine translation aims to construct a unified neural machine translation model to translate sentences across various domains. Nevertheless, previous studies have one limitation is the incapacity to acquire both domain-general and domain-specific representations concurrently. To this end, we propose an ensemble strategy with gradient conflict for multi-domain neural machine translation that automatically learns model parameters by identifying both domain-shared and domain-specific features. Specifically, our approach consists of (1) a parameter-sharing framework, where the parameters of all the layers are originally shared and equivalent to each domain, and (2) ensemble strategy, in which we design an Extra Ensemble strategy via a piecewise condition function to learn direction and distance-based gradient conflict. In addition, we give a detailed theoretical analysis of the gradient conflict to further validate the effectiveness of our approach. Experimental results on two multi-domain datasets show the superior performance of our proposed model compared to previous work.

Enhancing Energy Efficiency and Classification Modeling Through a Combined Approach of LightGBM and Stratified KFold Cross-Validation

This research article aims to develop a robust and interpretable model for energy efficiency and classification that overcomes the limitations of existing techniques. The study’s objectives are to address the challenges faced by the established methods such as linear regression, random forest, support vector machines, and others. These limitations include issues related to predictive accuracy, model interpretability, and computational efficiency. To bridge these gaps, we introduce a novel model, stratified KFold LightGBM ensemble (SKL-EEFE). SKL-EEFE leverages the strengths of stratified KFold cross-validation, LightGBM gradient boosting, domain-specific feature engineering, and ensemble learning to provide a comprehensive solution to classification tasks. Our key findings reveal that SKL-EEFE outperforms existing methods while maintaining model interpretability. The proposed model offers both accuracy and transparency, making it a valuable addition to the field of classification modeling. This research contributes to advancing the state of knowledge by offering an innovative approach to overcome the limitations of existing models and providing a practical solution for real-world classification problems.

Model-Agnostic Binary Patch Grouping for Bone Marrow Whole Slide Image Representation

Histopathology is the reference standard for pathology diagnosis, and has evolved with the digitization of glass slides, i.e. whole slide images (WSIs). Trained histopathologists can help to diagnose disease by examining WSIs visually, but this process is time-consuming and prone to variability. To address these issues, AI models are being developed to create slide-level representations of WSIs, summarizing the entire slide as a single vector. This enables various computational pathology applications, including inter-slide search, multi-modal training, and slide-level classification. Achieving expressive and robust slide-level representations hinges on patch feature extraction and aggregation steps. We propose integrating an additional Binary Patch Grouping (BPG) step, a plugin that can be integrated into various slide-level representation pipelines to enhance the quality of slide-level representation in bone marrow histopathology. BPG excludes patches with less clinical relevance through minimal interaction with the pathologist: a one-time human intervention for the entire process. We further investigated domain-general versus domain-specific feature extraction models based on convolution and attention and examined two different feature aggregation methods, with and without BPG, showing BPG’s generalizability. We show that BPG boosts the performance of WSI retrieval (mAP@10) by 4% and improves WSI classification (weighted-F1) by 5% relative to not using BPG. Additionally, we found that the pipeline with BPG, domain-general large models and parameterized pooling produced the best-quality slide-level representations.

Energy-guided diffusion model for CBCT-to-CT synthesis

Cone Beam Computed Tomography (CBCT) plays a crucial role in Image-Guided Radiation Therapy (IGRT), providing essential assurance of accuracy in radiation treatment by monitoring changes in anatomical structures during the treatment process. However, CBCT images often face interference from scatter noise and artifacts, posing a significant challenge when relying solely on CBCT for precise dose calculation and accurate tissue localization. There is an urgent need to enhance the quality of CBCT images, enabling a more practical application in IGRT. This study introduces EGDiff, a novel framework based on the diffusion model, designed to address the challenges posed by scatter noise and artifacts in CBCT images. In our approach, we employ a forward diffusion process by adding Gaussian noise to CT images, followed by a reverse denoising process using ResUNet with an attention mechanism to predict noise intensity, ultimately synthesizing CBCT-to-CT images. Additionally, we design an energy-guided function to retain domain-independent features and discard domain-specific features during the denoising process, enhancing the effectiveness of CBCT-CT generation. We conduct numerous experiments on the thorax dataset and pancreas dataset. The results demonstrate that EGDiff performs better on the thoracic tumor dataset with SSIM of 0.850, MAE of 26.87 HU, PSNR of 19.83 dB, and NCC of 0.874. EGDiff outperforms SoTA CBCT-to-CT synthesis methods on the pancreas dataset with SSIM of 0.754, MAE of 32.19 HU, PSNR of 19.35 dB, and NCC of 0.846. By improving the accuracy and reliability of CBCT images, EGDiff can enhance the precision of radiation therapy, minimize radiation exposure to healthy tissues, and ultimately contribute to more effective and personalized cancer treatment strategies.

Multi-source domain adaptation based tempo-spatial convolution network for cross-subject EEG classification in RSVP task

Objective. Many subject-dependent methods were proposed for electroencephalogram (EEG) classification in rapid serial visual presentation (RSVP) task, which required a large amount of data from new subject and were time-consuming to calibrate system. Cross-subject classification can realize calibration reduction or zero calibration. However, cross-subject classification in RSVP task is still a challenge. Approach. This study proposed a multi-source domain adaptation based tempo-spatial convolution (MDA-TSC) network for cross-subject RSVP classification. The proposed network consisted of three modules. First, the common feature extraction with multi-scale tempo-spatial convolution was constructed to extract domain-invariant features across all subjects, which could improve generalization of the network. Second, the multi-branch domain-specific feature extraction and alignment was conducted to extract and align domain-specific feature distributions of source and target domains in pairs, which could consider feature distribution differences among source domains. Third, the domain-specific classifier was exploited to optimize the network through loss functions and obtain prediction for the target domain. Main results. The proposed network was evaluated on the benchmark RSVP dataset, and the cross-subject classification results showed that the proposed MDA-TSC network outperformed the reference methods. Moreover, the effectiveness of the MDA-TSC network was verified through both ablation studies and visualization. Significance. The proposed network could effectively improve cross-subject classification performance in RSVP task, and was helpful to reduce system calibration time.