Domain-specific Features Research Articles

Exploring Latent Transferability of feature components

Feature disentanglement techniques have been widely employed to extract transferable (domain-invariant) features from non-transferable (domain-specific) features in Unsupervised Domain Adaptation (UDA). However, due to the complex interplay among high-dimensional features, the separated “non-transferable” features may still be partially informative. Suppressing or disregarding them, as commonly employed in previous methods, can overlook the inherent transferability. In this work, we introduce two concepts: Partially Transferable Class Features and Partially Transferable Domain Features (PTCF and PTDF), and propose a succinct feature disentanglement technique. Different with prior works, we don’t seek to thoroughly peel off the non-transferable features, as it is challenging practically. Instead, we take the two-stage strategy consisting of rough feature disentanglement and dynamic adjustment. We name our model as ELT because it can systematically Explore Latent Transferability of feature components. ELT can automatically evaluate the transferability of internal feature components, dynamically giving more attention to features with high transferability and less to features with low transferability, effectively solving the problem of negative transfer. Extensive experimental results have proved its efficiency. The code and supplementary file will be available at https://github.com/njtjmc/ELT.

An enhanced domain generalization method for object detection based on text guided feature disentanglement

The application scenarios of object detection models are constantly changing, due to the alternation of day and night and weather changes. Detector often suffers from the scarcity of training sets on potential domains. Recently, this challenge known as domain shift has been relieved by single domain generalization (SDG). To further generalize towards multiple unseen domains, this paper proposes a detector that uses text semantic gaps to enhance scene diversity and utilizes feature disentangling to extract domain-invariant features from different scenes, thereby improving detection accuracy. Firstly, random semantic augmentation (RSA) is adopted leveraging the text modality to capture semantically generalized representations, thereby augmenting the diversity of domain related information. Second, by broadening the decision boundary between domain-invariant and domain-specific features, feature disentangling (FD) branches are applied to improve the detector's object-background differentiation. Additionally, a cross modality alignment (CMA) is performed by estimating the relevances between domain-specific features and textual domain prompts. Experimental results show the proposed detector has excellent performance among existing baselines on diverse weather conditions, such as rainy, foggy and night rainy, which also confirms the enhanced generalization ability on multiple unseen domains.

A Hybrid Semantic Representation Method Based on Fusion Conceptual Knowledge and Weighted Word Embeddings for English Texts

The accuracy of traditional topic models may be compromised due to the sparsity of co-occurring vocabulary in the corpus, whereas conventional word embedding models tend to excessively prioritize contextual semantic information and inadequately capture domain-specific features in the text. This paper proposes a hybrid semantic representation method that combines a topic model that integrates conceptual knowledge with a weighted word embedding model. Specifically, we construct a topic model incorporating the Probase concept knowledge base to perform topic clustering and obtain topic semantic representation. Additionally, we design a weighted word embedding model to enhance the contextual semantic information representation of the text. The feature-based information fusion model is employed to integrate the two textual representations and generate a hybrid semantic representation. The hybrid semantic representation model proposed in this study was evaluated based on various English composition test sets. The findings demonstrate that the model presented in this paper exhibits superior accuracy and practical value compared to existing text representation methods.

The Domain-Specific Neural Basis of Auditory Statistical Learning in 5-7-Year-Old Children.

Statistical learning (SL) is the ability to rapidly track statistical regularities and learn patterns in the environment. Recent studies show that SL is constrained by domain-specific features, rather than being a uniform learning mechanism across domains and modalities. This domain-specificity has been reflected at the neural level, as SL occurs in regions primarily involved in processing of specific modalities or domains of input. However, our understanding of how SL is constrained by domain-specific features in the developing brain is severely lacking. The present study aims to identify the functional neural profiles of auditory SL of linguistic and nonlinguistic regularities among children. Thirty children between 5 and 7 years old completed an auditory fMRI SL task containing interwoven sequences of structured and random syllable/tone sequences. Using traditional group univariate analyses and a group-constrained subject-specific analysis, frontal and temporal cortices showed significant activation when processing structured versus random sequences across both linguistic and nonlinguistic domains. However, conjunction analyses failed to identify overlapping neural indices across domains. These findings are the first to compare brain regions supporting SL of linguistic and nonlinguistic regularities in the developing brain and indicate that auditory SL among developing children may be constrained by domain-specific features.

A real-world battery state of charge prediction method based on a lightweight mixer architecture

This paper proposes a lightweight model based on the improved Mixer architecture—TrendFlow-Mixer, designed for efficient prediction of the state of charge (SOC) of lithium-ion batteries. SOC accuracy is crucial for the safe operation, usage, and maintenance of power batteries. When predicting the SOC of real-world batteries, the algorithm's generalization, robustness, and computational efficiency across different battery types must be considered. To this end, TrendFlow-Mixer introduces targeted improvements to battery characteristics, significantly alleviating the Mixer architecture's lag in SOC prediction. The model's core structure integrates the Fast Fourier Transform and arrangement transformation convolutional structures. This allows for efficient extraction of domain-specific spatial features and enables cross-channel correlation interactions. This reduces the model's parameter size while enhancing its adaptability to new domains. The model quickly captures data trend changes and identifies data flow patterns, improving computational efficiency by 70 % compared to Transformer-based architectures. TrendFlow-Mixer employs a dual-prediction head, combining rapid feature change analysis with a nonlinear prediction output to generate SOC predictions. This design facilitates fine-tuning of the model's structure and parameters while enhancing overall flexibility. Experimental results show that in a laboratory dynamic condition dataset, the model achieves a minimum mean absolute error (MAE) of 0.11 %, with the maximum prediction error under extreme temperatures controlled within 4.66 %. In validation using real-world data from the BMW i3, the mean absolute percentage error remains around 1 %, with a minimum MAE of 0.465 %. The results demonstrate that TrendFlow-Mixer possesses high computational efficiency, excellent generalization, and adaptability, effectively predicting SOC trends.

How to create and use a national cross-domain ontology and data infrastructure on the Semantic Web

This paper presents a model and lessons learned for creating a cross-domain national ontology and Linked (Open) Data (LOD) infrastructure. The idea is to extend the global, domain agnostic “layer cake model” underlying the Semantic Web with domain specific and local features needed in applications. To test and demonstrate the infrastructure, a series of LOD services and portals in use have been created in 2002–2023 that cover a wide range of application domains. They have attracted millions of users in total suggesting feasibility of the proposed model. This line of research and development is unique due to its systematic national level nature and long time span of over twenty years.

A bearing fault diagnosis method for unknown operating conditions based on differentiated feature extraction

Under unknown operating conditions, the domain generalization approach based on domain metrics is commonly used for rolling bearing fault diagnostics. Nevertheless, in the event of equipment failure under unknown operating conditions, focusing solely on the transferable characteristics across domains may result in the unintentional neglect of domain-specific features. To address the problems mentioned, the present study introduces a feature decomposition learning method that simultaneously extracts inter-domain transferable and domain-specific features. This method aims to obtain richer feature information by constructing different feature extractors. For the extraction of transferable features, a joint metric method based on central moment differences is devised. A difference maximization method is employed to extract domain-specific features. The experimental findings demonstrate that the proposed technique exhibits greater defect detection capacity across two datasets.

Evaluation of a Task-Specific Self-Supervised Learning Framework in Digital Pathology Relative to Transfer Learning Approaches and Existing Foundation Models

An integral stage in typical digital pathology workflows involves deriving specific features from tiles extracted from a tessellated whole-slide image. Notably, various computer vision neural network architectures, particularly the ImageNet pretrained, have been extensively used in this domain. This study critically analyzes multiple strategies for encoding tiles to understand the extent of transfer learning and identify the most effective approach. The study categorizes neural network performance into 3 weight initialization methods: random, ImageNet-based, and self-supervised learning. Additionally, we propose a framework based on task-specific self-supervised learning, which introduces a shallow feature extraction method, employing a spatial-channel attention block to glean distinctive features optimized for histopathology intricacies. Across 2 different downstream classification tasks (patch classification and weakly supervised whole-slide image classification) with diverse classification data sets, including colorectal cancer histology, Patch Camelyon, prostate cancer detection, The Cancer Genome Atlas, and CIFAR-10, our task-specific self-supervised encoding approach consistently outperforms other convolutional neural network–based encoders. The better performances highlight the potential of task-specific attention-based self-supervised training in tailoring feature extraction for histopathology, indicating a shift from using pretrained models originating outside the histopathology domain. Our study supports the idea that task-specific self-supervised learning allows domain-specific feature extraction, encouraging a more focused analysis.

A privacy-preserving framework with multi-modal data for cross-domain recommendation

Cross-domain recommendation (CDR) aims to enhance the recommendation accuracy in a target domain with sparse data by leveraging rich information in a source domain, thereby addressing the data-sparsity problem. Some existing CDR methods highlight the advantages of extracting domain-common and domain-specific features to learn comprehensive user and item representations. However, these methods cannot effectively disentangle these components, as they often rely on simple user-item historical interaction information (such as ratings, clicks, and browsing), neglecting the rich multi-modal features. In addition, they do not protect user-sensitive data from potential leakage during knowledge transfer between domains. To address these challenges, we propose a Privacy-Preserving Framework with Multi-Modal Data for Cross-Domain Recommendation, called P2M2-CDR. Specifically, we first design a multi-modal disentangled encoder that utilizes multi-modal information to disentangle more informative domain-common and domain-specific embeddings. Furthermore, we introduce a privacy-preserving decoder to mitigate user privacy leakage during knowledge transfer. Local differential privacy (LDP) is used to obfuscate disentangled embeddings before the inter-domain exchange, thereby enhancing privacy protection. To ensure both consistency and differentiation among these obfuscated disentangled embeddings, we incorporate contrastive learning-based domain-inter and domain-intra losses. Extensive experiments conducted on six CDR tasks from two real-world datasets demonstrate that P2M2-CDR outperforms other state-of-the-art single- and cross-domain baselines. The code is available at https://github.com/Lili1013/P2M2-CDR.

Data-driven discovery of potent small molecule ice recrystallisation inhibitors

Controlling the formation and growth of ice is essential to successfully cryopreserve cells, tissues and biologics. Current efforts to identify materials capable of modulating ice growth are guided by iterative changes and human intuition, with a major focus on proteins and polymers. With limited data, the discovery pipeline is constrained by a poor understanding of the mechanisms and the underlying structure-activity relationships. In this work, this barrier is overcome by constructing machine learning models capable of predicting the ice recrystallisation inhibition activity of small molecules. We generate a new dataset via experimental measurements of ice growth, then harness predictive models combining state-of-the-art descriptors with domain-specific features derived from molecular simulations. The models accurately identify potent small molecule ice recrystallisation inhibitors within a commercial compound library. Identified hits can also mitigate cellular damage during transient warming events in cryopreserved red blood cells, demonstrating how data-driven approaches can be used to discover innovative cryoprotectants and enable next-generation cryopreservation solutions for the cold chain.

Domain-augmented meta ensemble learning for mechanical fault diagnosis from heterogeneous source domains to unseen target domains

Existing domain generalization (DG) fault diagnosis methods primarily use adversarial training to reduce shifts between source domains and learn domain-invariant features. However, such features are difficult to learn when dealing with substantial shifts between source domains. In addition, focusing only on domain-invariant and ignoring domain-specific information is not conducive to improving model generalization. Furthermore, these methods are typically developed under an assumption that source domains share the same label space. In situations where the source domains exhibit heterogeneity with inconsistent labels, as explored in this paper, aligning the source domains becomes more difficult due to severe class mismatches. To address the above challenges, this paper proposes a method named domain-augmented meta ensemble learning. Specifically, Dirichlet CutMix is developed to compensate for missing classes in a source domain by utilizing knowledge from other source domains. Moreover, a training strategy summarized as “learn to generalize to unseen domains through collaborative ensemble” is designed to balance the learning of domain-specific features with the ability to generalize across domains. The proposed method is applied to the bearing and gearbox fault diagnosis, and experimental results demonstrate the excellent generalization of the method from heterogeneous source domains to unseen target domains.

On the Nature of the Last Common Ancestor: A Story from its Translation Machinery.

The Last Common Ancestor (LCA) is understood as a hypothetical population of organisms from which all extant living creatures are thought to have descended. Its biology and environment have been and continue to be the subject of discussions within the scientific community. Since the first bacterial genomes were obtained, multiple attempts to reconstruct the genetic content of the LCA have been made. In this review, we compare 10 of the most extensive reconstructions of the gene content possessed by the LCA as they relate to aspects of the translation machinery. Although each reconstruction has its own methodological biases and many disagree in the metabolic nature of the LCA all, to some extent, indicate that several components of the translation machinery are among the most conserved genetic elements. The datasets from each reconstruction clearly show that the LCA already had a largely complete translational system with a genetic code already in place and therefore was not a progenote. Among these features several ribosomal proteins, transcription factors like IF2, EF-G, and EF-Tu and both class I and class II aminoacyl tRNA synthetases were found in essentially all reconstructions. Due to the limitations of the various methodologies, some features such as the occurrence of rRNA posttranscriptional modified bases are not fully addressed. However, conserved as it is, non-universal ribosomal features found in various reconstructions indicate that LCA's translation machinery was still evolving, thereby acquiring the domain specific features in the process. Although progenotes from the pre-LCA likely no longer exist recent results obtained by unraveling the early history of the ribosome and other genetic processes can provide insight to the nature of the pre-LCA world.

Unveiling the Power of Model-Agnostic Multiscale Analysis for Enhancing Artificial Intelligence Models in Breast Cancer Histopathology Images.

Developing AI models for digital pathology has traditionally relied on single-scale analysis of histopathology slides. However, a whole slide image is a rich digital representation of the tissue, captured at various magnification levels. Limiting our analysis to a single scale overlooks critical information, spanning from intricate high-resolution cellular details to broad low-resolution tissue structures. In this study, we propose a model-agnostic multiresolution feature aggregation framework tailored for the analysis of histopathology slides in the context of breast cancer, on a multicohort dataset of 2038 patient samples. We have adapted 9 state-of-the-art multiple instance learning models on our multi-scale methodology and evaluated their performance on grade prediction, TP53 mutation status prediction and survival prediction. The results prove the dominance of the multiresolution methodology, and specifically, concatenating or linearly transforming via a learnable layer the feature vectors of image patches from a high (20x) and low (10x) magnification factors achieve improved performance for all prediction tasks across domain-specific and imagenet-based features. On the contrary, the performance of uniresolution baseline models was not consistent across domain-specific and imagenet-based features. Moreover, we shed light on the inherent inconsistencies observed in models trained on whole-tissue-sections when validated against biopsy-based datasets. Despite these challenges, our findings underscore the superiority of multiresolution analysis over uniresolution methods. Finally, cross-scale analysis also benefits the explainability aspects of attention-based architectures, since one can extract attention maps at the tissue- and cell-levels, improving the interpretation of the model's decision.

Exploring Robust Features for Improving Adversarial Robustness.

While deep neural networks (DNNs) have revolutionized many fields, their fragility to carefully designed adversarial attacks impedes the usage of DNNs in safety-critical applications. In this article, we strive to explore the robust features that are not affected by the adversarial perturbations, that is, invariant to the clean image and its adversarial examples (AEs), to improve the model's adversarial robustness. Specifically, we propose a feature disentanglement model to segregate the robust features from nonrobust features and domain-specific features. The extensive experiments on five widely used datasets with different attacks demonstrate that robust features obtained from our model improve the model's adversarial robustness compared to the state-of-the-art approaches. Moreover, the trained domain discriminator is able to identify the domain-specific features from the clean images and AEs almost perfectly. This enables AE detection without incurring additional computational costs. With that, we can also specify different classifiers for clean images and AEs, thereby avoiding any drop in clean image accuracy.

A LiDAR-Based Recommendation System to Improve Sitting Posture Respecting Privacy.

Poor posture is becoming more widespread due to the rising number of jobs that require workers to sit for extended hours. Maintaining proper leg positioning is essential for good overall posture and long-term health. However, current monitoring methods involve multiple sensors and cameras, leading to discomfort and privacy concerns. This paper presents a privacy-preserving recommendation system using Light Detection and Ranging (LiDAR) to monitor leg positions. The system captures the horizontal leg outline at knee height and extracts domain-specific features, while a stacked autoencoder reproduces the latent representation of the leg outline. The system comprises three modules, including one-class and multi-class support vector machines, to identify 15 leg positions and reject unrecognized ones. Data from 30 healthy volunteers during work activities trained the system. When tested on new participants, the system achieved an accuracy of over 98%. In addition to monitoring leg positions while respecting privacy and ergonomics, our system alerts workers about poor leg positions and generates intuitive dashboards with posture statistics that help safety engineers identify at-risk workers and body parts. This offers a promising solution for improving desk workers' posture and reducing long-term health issues, respecting privacy.

Mathematics teachers’ multiple perspectives on adaptive tasks: task evaluation and selection as core practices for teaching quality

The selection of tasks based on the evaluation of task features can be considered a core practice of teaching and a relevant component of teaching quality. This is typically part of teachers’ preparation for their classroom teaching, which prompts the following question: What are the characteristics of the tasks that teachers use when selecting tasks for differentiated teaching? To answer this question, we analyzed systematic differences in the focus of 78 in-service high school and lower secondary school teachers during the evaluation of task features. The teachers had to select eight tasks about the practice of fractions with respect to their differentiation potential—operationalizing their adaptive teaching competence from a mathematics educational perspective. To analyze the differences, we performed a cluster analysis of the task features that the teachers drew upon. Three groups of teachers could be identified with variations in their focus on directly or indirectly relevant, domain-specific or domain-general task features. Taking into account such variations may explain differences in teaching quality and student outcomes and may be relevant when designing teacher professional development programs.

Disentanglement-inspired single-source domain-generalization network for cross-scene hyperspectral image classification

Cross-scene classification stands as a pivotal frontier in hyperspectral image (HSI) processing, aiming to enhance the generalization capabilities of classification models. However, the diversity of sensor type, shooting environments, and shooting times leads to the spectral heterogeneity problem in HSI. As a result, the same land cover may exhibit varying spectral traits in different domains, posing challenges for cross-scene HSI classification. Drawing inspiration from image disentanglement, we have identified that extracting the latent domain-invariant representation (DIR) of HSI could potentially mitigate the spectral heterogeneity issue. Therefore, we propose a Disentanglement-Inspired Single-Source Domain Generalization Network (DSDGnet) for cross-scene HSI classification in this paper. Firstly, a style transfer module based on a Transformer encoder-transfer-decoder is designed to expand the single source domain to an extended domain. Then, a progressive disentanglement module is proposed to decompose the domain-invariant features and domain-specific features of HSI. Furthermore, a domain combination module is designed to guarantee the accuracy of the progressive disentanglement module and ensure the effectiveness of the domain-invariant feature of HSI. Finally, the domain-invariant features are applied to the classification task, and the domain-specific features are separated to reduce their impact on the generalization ability of classification models. Extensive experiments on three HSI datasets have demonstrated the advanced classification performance of DSDGnet compared to existing domain-generalization methods.

Piston aero-engine fault cross-domain diagnosis based on unpaired generative transfer learning

As artificial intelligence (AI) and machine learning continue to advance, they have become key players in signal analysis and pattern recognition. However, challenges such as domain shift and a lack of labeled data hinder the application of machine learning to new, unfamiliar signals, particularly in the development of intelligent diagnostic methods for mechanical faults. In this study, we address these challenges by framing them as the extreme class imbalance problem. We introduce a novel generative transfer learning framework that leverages cycle-consistent adversarial networks with hard constraints (CycleGAN-HCs). This framework is designed to generate unpaired mechanical fault signals, which are then used to train classifiers that can operate across different domains. To enhance feature extraction capabilities for time series data, we developed a new generator model called U-Attention. Additionally, we refined the training loss function to better capture domain-specific and classification-specific features in mechanical vibration signals. The proposed method successfully facilitates feature transfer and data augmentation. Its effectiveness and reliability have been demonstrated through four cross-domain fault diagnosis tasks involving piston aero-engines. Comparative verification shows that the generative transfer learning diagnostic framework outperforms traditional methods, offering superior performance and better generalization in complex mechanical fault diagnosis scenarios.

A general multi-source ensemble transfer learning framework for health prognostic of lithium-ion batteries

This paper proposes a novel multi-source domain adaptation method for the health prediction in the absence of target batteries labels. First, a computationally efficient and effective source domain selection method is proposed. The proposed source domain selection method selects source domains with excellent transferability based on Euclidean distance and Sample Entropy thresholds, thereby improving the performance of multi-source domain adaptation. Second, improved maximum mean discrepancy is proposed to guide the representation alignment. Compared with the original maximum mean discrepancy, it matches the source data with the target data in the order of aging, thus preserving the time order of the distribution. Third, we develop a multi-source domain adaptation network that contains a domain-invariant feature generator and several domain-specific feature generators and estimators. Two out of 12 datasets are used as target LIBs to evaluate the performance of the proposed method. The mean absolute error of the proposed method on these two datasets is only 0.671% and 0.868%, respectively. While the maximum error of MDA-IM-GC on both datasets does not exceed 5.2%. The experiment results indicate that the proposed network provides impressive accuracy with only 1/10 of unlabeled target historical data accessible and outperforms the single-source domain adaptation approach.

Cost-efficient prompt engineering for unsupervised entity resolution in the product matching domain

Entity Resolution (ER) is the problem of semi-automatically determining when two entities refer to the same underlying entity, with applications ranging from healthcare to e-commerce. Traditional ER solutions required considerable manual expertise, including domain-specific feature engineering, as well as identification and curation of training data. Recently released large language models (LLMs) provide an opportunity to make ER more seamless and domain-independent. Because of LLMs’ pre-trained knowledge, the matching step in ER can be made easier by just prompting. However, it is also well known that LLMs can pose risks, that the quality of their outputs can depend on how prompts are engineered, and that the cost of using LLMs can be significant. Unfortunately, a systematic experimental study on the effects of different prompting methods and their respective cost for solving domain-specific entity matching using LLMs, like ChatGPT, has been lacking thus far. This paper aims to address this gap by conducting such a study. We consider some relatively simple and cost-efficient ER prompt engineering methods and apply them to perform product matching on two real-world datasets widely used in the community. We select two well-known e-commerce datasets and provide extensive experimental results to show that an LLM like GPT3.5 is viable for high-performing product matching and, interestingly, that more complicated and detailed (and hence, expensive) prompting methods do not necessarily outperform simpler approaches. We provide brief discussions on qualitative and error analysis, including a study of the inter-consistency of different prompting methods to determine whether they yield stable outputs. Finally, we consider some limitations of LLMs when used as a product matcher in potential real-world e-commerce applications.