Discriminative Features Research Articles

A Hierarchical Matrix Factorization-Based Method for Intelligent Industrial Fault Diagnosis.

Data-driven fault diagnosis, identifying abnormality causes using collected industrial data, is one of the challenging tasks for intelligent industry safety management. It is worth noting that practical industrial data are usually related to a mixture of several physical attributes, such as the operating environment, product quality and working conditions. However, the traditional models may not be sufficient to leverage the coherent information for diagnostic performance enhancement, due to their shallow architecture. This paper presents a hierarchical matrix factorization (HMF) that relies on a succession of matrix factoring to find an efficient representation of industrial data for fault diagnosis. Specifically, HMF consecutively decomposes data into several hierarchies. The intermediate hierarchies play the role of analysis operators which automatically learn implicit characteristics of industrial data; the final hierarchy outputs high-level and discriminative features. Furthermore, HMF is also extended in a nonlinear manner by introducing activation functions, referred as NHMF, to deal with nonlinearities in practical industrial processes. The applications of HMF and NHMF to fault diagnosis are evaluated by the multiple-phase flow process. The experimental results show that our models achieve competitive performance against the considered shallow and deep models, consuming less computing time than deep models.

HerbMet: Enhancing metabolomics data analysis for accurate identification of Chinese herbal medicines using deep learning.

Chinese herbal medicines have been utilized for thousands of years to prevent and treat diseases. Accurate identification is crucial since their medicinal effects vary between species and varieties. Metabolomics is a promising approach to distinguish herbs. However, current metabolomics data analysis and modeling in Chinese herbal medicines are limited by small sample sizes, high dimensionality, and overfitting. This study aims to use metabolomics data to develop HerbMet, a high-performance artificial intelligence system for accurately identifying Chinese herbal medicines, particularly those from different species of the same genus. We propose HerbMet, an AI-based system for accurately identifying Chinese herbal medicines. HerbMet employs a 1D-ResNet architecture to extract discriminative features from input samples and uses a multilayer perceptron for classification. Additionally, we design the double dropout regularization module to alleviate overfitting and improve model's performance. Compared to 10 commonly used machine learning and deep learning methods, HerbMet achieves superior accuracy and robustness, with an accuracy of 0.9571 and an F1-score of 0.9542 for distinguishing seven similar Panax ginseng species. After feature selection by 25 different feature ranking techniques in combination with prior knowledge, we obtained 100% accuracy and an F1-score for discriminating P. ginseng species. Furthermore, HerbMet exhibits acceptable inference speed and computational costs compared to existing approaches on both CPU and GPU. HerbMet surpasses existing solutions for identifying Chinese herbal medicines species. It is simple to use in real-world scenarios, eliminating the need for feature ranking and selection in classical machine learning-based methods.

MMDG-DTI: Drug–target interaction prediction via multimodal feature fusion and domain generalization

Recently, deep learning has become the essential methodology for Drug–Target Interaction (DTI) prediction. However, the existing learning-based representation methods embed the prior knowledge encapsulated by the training data and, as such, acquire redundant domain information irrelevant to the DTI prediction, compromising the generalization capability of a deep network to unfamiliar instances. To tackle this limitation, we propose a novel DTI prediction method using Multi-Modal feature fusion and Domain Generalization (MMDG-DTI). Specifically, we first use pre-trained Large Language Models (LLMs) to obtain generalized textual features covering nearly the whole biological text vocabulary, with the capacity to handle unseen samples and extract robust and discriminative features. In parallel, we propose a unified structural feature extractor based on a hybrid Graph Neural Network (GNN). The extractor improves the performance of MMDG-DTI by extracting features from another modality and discarding the representation of domain-specific prior substructures. Then, we integrate the complementary information of LLM and GNN features using an innovative classifier, while enhancing the generalization ability with the help of Domain Adversarial Training (DAT) and contrastive learning. Last, we propose a novel cross-domain evaluation protocol to enhance the predication fidelity in practical applications. The quantitative and visualization results obtained on several benchmarking datasets demonstrate that MMDG-DTI achieves state-of-the-art performance under both single-domain and cross-domain settings, confirming the superiority of the proposed method. The datasets and codes will be available on the project homepage: https://github.com/JU-HuaY/MMDG-DTI.

State of health estimation for lithium-ion batteries based on multi-scale frequency feature and time domain feature fusion method

Abstract Accurately estimating the state of health (SOH) of lithium-ion batteries is important for improving battery safety performance. The single time-domain feature extraction is hard to efficiently extract discriminative features from strongly nonlinear coupled data, leading to difficulties in accurately estimating the battery SOH. To this end, this paper proposes a multi-scale frequency domain feature and time domain feature fusion method for SOH estimation of lithium-ion batteries based on the Transformer model. Firstly, the voltage, current, temperature and time information of the battery are extracted as time domain features; secondly, the battery signal is processed by a multi-scale filter bank based on Mel-frequency cepstral coefficients (MFCC) to obtain the multi-scale frequency domain features; then, a parallel focusing network (PFN) is designed to fuse the time domain features with the frequency domain features, which yields low-coupling complementary discriminative features; finally, constructing the SOH estimation mechanism based on the Transformer deep network model. The algorithm is validated by NASA and Oxford datasets, the mean's absolute error (MAE) and root mean square error (RMSE) are as low as 0.06% and 0.23%, respectively.

Simulation-driven unsupervised fault diagnosis of rolling bearing under time-varying speeds

Simulation models incorporating fault mechanisms can acquire sufficient samples with rich fault information, providing an effective solution to deep learning-driven bearing fault diagnosis in case of sample scarcity. However, the simulation models in the previous studies are mainly designed for constant speed conditions and cannot generate effective source data aligning with the variable speed conditions. Moreover, the fault impacts of bearing exhibit time-varying characteristics under variable speed conditions, causing the obstacle of the fault feature representation of the deep learning model. Therefore, this article investigates an analytical fault simulation model-driven unsupervised fault diagnosis of rolling bearing under time-varying speeds. The simulation model can customize high-quality data that match the specific variable speed conditions. The proposed network can extract robust discriminable feature representation by designing multiscale enhanced temporal convolution transformer network and enables the feature alignment of the target and simulation samples under a pseudolabel guided domain adaptation training strategy. The effectiveness and superiority of the proposed method in addressing the bottleneck problems of bearing unsupervised fault diagnosis under variable speed conditions, including condition matching data generation and time-varying feature representation, is demonstrated using two different bearing datasets.

Characteristic discriminative prototype network with detailed interpretation for classification

Existing prototype learning methods provide limited interpretation on which patches from input images are similar to the corresponding prototypes. Moreover, these methods do not consider the diversities among the prototypes, which leads to low classification accuracy. To address these problems, this paper proposes Characteristic Prototype Network (CDPNet) with clear interpretation of local regions and characteristic. The network designs the feature prototype to represent the discriminative feature and the characteristic prototype to characterize the prototype’s properties among different individuals. In addition, two novel strategies, dynamic region learning and similarity score minimization among similar intra-class prototypes, are designed to learn the prototypes so as to improve their diversity. Therefore, CDPNet can explain which kind of characteristic within the image is the most important one for classification tasks. The experimental results on well-known datasets show that CDPNet can provide clearer interpretations and obtain state-of-the-art classification performance in prototype learning.

Wind Turbine Blade Fault Diagnosis: Approximate Entropy as a Tool to Detect Erosion and Mass Imbalance

Wind energy is a clean, sustainable, and renewable source. It is receiving a large amount of attention from governments and energy companies worldwide as it plays a significant role as an alternative source of energy in reducing carbon emissions. However, due to long-term operation in reduced and difficult weather conditions, wind turbine blades are always seriously damaged. Hence, damage detection in blade structure is essential to evaluate its operational condition and ensure its structural integrity and safety. We aim to use fractal, entropy, and chaos concepts as descriptors for the diagnosis of wind turbine blade condition. They are, respectively, estimated by the correlation dimension, approximate entropy, and the Lyapunov exponent. Formal statistical tests are performed to check how they are different across wind turbine blade conditions. The experimental results follow. First, the correlation dimension is not able to distinguish between all conditions of wind turbine blades. Second, approximate entropy is suitable to distinguish between healthy and erosion conditions and between healthy and mass imbalance conditions. Third, chaos is not a discriminative feature to distinguish between wind turbine blade conditions. Fourth, wind turbine blades with either erosion or mass imbalance exhibit less irregularity in their respective signals than healthy wind turbine blades.

Advances in the Application of Protein Language Modeling for Nucleic Acid Protein Binding Site Prediction.

Protein and nucleic acid binding site prediction is a critical computational task that benefits a wide range of biological processes. Previous studies have shown that feature selection holds particular significance for this prediction task, making the generation of more discriminative features a key area of interest for many researchers. Recent progress has shown the power of protein language models in handling protein sequences, in leveraging the strengths of attention networks, and in successful applications to tasks such as protein structure prediction. This naturally raises the question of the applicability of protein language models in predicting protein and nucleic acid binding sites. Various approaches have explored this potential. This paper first describes the development of protein language models. Then, a systematic review of the latest methods for predicting protein and nucleic acid binding sites is conducted by covering benchmark sets, feature generation methods, performance comparisons, and feature ablation studies. These comparisons demonstrate the importance of protein language models for the prediction task. Finally, the paper discusses the challenges of protein and nucleic acid binding site prediction and proposes possible research directions and future trends. The purpose of this survey is to furnish researchers with actionable suggestions for comprehending the methodologies used in predicting protein-nucleic acid binding sites, fostering the creation of protein-centric language models, and tackling real-world obstacles encountered in this field.

Feature extractor optimization for discriminative representations in Generalized Category Discovery

Generalized Category Discovery (GCD) task involves transferring knowledge from labeled known categories to recognize both known and novel categories within an unlabeled dataset. A significant challenge arises from the lack of prior information for novel categories. To address this, we develop a feature extractor that can learn discriminative features for both known and novel categories. Our approach leverages the observation that similar samples often belong to the same class. We construct a similarity matrix and employ similarity contrastive loss to increase the similarity between similar samples in the feature space. Additionally, we incorporate cluster labels to further refine the feature extractor, utilizing K-means clustering to assign these labels to unlabeled data, providing valuable supervision. Our feature extractor is optimized through the utilization of instance-level contrastive learning and class-level contrastive learning constraints. These constraints promote similarity maximization in both the instance space and the label space for instances sharing the same pseudo-labels. These three components complement each other, facilitating the learning of discriminative representations for both known and novel categories. Through comprehensive evaluations of generic image recognition datasets and challenging fine-grained datasets, we demonstrate that our proposed method achieves state-of-the-art performance.

ACAN: A plug-and-play Adaptive Center-Aligned Network for unsupervised domain adaptation

Domain adaptation is an important topic due to its capability in transferring knowledge from source domain to target domain. However, many existing domain adaptation methods primarily concentrate on aligning the data distributions between the source and target domains, often neglecting discriminative feature learning. As a result, target samples with low confidence are embedded near the decision boundary, where they are susceptible to being misclassified, resulting in negative transfer. To address this problem, a novel Adaptive Center-Aligned Network dubbed ACAN is proposed for unsupervised domain adaptation in this work. The main innovations of ACAN are fourfold. Firstly, it is a plug-and-play module and can be easily incorporated into any domain alignment methods without increasing the model complexity and computational burden. Secondly, in contrast to conventional softmax plus cross-entropy loss, angular margin loss is called to enhance the discrimination power for classifier. Thirdly, entropy regularization is exploited to highlight the probability of potential related class, which renders our learned feature representation far away from the decision boundary. Fourthly, to improve the discriminative capacity of model to the target domain, we propose to align the target domain samples to the corresponding class center via pseudo labels. Incorporating ACAN, the performance of baseline domain alignment methods is significantly improved. Extensive ablation and comparison experiments on four widely adopted databases demonstrate the effectiveness of our ACAN. Code is available at: https://github.com/Cloudfly-Z/ACAN

D2Net: discriminative feature extraction and details preservation network for salient object detection

D2Net: discriminative feature extraction and details preservation network for salient object detection

EEG-fNIRS-Based Emotion Recognition Using Graph Convolution and Capsule Attention Network.

Electroencephalogram (EEG) and functional near-infrared spectroscopy (fNIRS) can objectively reflect a person's emotional state and have been widely studied in emotion recognition. However, the effective feature fusion and discriminative feature learning from EEG-fNIRS data is challenging. In order to improve the accuracy of emotion recognition, a graph convolution and capsule attention network model (GCN-CA-CapsNet) is proposed. Firstly, EEG-fNIRS signals are collected from 50 subjects induced by emotional video clips. And then, the features of the EEG and fNIRS are extracted; the EEG-fNIRS features are fused to generate higher-quality primary capsules by graph convolution with the Pearson correlation adjacency matrix. Finally, the capsule attention module is introduced to assign different weights to the primary capsules, and higher-quality primary capsules are selected to generate better classification capsules in the dynamic routing mechanism. We validate the efficacy of the proposed method on our emotional EEG-fNIRS dataset with an ablation study. Extensive experiments demonstrate that the proposed GCN-CA-CapsNet method achieves a more satisfactory performance against the state-of-the-art methods, and the average accuracy can increase by 3-11%.

Multi-level interactive fusion network based on adversarial learning for fusion classification of hyperspectral and LiDAR data

With the growing awareness of the limitations of unimodal data, research on joint classification using multimodal remote sensing data has garnered increasing attention. However, existing methods still have certain deficiencies in dealing with feature extraction and feature fusion of multimodal remote sensing data. In order to obtain more accurate classification results, it is crucial to fully exploit and utilize the complementary information in multimodal remote sensing data. In this paper, a multi-level interactive fusion network based on adversarial learning (MLIF-AL) is proposed for hyperspectral image (HSI) and light detection and ranging (LiDAR) data fusion classification. First, the feature extraction part based on generative adversarial network (GAN) utilizes adversarial training between the generator and the discriminator to guide the network to learn more discriminative and distinguished feature representations. Meanwhile, a cross-modal interactive information extraction (CMIIE) module is also designed for the generator encoding part to promote the interaction between multimodal data from a global perspective and realize the full utilization of multimodal complementary information. In addition, in the multi-level feature fusion classification (MLFFC) part, the complementarities between the features learned by the neural networks at each layer are comprehensively utilized to obtain higher-level multimodal fusion semantic information. Finally, the adversarial loss and classification loss are combined for network training. Extensive experiments on three commonly used HSI and LiDAR datasets and comparisons with state-of-the-art methods demonstrate the effectiveness and superiority of the model.

Intelligent fault diagnosis of multi-source cross-machine bearings based on center-weighted optimal transport and class-level alignment domain adaptation

Abstract Most of the current domain adaptation research primarily focuses on the single-source or multi-source domain transfer constructed under different working conditions of the same machine. However, when faced with cross-machine tasks with significant domain discrepancies, forcing the direct feature alignment between source and target domain samples may lead to negative transfer, thereby reducing the model’s diagnostic performance. To overcome the above limitations, this paper proposes a multi-source deep transfer model based on center-weighted optimal transport (CWOT) and class-level alignment domain adaptation. Firstly, to enhance the representation capability of deep features, a multi-structure feature representation network is constructed to enrich the information capacity embedded within the deep features, thereby achieving better domain adaptation capabilities. Then, the local maximum mean discrepancy is introduced to fully exploit fine-grained information and discriminative features among different source domains, minimizing the distribution differences among the source domains to the greatest extent, thus capturing reliable and highly generalized multi-source domain invariant features. On this basis, a CWOT strategy is designed, which comprehensively considers the transport cost of intra-domain uncertainty and inter-domain correlation among samples, establishing a more effective transport between source and target domains, alleviating the problem of sample negative transfer, and improving the model’s cross-machine diagnostic performance. Finally, instance studies are conducted through multiple cross-machine transfer diagnostic tasks, demonstrating that the proposed method outperforms existing domain adaptation methods in terms of diagnostic accuracy and fault transfer capability. This research provides a reliable fault diagnosis method for detecting the health status of rotating machinery equipment, promoting the application of domain adaptation technology in practical industry.

Hard sample mining-enabled supervised contrastive feature learning for wind turbine pitch system fault diagnosis

Abstract The presence of multiple failure severities in the wind turbine pitch system due to the long-term wear and tear poses challenges in accurately classifying the system’s health condition, thus increasing maintenance costs or damage risks. This paper proposes a novel method based on hard sample mining (HSM)-enabled supervised contrastive learning to address this problem. The proposed method leverages the powerful feature extraction capabilities of supervised contrastive learning to extract discriminative features from highly imbalanced data. Additionally, the method incorporates a cosine similarity-based HSM framework that constructs hard sample pairs within mini-batches during both the representation learning and classifier training phases, thereby improving the diagnostic performance of the model for hard samples. The proposed method achieves macro G-mean of 0.9991 and 0.9971 on two real datasets containing data on wind turbine pitch system cog belt fractures. These results indicate significantly superior fault diagnosis performance compared to existing methods, highlighting its potential for enhancing the reliability and efficiency of fault diagnosis in wind turbine pitch systems.

Kernel Extreme Learning Machine with Discriminative Transfer Feature and Instance Selection for Unsupervised Domain Adaptation

The goal of domain adaptation (DA) is to develop a robust decision model on the source domain effectively generalize to the target domain data. State-of-the-art domain adaptation methods typically focus on finding an optimal inter-domain invariant feature representation or helpful instances from the source domain. In this paper, we propose a kernel extreme learning machine with discriminative transfer features and instance selection (KELM-DTF-IS) for unsupervised domain adaptation tasks, which consists of two steps: discriminative transfer feature extraction and classification with instance selection. At the feature extraction stage, we extend cross domain mean approximation(CDMA) by incorporating a penalty term and develop discriminative cross domain mean approximation (d-CDMA) to optimize the category separability between instances. Subsequently, d-CDMA is integrated into kernel ELM-AutoEncoder(KELM-AE) for extracting inter-domain invariant features. During the classification process, our approach uses CDMA metrics to compute a weights to each source instances based on their impact in reducing distribution differences between domains. Instances with a greater effect receive higher weights and vice versa. These weights are then used to distinguish and select source domain instances before incorporating them into weight KELM for proposing an adaptive classifier. Finally, we apply our approach to conduct classification experiments on publicly available domain adaptation datasets, and the results demonstrate its superiority over KELM and numerous other domain adaptation approaches.

A novel CatractNetDetect deep learning model for effective cataract classification through data fusion of fundus images

Cataracts are common eye disorders characterized by the clouding of the lens, preventing light from passing through and impairing vision. Various factors, including changes in the lens’s hydration or alterations in its proteins, may contribute to their development. Regular eye examinations conducted by an ophthalmologist or optometrist are imperative for detecting cataracts and other ocular conditions early on. Manual checks by caregivers pose several problems, including subjectivity, human error, and a lack of expertise. Biomedical fusion involves combining or linking various characteristics specific to certain diseases from different medical imaging resources. The primary objectives of this approach in disease classification are to reduce the error rate and increase the number of retrieved features. The aim of this study is to evaluate the outcomes associated with fusing visual features related to left and right eye cataract characteristics. Additionally, we investigate the impact of limited variability in deep learning models, specifically in the classification of cataract fundus versus normal fundus images. To address this issue, this study introduces CataractNetDetect, an innovative multi-label deep learning classification system that fuses feature representations from pairs of fundus images (e.g., left and right eyes) for the automatic diagnosis of various ocular disorders. Our focus is on achieving improved performance by stacking discriminative deep feature representations to combine two fundus images into a unified feature representation. Several deep learning architectures are utilized as feature descriptors, including ResNet-50, DenseNet-121, and Inception-V3, enhancing the resilience and quality of representations. Fine-tuning of these DL architectures is conducted using the ImageNet dataset, followed by an integrated stacking approach combining ResNet-50, DenseNet-121, and Inception-V3 models. The model is trained on the publicly available ODIR-5k dataset, which includes 5000 left/right eye images depicting eight different ocular conditions, ranging from healthy states to uncommon ailments such as cataracts, glaucoma, age-related macular degeneration (AMD), diabetes, hypertension, and myopia abnormalities. Moreover, extensive preprocessing of the images is performed, including data augmentation, noise reduction, contrast enhancement, scaling, and circular border cropping. The CataractNetDetect system demonstrates F1-scores, AUC, and maximum validation scores of 98.0%, 97.9%, and 100%, respectively. This ensemble-based model distinguishes itself by surpassing the performance of conventional established methodologies, including ResNet-50, DenseNet-121, and Inception-V3, thereby underscoring its efficacy in diagnostic applications.

UDANet: An unsupervised domain adaptive vehicle density estimation network based on joint adversarial learning

Unsupervised domain adaptive vehicle density estimation aims to transfer the knowledge learned from the labeled source domain to the unlabeled target domain, which has received extensive attention due to its practicality and effectiveness. However, the existing methods often rely on a large amount of expensive labeled data, and ignore the large differences between different data domains, making the network trained in one scene difficult to apply to the target scene. To solve these problems, this paper proposes an unsupervised domain adaptive vehicle density estimation network based on joint adversarial learning (UDANet), which effectively extends the model to unlabeled target domain data by adding two domain adaptive components to the pre-trained backbone. Specifically, a multi-level feature discriminator is designed to align vehicle features from the image level and refine features from the pixel level by joint adversarial learning, thus achieving cross-domain transfer of features from the source domain to the target domain. In addition, to address the shared convolutional structure that incorrectly highlights irrelevant information, a feature correction module is designed to improve the adaptability of the network to target domain data by re-modeling the source and target data according to channel independence. Several well-known backbones are used to prove the superiority of the backbone and the mobility of the two domain adaptive components. The domain adaptation experimental results on four vehicle datasets show that UDANet achieves high estimation accuracy compared to current state-of-the-art unsupervised methods, and is suitable for the task of vehicle density estimation in complex scenes.

Sparse feature selection using hypergraph Laplacian-based semi-supervised discriminant analysis

Feature selection, as a dimension reduction technique in data mining and pattern recognition, aims to select the most discriminative features and improve the learning performance. With an abundance of unlabeled data readily available across various applications, semi-supervised feature selection has emerged as a promising approach. While most semi-supervised feature selection methods rely on simple graphs to preserve the geometrical structure of data, this approach often fails in capturing the high-order relationships present in many real-world applications. In contrast, hypergraphs offer the ability to encode more complex structures of data beyond what a simple graph can achieve. In this paper, we propose a feature selection method formulated in the trace ratio form, integrating hypergraph Laplacian-based semi-supervised discriminant analysis (SDA) and the mixed convex and non-convex ℓ2,p-norm (0<p≤1) regularization. The proposed trace ratio-based method, called HSDAFS, leverages the discriminative information from labeled data to maximize class separability while also utilizing the hypergraph Laplacian to capture the geometrical structure and high-order relationships within both labeled and unlabeled data. The ℓ2,p-norm regularization in the proposed HSDAFS provides improved sparsity over the ℓ2,1-norm. It ensures that the projection matrix is row-sparse, enabling the effective joint selection of discriminative features across all data. To solve the trace ratio-based HSDAFS method, we convert it into a trace difference method and propose an iterative algorithm. Experiments on several datasets demonstrate that HSDAFS is more effective in selecting the most discriminative features compared to other methods.

New oscillatory features of electrodermal activity signals evaluated in automated mental workload monitoring

Oscillatory components are discriminative features of mental workload monitoring in electrodermal activity (EDA) signals. Currently, most feature extraction techniques mainly focus on time domain or linear techniques but rarely consider the non-stationary nature, individual differences, and recording noise. This study has proposed a new time–frequency feature extraction method based on Tunable Q-factor Wavelet Transform providing an effective quantification of oscillatory behavior of a single transient. The extracted features have been classified using a recurrent neural network with the advantage of dynamic mapping procedure. The proposed method has been evaluated using EDA data acquired during an arithmetic task of five workload levels with both subtle and distinct differences. In a comparative study, the effects of several decomposition parameters and cognitive factors have been explored. Experimental results have shown that it achieves an average accuracy rate of 98.52% for three workload levels discrimination by optimizing the number of features to 3 (features extracted from a low frequency sub-band) from 15 (features extracted from five frequency sub-bands), i.e., 80% feature reduction. The greatest contribution coming fromthe smallest frequency sub-bands has also been obtained. This method is superior to other existing methods in separating 3 to 5 workload levels with both distinct and subtle differences extracting effective oscillatory characteristics. Considering the merits of a simple feature extraction procedure and cost-effectiveness of a single lead EDA signal as well as high discriminability between several workload levels, it provides a trade-off between performance and computational complexity, thus demonstrating its potential for online human-error prevention systems.