Discriminative Features For Classification Research Articles

ALFREDO: Active Learning with FeatuRe disEntangelement and DOmain adaptation for medical image classification

State-of-the-art deep learning models often fail to generalize in the presence of distribution shifts between training (source) data and test (target) data. Domain adaptation methods are designed to address this issue using labeled samples (supervised domain adaptation) or unlabeled samples (unsupervised domain adaptation). Active learning is a method to select informative samples to obtain maximum performance from minimum annotations. Selecting informative target domain samples can improve model performance and robustness, and reduce data demands. This paper proposes a novel pipeline called ALFREDO (Active Learning with FeatuRe disEntangelement and DOmain adaptation) that performs active learning under domain shift. We propose a novel feature disentanglement approach to decompose image features into domain specific and task specific components. Domain specific components refer to those features that provide source specific information, e.g., scanners, vendors or hospitals. Task specific components are discriminative features for classification, segmentation or other tasks. Thereafter we define multiple novel cost functions that identify informative samples under domain shift. We test our proposed method for medical image classification using one histopathology dataset and two chest X-ray datasets. Experiments show our method achieves state-of-the-art results compared to other domain adaptation methods, as well as state of the art active domain adaptation methods.

Multibranch CNN With MLP-Mixer-Based Feature Exploration for High-Performance Disease Diagnosis.

Deep learning-based diagnosis is becoming an indispensable part of modern healthcare. For high-performance diagnosis, the optimal design of deep neural networks (DNNs) is a prerequisite. Despite its success in image analysis, existing supervised DNNs based on convolutional layers often suffer from their rudimentary feature exploration ability caused by the limited receptive field and biased feature extraction of conventional convolutional neural networks (CNNs), which compromises the network performance. Here, we propose a novel feature exploration network named manifold embedded multilayer perceptron (MLP) mixer (ME-Mixer), which utilizes both supervised and unsupervised features for disease diagnosis. In the proposed approach, a manifold embedding network is employed to extract class-discriminative features; then, two MLP-Mixer-based feature projectors are adopted to encode the extracted features with the global reception field. Our ME-Mixer network is quite general and can be added as a plugin to any existing CNN. Comprehensive evaluations on two medical datasets are performed. The results demonstrate that their approach greatly enhances the classification accuracy in comparison with different configurations of DNNs with acceptable computational complexity.

Feature disentanglement based domain adaptation network for cross-scene coastal wetland hyperspectral image classification

At present, domain adaptation (DA) methods have made noteworthy advancements in cross-scene hyperspectral image (HSI) classification. Their success largely hinges on the alignment of distributions between source and target domains, which is a critical step in extracting domain-invariant features. However, this intense focus on domain-invariant feature extraction frequently leads to the neglect of class-discriminative features, limiting their utility in cross-scene coastal wetland classification, where the nuanced identification of different classes is crucial. In this paper, a feature disentanglement based domain adaptation network (FDDAN) is proposed to disentangle and exclude domain-specific features and class-invariant features, thereby obtaining class-specific domain-invariant features for classification tasks. Specifically, a transformer and convolution fusion-based feature extraction network is designed to capture global–local mixed features. To align domain distributions and learn shared features, two corresponding disentanglers separate domain-invariant features and domain-specific features from mixed features. Furthermore, to allow domain-invariant features containing purer category discriminative information, class-invariant features are also segregated. In addition, an adversarial learning strategy between three features is utilized to simultaneously enhance the transferability and discriminability of domain-invariant features. The effectiveness of FDDAN is demonstrated by the experimental results obtained from three cross-scene unmanned aerial vehicle datasets collected in coastal wetlands and a publicly available dataset. We will make our datasets and source code publicly available upon acceptance of the paper.

A dual-weight domain adversarial network for partial domain fault diagnosis of feedwater heater system

Domain adaptation (DA) approaches have received significant attention in industrial cross-domain fault diagnosis. However, the scarcity of sufficient labeled fault data, particularly under varying loading conditions and harsh operational environments, can give rise to distinct label spaces between two domains, thereby impeding the application of DA-based diagnosis methods. In this paper, we propose a novel dual-weight domain adversarial network (DWDAN) for diagnosing partial domain faults of feedwater heater system in a large-scale power unit, where the target label space is a subset of the source domain. Firstly, domain adversarial network with an instance-based feature learning strategy is constructed to capture domain-invariant and class-discriminative features hidden in raw process data, thereby enhancing feature extraction and generalization abilities of fault diagnosis. Furthermore, a dual-stage reweighted induction module is designed to quantify the contribution of samples from both class-level and sample-level for selective adaptation. This module can automatically eliminate outlier fault categories in the source domain and facilitates alignment of feature distributions for shared fault categories. Comprehensive experiments conducted on the feedwater heater system of a 600 MW coal-fired generating unit demonstrate the outstanding performance of DWDAN.

SleePyCo: Automatic sleep scoring with feature pyramid and contrastive learning

Automatic sleep scoring is essential for the diagnosis and treatment of sleep disorders and enables longitudinal sleep tracking in home environments. Conventionally, learning-based automatic sleep scoring on single-channel electroencephalogram (EEG) is actively studied because obtaining multi-channel signals during sleep is difficult. However, learning representation from raw EEG signals is challenging owing to the following issues: (1) sleep-related EEG patterns occur on different temporal and frequency scales and (2) sleep stages share similar EEG patterns. To address these issues, we propose an automatic Sleep scoring framework that incorporates (1) a feature Pyramid and (2) supervised Contrastive learning, named SleePyCo. For the feature pyramid, we propose a backbone network named SleePyCo-backbone to consider multiple feature sequences on different temporal and frequency scales. Supervised contrastive learning allows the network to extract class discriminative features by minimizing the distance between intra-class features and simultaneously maximizing that between inter-class features. Comparative analyses on four public datasets demonstrate that SleePyCo consistently outperforms existing frameworks based on single-channel EEG. Extensive ablation experiments show that SleePyCo exhibited an enhanced overall performance, with significant improvements in discrimination between sleep stages, especially for N1 and rapid eye movement (REM). Source code is available at https://github.com/gist-ailab/SleePyCo.

A unified framework for visual domain adaptation with covariance matching

In the field of Machine Learning, it is widely acknowledged that training and test data should ideally stem from the same source and distribution. However, in the real world, this is not always feasible. Domain Adaptation (DA) techniques address this issue by adapting a classifier trained on annotated source domain data on unannotated target domain data while minimizing the impact of domain shift. Many recent DA approaches concentrate on learning a latent feature space that remains invariant to domain shift by mitigating various statistical and geometrical divergences. Although these methods have demonstrated effectiveness, they frequently overlook a crucial aspect of learning a domain-invariant discriminative latent feature space across diverse domains. To address this issue, we propose a novel framework called Unified Framework for Visual Domain Adaptation with Covariance Matching (UDACM) that aims to learn a domain invariant and class discriminative latent feature space by incorporating multiple objectives such as maximizing target domain variance, minimizing distribution and subspace divergence, performing manifold learning, preserving discriminative information of both the source and target domains, and measuring and matching covariance simultaneously. In our proposed framework, the measuring and matching of covariance play a crucial role in ensuring the learning of discriminative latent feature space by aligning the within-class and between-class covariance matrices of the source and target domains simultaneously. Detailed experiments demonstrate the outperformance of UDACM on baseline datasets such as CMU-PIE, Office+Caltech10, USPS+MNIST, VLCS, and Office–Home, compared to various established primitive, shallow, and deep domain adaptation methods on several image classification tasks.

Mutual Information-Driven Subject-Invariant and Class-Relevant Deep Representation Learning in BCI.

In recent years, deep learning-based feature representation methods have shown a promising impact on electroencephalography (EEG)-based brain-computer interface (BCI). Nonetheless, owing to high intra- and inter-subject variabilities, many studies on decoding EEG were designed in a subject-specific manner by using calibration samples, with no concern of its practical use, hampered by time-consuming steps and a large data requirement. To this end, recent studies adopted a transfer learning strategy, especially domain adaptation techniques. Among those, we have witnessed the potential of adversarial learning-based transfer learning in BCIs. In the meantime, it is known that adversarial learning-based domain adaptation methods are prone to negative transfer that disrupts learning generalized feature representations, applicable to diverse domains, for example, subjects or sessions in BCIs. In this article, we propose a novel framework that learns class-relevant and subject-invariant feature representations in an information-theoretic manner, without using adversarial learning. To be specific, we devise two operational components in a deep network that explicitly estimate mutual information between feature representations: 1) to decompose features in an intermediate layer into class-relevant and class-irrelevant ones and 2) to enrich class-discriminative feature representation. On two large EEG datasets, we validated the effectiveness of our proposed framework by comparing with several comparative methods in performance. Furthermore, we conducted rigorous analyses by performing an ablation study in regard to the components in our network, explaining our model's decision on input EEG signals via layer-wise relevance propagation, and visualizing the distribution of learned features via t-SNE.

Probability-based Graph Embedding Cross-domain and Class Discriminative Feature Learning for Domain Adaptation.

Feature-based domain adaptation methods project samples from different domains into the same feature space and try to align the distribution of two domains to learn an effective transferable model. The vital problem is how to find a proper way to reduce the domain shift and improve the discriminability of features. To address the above issues, we propose a unified Probability-based Graph embedding Cross-domain and class Discriminative feature learning framework for unsupervised domain adaptation (PGCD). Specifically, we propose novel graph embedding structures to be the class discriminative transfer feature learning item and cross-domain alignment item, which can make the same-category samples compact in each domain, and fully align the local and global geometric structure across domains. Besides, two theoretical analyses are given to prove the interpretability of the proposed graph structures, which can further describe the relationships between samples to samples in single-domain and cross-domain transfer feature learning scenarios. Moreover, we adopt novel weight strategies via probability information to generate robust centroids in each proposed item to enhance the accuracy of transfer feature learning and reduce the error accumulation. Compared with the advanced approaches by comprehensive experiments, the promising performance on the benchmark datasets verify the effectiveness of the proposed model.

NIRS Data Augmentation Technique to Detect Hemodynamic Peaks During Self-Paced Motor Imagery

Optical brain monitoring, such as near-infrared spectroscopy (NIRS), has facilitated numerous brain studies, including those based on machine learning techniques. A large and diverse dataset is necessary for training machine learning algorithms to avoid overfitting a limited amount of data. However, recruiting sufficient subjects is challenging owing to time and budget constraints. Therefore, we propose an NIRS data generation algorithm that scales NIRS signal components, such as hemodynamic response function, physiological noise, and system spike noise, based on the source-detector distance to augment the training data. The experimental data were augmented with the generated NIRS data to train a convolutional neural network to classify self-paced left- and right-hand motor imagery. Augmenting the training dataset with 1000 generated data points increased the classification accuracy to 86.3 ± 4.1%, indicating a 26% increase compared with training on experimental data only. In addition, we applied Guided Gradient-weighted Class Activation Mapping (Grad-CAM) to visualize the class discriminative features of the input data. The Guided Grad-CAM heatmaps aligned well with the oxy-hemoglobin peaks during self-paced motor imagery. We concluded that the increased cerebral oxygenation, especially in the contralateral hemisphere, was the class-discriminative feature for classifying left- and right-hand motor imagery.

Failure to Achieve Domain Invariance With Domain Generalization Algorithms: An Analysis in Medical Imaging

One prominent issue in the application of deep learning is the failure to generalize to data that lies on a different distribution to the training data. While many methods have been proposed to address this, prior work has shown that when operating under the same conditions most algorithms perform almost equally. As such, more work needs to be done to validate past and future methods before they are put into important scenarios like medical imaging. Our work analyses eight domain generalization algorithms across four important medical imaging classification datasets along with three standard natural image classification problems to discover the differences in how these methods operate in these different contexts. We assess these algorithms in terms of generalization capability, domain invariance, and representational sensitivity. Through this, we show that despite the differences between domain and content variations between natural and medical imaging there is little deviation in the operation of each method between natural images and medical images. Additionally, we show that all tested algorithms retain significant amounts of domain-specific information in their feature representations despite explicit training to remove it. Thus, revealing the failure point of all these methods is a lack of class-discriminative features extracted from out-of-distribution data. While these results show that methods that work well on natural imaging work similarly in medical imaging, no method outperforms baseline methods, highlighting the continuing gap of achieving adequate domain generalization. Similarly, the results also question the efficacy of optimizing for domain invariant representations as a method for generalizing to unseen domains.

Push Stricter to Decide Better: A Class-Conditional Feature Adaptive Framework for Improving Adversarial Robustness

In response to the threat of adversarial examples, adversarial training provides an attractive option for improving robustness by training models on online-augmented adversarial examples. However, most existing adversarial training methods focus on improving the model’s robust accuracy by strengthening the adversarial examples but neglecting the increasing shift between natural data and adversarial examples, leading to a decrease in natural accuracy. To maintain the trade-off between natural and robust accuracy, we alleviate the shift from the perspective of feature adaption and propose a Feature Adaptive Adversarial Training (FAAT) optimizing the class-conditional feature adaption across natural data and adversarial examples. Specifically, we propose to incorporate a class-conditional discriminator to encourage the features to become <xref ref-type="disp-formula" rid="deqn1" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">(1)</xref> class-discriminative and <xref ref-type="disp-formula" rid="deqn2" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">(2)</xref> invariant to the change of adversarial attacks. The novel FAAT framework enables the trade-off between natural and robust accuracy by generating features with similar distribution across natural and adversarial data within the same class and achieves higher overall robustness benefiting from the class-discriminative feature characteristics. Experiments on various datasets demonstrate that FAAT produces more discriminative features and performs favorably against state-of-the-art methods.

TMLP+SRDANN: A domain adaptation method for EEG-based emotion recognition

Electroencephalogram (EEG) contains emotion information, but usually undergoes severe signal variations. In the literature, for the issue of EEG-based emotion recognition, not only have diverse features been designed to capture the representative characteristics of EEG, but also various domain adaptation techniques have been devised to reduce the distributional discrepancy between the source and target domains. However, existing domain adaptation techniques for this issue treat all the source samples equally, but ignore their transferability difference, which more or less limits the performance of learned domain-invariant and class-discriminative EEG features for emotion recognition. To cope with this problem, we propose Transposition Multi-Layer Perceptron (TMLP) and Sample-Reweighted Domain Adaptation Neural Network (SRDANN) in one whole learning framework. TMLP extracts the robust multi-channel EEG features related to emotions; SRDANN transfers the discriminative knowledge from the source domain to the target domain for classifying emotions. By concentrating on the source samples with stronger transferability in domain adaptation, TMLP+SRDANN can learn the more domain-invariant and class-discriminative features from EEG, thus providing a more effective solution to emotion recognition in the challenging cross-subject scenario. The subject-independent emotion recognition experiments on two benchmark datasets have demonstrated the effectiveness of our method: an accuracy of 81.04% for emotion classification on SEED, 61.88% for valence classification and 57.70% for arousal classification on DEAP.

Transferable Feature Selection for Unsupervised Domain Adaptation

Domain adaptation aims at extracting knowledge from auxiliary source domains to assist the learning task in a target domain. In classification problems, since the distributions of the source and target domains are different, directly using source data to build a classifier for the target domain may hamper the classification performance on the target data. Fortunately, in many tasks, there can be some features that are transferable, i.e., the source and target domains share similar properties. On the other hand, it is common that the source data contain noisy features which may degrade the learning performance in the target domain. This issue, however, is barely studied in existing works. In this paper, we propose to find a feature subset that is transferable across the source and target domains. As a result, the domain discrepancy measured on the selected features can be reduced. Moreover, we seek to find the most discriminative features for classification. To achieve the above goals, we formulate a new sparse learning model that is able to jointly reduce the domain discrepancy and select informative features for classification. We develop two optimization algorithms to address the derived learning problem. Extensive experiments on real-world data sets demonstrate the effectiveness of the proposed method.

Towards Prediction Constraints: A Novel Domain Adaptation Method for Machine Fault Diagnosis

Domain adaptation technologies have been extensively explored and successfully applied to machine fault diagnosis, aiming to address problems that target data are unlabeled and have a certain distribution bias with source data. Nonetheless, existing fault diagnosis methods mainly explore feature-level alignment strategies to reduce domain discrepancies, which not only fails to directly ascertain the relationship between the target output and domain deviation, but also cannot guarantee accurate diagnosis results (i.e., learning class-discriminative features) when only relying on feature adaptation. In light of these issues, a more intuitive and effective domain adaptation method is developed for intelligent diagnosis of machinery in this article, in which the minimum class confusion and maximum nuclear norm-based target prediction constraints are simultaneously designed to promote learning reliable domain-invariant and discriminative features for accurate fault diagnosis. We conduct extensive experiments based on two different mechanical systems to evaluate the proposed method. Comprehensive results and discussions demonstrate the promising performance of our approach.

Individual-specific characterization of event-related hemodynamic responses during an auditory task: An exploratory study

Functional near-infrared spectroscopy (fNIRS) has been established as an informative modality for understanding the hemodynamic-metabolic correlates of cortical auditory processing. To date, such knowledge has shown broad clinical applications in the diagnosis, treatment, and intervention procedures in disorders affecting auditory processing; however, exploration of the hemodynamic response to auditory tasks is yet incomplete. This holds particularly true in the context of auditory event-related fNIRS experiments, where preliminary work has shown the presence of valid responses while leaving the need for more comprehensive explorations of the hemodynamic correlates of event-related auditory processing. In this study, we apply an individual-specific approach to characterize fNIRS-based hemodynamic changes during an auditory task in healthy adults. Oxygenated hemoglobin (HbO2) concentration change time courses were acquired from eight participants. Independent component analysis (ICA) was then applied to isolate individual-specific class discriminative spatial filters, which were then applied to HbO2 time courses to extract auditory-related hemodynamic features. While six of eight participants produced significant class discriminative features before ICA-based spatial filtering, the proposed method identified significant auditory hemodynamic features in all participants. Furthermore, ICA-based filtering improved correlation between trial labels and extracted features in every participant. For the first time, this study demonstrates hemodynamic features important in experiments exploring auditory processing as well as the utility of individual-specific ICA-based spatial filtering in fNIRS-based feature extraction techniques in auditory experiments. These outcomes provide insights for future studies exploring auditory hemodynamic characteristics and may eventually provide a baseline framework for better understanding auditory response dysfunctions in clinical populations.

Semi-supervised generative and discriminative adversarial learning for motor imagery-based brain\u2013computer interface

Convolutional neural networks (CNNs), which can recognize structural/configuration patterns in data with different architectures, have been studied for feature extraction. However, challenges remain regarding leveraging advanced deep learning methods in BCIs. We focus on problems of small-sized training samples and interpretability of the learned parameters and leverages a semi-supervised generative and discriminative learning framework that effectively utilizes synthesized samples with real samples to discover class-discriminative features. Our framework learns the distributional characteristics of EEG signals in an embedding space using a generative model. By using artificially generated and real EEG signals, our framework finds class-discriminative spatio-temporal feature representations that help to correctly discriminate input EEG signals. It is noteworthy that the framework facilitates the exploitation of real, unlabeled samples to better uncover the underlying patterns inherent in a user’s EEG signals. To validate our framework, we conducted experiments comparing our method with conventional linear models by utilizing variants of three existing CNN architectures as generator networks and measuring the performance on three public datasets. Our framework exhibited statistically significant improvements over the competing methods. We investigated the learned network via activation pattern maps and visualized generated artificial samples to empirically justify the stability and neurophysiological plausibility of our model.

Improving Alzheimer's Disease Detection for Speech Based on Feature Purification Network.

Alzheimer's disease (AD) is a neurodegenerative disease involving the decline of cognitive ability with illness progresses. At present, the diagnosis of AD mainly depends on the interviews between patients and doctors, which is slow, expensive, and subjective, so it is not a better solution to recognize AD using the currently available neuropsychological examinations and clinical diagnostic criteria. A recent study has indicated the potential of language analysis for AD diagnosis. In this study, we proposed a novel feature purification network that can improve the representation learning of transformer model further. Though transformer has made great progress in generating discriminative features because of its long-distance reasoning ability, there is still room for improvement. There exist many common features that are not indicative of any specific class, and we rule out the influence of common features from traditional features extracted by transformer encoder and can get more discriminative features for classification. We apply this method to improve transformer's performance on three public dementia datasets and get improved classification results markedly. Specifically, the method on Pitt datasets gets state-of-the-art (SOTA) result.

MStrans: Multiscale Vision Transformer for Aerial Objects Detection

Detecting objects in aerial images is a challenging task due to the large-scale variations and arbitrary orientations with tiny instances. A new multi-scale transformer-based aerial objects detector called MStrans is proposed in this paper to deal with the challenges in aerial detection. To detect remote instances, MStrans adopts a multi-scale patch embedding transformer (MViT) to extract the global features of the image effectively. Furthermore, to capture the different discriminant features for classification and regression branch tasks, the partial interactive fusion module (PIFM) is designed to enhance the semantic expression of the key features of classification and regression tasks by using the strategy of interactive modeling of adjacent layer features. In addition, considering that the transformer may worsen the local feature details while capturing long-distance feature dependencies, this paper designs a global to local interactive fusion module (GLIFM). It uses the advantage of convolution to extract local features to enrich the detailed information in the transformer. Experiments were carried out on DOTA and DIOR datasets, and the MStrans achieves superior detection performances compared with other approaches.

Multi-label feature selection via manifold regularization and dependence maximization

Feature selection is able to select more discriminative features for classification and plays an important role in multi-label learning to alleviate the effect of the curse of dimensionality. Recently, the multi-label feature selection methods based on the sparse regression model have received increasing attentions. However, most of these methods directly project original data space to label space in the regression model, which is inappropriate because the linear assumption between data space and label space doesn't hold in most cases. In the paper, we propose a feature selection method named multi-label feature selection via manifold regularization and dependence maximization (MRDM). In the regression model of MRDM, the original data space is projected to a low-dimensional manifold space, which not only has the same topological structure with the original data, but also has a strong dependence with the class labels. Then, an objective function involving l2,1-norm regularization is formulated, and an alternating optimization-based iterative algorithm is designed to obtain the sparse coefficients for multi-label feature selection. Extensive experiments on various multi-label data sets demonstrate the superiority of the proposed method compared with some state-of-the-art multi-label feature selection methods.

Interpatient ECG Heartbeat Classification with an Adversarial Convolutional Neural Network.

Discovering shared, invariant feature representations across subjects in electrocardiogram (ECG) classification tasks is crucial for improving the generalization of models to unknown patients. Although deep neural networks have recently been emerging in extracting generalizable ECG features, they usually rely on labeled samples from a large number of subjects to guarantee generalization. Extracting invariant representations to intersubject variabilities from a small number of subjects is still a challenge today due to individual physical differences. To address this problem, we propose an adversarial deep neural network framework for interpatient heartbeat classification by integrating adversarial learning into a convolutional neural network to learn subject-invariant, class-discriminative features. The proposed method was evaluated on the MIT-BIH arrhythmia database which is a publicly available ECG dataset collected from 47 patients. Compared with the state-of-the-art methods, the proposed method achieves the highest performance for detecting supraventricular ectopic beats (SVEBs), which are very challenging to identify, and also gains comparable performance on the detection of ventricular ectopic beats (VEBs). The sensitivities of SVEBs and VEBs are 78.8% and 92.5%, respectively. The precisions of SVEBs and VEBs are 90.8% and 94.3%, respectively. With high performance in the detection of pathological classes (i.e., SVEBs and VEBs), this work provides a promising method for ECG classification tasks when the number of patients is limited.