Discriminative Features For Classification Research Articles

Domain adaptation with feature and label adversarial networks

Learning a cross-domain representation from labeled source domains to unlabeled target domains is an important research problem in representation learning. Despite the success of traditional adversarial methods, they proposed to align features from each domain only while neglecting the importance of labels, when fooling a special domain discriminator network. Thus, the discriminator of these approaches merely distinguishes whether the generated features are in-domain or not, which may lead to less class-discriminative features. In this paper, by considering the joint distributions of features and labels in both domains, we present Feature and Label Adversarial Networks (FLAN). As a result, FLAN can generate more discriminative features in both domains. Experimental results on standard unsupervised domain adaptation benchmarks have demonstrated that FLAN can outperform the state-of-art domain invariant representation learning methods.

Optimal Transport Based Deep Domain Adaptation Approach for Fault Diagnosis of Rotating Machine

Rotating machinery working under changing operation conditions is prone to failure. In recent years, domain adaptation has been successfully used for fault diagnosis. However, the existing fault diagnosis methods based on domain adaptation have two main disadvantages: 1) with these methods, it is difficult to precisely measure and estimate the differences between the source and target domains and 2) they only consider the discrepancies in the feature space, but not in the label space. In this article, a new optimal transport (OT)-based deep domain adaptation model is proposed for rotating machine fault diagnosis. The framework of the proposed method comprises three main components. First, an autoencoder network is designed to extract compact and class discriminative features from the raw data. Second, the domain-invariant representation features are trained by searching an OT plan with a predefined cost function between source and target domains and by minimizing the discrepancies of a joint distribution of the feature and label spaces based on OT. Finally, the classifier trained with data in the source domain is directly used to perform the classification task in the target domain. In addition, the optimal selection of the model hyper-parameters is verified through empirical analysis, and the transfer ability of the proposed model is visually illustrated in a reduced feature space. The experimental results show that the proposed method outperforms the existing machine learning and domain adaptation fault diagnosis methods, in terms of, e.g., classification accuracy and generalization ability.

Joint Clustering and Discriminative Feature Alignment for Unsupervised Domain Adaptation.

Unsupervised Domain Adaptation (UDA) aims to learn a classifier for the unlabeled target domain by leveraging knowledge from a labeled source domain with a different but related distribution. Many existing approaches typically learn a domain-invariant representation space by directly matching the marginal distributions of the two domains. However, they ignore exploring the underlying discriminative features of the target data and align the cross-domain discriminative features, which may lead to suboptimal performance. To tackle these two issues simultaneously, this paper presents a Joint Clustering and Discriminative Feature Alignment (JCDFA) approach for UDA, which is capable of naturally unifying the mining of discriminative features and the alignment of class-discriminative features into one single framework. Specifically, in order to mine the intrinsic discriminative information of the unlabeled target data, JCDFA jointly learns a shared encoding representation for two tasks: supervised classification of labeled source data, and discriminative clustering of unlabeled target data, where the classification of the source domain can guide the clustering learning of the target domain to locate the object category. We then conduct the cross-domain discriminative feature alignment by separately optimizing two new metrics: 1) an extended supervised contrastive learning, i.e., semi-supervised contrastive learning 2) an extended Maximum Mean Discrepancy (MMD), i.e., conditional MMD, explicitly minimizing the intra-class dispersion and maximizing the inter-class compactness. When these two procedures, i.e., discriminative features mining and alignment are integrated into one framework, they tend to benefit from each other to enhance the final performance from a cooperative learning perspective. Experiments are conducted on four real-world benchmarks (e.g., Office-31, ImageCLEF-DA, Office-Home and VisDA-C). All the results demonstrate that our JCDFA can obtain remarkable margins over state-of-the-art domain adaptation methods. Comprehensive ablation studies also verify the importance of each key component of our proposed algorithm and the effectiveness of combining two learning strategies into a framework.

Group visualization of class-discriminative features

Research explaining the behavior of convolutional neural networks (CNNs) has gained a lot of attention over the past few years. Although many visualization methods have been proposed to explain network predictions, most fail to provide clear correlations between the target output and the features extracted by convolutional layers. In this work, we define a concept, i.e., class-discriminative feature groups, to specify features that are extracted by groups of convolutional kernels correlated with a particular image class. We propose a detection method to detect class-discriminative feature groups and a visualization method to highlight image regions correlated with particular output and to interpret class-discriminative feature groups intuitively. The experiments showed that the proposed method can disentangle features based on image classes and shed light on what feature groups are extracted from which regions of the image. We also applied this method to visualize “lost” features in adversarial samples and features in an image containing a non-class object to demonstrate its ability to debug why the network failed or succeeded.

RefineFace: Refinement Neural Network for High Performance Face Detection.

Face detection has achieved significant progress in recent years. However, high performance face detection still remains a very challenging problem, especially when there exists many tiny faces. In this paper, we present a single-shot refinement face detector namely RefineFace to achieve high performance. Specifically, it consists of five modules: selective two-step regression (STR), selective two-step classification (STC), scale-aware margin loss (SML), feature supervision module (FSM) and receptive field enhancement (RFE). To enhance the regression ability for high location accuracy, STR coarsely adjusts locations and sizes of anchors from high level detection layers to provide better initialization for subsequent regressor. To improve the classification ability for high recall efficiency, STC first filters out most simple negatives from low level detection layers to reduce search space for subsequent classifier, then SML is applied to better distinguish faces from background at various scales and FSM is introduced to let the backbone learn more discriminative features for classification. Besides, RFE is presented to provide more diverse receptive field to better capture faces in some extreme poses. Extensive experiments conducted on WIDER FACE, AFW, PASCAL Face, FDDB, MAFA demonstrate that our method achieves state-of-the-art results and runs at 37.3 FPS with ResNet-18 for VGA-resolution images.

A Novel Weighted Adversarial Transfer Network for Partial Domain Fault Diagnosis of Machinery

Recently, domain adaptation techniques have achieved great attention in solving domain-shift problems of mechanical fault diagnosis. However, existing methods mostly work under assumption that source domain and target domain share identical label spaces, which fail to handle those issues, where a large set of source data classes are available and target data only cover a subset of classes. To address this problem, a novel weighted adversarial transfer network (WATN) is proposed for partial domain fault diagnosis, in this article. Adversarial training is introduced to learn both class discriminative and domain invariant features, and a weighting learning strategy is adopted to weigh their contributions to both source classifier and domain discriminator. As such, the irrelevant source examples can be identified and filtered out, and the distribution discrepancy of shared classes between domains can be reduced. Experiments on two diagnosis data sets demonstrate that the proposed WATN achieves satisfactory performance and outperforms state-of-the-art methods.

Hand-crafted and deep convolutional neural network features fusion and selection strategy: An application to intelligent human action recognition

Human action recognition (HAR) has gained much attention in the last few years due to its enormous applications including human activity monitoring, robotics, visual surveillance, to name but a few. Most of the previously proposed HAR systems have focused on using hand-crafted images features. However, these features cover limited aspects of the problem and show performance degradation on a large and complex datasets. Therefore, in this work, we propose a novel HAR system which is based on the fusion of conventional hand-crafted features using histogram of oriented gradients (HoG) and deep features. Initially, human silhouette is extracted with the help of saliency-based method - implemented in two phases. In the first phase, motion and geometric features are extracted from the selected channel, whilst, second phase calculates the Chi-square distance between the extracted and threshold-based minimum distance features. Afterwards, extracted deep CNN and hand-crafted features are fused to generate a resultant vector. Moreover, to cope with the curse of dimensionality, an entropy-based feature selection technique is also proposed to identify the most discriminant features for classification using multi-class support vector machine (M-SVM). All the simulations are performed on five publicly available benchmark datasets including Weizmann, UCF11 (YouTube), UCF Sports, IXMAS, and UT-Interaction. A comparative evaluation is also presented to show that our proposed model achieves superior performances in comparison to a few exiting methods.

Global-Local Mutual Attention Model for Text Classification

Text classification is a central field of inquiry in natural language processing (NLP). Although some models learn local semantic features and global long-term dependencies simultaneously, they simply combine them through concatenation either in a cascade way or in parallel while mutual effects between them are ignored. In this paper, we propose the Global-Local Mutual Attention (GLMA) model for text classification problems, which introduces a mutual attention mechanism for mutual learning between local semantic features and global long-term dependencies. The mutual attention mechanism consists of a Local-Guided Global-Attention (LGGA) and a Global-Guided Local-Attention (GGLA). The LGGA allows to assign weights and combine global long-term dependencies of word positions that are semantic related. It captures combined semantics and alleviates the gradient vanishing problem. The GGLA automatically assigns more weights to relevant local semantic features, which captures key local semantic information and filters both noises and irrelevant words/phrases. Furthermore, a weighted-over-time pooling operation is developed to aggregate the most informative and discriminative features for classification. Extensive experiments demonstrate that our model obtains the state-of-the-art performance on seven benchmark datasets and sixteen Amazon product reviews datasets. Both the result analysis and the mutual attention weights visualization further demonstrate the effectiveness of the proposed model.

A Hand-Based Multi-Biometrics via Deep Hashing Network and Biometric Graph Matching

At present, the fusion of different unimodal biometrics has attracted increasing attention from researchers, who are dedicated to the practical application of biometrics. In this paper, we explored a multi-biometric algorithm that integrates palmprints and dorsal hand veins (DHV). Palmprint recognition has a rather high accuracy and reliability, and the most significant advantage of DHV recognition is the biopsy (Liveness detection). In order to combine the advantages of both and implement the fusion method, deep learning and graph matching were, respectively, introduced to identify palmprint and DHV. Upon using the deep hashing network (DHN), biometric images can be encoded as 128-bit codes. Then, the Hamming distances were used to represent the similarity of two codes. Biometric graph matching (BGM) can obtain three discriminative features for classification. In order to improve the accuracy of open-set recognition, in multi-modal fusion, the score-level fusion of DHN and BGM was performed and authentication was provided by support vector machine (SVM). Furthermore, based on DHN, all four levels of fusion strategies were used for multi-modal recognition of palmprint and DHV. Evaluation experiments and comprehensive comparisons were conducted on various commonly used datasets, and the promising results were obtained in this case where the equal error rates (EERs) of both palmprint recognition and multi-biometrics equal 0, demonstrating the great superiority of DHN in biometric verification.

High-Resolution Aerial Images Semantic Segmentation Using Deep Fully Convolutional Network With Channel Attention Mechanism

Semantic segmentation is one of the fundamental tasks in understanding high-resolution aerial images. Recently, convolutional neural network (CNN) and fully convolutional network (FCN) have achieved excellent performance in general images’ semantic segmentation tasks and have been introduced to the field of aerial images. In this paper, we propose a novel deep FCN with channel attention mechanism (CAM-DFCN) for high-resolution aerial images’ semantic segmentation. The CAM-DFCN architecture follows the mode of encoder–decoder. In the encoder, two identical deep residual networks are both divided into multiple levels and acted on spectral images and auxiliary data, respectively. Then, the feature map concatenation is carried out at each level. In the decoder, the channel attention mechanism (CAM) is introduced to automatically weigh the channels of feature maps to perform feature selection. On the one hand, the CAM follows the concatenated feature maps at each level to select more discriminative features for classification. On the other hand, the CAM is used to further weigh the semantic information and spatial location information in the adjacent-level concatenated feature maps for more accurate predictions. We evaluate the proposed CAM-DFCN by using two benchmarks (the Potsdam set and the Vaihingen set) provided by the International Society for Photogrammetry and Remote Sensing. Experimental results show that the proposed method has considerable improvement.

Underwater target classification at greater depths using deep neural network with joint multiple‐domain feature

For underwater target classification which is supposed to recognise different ships with the radiated acoustic signal, it is the most challenging task to provide excellent classification accuracy in a variety of environments. However, most of the existing systems are optimised to get the best performance on the data set from certain situations which they are trained in, which may lead to generalisation risks when applied to new environments. Here, the authors introduce an underwater target classification framework using a deep neural network to learn deep features from a large joint multiple-domain input. The authors propose to incorporate spectral and wavelet domain information with different resolutions to grasp the `global' structure and the `local' transient variation of the raw radiated signals. In contrast to shallow models, a stacked sparse autoencoder (SSAE) model, which is composed of multiple hidden layers and a softmax classifier, is adopted to learn more discriminating features for classification. In the authors' experiments, the proposed SSAE model is evaluated on the data set consisting of underwater acoustic signal received at different ocean depths. The authors' results show that the proposed SSAE model with joint input features achieved a 5% improvement in classification accuracy compared to the state-of-the-art DBN approach.

Automatic lung nodule detection using multi-scale dot nodule-enhancement filter and weighted support vector machines in chest computed tomography.

A novel CAD scheme for automated lung nodule detection is proposed to assist radiologists with the detection of lung cancer on CT scans. The proposed scheme is composed of four major steps: (1) lung volume segmentation, (2) nodule candidate extraction and grouping, (3) false positives reduction for the non-vessel tree group, and (4) classification for the vessel tree group. Lung segmentation is performed first. Then, 3D labeling technology is used to divide nodule candidates into two groups. For the non-vessel tree group, nodule candidates are classified as true nodules at the false positive reduction stage if the candidates survive the rule-based classifier and are not screened out by the dot filter. For the vessel tree group, nodule candidates are extracted using dot filter. Next, RSFS feature selection is used to select the most discriminating features for classification. Finally, WSVM with an undersampling approach is adopted to discriminate true nodules from vessel bifurcations in vessel tree group. The proposed method was evaluated on 154 thin-slice scans with 204 nodules in the LIDC database. The performance of the proposed CAD scheme yielded a high sensitivity (87.81%) while maintaining a low false rate (1.057 FPs/scan). The experimental results indicate the performance of our method may be better than the existing methods.

Computer-Assisted Diagnosis for Diabetic Retinopathy Based on Fundus Images Using Deep Convolutional Neural Network

Diabetic retinopathy (DR) is a complication of long-standing diabetes, which is hard to detect in its early stage because it only shows a few symptoms. Nowadays, the diagnosis of DR usually requires taking digital fundus images, as well as images using optical coherence tomography (OCT). Since OCT equipment is very expensive, it will benefit both the patients and the ophthalmologists if an accurate diagnosis can be made, based solely on reading digital fundus images. In the paper, we present a novel algorithm based on deep convolutional neural network (DCNN). Unlike the traditional DCNN approach, we replace the commonly used max-pooling layers with fractional max-pooling. Two of these DCNNs with a different number of layers are trained to derive more discriminative features for classification. After combining features from metadata of the image and DCNNs, we train a support vector machine (SVM) classifier to learn the underlying boundary of distributions of each class. For the experiments, we used the publicly available DR detection database provided by Kaggle. We used 34,124 training images and 1,000 validation images to build our model and tested with 53,572 testing images. The proposed DR classifier classifies the stages of DR into five categories, labeled with an integer ranging between zero and four. The experimental results show that the proposed method can achieve a recognition rate up to 86.17%, which is higher than previously reported in the literature. In addition to designing a machine learning algorithm, we also develop an app called “Deep Retina.” Equipped with a handheld ophthalmoscope, the average person can take fundus images by themselves and obtain an immediate result, calculated by our algorithm. It is beneficial for home care, remote medical care, and self-examination.

Structured sparse multi-view feature selection based on weighted hinge loss

In applications, using features obtained from multiple views to describe objects has become popular because multiple views contain much more information than the single view. As the dimensions of the data sets are high, which may cause expensive time consumption and memory space, how to identify the representative views and features becomes a crucial problem. Multi-view feature selection that can integrate multiple views to select important and relevant features to improve performance has attracted more and more attentions in recent years. Previous supervised multi-view feature selection methods usually establish the models by concatenating multiple views into long vectors. However, this concatenation is not physically meaningful and implies that different views play the similar roles for specific tasks. In this paper, we propose a novel supervised multi-view feature selection method based on the weighted hinge loss (WHMVFS) that can learn the corresponding weight for each view and implement sparsity from the group and individual point of views under the structured sparsity framework. The newly proposed multi-view weighted hinge loss penalty not only has the ability to select more discriminative features for classification, but also can make the involved optimization problem be decomposed into several small scale subproblems, which can be easily solved by an iterative algorithm, and the convergence of the iterative algorithm is also proved. Experimental results conducted on real-world data sets show the effectiveness of the proposed method.

Sparse binarised statistical dynamic features for spatio-temporal texture analysis

The paper presents a new spatio-temporal learning-based descriptor called binarised statistical dynamic features (BSDF) for representation and classification of dynamic texture. The BSDF descriptor operates by applying three-dimensional spatio-temporal filters on local voxels of an image sequence where the filters are learned via an independent component analysis, maximising independence over spatial and temporal domains concurrently. The BSDF representation is formed by binarising filter responses which are then converted into codewords and summarised using histograms. A robust representation of the BSDF descriptor is finally obtained via a sparse representation approach yielding very discriminative features for classification. The effects of different hyper-parameters on performance including the number of filters, the number of scales, temporal depth, number of samples drawn are also investigated. The proposed approach is evaluated on the most commonly used dynamic texture databases and shown to perform very well compared to the existing methods.

Domain Invariant and Class Discriminative Feature Learning for Visual Domain Adaptation.

Domain adaptation manages to build an effective target classifier or regression model for unlabeled target data by utilizing the well-labeled source data but lying different distributions. Intuitively, to address domain shift problem, it is crucial to learn domain invariant features across domains, and most existing approaches have concentrated on it. However, they often do not directly constrain the learned features to be class discriminative for both source and target data, which is of vital importance for the final classification. Therefore, in this paper, we put forward a novel feature learning method for domain adaptation to construct both domain invariant and class discriminative representations, referred to as DICD. Specifically, DICD is to learn a latent feature space with important data properties preserved, which reduces the domain difference by jointly matching the marginal and class-conditional distributions of both domains, and simultaneously maximizes the inter-class dispersion and minimizes the intra-class scatter as much as possible. Experiments in this paper have demonstrated that the class discriminative properties will dramatically alleviate the cross-domain distribution inconsistency, which further boosts the classification performance. Moreover, we show that exploring both domain invariance and class discriminativeness of the learned representations can be integrated into one optimization framework, and the optimal solution can be derived effectively by solving a generalized eigen-decomposition problem. Comprehensive experiments on several visual cross-domain classification tasks verify that DICD can outperform the competitors significantly.

Semi-Supervised Maximum Discriminative Local Margin for Gene Selection

In the present study, we introduce a novel semi-supervised method called the semi-supervised maximum discriminative local margin (semiMM) for gene selection in expression data. The semiMM is a “filter” approach that exploits local structure, variance, and mutual information. We first constructed a local nearest neighbour graph and divided this information into within-class and between-class local nearest neighbour graphs by weighing the edge between the two data points. The semiMM aims to discover the most discriminative features for classification via maximizing the local margin between the within-class and between-class data, the variance of all data, and the mutual information of features with class labels. Experiments on five publicly available gene expression datasets revealed the effectiveness of the proposed method compared to three state-of-the-art feature selection algorithms.

Automatic classification of first-episode, drug-naive schizophrenia with multi-modal magnetic resonance imaging

A great number of studies have demonstrated the structural and functional abnormalities in chronic schizophrenia (SZ) patients. However, few studies analyzed the differences between first-episode, drug-naive SZ (FESZ) patients and normal controls (NCs). In this study, we recruited 44 FESZ patients and 56 NCs, and acquired their multi-modal magnetic resonance imaging (MRI) data, including structural and resting-state functional MRI data. We calculated gray matter volume (GMV), regional homogeneity (ReHo), amplitude of low frequency fluctuation (ALFF), and degree centrality (DC) of 90 brain regions, basing on an automated anatomical labeling (AAL) atlas. We then applied these features into support vector machine (SVM) combined with recursive feature elimination (RFE) to discriminate FESZ patients from NCs. Our results showed that the classifier using the combination of ReHo and ALFF as input features achieved the best performance (an accuracy of 96.97%). Moreover, the most discriminative features for classification were predominantly located in the frontal lobe. Our findings may provide potential information for understanding the neuropathological mechanism of SZ and facilitate the development of biomarkers for computer-aided diagnosis of SZ patients.

Discriminative self-representation sparse regression for neuroimaging-based alzheimer's disease diagnosis.

In this paper, we propose a novel feature selection method by jointly considering (1) 'task-specific' relations between response variables (e.g., clinical labels in this work) and neuroimaging features and (2) 'self-representation' relations among neuroimaging features in a sparse regression framework. Specifically, the task-specific relation is devised to learn the relative importance of features for representation of response variables by a linear combination of the input features in a supervised manner, while the self-representation relation is used to take into account the inherent information among neuroimaging features such that any feature can be represented by a weighted sum of the other features, regardless of the label information, in an unsupervised manner. By integrating these two different relations along with a group sparsity constraint, we formulate a new sparse linear regression model for class-discriminative feature selection. The selected features are used to train a support vector machine for classification. To validate the effectiveness of the proposed method, we conducted experiments on the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset; experimental results showed superiority of the proposed method over the state-of-the-art methods considered in this work.

A support vector machine-based method to identify mild cognitive impairment with multi-level characteristics of magnetic resonance imaging

Mild cognitive impairment (MCI) represents a transitional state between normal aging and Alzheimer’s disease (AD). Non-invasive diagnostic methods are desirable to identify MCI for early therapeutic interventions. In this study, we proposed a support vector machine (SVM)-based method to discriminate between MCI patients and normal controls (NCs) using multi-level characteristics of magnetic resonance imaging (MRI). This method adopted a radial basis function (RBF) as the kernel function, and a grid search method to optimize the two parameters of SVM. The calculated characteristics, i.e., the Hurst exponent (HE), amplitude of low-frequency fluctuations (ALFF), regional homogeneity (ReHo) and gray matter density (GMD), were adopted as the classification features. A leave-one-out cross-validation (LOOCV) was used to evaluate the classification performance of the method. Applying the proposed method to the experimental data from 29 MCI patients and 33 healthy subjects, we achieved a classification accuracy of up to 96.77%, with a sensitivity of 93.10% and a specificity of 100%, and the area under the curve (AUC) yielded up to 0.97. Furthermore, the most discriminative features for classification were found to predominantly involve default-mode regions, such as hippocampus (HIP), parahippocampal gyrus (PHG), posterior cingulate gyrus (PCG) and middle frontal gyrus (MFG), and subcortical regions such as lentiform nucleus (LN) and amygdala (AMYG). Therefore, our method is promising in distinguishing MCI patients from NCs and may be useful for the diagnosis of MCI.