Class Discriminatory Information Research Articles

Contrastive Learning Assisted-Alignment for Partial Domain Adaptation.

This work addresses unsupervised partial domain adaptation (PDA), in which classes in the target domain are a subset of the source domain. The key challenges of PDA are how to leverage source samples in the shared classes to promote positive transfer and filter out the irrelevant source samples to mitigate negative transfer. Existing PDA methods based on adversarial DA do not consider the loss of class discriminative representation. To this end, this article proposes a contrastive learning-assisted alignment (CLA) approach for PDA to jointly align distributions across domains for better adaptation and to reweight source instances to reduce the contribution of outlier instances. A contrastive learning-assisted conditional alignment (CLCA) strategy is presented for distribution alignment. CLCA first exploits contrastive losses to discover the class discriminative information in both domains. It then employs a contrastive loss to match the clusters across the two domains based on adversarial domain learning. In this respect, CLCA attempts to reduce the domain discrepancy by matching the class-conditional and marginal distributions. Moreover, a new reweighting scheme is developed to improve the quality of weights estimation, which explores information from both the source and the target domains. Empirical results on several benchmark datasets demonstrate that the proposed CLA outperforms the existing state-of-the-art PDA methods.

Discriminative subspace embedded dynamic geometrical and statistical alignment based on pseudo-label correction for cross-domain bearing fault diagnosis

Many domain adaptation (DA) approaches have been developed to address the challenge of domain divergence in cross-domain fault diagnosis. However, most of them only attempt to align statistical distribution while neglecting geometric alignment between source and target data. Furthermore, the use of some unreliable pseudo-labels may cause geometrical and statistical distributions mismatching and interfere with the DA model generating correct pseudo-labels during the iterative learning. In this paper, we propose a new model called discriminative subspace embedded dynamic geometrical and statistical alignment based on pseudo-label correction (DSDGSA-PC) for bearing fault diagnosis. Firstly, discriminative subspace alignment is proposed to mitigate feature redundancy and divergence by generating aligned subspaces for two domains, while preserving class discriminative information and global structures of data. Then, DSDGSA-PC leverages the representer theorem and the principle of structural risk minimization to learn a domain-invariant classifier in the subspace, while minimizing statistical and geometrical shift by jointly optimizing dynamic graph embedding and dynamic weighted distribution alignment strategies. Finally, a novel pseudo-label correction mechanism is integrated into DSDGSA-PC to evaluate the credibility of pseudo-labels and rectify the unreliable ones during the iterations. The experimental results illustrate that DSDGSA-PC has higher transfer performance compared to several advanced methods on 24 transfer tasks.

Sparse representation learning using [formula omitted] compressed sensing and rank-revealing QR factorization

Compressive sensing can be conceptualized for a classification problem in the context of representation learning, which is frequently applied for signal reconstruction via a few measurements. A novel compressive sensing-based approach is presented to substantially reduce the number of required samples for classification. This technique considers the pixels of image as sensors and it seeks to identify the most optimum sensors as decision variables in feature space. In this study, Spatial sensor locations are acquired through learning to identify the regions within an image where the most discriminative information for classification is embedded. ℓ1−2 minimization, as nonconvex but Lipschitz continuous, is solved to obtain the least non-zero elements of the full measurement vector to completely reconstruct the discriminative vector in feature space. Optimal sensors are localized from the training-set and subsequent test images are categorized based on learned sensors. The algorithm consists of three primary components: sparse minimization, feature space, and discrimination vector. ℓ1−2 minimization, rank-revealing QR factorization (RRQR) and SVM are considered for enhanced sparsity exploitation, feature space extraction and discrimination vector acquiring, respectively. The proposed method is evaluated on four different experiments and is compared against a state-of-art technique. The results demonstrate the superiority of proposed method to the compared method.

A Novel Parameter-Optimized Recurrent Attention Network for Pipeline Leakage Detection

Accurate detection of pipeline leakage is essential to maintain the safety of pipeline transportation. Recently, deep learning (DL) has emerged as a promising tool for pipeline leakage detection (PLD). However, most existing DL methods have difficulty in achieving good performance in identifying leakage types due to the complex time dynamics of pipeline data. On the other hand, the initial parameter selection in the detection model is generally random, which may lead to unstable recognition performance. For this reason, a hybrid DL framework referred to as parameter-optimized recurrent attention network (PRAN) is presented in this paper to improve the accuracy of PLD. First, a parameter-optimized long short-term memory (LSTM) network is introduced to extract effective and robust features, which exploits a particle swarm optimization (PSO) algorithm with cross-entropy fitness function to search for globally optimal parameters. With this framework, the learning representation capability of the model is improved and the convergence rate is accelerated. Moreover, an anomaly-attention mechanism (AM) is proposed to discover class discriminative information by weighting the hidden states, which contributes to amplifying the normal-abnormal distinguishable discrepancy, further improving the accuracy of PLD. After that, the proposed PRAN not only implements the adaptive optimization of network parameters, but also enlarges the contribution of normal-abnormal discrepancy, thereby overcoming the drawbacks of instability and poor generalization. Finally, the experimental results demonstrate the effectiveness and superiority of the proposed PRAN for PLD.

Domain Adaptation via Low Rank and Class Discriminative Representation for Autism Spectrum Disorder identification: A Multi-site fMRI Study.

To construct a more effective model with good generalization performance for inter-site autism spectrum disorder (ASD) diagnosis, domain adaptation based ASD diagnostic models are proposed to alleviate the inter-site heterogeneity. However, most existing methods only reduce the marginal distribution difference without considering class discriminative information, and are difficult to achieve satisfactory results. In this paper, we propose a low rank and class discriminative representation (LRCDR) based multi-source unsupervised domain adaptation method to reduce the marginal and conditional distribution differences synchronously for improving ASD identification. Specifically, LRCDR adopts low rank representation to alleviate the marginal distribution difference between domains by aligning the global structure of the projected multi-site data. To reduce the conditional distribution difference of data from all sites, LRCDR learns the class discriminative representation of data from multiple source domains and the target domain to enhance the intra-class compactness and inter-class separability of the projected data. For inter-site prediction on all ABIDE data (1102 subjects from 17 sites), LRCDR obtains the mean accuracy of 73.1%, superior to the results of the compared state-of-the-art domain adaptation methods and multi-site ASD identification methods. In addition, we locate some meaningful biomarkers: Most of the top important biomarkers are inter-network resting-state functional connectivities (RSFCs). The proposed LRCDR method can effectively improve the identification of ASD, which has great potential as a clinical diagnostic tool.

GA-Net: A Guidance Aware Network for Skeleton-Based Early Activity Recognition

Early activity prediction, which aims to recognize class labels before actions are fully performed, is a very challenging task since partially observed action sequences contain insufficient class-discrimination information, and thus, many partial action sequences belonging to different categories may look very similar. Therefore, in this paper, we propose a novel <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">guidance aware network (GA-Net)</i> to boost the ability to distinguish different activities in diversified partially observed action sequences via metric learning. To mitigate the similarity problem of action segments at very early stages, the proposed <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">guided metric learning module (GMLM)</i> is able to encourage the feature extractor to mine class-discriminative information given partially observed sequences. Specifically, the GMLM is able to minimize the intraclass distance with a full-length guided direction approach and maximize the difference between interclass categories with different observation ratios. To enhance the similarities between the partial- and full-length sequences in the same action categories, we further introduce a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">distribution alignment module (DAM)</i> that employs full-length guidance to pull the partially observed features closer to the global features. We evaluate our proposed method on three public human activity datasets and achieve competitive results compared with the state-of-the-art approaches.

Fuzzy-rough hybrid dimensionality reduction

Dimensionality reduction including feature selection and feature extraction is a significant preprocessing method for data analysis, which is an important application of fuzzy rough set theory. Linear discriminant analysis (LDA) is a commonly used supervised feature extraction method, which can generate transformations while retaining as much class discrimination information as possible. However, it is worth noting that linear discriminant analysis is not interpretable. To improve the interpretability of the model and avoid losing part of the original feature information when only using feature selection for dimensionality reduction, this paper proposes a hybrid dimensionality reduction method based on fuzzy rough set and linear discriminant analysis. It can obtain a reduction set that contains complementary transformation features and original features. First, the semi-feature set is generated by criterion based on the normalized fuzzy joint mutual information, and the hybrid feature set is further induced by applying LDA to the feature set. Then, the redundancy between features is evaluated by the normalized fuzzy mutual information. The significance of a feature is defined by the proposed normalized effective classification information amount and the symmetric effective classification information amount. Next, the priority value between features according to the significance of them is defined to delete the redundant feature from the hybrid feature set. Furthermore, a hybrid dimensionality reduction algorithm based on fuzzy rough set is designed further. Finally, the performance of the proposed method is compared with some methods. The experimental results show that the proposed algorithm can obtain effective feature subset, which improve the performance of a classifier.

Dual-Representation-Based Autoencoder for Domain Adaptation.

Domain adaptation aims to facilitate the learning task in an unlabeled target domain by leveraging the auxiliary knowledge in a well-labeled source domain from a different distribution. Almost existing autoencoder-based domain adaptation approaches focus on learning domain-invariant representations to reduce the distribution discrepancy between source and target domains. However, there is still a weakness existing in these approaches: the class-discriminative information of the two domains may be damaged while aligning the distributions of the source and target domains, which makes the samples with different classes close to each other, leading to performance degradation. To tackle this issue, we propose a novel dual-representation autoencoder (DRAE) to learn dual-domain-invariant representations for domain adaptation. Specifically, DRAE consists of three learning phases. First, DRAE learns global representations of all source and target data to maximize the interclass distance in each domain and minimize the marginal distribution and conditional distribution of both domains simultaneously. Second, DRAE extracts local representations of instances sharing the same label in both domains to maintain class-discriminative information in each class. Finally, DRAE constructs dual representations by aligning the global and local representations with different weights. Using three text and two image datasets and 12 state-of-the-art domain adaptation methods, the extensive experiments have demonstrated the effectiveness of DRAE.

Recognizing micro actions in videos by learning multi-layer local features

Recognizing micro actions such as slight head shaking or hand clapping from videos can be challenging since they only involve small movements of local body parts. In this paper, we propose to fuse features from both higher-level and lower-level layers of convolutional neural networks for improving the accuracy of micro-action recognition. Deep features in higher-level layers have been shown to be effective in recognizing general actions, such as biking and jumping, that involve relatively large movements. Different from features in higher-level layers, features in lower-level layers are usually of higher resolution and can help capture small motions in micro actions. In this paper, we employ class-discriminative information as a guidance in lower-level layers to learn local features that are highly associated with micro-action regions. In the experiments, we evaluate the proposed method on two micro-action video datasets and achieve new state-of-the-art performance. We also test the proposed method on two general-action video datasets with promising performance.

Discriminative Invariant Alignment for Unsupervised Domain Adaptation

As one of the most prevalent branches of transfer learning, domain adaptation is dedicated to generalizing the knowledge of a source domain to a target domain to perform machine learning tasks. In domain adaptation, the key strategy is to overcome the shift between different domains and learn shared features with domain invariance. However, most existing methods focus on extracting the common features of the source and target domains, and do not consider the shift problem of class center in the target domain caused by this process. Specifically, when we align the domain distributions, we often ignore the inherent feature attributes of the data, or under the guidance of false pseudo-labels, cause the target domain data to be far away from the class center after projection. This is not conducive to classification task. To address these problems, in this study, we propose a novel domain adaptation method, referred to as discriminative invariant alignment (DIA), for image representation. DIA enriches the knowledge matrix by combining the class discriminative information of the source domain and local data structure information of the target domain into a new framework. By introducing the maximum margin criterion of the source domain, the classification boundaries are expanded. To verify the performance of the proposed method, we compared DIA with several state-of-the-art methods on five benchmark databases. The experimental results show that DIA is superior to the state-of-the-art methods.

Hyper-Laplacian Regularized Multi-View Subspace Clustering With a New Weighted Tensor Nuclear Norm

In this paper, we present a hyper-Laplacian regularized method WHLR-MSC with a new weighted tensor nuclear norm for multi-view subspace clustering. Specifically, we firstly stack the subspace representation matrices of the different views into a tensor, which neatly captures the higher-order correlations between the different views. Secondly, in order to make all the singular values have different contributions in tensor nuclear norm based on tensor-Singular Value Decomposition (t-SVD), we use weighted tensor nuclear norm to constrain the constructed tensor, which can obtain the class discrimination information of the sample distribution more accurately. Third, from a geometric point of view, the data are usually sampled from a low-dimensional manifold embedded in a high-dimensional ambient space, the WHLR-MSC model uses hyper-Laplacian graph regularization to capture the local geometric structure of the data. An effective algorithm for solving the optimization problem of WHLR-MSC model is proposed. Extensive experiments on five benchmark image datasets show the effectiveness of our proposed WHLR-MSC method.

Discriminative Transfer Feature and Label Consistency for Cross-Domain Image Classification.

Visual domain adaptation aims to seek an effective transferable model for unlabeled target images by benefiting from the well-labeled source images following different distributions. Many recent efforts focus on extracting domain-invariant image representations via exploring target pseudo labels, predicted by the source classifier, to further mitigate the conditional distribution shift across domains. However, two essential factors are overlooked by most existing methods: 1) the learned transferable features should be not only domain invariant but also category discriminative; and 2) the target pseudo label is a two-edged sword to cross-domain alignment. In other words, the wrongly predicted target labels may hinder the class-wise domain matching. In this article, to address these two issues simultaneously, we propose a discriminative transfer feature and label consistency (DTLC) approach for visual domain adaptation problems, which can naturally unify cross-domain alignment with discriminative information preserved and label consistency of source and target data into one framework. To be specific, DTLC first incorporates class discriminative information by penalizing the maximum distance of data pair in the same class and the minimum distance of data pair sharing the different labels for each data into the distribution alignment of both domains. The target pseudo labels are then refined based on the label consistency within the domains. Thus, the transfer feature learning and coarse-to-fine target labels would be coupled to benefit each other in an iterative way. Comprehensive experiments on several visual cross-domain benchmarks verify that DTLC can gain remarkable margins over state-of-the-art (SOTA) nondeep visual domain adaptation methods and even be comparable to competitive deep domain adaptation ones.

Double Competitive Constraints-Based Collaborative Representation for Pattern Classification

Representation-based classification (RBC) has attracted much attention in pattern recognition. As a linear representative RBC method, collaborative representation-based classification (CRC) is very promising for classification. Although many extensions of CRC have been developed recently, the discriminative and competitive representations of different classes for favorable classification has not yet been fully explored. To design the discriminative and competitive collaborative representations for enhancing the power of pattern discrimination, we propose a novel double competitive constraints-based collaborative representation for classification (DCCRC). In the proposed DCCRC, one competitive constraint is the l2-norm regularization of residuals between each query sample and the class-specific representations, the other one is the l2-norm regularization of the representations of all the classes excluding any one class. In two competitive constraints, the class discrimination information is employed to generate competitive representations. Moreover, the proposed method integrates both the representation learning and classification into the unified model. We study the effectiveness and robustness of the proposed method by comparing it with the state-of-the-art CRC methods on six face databases and twelve UCI data sets. The experimental results demonstrate the promising classification performance of the proposed method.

Learning Cross-Aligned Latent Embeddings for Zero-Shot Cross-Modal Retrieval

Zero-Shot Cross-Modal Retrieval (ZS-CMR) is an emerging research hotspot that aims to retrieve data of new classes across different modality data. It is challenging for not only the heterogeneous distributions across different modalities, but also the inconsistent semantics across seen and unseen classes. A handful of recently proposed methods typically borrow the idea from zero-shot learning, i.e., exploiting word embeddings of class labels (i.e., class-embeddings) as common semantic space, and using generative adversarial network (GAN) to capture the underlying multimodal data structures, as well as strengthen relations between input data and semantic space to generalize across seen and unseen classes. In this paper, we propose a novel method termed Learning Cross-Aligned Latent Embeddings (LCALE) as an alternative to these GAN based methods for ZS-CMR. Unlike using the class-embeddings as the semantic space, our method seeks for a shared low-dimensional latent space of input multimodal features and class-embeddings by modality-specific variational autoencoders. Notably, we align the distributions learned from multimodal input features and from class-embeddings to construct latent embeddings that contain the essential cross-modal correlation associated with unseen classes. Effective cross-reconstruction and cross-alignment criterions are further developed to preserve class-discriminative information in latent space, which benefits the efficiency for retrieval and enable the knowledge transfer to unseen classes. We evaluate our model using four benchmark datasets on image-text retrieval tasks and one large-scale dataset on image-sketch retrieval tasks. The experimental results show that our method establishes the new state-of-the-art performance for both tasks on all datasets.

A group lasso based sparse KNN classifier

Sparse features have been shown effective in applications of computer vision, machine learning, signal processing, etc. Group sparsity was proposed by considering that there exist natural group structures in many problems. Previous research mainly focuses on improving the way to extract sparse features, such as lasso, group lasso, overlapped group lasso, sparse group lasso. In existing work, sparse features are usually taken as input for classifiers, such as SVM, KNN, or SRC (Sparse Representation based Classification). In this paper, we find that, instead of using sparse group features as input for classifiers, sparse group features are good candidates for selection of most relevant classes/groups. We design a new classifier to improve classification accuracy: (1) we use sparse group lasso to select K most relevant classes/groups, which makes this approach robust, because it filters out unrelated classes/groups in group level, instead of individual sample level; (2) KSVD is used to get exact desired sparsity (k nonzero entries) and thus eliminates the difficulty of hyperparameter tuning; (3) simple summation of regression weights within each class/group contains sufficient class discriminant information, and the chance of a sample belonging to a specific class is denoted simply by the summation of corresponding regression weights within each class, which is in line with the need of Explainable AI (XAI). The K most relevant groups/classes can be considered as K neighbors of the correct class. Thus, we call this classifier Group Lasso based Sparse KNN (GLSKNN). Compared to 8 other approaches, GLSKNN classifier outperforms other methods in term of classification accuracy for two public image datasets and images with different occlusion/noise levels.

Deterministic dropout for deep neural networks using composite random forest

Dropout prevents overfitting in deep neural networks. Typical strategy of dropout involves random termination of connections irrespective of their importance. Termination blocks the propagation of class discriminative information across the network. As a result, dropout may lead to inferior performance. We propose a deterministic dropout where only unimportant connections are dropped ensuring propagation of class discriminative information. We identify the unimportant connections using a novel composite random forest, integrated into the network. We prove that better generalization is achieved by terminating these unimportant connections. The proposed algorithm is useful in preventing overfitting in noisy datasets. The proposal is equally good for datasets with smaller number of training examples. Experiments on several benchmark datasets show up to 8% improvement in classification accuracy.

Discriminative, Competitive, and Collaborative Representation-Based Classification with l2-Norm Regularizations

Recently, collaborative representation-based classification (CRC) and its many variations have been widely applied for various classification tasks in pattern recognition. To further enhance the pattern discrimination of CRC, in this article we propose a novel extension of CRC, entitled discriminative, competitive, and collaborative representation-based classification (DCCRC). In the proposed DCCRC, the class discrimination information is fully utilized for promoting the true class of each testing sample to dominantly represent the testing sample during collaborative representation. The class discrimination information is well considered in the newly designed discriminative l2-norm regularization that can decrease the ability of representation from the interclasses of each testing sample. Simultaneously, a competitive l2-norm regularization is introduced to the DCCRC model with the class discrimination information with the aim of enhancing the competitive ability of representation from the true class of each testing sample. The effectiveness of the proposed DCCRC is explored by extensive experiments on the several public face databases and some real numerical UCI data sets. The experimental results demonstrate that the proposed DCCRC achieves the superior performance over the state-of-the-art representation-based classification methods.

Crack Fault Classification for Planetary Gearbox Based on Feature Selection Technique and K-means Clustering Method

During the condition monitoring of a planetary gearbox, features are extracted from raw data for a fault diagnosis. However, different features have different sensitivity for identifying different fault types, and thus, the selection of a sensitive feature subset from an entire feature set and retaining as much of the class discriminatory information as possible has a directly effect on the accuracy of the classification results. In this paper, an improved hybrid feature selection technique (IHFST) that combines a distance evaluation technique (DET), Pearson’s correlation analysis, and an ad hoc technique is proposed. In IHFST, a temporary feature subset without irrelevant features is first selected according to the distance evaluation criterion of DET, and the Pearson’s correlation analysis and ad hoc technique are then employed to find and remove redundant features in the temporary feature subset, respectively, and hence, a sensitive feature subset without irrelevant or redundant features is selected from the entire feature set. Further, the k-means clustering method is applied to classify the different kinds of health conditions. The effectiveness of the proposed method was validated through several experiments carried out on a planetary gearbox with incipient cracks seeded in the tooth root of the sun gear, planet gear, and ring gear. The results show that the proposed method can successfully distinguish the different health conditions of a planetary gearbox, and achieves a better classification performance than other methods. This study proposes a sensitive feature subset selection method that achieves an obvious improvement in terms of the accuracy of the fault classification.

Semisupervised local preserving embedding algorithm based on maximum margin criterion for large‐scale data streams

SummaryIn the field of machine learning, feature extraction is one of the most important preprocessing in data classification for its effectiveness, and now it has attracted much extensive attention for large‐scale data stream preprocessing step, especially in the era of big data. Motivated by the advantages of unsupervised and supervised feature extraction, which are two desirable and promising characteristics for dimension reduction, a new semisupervised local preserving embedding algorithm based on maximum margin criterion (SLPE/MMC) is proposed in this paper. First, the objective functions of maximum margin criterion (MMC) and neighborhood preserving embedding (NPE) are combined to get the first objective function of SLPE/MMC. Then, in order to overcome the out‐of‐sample problem, a linear transformation is introduced to construct the second objective function. At last, the whole optimal objective function is constructed by combing the two objective functions together. The proposed algorithm has effectively taken advantage of the sample's supervised information and keeps the geometry structure and the class discrimination information of the manifold. Experiments on face datasets Yale, CMU PIE, and AR datasets are performed to evaluate the classification accuracy of SLPE/MMC. The experimental results and time complexity comparisons have demonstrated the effectiveness of the proposed method.

Machinery running state identification based on discriminant semi-supervised local tangent space alignment for feature fusion and extraction

Extraction of sensitive features is a challenging but key task in data-driven machinery running state identification. Aimed at solving this problem, a method for machinery running state identification that applies discriminant semi-supervised local tangent space alignment (DSS-LTSA) for feature fusion and extraction is proposed. Firstly, in order to extract more distinct features, the vibration signals are decomposed by wavelet packet decomposition WPD, and a mixed-domain feature set consisted of statistical features, autoregressive (AR) model coefficients, instantaneous amplitude Shannon entropy and WPD energy spectrum is extracted to comprehensively characterize the properties of machinery running state(s). Then, the mixed-dimension feature set is inputted into DSS-LTSA for feature fusion and extraction to eliminate redundant information and interference noise. The proposed DSS-LTSA can extract intrinsic structure information of both labeled and unlabeled state samples, and as a result the over-fitting problem of supervised manifold learning and blindness problem of unsupervised manifold learning are overcome. Simultaneously, class discrimination information is integrated within the dimension reduction process in a semi-supervised manner to improve sensitivity of the extracted fusion features. Lastly, the extracted fusion features are inputted into a pattern recognition algorithm to achieve the running state identification. The effectiveness of the proposed method is verified by a running state identification case in a gearbox, and the results confirm the improved accuracy of the running state identification.