Discriminative Feature Learning Research Articles

Multi-granularity tooth analysis via YOLO-based object detection models for effective tooth detection and classification

Accurate detection and classification of teeth is the first step in dental disease diagnosis. However, the same class of tooth exhibits significant variations in surface appearance. Moreover, the complex geometrical structure poses challenges in learning discriminative features among the different tooth classes. Due to these complex features, tooth classification is one of the challenging research domains in deep learning. To address the aforementioned issues, the presented study proposes discriminative local feature extraction at different granular levels using YOLO models. However, this necessitates a granular intra-oral image dataset. To facilitate this requirement, a dataset at three granular levels (two, four, and seven teeth classes) is developed. YOLOv5, YOLOv6, and YOLOv7 models were trained using 2,790 images. The results indicate superior performance of YOLOv6 for two-class classification problems. The model generated a mean average precision (mAP) value of 94%. However, as the granularity level is increased, the performance of YOLO models decreases. For, four and seven-class classification problems, the highest mAP value of 87% and 79% was achieved by YOLOv5 respectively. The results indicate that different levels of granularity play an important role in tooth detection and classification. The YOLO’s performance gradually decreased as the granularity decreased especially at the finest granular level.

MuLA-GAN: Multi-Level Attention GAN for Enhanced Underwater Visibility

The underwater environment presents unique challenges (color distortions, reduced contrast, blurriness) hindering accurate analysis. This work introduces MuLA-GAN, a novel approach leveraging Generative Adversarial Networks (GANs) and specifically adapted Multi-Level Attention for comprehensive underwater image enhancement. MuLA-GAN integrates Multi-Level Attention within the GAN architecture to prioritize learning discriminative features crucial for precise image restoration. These relevant features encompass information on local details within image regions leveraged by spatial attention and features at various scales across the entire image captured by multi-level attention. This allows MuLA-GAN to identify and enhance objects, textures, and edges obscured by underwater distortions while also reconstructing a more accurate and visually clear representation of the underwater scene by analyzing low-level information like edges and textures, as well as high-level information like object shapes and global scene information. By selectively focusing on these relevant features, MuLA-GAN excels at capturing and preserving intricate details in underwater imagery, which is essential for various marine research, exploration, and resource management applications. Extensive evaluations on diverse datasets (UIEB test, UIEB challenge, U45, UCCS) demonstrate MuLA-GAN's superior performance compared to existing methods. Additionally, a specialized bio-fouling and aquaculture dataset confirms the model's robustness in challenging environments. On the UIEB test dataset, MuLA-GAN achieves exceptional Peak Signal-to-Noise Ratio (PSNR) (25.59) and Structural Similarity Index (SSIM) (0.893) scores, surpassing Water-Net (24.36 PSNR, 0.885 SSIM). This work addresses a significant research gap in underwater image enhancement by demonstrating the effectiveness of combining GANs with specifically adapted Multi-Level Attention mechanisms. This tailored approach offers a novel and comprehensive framework for restoring underwater image quality, providing valuable insights for accurate underwater scene analysis. The source code for MuLA-GAN is publicly available on GitHub at https://github.com/AhsanBaidar/MuLA_GAN.git

Discriminative Feature Learning With Co-Occurrence Attention Network for Vehicle ReID

Discriminative Feature Learning With Co-Occurrence Attention Network for Vehicle ReID

Clustering-Guided Twin Contrastive Learning for Endomicroscopy Image Classification.

Learning better representations is essential in medical image analysis for computer-aided diagnosis. However, learning discriminative semantic features is a major challenge due to the lack of large-scale well-annotated datasets. Thus, how can we learn a well-structured categorizable embedding space in limited-scale and unlabeled datasets? In this paper, we proposed a novel clustering-guided twin-contrastive learning framework (CTCL) that learns the discriminative representations of probe-based confocal laser endomicroscopy (pCLE) images for gastrointestinal (GI) tumor classification. Compared with traditional contrastive learning, in which only two randomly augmented views of the same instance are considered, the proposed CTCL aligns more semantically related and class-consistent samples by clustering, which improved intra-class tightness and inter-class variability to produce more informative representations. Furthermore, based on the inherent properties of CLE (geometric invariance and intrinsic noise), we proposed to regard CLE images with any angle rotation and CLE images with different noises as the same instance, respectively, for increased variability and diversity of samples. By optimizing CTCL in an end-to-end expectation-maximization framework, comprehensive experimental results demonstrated that CTCL-based visual representations achieved competitive performance on each downstream task as well as more robustness and transferability compared with existing state-of-the-art SSL and supervised methods. Notably, CTCL achieved 75.60%/78.45% and 64.12%/77.37% top-1 accuracy on the linear evaluation protocol and few-shot classification downstream tasks, respectively, which outperformed the previous best results by 1.27%/1.63% and 0.5%/3%, respectively. The proposed method holds great potential to assist pathologists in achieving an automated, fast, and high-precision diagnosis of GI tumors and accurately determining different stages of tumor development based on CLE images.

Dynamic Weighting Translation Transfer Learning for Imbalanced Medical Image Classification.

Medical image diagnosis using deep learning has shown significant promise in clinical medicine. However, it often encounters two major difficulties in real-world applications: (1) domain shift, which invalidates the trained model on new datasets, and (2) class imbalance problems leading to model biases towards majority classes. To address these challenges, this paper proposes a transfer learning solution, named Dynamic Weighting Translation Transfer Learning (DTTL), for imbalanced medical image classification. The approach is grounded in information and entropy theory and comprises three modules: Cross-domain Discriminability Adaptation (CDA), Dynamic Domain Translation (DDT), and Balanced Target Learning (BTL). CDA connects discriminative feature learning between source and target domains using a synthetic discriminability loss and a domain-invariant feature learning loss. The DDT unit develops a dynamic translation process for imbalanced classes between two domains, utilizing a confidence-based selection approach to select the most useful synthesized images to create a pseudo-labeled balanced target domain. Finally, the BTL unit performs supervised learning on the reassembled target set to obtain the final diagnostic model. This paper delves into maximizing the entropy of class distributions, while simultaneously minimizing the cross-entropy between the source and target domains to reduce domain discrepancies. By incorporating entropy concepts into our framework, our method not only significantly enhances medical image classification in practical settings but also innovates the application of entropy and information theory within deep learning and medical image processing realms. Extensive experiments demonstrate that DTTL achieves the best performance compared to existing state-of-the-art methods for imbalanced medical image classification tasks.

Sequence based model using deep neural network and hybrid features for identification of 5-hydroxymethylcytosine modification

RNA modifications are pivotal in the development of newly synthesized structures, showcasing a vast array of alterations across various RNA classes. Among these, 5-hydroxymethylcytosine (5HMC) stands out, playing a crucial role in gene regulation and epigenetic changes, yet its detection through conventional methods proves cumbersome and costly. To address this, we propose Deep5HMC, a robust learning model leveraging machine learning algorithms and discriminative feature extraction techniques for accurate 5HMC sample identification. Our approach integrates seven feature extraction methods and various machine learning algorithms, including Random Forest, Naive Bayes, Decision Tree, and Support Vector Machine. Through K-fold cross-validation, our model achieved a notable 84.07% accuracy rate, surpassing previous models by 7.59%, signifying its potential in early cancer and cardiovascular disease diagnosis. This study underscores the promise of Deep5HMC in offering insights for improved medical assessment and treatment protocols, marking a significant advancement in RNA modification analysis.

SaliencyCut: Augmenting plausible anomalies for anomaly detection

Anomaly detection under the open-set scenario is a challenging task that requires learning discriminative features to detect anomalies that were even unseen during training. As a cheap yet effective approach, data augmentation has been widely used to create pseudo anomalies for better training of such models. Recent wisdom of augmentation methods focuses on generating random pseudo instances that may lead to a mixture of augmented instances with seen anomalies, or out of the typical range of anomalies. To address this issue, we propose a novel saliency-guided data augmentation method, SaliencyCut, to produce pseudo but more common anomalies that tend to stay in the plausible range of anomalies. Furthermore, we deploy a two-head learning strategy consisting of normal and anomaly learning heads to learn the anomaly score of each sample. Theoretical analyses show that this mechanism offers a more tractable and tighter lower bound of the data log-likelihood. We then design a novel patch-wise residual module in the anomaly learning head to extract and assess anomaly features from each sample, facilitating the learning of discriminative representations of anomaly instances. Extensive experiments conducted on six real-world anomaly detection datasets demonstrate the superiority of our method to competing methods under various settings. Codes are available at: https://github.com/yjnanan/SaliencyCut.

Transfer and Alignment Network for Generalized Category Discovery

Generalized Category Discovery (GCD) is a crucial real-world task that aims to recognize both known and novel categories from an unlabeled dataset by leveraging another labeled dataset with only known categories. Despite the improved performance on known categories, current methods perform poorly on novel categories. We attribute the poor performance to two reasons: biased knowledge transfer between labeled and unlabeled data and noisy representation learning on the unlabeled data. The former leads to unreliable estimation of learning targets for novel categories and the latter hinders models from learning discriminative features. To mitigate these two issues, we propose a Transfer and Alignment Network (TAN), which incorporates two knowledge transfer mechanisms to calibrate the biased knowledge and two feature alignment mechanisms to learn discriminative features. Specifically, we model different categories with prototypes and transfer the prototypes in labeled data to correct model bias towards known categories. On the one hand, we pull instances with known categories in unlabeled data closer to these prototypes to form more compact clusters and avoid boundary overlap between known and novel categories. On the other hand, we use these prototypes to calibrate noisy prototypes estimated from unlabeled data based on category similarities, which allows for more accurate estimation of prototypes for novel categories that can be used as reliable learning targets later. After knowledge transfer, we further propose two feature alignment mechanisms to acquire both instance- and category-level knowledge from unlabeled data by aligning instance features with both augmented features and the calibrated prototypes, which can boost model performance on both known and novel categories with less noise. Experiments on three benchmark datasets show that our model outperforms SOTA methods, especially on novel categories. Theoretical analysis is provided for an in-depth understanding of our model in general. Our code and data are available at https://github.com/Lackel/TAN.

PICNN: A Pathway towards Interpretable Convolutional Neural Networks

Convolutional Neural Networks (CNNs) have exhibited great performance in discriminative feature learning for complex visual tasks. Besides discrimination power, interpretability is another important yet under-explored property for CNNs. One difficulty in the CNN interpretability is that filters and image classes are entangled. In this paper, we introduce a novel pathway to alleviate the entanglement between filters and image classes. The proposed pathway groups the filters in a late conv-layer of CNN into class-specific clusters. Clusters and classes are in a one-to-one relationship. Specifically, we use the Bernoulli sampling to generate the filter-cluster assignment matrix from a learnable filter-class correspondence matrix. To enable end-to-end optimization, we develop a novel reparameterization trick for handling the non-differentiable Bernoulli sampling. We evaluate the effectiveness of our method on ten widely used network architectures (including nine CNNs and a ViT) and five benchmark datasets. Experimental results have demonstrated that our method PICNN (the combination of standard CNNs with our proposed pathway) exhibits greater interpretability than standard CNNs while achieving higher or comparable discrimination power.

Discriminative multi-scale adjacent feature for person re-identification

AbstractRecently, discriminative and robust identification information has played an increasingly critical role in Person Re-identification (Re-ID). It is a fact that the existing part-based methods demonstrate strong performance in the extraction of fine-grained features. However, their intensive partitions lead to semantic information ambiguity and background interference. Meanwhile, we observe that the body with different structural proportions. Hence, we assume that aggregation with the multi-scale adjacent features can effectively alleviate the above issues. In this paper, we propose a novel Discriminative Multi-scale Adjacent Feature (MSAF) learning framework to enrich semantic information and disregard background. In summary, we establish multi-scale interaction in two stages: the feature extraction stage and the feature aggregation stage. Firstly, a Multi-scale Feature Extraction (MFE) module is designed by combining CNN and Transformer structure to obtain the discriminative specific feature, as the basis for the feature aggregation stage. Secondly, a Jointly Part-based Feature Aggregation (JPFA) mechanism is revealed to implement adjacent feature aggregation with diverse scales. The JPFA contains Same-scale Feature Correlation (SFC) and Cross-scale Feature Correlation (CFC) sub-modules. Finally, to verify the effectiveness of the proposed method, extensive experiments are performed on the common datasets of Market-1501, CUHK03-NP, DukeMTMC, and MSMT17. The experimental results achieve better performance than many state-of-the-art methods.

SigCo: Eliminate the inter-class competition via sigmoid for learning with noisy labels

Accurate predictions from deep neural networks are crucial for distinguishing clean data and correcting noisy labels in current label noise learning methods. However, the conventional Softmax classifier used in most relevant works is highly sensitive to label noise due to its inherent competition-prompting mechanism, i.e., similar categories are encouraged to compete for limited confidence scores during class activation, especially between the noisy classes and the ground-truth, which can inevitably lead to suboptimal predictions and eventually hamper model performance. To address this inter-class competition problem, we propose a novel Sigmoid-based Sample Selection and Correction method named SigCo for learning with noisy labels. Different from previous works, we develop a Sigmoid-based network in which each Sigmoid classifier independently predicts its respective class, improving the reliability of the selection and correction process through more accurate predictions. Besides, in order to mitigate the negative impact of noisy labels, we design a noise-adaptive learning strategy by imposing stringent class masking constraints on clean samples to enhance the learning of discriminative features, while adopting a loose masking strategy for noisy data to improve the robustness to label noise. Additionally, we introduce a co-training strategy between our Sigmoid-based network and the conventional Softmax-based network to implicitly boost the generalization capability of the model. Extensive experiments on synthetic and real-world benchmarks show that SigCo consistently outperforms state-of-the-art methods. Especially on CIFAR-100N with 80% and 90% symmetric noise ratios, it improves test accuracy by 5.10% and 18.44%, respectively.

Semantics-enhanced discriminative descriptor learning for LiDAR-based place recognition

LiDAR-based place recognition (LPR) aims to localize autonomous vehicles and mobile robots relative to pre-built maps or retrieve previously visited places. However, the complexity of real-world scenes and changes in viewpoint are significant challenges for place recognition. As high-level information, semantics makes it easier to distinguish geometrically similar scene situations. Unlike most existing methods that rely solely on a single type of information (geometric or semantic) to construct scene descriptors, we consider the complementary nature of the semantic and geometric information and propose a semantics-enhanced discriminative feature learning method for LPR. Specifically, we first develop a Multi-layer Fusion Feature Extraction Network (MFFEN) based on the transformer encoder to hierarchically fuse local geometric and semantic information and utilize contextual information for extracting discriminative local features. To obtain semantic information, we introduce the dynamic graph convolution network to extract local semantic features with local relations. In addition, to weaken the interference of redundancy and dynamic objects in the scene, we design a semantics-guided local attention network (SLAN) to focus on salient local features that are helpful for recognizing scenes, thereby enhancing the descriptive ability of the global descriptor. Extensive experiments on public datasets KITTI and KITTI-360 demonstrate that the proposed method performs better than recent LiDAR-based methods on the 3D place recognition task. For instance, it achieves the mean F1max score of 96.9% on the KITTI dataset, surpassing the strongest prior model by 2.7%.

An efficient deep learning network with orthogonal softmax layer for automatic detection of tuberculosis

Tuberculosis (TB) is a persistent bacterial lung infection that affects the lungs severely. It is one of the fatal diseases among the top ten leading causes of death. Thus, early and accurate diagnosis of TB precisely has vital role in saving lives. In this work, a novel Deep Learning Network with Orthogonal SoftMax Layer (OSL) is proposed for the detection of Tuberculosis (TB) that yields admirable performance in small medical datasets. The integrated merits of deep learning-based efficient feature extraction and OSL-based classification boost the classification performance. OSL enhances discriminative feature learning capability by maintaining orthogonality among weight vectors in this layer. It is also responsible for reducing the co-adaption among parameters and making optimization easier. In addition, data augmentation is done in the training stage using a customized version of RandAugment to train the network properly, even with small medical datasets. The experimental results depict the proposed deep learning network outperforms others with the best 98.67% accuracy, 98.99% precision, 98.33% sensitivity, 99.00% specificity, and 0.9866 F1 score.

Universal representation learning for multivariate time series using the instance-level and cluster-level supervised contrastive learning

AbstractThe multivariate time series classification (MTSC) task aims to predict a class label for a given time series. Recently, modern deep learning-based approaches have achieved promising performance over traditional methods for MTSC tasks. The success of these approaches relies on access to the massive amount of labeled data (i.e., annotating or assigning tags to each sample that shows its corresponding category). However, obtaining a massive amount of labeled data is usually very time-consuming and expensive in many real-world applications such as medicine, because it requires domain experts’ knowledge to annotate data. Insufficient labeled data prevents these models from learning discriminative features, resulting in poor margins that reduce generalization performance. To address this challenge, we propose a novel approach: supervised contrastive learning for time series classification (SupCon-TSC). This approach improves the classification performance by learning the discriminative low-dimensional representations of multivariate time series, and its end-to-end structure allows for interpretable outcomes. It is based on supervised contrastive (SupCon) loss to learn the inherent structure of multivariate time series. First, two separate augmentation families, including strong and weak augmentation methods, are utilized to generate augmented data for the source and target networks, respectively. Second, we propose the instance-level, and cluster-level SupCon learning approaches to capture contextual information to learn the discriminative and universal representation for multivariate time series datasets. In the instance-level SupCon learning approach, for each given anchor instance that comes from the source network, the low-variance output encodings from the target network are sampled as positive and negative instances based on their labels. However, the cluster-level approach is performed between each instance and cluster centers among batches, as opposed to the instance-level approach. The cluster-level SupCon loss attempts to maximize the similarities between each instance and cluster centers among batches. We tested this novel approach on two small cardiopulmonary exercise testing (CPET) datasets and the real-world UEA Multivariate time series archive. The results of the SupCon-TSC model on CPET datasets indicate its capability to learn more discriminative features than existing approaches in situations where the size of the dataset is small. Moreover, the results on the UEA archive show that training a classifier on top of the universal representation features learned by our proposed method outperforms the state-of-the-art approaches.

Discriminative spatial-temporal feature learning for modeling network intrusion detection systems

Increasing interest and advancement of internet and communication technologies have made network security rise as a vibrant research domain. Network intrusion detection systems (NIDSs) have developed as indispensable defense mechanisms in cybersecurity that are employed in discovery and prevention of malicious network activities. In the recent years, researchers have proposed deep learning approaches in the development of NIDSs owing to their ability to extract better representations from large corpus of data. In the literature, convolutional neural network architecture is extensively used for spatial feature learning, while the long short term memory networks are employed to learn temporal features. In this paper, a novel hybrid method that learn the discriminative spatial and temporal features from the network flow is proposed for detecting network intrusions. A two dimensional convolution neural network is proposed to intelligently extract the spatial characteristics whereas a bi-directional long short term memory is used to extract temporal features of network traffic data samples consequently, forming a deep hybrid neural network architecture for identification and classification of network intrusion samples. Extensive experimental evaluations were performed on two well-known benchmarks datasets: CIC-IDS 2017 and the NSL-KDD datasets. The proposed network model demonstrated state-of-the-art performance with experimental results showing that the accuracy and precision scores of the intrusion detection model are significantly better than those of other existing models. These results depicts the applicability of the proposed model in the spatial-temporal feature learning in network intrusion detection systems.

Effect Predictor of Driver Synonymous Mutations Based on Multi-Feature Fusion and Iterative Feature Representation Learning.

Accurate identification of driver mutations is crucial in genetic studies of human cancers. While numerous cancer driver missense mutations have been identified, research into potential cancer drivers for synonymous mutations has shown limited success to date. Here, we developed a novel machine learning framework, epSMic, for predicting cancer driver synonymous mutations. epSMic employs an iterative feature representation scheme that facilitates the learning of discriminative features from various sequential models in a supervised iterative mode. We constructed the benchmark datasets and encoded the embedding sequence, physicochemical property, and basic information such as conservation and splicing feature. The evaluation results on benchmark test datasets demonstrate that epSMic outperforms existing methods, making it a valuable tool for researchers in identifying functional synonymous mutations in cancer. We hope epSMic can enable researchers to concentrate on synonymous mutations that have a functional impact on cancer.

Learning informative and discriminative semantic features for robust facial expression recognition

Facial expression recognition (FER) becomes challenging in real-world scenarios, which requires learning informative and discriminative features from challenging datasets to obtain robust facial expression recognition. In this paper, we propose an Informative and Discriminative Semantic Features Learning (IDSFL) network for FER against occlusion and head pose in the wild. Specifically, IDSFL aims to mine informative and discriminative semantic features from both low and high levels learned features to learn robust representations. First, a multi-channel feature (MCF) modulator incorporating low-level Gabor features is introduced to learn informative semantic features by capturing adequate diverse and detailed information. Additionally, a specific emotion-aware (SEA) module is proposed to learn discriminative semantic features by aggregating high-level emotion-specific features to focus on each expression category. Thus, IDSFL can collaboratively learn informative and discriminative representations. Extensive experiments on challenging in-the-wild datasets, including RAF-DB, FERPlus and AffectNet-7, demonstrate that our proposed method outperforms most state-of-the-art FER methods.

Frontal-view gait recognition using discriminative dynamics feature representations and learning

Frontal-view gait recognition using discriminative dynamics feature representations and learning

Label correlation guided discriminative label feature learning for multi-label chest image classification

Background and ObjectiveMulti-label Chest X-ray (CXR) images often contain rich label relationship information, which is beneficial to improve classification performance. However, because of the intricate relationships among labels, most existing works fail to effectively learn and make full use of the label correlations, resulting in limited classification performance. In this study, we propose a multi-label learning framework that learns and leverages the label correlations to improve multi-label CXR image classification. MethodsIn this paper, we capture the global label correlations through the self-attention mechanism. Meanwhile, to better utilize label correlations for guiding feature learning, we decompose the image-level features into label-level features. Furthermore, we enhance label-level feature learning in an end-to-end manner by a consistency constraint between global and local label correlations, and a label correlation guided multi-label supervised contrastive loss. ResultsTo demonstrate the superior performance of our proposed approach, we conduct three times 5-fold cross-validation experiments on the CheXpert dataset. Our approach obtains an average F1 score of 44.6% and an AUC of 76.5%, achieving a 7.7% and 1.3% improvement compared to the state-of-the-art results. ConclusionMore accurate label correlations and full utilization of the learned label correlations help learn more discriminative label-level features. Experimental results demonstrate that our approach achieves exceptionally competitive performance compared to the state-of-the-art algorithms.

Electromyographic hand gesture recognition using convolutional neural network with multi-attention

Electromyographic hand gesture recognition can be used to develop myoelectric prosthetic hands for disabled people when the acquired myoelectric signals are accurately classified. Recently, the developed end-to-end deep learning methods are widely used in hand gesture recognition for their automatic extraction of features from raw signals. But not all extracted features are useful and unimportant information can impede gesture recognition. In this study, a deep learning model using multi-attention for electromyographic hand gesture recognition is proposed. The attention mechanism inspired by the biological system enables deep learning models to focus on the important parts of input data while ignoring unimportant information. At first, a convolutional neural network is employed as the feature extractor, enabling the learning of high-level discriminative features from input signals. Then, multi-attention blocks, including channel attention, spatial attention, and temporal attention, are integrated into the convolutional layers, enhancing the extraction of pertinent features by emphasizing critical information through weight recalibration. Finally, the proposed model is evaluated on both the public NinaPro dataset and the public Myo dataset, demonstrating superior performance compared to baseline models in terms of recognition accuracy. The experimental results reveal that the model with attention mechanism achieves an average recognition accuracy of 91.64%, representing a 1.1% improvement over the model without attention mechanism.