Discrimination Learning Research Articles

The Keys to the Future? An Examination of Statistical Versus Discriminative Accounts of Serial Pattern Learning.

Sequence learning is fundamental to a wide range of cognitive functions. Explaining how sequences-and the relations between the elements they comprise-are learned is a fundamental challenge to cognitive science. However, although hundreds of articles addressing this question are published each year, the actual learning mechanisms involved in the learning of sequences are rarely investigated. We present three experiments that seek to examine these mechanisms during a typing task. Experiments 1 and 2 tested learning during typing single letters on each trial. Experiment 3 tested for "chunking" of these letters into "words." The results of these experiments were used to examine the mechanisms that could best account for them, with a focus on two particular proposals: statistical transitional probability learning and discriminative error-driven learning. Experiments 1 and 2 showed that error-driven learning was a better predictor of response latencies than either n-gram frequencies or transitional probabilities. No evidence for chunking was found in Experiment 3, probably due to interspersing visual cues with the motor response. In addition, learning occurred across a greater distance in Experiment 1 than Experiment 2, suggesting that the greater predictability that comes with increased structure leads to greater learnability. These results shed new light on the mechanism responsible for sequence learning. Despite the widely held assumption that transitional probability learning is essential to this process, the present results suggest instead that the sequences are learned through a process of discriminative learning, involving prediction and feedback from predictionerror.

Fuzzified Deep Learning based Forgery Detection of Signatures in the Healthcare Mission Records

In an era subjected to digital solutions, handwritten signatures continue playing a crucial role in identity verification and document authentication. These signatures, a form of bio-metric verification, are unique to every individual, serving as a primitive method for confirming identity and ensuring security of an individual. Signatures, apart from being a means of personal authentication, are often considered a cornerstone in the validation of critical documents and processes, especially within the healthcare sector. In healthcare missions, particularly in the regions that are underdeveloped, hand-written records persist as the primary mode of documentation. The credibility of these handwritten documents hinges on the authenticity of the accompanying signatures, making signature verification a paramount safeguard for the integrity and security of medical information. Nonetheless, traditional offline methods of signature identification can be time-consuming and inefficient, particularly while dealing with a massive volume of documents. This arises the evident need for automated signature verification systems. Our research introduces an innovative signature verification system which synthesizes the strengths of fuzzy logic and CNN (Convolutional Neural Networks) to deliver precise and efficient signature verification. Leveraging the capabilities of Fuzzy Logic for feature representation and CNNs for discriminative learning, our proposed hybrid model offers a compelling solution. Through rigorous training, spanning a mere 28 epochs, our hybrid model exhibits remarkable performance by attaining a training accuracy of 91.29% and a test accuracy of 88.47%, underscoring its robust generalization capacity. In an era of evolving security requirements and the persistent relevance of handwritten signatures, our research links the disparity between tradition and modernity.

Learning informative and discriminative semantic features for robust facial expression recognition

Facial expression recognition (FER) becomes challenging in real-world scenarios, which requires learning informative and discriminative features from challenging datasets to obtain robust facial expression recognition. In this paper, we propose an Informative and Discriminative Semantic Features Learning (IDSFL) network for FER against occlusion and head pose in the wild. Specifically, IDSFL aims to mine informative and discriminative semantic features from both low and high levels learned features to learn robust representations. First, a multi-channel feature (MCF) modulator incorporating low-level Gabor features is introduced to learn informative semantic features by capturing adequate diverse and detailed information. Additionally, a specific emotion-aware (SEA) module is proposed to learn discriminative semantic features by aggregating high-level emotion-specific features to focus on each expression category. Thus, IDSFL can collaboratively learn informative and discriminative representations. Extensive experiments on challenging in-the-wild datasets, including RAF-DB, FERPlus and AffectNet-7, demonstrate that our proposed method outperforms most state-of-the-art FER methods.

Frontal-view gait recognition using discriminative dynamics feature representations and learning

Frontal-view gait recognition using discriminative dynamics feature representations and learning

A multi-modal feature fusion classification model based on distance matching and discriminative representation learning for differentiation of high-grade glioma from solitary brain metastasis

To explore the performance of a new multimodal feature fusion classification model based on distance matching and discriminative representation learning for differentiating high-grade glioma (HGG) from solitary brain metastasis (SBM). We collected multi-parametric magnetic resonance imaging (MRI) data from 61 patients with HGG and 60 with SBM, and delineated regions of interest (ROI) on T1WI, T2WI, T2-weighted fluid attenuated inversion recovery (T2_FLAIR) and post-contrast enhancement T1WI (CE_T1WI) images. The radiomics features were extracted from each sequence using Pyradiomics and fused using a multimodal feature fusion classification model based on distance matching and discriminative representation learning to obtain a classification model. The discriminative performance of the classification model for differentiating HGG from SBM was evaluated using five-fold cross-validation with metrics of specificity, sensitivity, accuracy, and the area under the ROC curve (AUC) and quantitatively compared with other feature fusion models. Visual experiments were conducted to examine the fused features obtained by the proposed model to validate its feasibility and effectiveness. The five-fold cross-validation results showed that the proposed multimodal feature fusion classification model had a specificity of 0.871, a sensitivity of 0.817, an accuracy of 0.843, and an AUC of 0.930 for distinguishing HGG from SBM. This feature fusion method exhibited excellent discriminative performance in the visual experiments. The proposed multimodal feature fusion classification model has an excellent ability for differentiating HGG from SBM with significant advantages over other feature fusion classification models in discrimination and classification tasks between HGG and SBM.

Multimodal classification of Alzheimer's disease and mild cognitive impairment using custom MKSCDDL kernel over CNN with transparent decision-making for explainable diagnosis

The study presents an innovative diagnostic framework that synergises Convolutional Neural Networks (CNNs) with a Multi-feature Kernel Supervised within-class-similar Discriminative Dictionary Learning (MKSCDDL). This integrative methodology is designed to facilitate the precise classification of individuals into categories of Alzheimer's Disease, Mild Cognitive Impairment (MCI), and Cognitively Normal (CN) statuses while also discerning the nuanced phases within the MCI spectrum. Our approach is distinguished by its robustness and interpretability, offering clinicians an exceptionally transparent tool for diagnosis and therapeutic strategy formulation. We use scandent decision trees to deal with the unpredictability and complexity of neuroimaging data. Considering that different people's brain scans are different, this enables the model to make more detailed individualised assessments and explains how the algorithm illuminates the specific neuroanatomical regions that are indicative of cognitive impairment. This explanation is beneficial for clinicians because it gives them concrete ideas for early intervention and targeted care. The empirical review of our model shows that it makes diagnoses with a level of accuracy that is unmatched, with a classification efficacy of 98.27%. This shows that the model is good at finding important parts of the brain that may be damaged by cognitive diseases.

Reconsidering repurposing: long-term metformin treatment impairs cognition in Alzheimer’s model mice

Metformin, a primary anti-diabetic medication, has been anticipated to provide benefits for Alzheimer’s disease (AD), also known as “type 3 diabetes”. Nevertheless, some studies have demonstrated that metformin may trigger AD pathology and even elevate AD risk in humans. Despite this, limited research has elucidated the behavioral outcomes of metformin treatment, which would hold significant translational value. Thus, we aimed to perform thorough behavioral research on the prolonged administration of metformin to mice: We administered metformin (300 mg/kg/day) to transgenic 3xTg-AD and non-transgenic (NT) C57BL/6 mice over 1 and 2 years, respectively, and evaluated their behaviors across multiple domains via touchscreen operant chambers, including motivation, attention, memory, visual discrimination, and cognitive flexibility. We found metformin enhanced attention, inhibitory control, and associative learning in younger NT mice (≤16 months). However, chronic treatment led to impairments in memory retention and discrimination learning at older age. Furthermore, metformin caused learning and memory impairment and increased levels of AMPKα1-subunit, β-amyloid oligomers, plaques, phosphorylated tau, and GSK3β expression in AD mice. No changes in potential confounding factors on cognition, including levels of motivation, locomotion, appetite, body weight, blood glucose, and serum vitamin B12, were observed in metformin-treated AD mice. We also identified an enhanced amyloidogenic pathway in db/db mice, as well as in Neuro2a-APP695 cells and a decrease in synaptic markers, such as PSD-95 and synaptophysin in primary neurons, upon metformin treatment. Our findings collectively suggest that the repurposing of metformin should be carefully reconsidered when this drug is used for individuals with AD.

Label correlation guided discriminative label feature learning for multi-label chest image classification

Background and ObjectiveMulti-label Chest X-ray (CXR) images often contain rich label relationship information, which is beneficial to improve classification performance. However, because of the intricate relationships among labels, most existing works fail to effectively learn and make full use of the label correlations, resulting in limited classification performance. In this study, we propose a multi-label learning framework that learns and leverages the label correlations to improve multi-label CXR image classification. MethodsIn this paper, we capture the global label correlations through the self-attention mechanism. Meanwhile, to better utilize label correlations for guiding feature learning, we decompose the image-level features into label-level features. Furthermore, we enhance label-level feature learning in an end-to-end manner by a consistency constraint between global and local label correlations, and a label correlation guided multi-label supervised contrastive loss. ResultsTo demonstrate the superior performance of our proposed approach, we conduct three times 5-fold cross-validation experiments on the CheXpert dataset. Our approach obtains an average F1 score of 44.6% and an AUC of 76.5%, achieving a 7.7% and 1.3% improvement compared to the state-of-the-art results. ConclusionMore accurate label correlations and full utilization of the learned label correlations help learn more discriminative label-level features. Experimental results demonstrate that our approach achieves exceptionally competitive performance compared to the state-of-the-art algorithms.

Highly Discriminative Driver Distraction Detection Method Based on Swin Transformer

Driver distraction detection not only helps to improve road safety and prevent traffic accidents, but also promotes the development of intelligent transportation systems, which is of great significance for creating a safer and more efficient transportation environment. Since deep learning algorithms have very strong feature learning abilities, more and more deep learning-based driver distraction detection methods have emerged in recent years. However, the majority of existing deep learning-based methods are optimized only through the constraint of classification loss, making it difficult to obtain features with high discrimination, so the performance of these methods is very limited. In this paper, to improve the discrimination between features of different classes of samples, we propose a high-discrimination feature learning strategy and design a driver distraction detection model based on Swin Transformer and the highly discriminative feature learning strategy (ST-HDFL). Firstly, the features of input samples are extracted through the powerful feature learning ability of Swin Transformer. Then, the intra-class distance of samples of the same class in the feature space is reduced through the constraint of sample center distance loss (SC loss), and the inter-class distance of samples of different classes is increased through the center vector shift strategy, which can greatly improve the discrimination of different class samples in the feature space. Finally, we have conducted extensive experiments on two publicly available datasets, AUC-DD and State-Farm, to demonstrate the effectiveness of the proposed method. The experimental results show that our method can achieve better performance than many state-of-the-art methods, such as Drive-Net, MobileVGG, Vanilla CNN, and so on.

Frequency effects in linear discriminative learning.

Word frequency is a strong predictor in most lexical processing tasks. Thus, any model of word recognition needs to account for how word frequency effects arise. The Discriminative Lexicon Model (DLM) models lexical processing with mappings between words' forms and their meanings. Comprehension and production are modeled via linear mappings between the two domains. So far, the mappings within the model can either be obtained incrementally via error-driven learning, a computationally expensive process able to capture frequency effects, or in an efficient, but frequency-agnostic solution modeling the theoretical endstate of learning (EL) where all words are learned optimally. In the present study we show how an efficient, yet frequency-informed mapping between form and meaning can be obtained (Frequency-informed learning; FIL). We find that FIL well approximates an incremental solution while being computationally much cheaper. FIL shows a relatively low type- and high token-accuracy, demonstrating that the model is able to process most word tokens encountered by speakers in daily life correctly. We use FIL to model reaction times in the Dutch Lexicon Project by means of a Gaussian Location Scale Model and find that FIL predicts well the S-shaped relationship between frequency and the mean of reaction times but underestimates the variance of reaction times for low frequency words. FIL is also better able to account for priming effects in an auditory lexical decision task in Mandarin Chinese, compared to EL. Finally, we used ordered data from CHILDES to compare mappings obtained with FIL and incremental learning. We show that the mappings are highly correlated, but that with FIL some nuances based on word ordering effects are lost. Our results show how frequency effects in a learning model can be simulated efficiently, and raise questions about how to best account for low-frequency words in cognitive models.

Interactive and discriminative analysis dictionary learning for image classification

Interactive and discriminative analysis dictionary learning for image classification

Wasserstein Discriminant Dictionary Learning for Graph Representation.

Mining discriminative graph topological information plays an important role in promoting graph representation ability. However, it suffers from two main issues: (1) the difficulty/complexity of computing global inter-class/intra-class scatters, commonly related to mean and covariance of graph samples, for discriminant learning; (2) the huge complexity and variety of graph topological structure that is rather challenging to robustly characterize. In this paper, we propose the Wasserstein Discriminant Dictionary Learning (WDDL) framework to achieve discriminant learning on graphs with robust graph topology modeling, and hence facilitate graph-based pattern analysis tasks. Considering the difficulty of calculating global inter-class/intra-class scatters, a reference set of graphs (aka graph dictionary) is first constructed by generating representative graph samples (aka graph keys) with expressive topological structure. Then, a Wasserstein Graph Representation (WGR) process is proposed to project input graphs into a succinct dictionary space through the graph dictionary lookup. To further achieve discriminant graph learning, a Wasserstein discriminant loss (WD-loss) is defined on the graph dictionary, in which the graph keys are optimizable, to make the intra-class keys more compact and inter-class keys more dispersed. Hence, the calculation of global Wasserstein metric (W-metric) centers can be bypassed. For sophisticated topology mining in the WGR process, a joint-Wasserstein graph embedding module is constructed to model both between-node and between-edge relationships across inputs and graph keys by encapsulating both the Wasserstein metric (between cross-graph nodes) and proposed novel Kron-Gromov-Wasserstein (KGW) metric (between cross-graph adjacencies). Specifically, the KGW-metric comprehensively characterizes the cross-graph connection patterns with the Kronecker operation, then adaptively captures those salient patterns through connection pooling. To evaluate the proposed framework, we study two graph-based pattern analysis problems, i.e. graph classification and cross-modal retrieval, with the graph dictionary flexibly adjusted to cater to these two tasks. Extensive experiments are conducted to comprehensively compare with existing advanced methods, as well as dissect the critical component of our proposed architecture. The experimental results validate the effectiveness of the WDDL framework.

Toward Robust Discriminative Projections Learning Against Adversarial Patch Attacks.

As one of the most popular supervised dimensionality reduction methods, linear discriminant analysis (LDA) has been widely studied in machine learning community and applied to many scientific applications. Traditional LDA minimizes the ratio of squared l2 norms, which is vulnerable to the adversarial examples. In recent studies, many l1 -norm-based robust dimensionality reduction methods are proposed to improve the robustness of model. However, due to the difficulty of l1 -norm ratio optimization and weakness on defending a large number of adversarial examples, so far, scarce works have been proposed to utilize sparsity-inducing norms for LDA objective. In this article, we propose a novel robust discriminative projections learning (rDPL) method based on the l1,2 -norm trace-ratio minimization optimization algorithm. Minimizing the l1,2 -norm ratio problem directly is a much more challenging problem than the traditional methods, and there is no existing optimization algorithm to solve such nonsmooth terms ratio problem. We derive a new efficient algorithm to solve this challenging problem and provide a theoretical analysis on the convergence of our algorithm. The proposed algorithm is easy to implement and converges fast in practice. Extensive experiments on both synthetic data and several real benchmark datasets show the effectiveness of the proposed method on defending the adversarial patch attack by comparison with many state-of-the-art robust dimensionality reduction methods.

Pseudo-Label Guided Structural Discriminative Subspace Learning for Unsupervised Feature Selection.

In this article, we propose a new unsupervised feature selection method named pseudo-label guided structural discriminative subspace learning (PSDSL). Unlike the previous methods that perform the two stages independently, it introduces the construction of probability graph into the feature selection learning process as a unified general framework, and therefore the probability graph can be learned adaptively. Moreover, we design a pseudo-label guided learning mechanism, and combine the graph-based method and the idea of maximizing the between-class scatter matrix with the trace ratio to construct an objective function that can improve the discrimination of the selected features. Besides, the main existing strategies of selecting features are to employ l2,1 -norm for feature selection, but this faces the challenges of sparsity limitations and parameter tuning. For addressing this issue, we employ the l2,0 -norm constraint on the learned subspace to ensure the row sparsity of the model and make the selected feature more stable. Effective optimization strategy is given to solve such NP-hard problem with the determination of parameters and complexity analysis in theory. Ultimately, extensive experiments conducted on nine real-world datasets and three biological ScRNA-seq genes datasets verify the effectiveness of the proposed method on the data clustering downstream task.

Worst-Case Discriminative Feature Learning via Max-Min Ratio Analysis.

We propose a novel discriminative feature learning method via Max-Min Ratio Analysis (MMRA) for exclusively dealing with the long-standing "worst-case class separation" problem. Existing technologies simply consider maximizing the minimal pairwise distance on all class pairs in the low-dimensional subspace, which is unable to separate overlapped classes entirely especially when the distribution of samples within same class is diverging. We propose a new criterion, i.e., Max-Min Ratio Analysis (MMRA) that focuses on maximizing the minimal ratio value of between-class and within-class scatter to extremely enlarge the separability on the overlapped pairwise classes. Furthermore, we develop two novel discriminative feature learning models for dimensionality reduction and metric learning based on our MMRA criterion. However, solving such a non-smooth non-convex max-min ratio problem is challenging. As an important theoretical contribution in this paper, we systematically derive an alternative iterative algorithm based on a general max-min ratio optimization framework to solve a general max-min ratio problem with rigorous proofs of convergence. More importantly, we also present another solver based on bisection search strategy to solve the SDP problem efficiently. To evaluate the effectiveness of proposed methods, we conduct extensive pattern classification and image retrieval experiments on several artificial datasets and real-world ScRNA-seq datasets, and experimental results demonstrate the effectiveness of proposed methods.

Unsupervised Domain Adaptation via Weighted Sequential Discriminative Feature Learning for Sentiment Analysis

Unsupervised domain adaptation (UDA) presents a significant challenge in sentiment analysis, especially when faced with differences between source and target domains. This study introduces Weighted Sequential Unsupervised Domain Adaptation (WS-UDA), a novel sequential framework aimed at discovering more profound features and improving target representations, even in resource-limited scenarios. WS-UDA utilizes a domain-adversarial learning model for sequential discriminative feature learning. While recent UDA techniques excel in scenarios where source and target domains are closely related, they struggle with substantial dissimilarities. This potentially leads to instability during shared-feature learning. To tackle this issue, WS-UDA employs a two-stage transfer process concurrently, significantly enhancing model stability and adaptability. The sequential approach of WS-UDA facilitates superior adaptability to varying levels of dissimilarity between source and target domains. Experimental results on benchmark datasets, including Amazon reviews, FDU-MTL datasets, and Spam datasets, demonstrate the promising performance of WS-UDA. It outperforms state-of-the-art cross-domain unsupervised baselines, showcasing its efficacy in scenarios with dissimilar domains. WS-UDA’s adaptability extends beyond sentiment analysis, making it a versatile solution for diverse text classification tasks.

Application of PCA and Unsupervised Deep Learning in Bird and Drone Discrimination Based on FMCW Radar Measurements

Application of PCA and Unsupervised Deep Learning in Bird and Drone Discrimination Based on FMCW Radar Measurements

Unsupervised Feature Selection via Controllable Adaptive Graph Learning and Discriminative Feature Learning.

Unsupervised feature selection is challenging in machine learning, pattern recognition, and data mining. The crucial difficulty is to learn a moderate subspace that preserves the intrinsic structure and to find uncorrelated or independent features simultaneously. The most common solution is first to project the original data into a lower dimensional space and then force them to preserve the similar intrinsic structure under linear uncorrelation constraint. However, there are three shortcomings. First, the final graph generated by the iterative learning process differs significantly from the initial graph in which the original intrinsic structure is embedded. Second, it requires prior knowledge about a moderate dimension of subspace. Third, it is inefficient when dealing with high-dimensional datasets. The first shortcoming, which is longstanding and undiscovered, makes the previous methods fail to achieve their expected results. The last two ones increase the difficulty of applying in different fields. Therefore, two unsupervised feature selection methods are proposed based on controllable adaptive graph learning and uncorrelated/independent feature learning (CAG-U and CAG-I) to address the abovementioned issues. In the proposed methods, the final graph that preserves intrinsic structure can be adaptively learned while the difference between the two graphs can be well controlled. Besides, relatively uncorrelated/independent features can be selected using a discrete projection matrix. The experimental results on 12 datasets in different fields show the superiority of CAG-U and CAG-I.

DeepMulticut: Deep Learning of Multicut Problem for Neuron Segmentation from Electron Microscopy Volume.

Superpixel aggregation is a powerful tool for automated neuron segmentation from electron microscopy (EM) volume. However, existing graph partitioning methods for superpixel aggregation still involve two separate stages-model estimation and model solving, and therefore model error is inherent. To address this issue, we integrate the two stages and propose an end-to-end aggregation framework based on deep learning of the minimum cost multicut problem called DeepMulticut. The core challenge lies in differentiating the NPhard multicut problem, whose constraint number is exponential in the problem size. With this in mind, we resort to relaxing the combinatorial solver-the greedy additive edge contraction (GAEC)-to a continuous Soft-GAEC algorithm, whose limit is shown to be the vanilla GAEC. Such relaxation thus allows the DeepMulticut to integrate edge cost estimators, Edge-CNNs, into a differentiable multicut optimization system and allows a decision-oriented loss to feed decision quality back to the Edge-CNNs for adaptive discriminative feature learning. Hence, the model estimators, Edge-CNNs, can be trained to improve partitioning decisions directly while beyond the NP-hardness. Also, we explain the rationale behind the DeepMulticut framework from the perspective of bi-level optimization. Extensive experiments on three public EM datasets demonstrate the effectiveness of the proposed DeepMulticut.

Deep Multiview Clustering by Pseudo-Label Guided Contrastive Learning and Dual Correlation Learning.

Deep multiview clustering (MVC) is to learn and utilize the rich relations across different views to enhance the clustering performance under a human-designed deep network. However, most existing deep MVCs meet two challenges. First, most current deep contrastive MVCs usually select the same instance across views as positive pairs and the remaining instances as negative pairs, which always leads to inaccurate contrastive learning (CL). Second, most deep MVCs only consider learning feature or cluster correlations across views, failing to explore the dual correlations. To tackle the above challenges, in this article, we propose a novel deep MVC framework by pseudo-label guided CL and dual correlation learning. Specifically, a novel pseudo-label guided CL mechanism is designed by using the pseudo-labels in each iteration to help removing false negative sample pairs, so that the CL for the feature distribution alignment can be more accurate, thus benefiting the discriminative feature learning. Different from most deep MVCs learning only one kind of correlation, we investigate both the feature and cluster correlations among views to discover the rich and comprehensive relations. Experiments on various datasets demonstrate the superiority of our method over many state-of-the-art compared deep MVCs. The source implementation code will be provided at https://github.com/ShizheHu/Deep-MVC-PGCL-DCL.