Discriminative Learning Model Research Articles

Background covariance discriminative dictionary learning for hyperspectral target detection

Hyperspectral target detection (HTD) aims to identifying targets within a hyperspectral image (HSI) based on provided target spectra. In the current HTD field, representation-based detectors have attracted much attention. However, there are two prominent challenges that are particularly noteworthy. First, the background class encompasses diverse land covers, making its accurate representation challenging. Second, the detection ability can be significantly influenced by the abnormalities and noise in HSI. To tackle these concerns, we propose a novel background covariance discriminative dictionary learning (BCDDL) model for HTD. To enhance the background representation ability and overcome the sparse noise, we combine the dictionary learning with spectral covariance descriptors and undertake background reconstruction in regional scale. Specifically, the input HSI is pre-processed into superpixels, the spectral covariance of each superpixel is used to provide a compact and flexible description of local regional statistical properties. Further, a novel spatial clustering-based dictionary learning method is proposed to learn the background discriminative covariances dictionary. The collaborative representation model within symmetric positive definite (SPD) manifold is utilized to reconstruct background region and get the residual. By merging the background residual with pixel-wise target reconstruction residual, we derive final detection output. Comprehensive experiments on two public hyperspectral datasets and two novel GaoFen-5 datasets demonstrate the superiority of our BCDDL approach over 10 state-of-the-art methods, especially in terms of suppressing background.

Augmented Sparse Representation for Incomplete Multiview Clustering.

Incomplete multiview data are collected from multiple sources or characterized by multiple modalities, where the features of some samples or some views may be missing. Incomplete multiview clustering (IMVC) aims to partition the data into different groups by taking full advantage of the complementary information from multiple incomplete views. Most existing methods based on matrix factorization or subspace learning attempt to recover the missing views or perform imputation of the missing features to improve clustering performance. However, this problem is intractable due to a lack of prior knowledge, e.g., label information or data distribution, especially when the missing views or features are completely damaged. In this article, we proposed an augmented sparse representation (ASR) method for IMVC. We first introduce a discriminative sparse representation learning (DSRL) model, which learns the sparse representations of multiple views as applied to measure the similarity of the existing features. The DSRL model explores complementary and consistent information by integrating the sparse regularization item and a consensus regularization item, respectively. Simultaneously, it learns a discriminative dictionary from the original samples. The sparsity constrained optimization problem in the DSRL model can be efficiently solved by the alternating direction method of multipliers (ADMM). Then, we present a similarity fusion scheme, namely, a sparsity augmented fusion of sparse representations, to obtain a sparsity augmented similarity matrix across different views for spectral clustering. Experimental results on several datasets demonstrate the effectiveness of the proposed ASR method for IMVC.

Worst-Case Discriminative Feature Learning via Max-Min Ratio Analysis.

We propose a novel discriminative feature learning method via Max-Min Ratio Analysis (MMRA) for exclusively dealing with the long-standing "worst-case class separation" problem. Existing technologies simply consider maximizing the minimal pairwise distance on all class pairs in the low-dimensional subspace, which is unable to separate overlapped classes entirely especially when the distribution of samples within same class is diverging. We propose a new criterion, i.e., Max-Min Ratio Analysis (MMRA) that focuses on maximizing the minimal ratio value of between-class and within-class scatter to extremely enlarge the separability on the overlapped pairwise classes. Furthermore, we develop two novel discriminative feature learning models for dimensionality reduction and metric learning based on our MMRA criterion. However, solving such a non-smooth non-convex max-min ratio problem is challenging. As an important theoretical contribution in this paper, we systematically derive an alternative iterative algorithm based on a general max-min ratio optimization framework to solve a general max-min ratio problem with rigorous proofs of convergence. More importantly, we also present another solver based on bisection search strategy to solve the SDP problem efficiently. To evaluate the effectiveness of proposed methods, we conduct extensive pattern classification and image retrieval experiments on several artificial datasets and real-world ScRNA-seq datasets, and experimental results demonstrate the effectiveness of proposed methods.

Parametric simulation of electron backscatter diffraction patterns through generative models

Recently, discriminative machine learning models have been widely used to predict various attributes from Electron Backscatter Diffraction (EBSD) patterns. However, there has never been any generative model developed for EBSD pattern simulation. On one hand, the training of generative models is much harder than that of discriminative ones; On the other hand, numerous variables affecting EBSD pattern formation make the input space high-dimensional and its relationship with the distribution of backscattered electrons complicated. In this study, we propose a framework (EBSD-CVAE/GAN) with great flexibility and scalability to realize parametric simulation of EBSD patterns. Compared with the frequently used forward model, EBSD-CVAE/GAN can take variables more than just orientation and generate corresponding EBSD patterns in a single run. The accuracy and quality of generated patterns are systematically evaluated. The model does not only summarize a distribution of backscattered electrons at a higher level, but also mitigates data scarcity in this field.

Flexible loss functions for binary classification in gradient-boosted decision trees: An application to credit scoring

This paper introduces new flexible loss functions for binary classification in Gradient-Boosted Decision Trees (GBDT) that combine Dice-based and cross-entropy-based losses and offer link functions from either a generalized extreme value (GEV) or exponentiated exponential logistic (EEL) distribution. Testing 27 different GBDT models using XGBoost on a Freddie Mac mortgage loan database showed that the choice of the loss function is useful. Specifically, when the class imbalance ratio (IR) is less than 99, using a skewed GEV distribution-based link function in XGBoost enhances discriminatory power and classification accuracy while retaining a simple model structure, which is particularly important in credit scoring applications. In cases where class imbalances are severe, typically between IRs of 99 and 200, we found that an advanced loss function, which is composed of a symmetric hybrid loss function and a link derived from a positively skewed EEL distribution, outperforms other XGBoost variants. Based on our findings, the accuracy improvements of these proposed extensions result in lower misclassification costs, which are especially evident when IR is below 99, which results in higher profitability for the business. Furthermore, the study highlights the transparency associated with GBDT, which is also an integral component of financial applications. Researchers and practitioners can use these insights to create more accurate and discriminative machine learning models, with possible extensions to other GBDT implementations and machine learning techniques that take into account loss functions. The source code for the proposed approach is publicly available at https://github.com/jm-ml/flexible-losses-for-binary-classification-with-GBDT.

Adaptive Optimized Discriminative Learning based Image Deblurring using Deep CNN

Image degradation plays a major problem in many image processing applications. Due to blurring, the quality of an image is degraded and there will be a reduction in bandwidth. Blur in an image is due to variations in atmospheric turbulence, focal length, camera settings, etc. Various types of blurs include Gaussian blur, Motion blur, Out-of-focus blur. The effect of noise along with blur further corrupts the captured image. Many techniques have evolved to deblur the degraded image. The leading approach to solve various degraded images are either based on discriminative learning models or on optimization models. Each method has its own advantages and disadvantages. Learning by discriminative methods is faster but restricted to a specific task whereas optimization models handle flexibly but consume more time. Integrating optimization models suitably by learning with discriminative manner results in effective image restoration. In this paper, a set of effective and fast Convolutional Neural Networks (CNNs) are employed to deblur the Gaussian, motion and out-of-focus blurred images that integrate with optimization models to further avoid noise effects. The proposed methods work more efficiently for applications with low-level vision.

Graph adaptive semi-supervised discriminative subspace learning for EEG emotion recognition

Since Electroencephalogram (EEG) is resistant to camouflage and contains abundant neurophysiological information, it shows significant superiorities in objective emotion recognition, making EEG-based emotion recognition become a hot research field in brain-computer interface research. However, EEG is generally non-stationary and has a low signal-to-noise ratio, which is difficult to analyze. Inspired by the consensus that exploring a discriminative subspace representation usually helps to capture the semantic information of EEG data, in this paper we propose a Graph Adaptive Semi-supervised Discriminative Subspace Learning (GASDSL) model for EEG-based emotion recognition. GASDSL aims to explore a discriminative subspace in which the intra-class scatter decreases while the inter-class separability increases. The adaptive maximum entropy graph construction and semi-supervised subspace emotional state prediction are adopted to mediate the discriminative subspace learning. Extensive comparative studies on the SEED-IV and SEED-V datasets depict that 1) GASDSL achieved satisfactory emotion recognition accuracy compared with other semi-supervised learning models, 2) the discriminative abilities of both the learned maximum entropy graph and subspace are improved as the model iterates, and 3) the features extracted from the Gamma band, the left/right temporal, prefrontal, and (central) parietal lobes contributed more to emotion recognition based on the spatial-frequency pattern analysis results.

Bayesian and Discriminative Models for Active Visual Perception across Saccades.

The brain interprets sensory inputs to guide behavior, but behavior itself disrupts sensory inputs. Perceiving a coherent world while acting in it constitutes active perception. For example, saccadic eye movements displace visual images on the retina and yet the brain perceives visual stability. Because this percept of visual stability has been shown to be influenced by prior expectations, we tested the hypothesis that it is Bayesian. The key prediction was that priors would be used more as sensory uncertainty increases. Humans and rhesus macaques reported whether an image moved during saccades. We manipulated both prior expectations and levels of sensory uncertainty. All psychophysical data were compared with the predictions of Bayesian ideal observer models. We found that humans were Bayesian for continuous judgments. For categorical judgments, however, they were anti-Bayesian: they used their priors less with greater uncertainty. We studied this categorical result further in macaques. The animals' judgments were similarly anti-Bayesian for sensory uncertainty caused by external, image noise, but Bayesian for uncertainty due to internal, motor-driven noise. A discriminative learning model explained the anti-Bayesian effects. We conclude that active vision uses both Bayesian and discriminative models depending on task requirements (continuous vs categorical) and the source of uncertainty (image noise vs motor-driven noise). In the context of previous knowledge about the saccadic system, our results provide an example of how the comparative analysis of Bayesian versus non-Bayesian models of perception offers novel insights into underlying neural organization.

Modelling L1 and the artificial language during artificial language learning

Artificial language learning research has become a popular tool to investigate universal mechanisms in language learning. However, often it is unclear whether the found effects are due to learning, or due to artefacts of the native language or the artificial language, and whether findings in only one language will generalise to speakers of other languages. The present study offers a new approach to model the influence of both the L1 and the target artificial language on language learning. The idea is to control for linguistic factors of the artificial and the native language by incorporating measures of wordlikeness into the statistical analysis as covariates. To demonstrate the approach, we extend Linzen and Gallagher (2017)’s study on consonant identity pattern to evaluate whether speakers of German and Mandarin rapidly learn the pattern when influences of L1 and the artificial language are accounted for by incorporating measures assessed by analogical and discriminative learning models over the L1 and artificial lexicon. Results show that nonwords are more likely to be accepted as grammatical if they are more similar to the trained artificial lexicon and more different from the L1 and, crucially, the identity effect is still present. The proposed approach is helpful for designing cross-linguistic studies.

Regularization oversampling for classification tasks: To exploit what you do not know

In numerous binary classification tasks, the two groups of instances are not equally represented, which often implies that the training data lack sufficient information to model the minority class correctly. Furthermore, many traditional classification models make arbitrarily overconfident predictions outside the range of the training data. These issues severely impact the deployment and usefulness of these models in real life. In this paper, we propose the boundary regularizing out-of-distribution (BROOD) sampler, which adds artificial data points on the edge of the training data. By exploiting these artificial samples, we are able to regularize the decision surface of discriminative machine learning models and make more prudent predictions. Next, it is crucial to correctly classify many positive instances in a limited pool of instances that can be investigated with the available resources. By smartly assigning predetermined nonuniform class probabilities outside the training data, we can emphasize certain data regions and improve classifier performance on various material classification metrics. The good performance of the proposed methodology is illustrated in a case study that consists of both benchmark balanced and imbalanced classification data sets.

Discriminative Local Representation Learning for Cross-Modality Visible-Thermal Person Re-Identification

Visible-thermal person re-identification (VTReID) is a rising and challenging cross-modality retrieval task in intelligent video surveillance systems. Most attention architectures cannot explore the discriminative person representations for VTReID, especially in the thermal modality. In addition, the fine-grained middle-level semantic information has received much less attention in the part-based approaches for the cross-modality pedestrian retrieval task, resulting in limited generalization capability and poor representation robustness. This paper proposes a simple yet powerful discriminative local representation learning (DLRL) model to capture the robust local fine-grained feature representations and explore the rich semantic relationship between the learned part features. Specifically, an efficient contextual attention aggregation module (CAAM) is designed to strengthen the discriminative capability of the feature representations and explore the contextual cues for visible and thermal modalities. Then, an integrated middle-high feature learning (IMHF) method is introduced to capture the part-level salient representations, which handles the ambiguous modality discrepancy in both discriminative middle-level and robust high-level information. Moreover, a part-guided graph convolution module (PGCM) is constructed to mine the structural relationship among the part representations within each modality. The quantitative and qualitative experiments on the two benchmark datasets demonstrate that the proposed DLRL model significantly outperforms state-of-the-art methods and achieves rank-1/mAP accuracy of 92.77%/82.05% on the RegDB dataset and 63.04%/60.58% on the SYSU-MM01 dataset.

Automatic proficiency scoring for early-stage writing

In this work, we study the feasibility of using machine learning and natural language processing methods for assessing writing proficiency in Danish with respect to text construction, sentence construction, and use of modifiers. Our work is based on the analytical framework for scoring early writing proposed by Kabel et al. (2022), where each text is first annotated by a human expert according to a predefined coding scheme and subsequently scored using statistical Rasch modeling (Rasch, 1960). We investigate two different strategies for estimating these scores automatically: 1) we propose a system for identifying the central linguistic features automatically mimicking the role of the human experts and 2) we train state-of-the-art discriminative machine learning models to predict the proficiency scores directly from the texts. We conduct a number of experiments to evaluate and compare the two approaches. Our results show strong and statistically significant correlations between the scores generated using the automatic system and scores based on human experts. We also estimate and report the reliability of the individual linguistic features in the automatic annotation system. Finally, we also propose and evaluate an extension of the statistical model, which allows the model to compensate for potential systematic errors in the automatic annotations. The article thereby contributes to the area of automated essay scoring (AES) and shows that it is possible to provide teachers with automated valid and reliable knowledge about the development of their students' writing competences, which they can use in their feedback to students.

Antenatal prediction models for outcomes of extremely and very preterm infants based on machine learning.

Predicting individual risks for adverse outcomes in preterm infants is necessary for perinatal management and antenatal counseling for their parents. To evaluate whether a machine learning approach can improve the prediction of severe infant outcomes beyond the performance of conventional logistic models, and to identify maternal and fetal factors that largely contribute to these outcomes. A population-based retrospective study was performed using clinical data of 31,157 infants born at < 32weeks of gestation and weighing ≤ 1500g, registered in the Neonatal Research Network of Japan between 2006 and 2015. We developed a conventional logistic model and 6 types of machine learning models based on 12 maternal and fetal factors. Discriminative ability was evaluated using the area under the receiver operating characteristic curves (AUROCs), and the importance of each factor in terms of its contribution to outcomes was evaluated using the SHAP (SHapley Additive exPlanations) value. The AUROCs of the most discriminative machine learning models were better than those of the conventional models for all outcomes. The AUROCs for in-hospital death and short-term adverse outcomes in the gradient boosting decision tree were significantly higher than those in the conventional model (p = 0.015 and p = 0.002, respectively). The SHAP value analyses showed that gestational age, birth weight, and antenatal corticosteroid treatment were the three most important factors associated with severe infant outcomes. Machine learning models improve the prediction of severe infant outcomes. Moreover, the machine learning approach provides insight into the potential risk factors for severe infant outcomes.

Multi-Attribute Discriminative Representation Learning for Prediction of Adverse Drug-Drug Interaction.

Adverse drug-drug interaction (ADDI) is a significant life-threatening issue, posing a leading cause of hospitalizations and deaths in healthcare systems. This paper proposes a unified Multi-Attribute Discriminative Representation Learning (MADRL) model for ADDI prediction. Unlike the existing works that equally treat features of each attribute without discrimination and do not consider the underlying relationship among drugs, we first develop a regularized optimization problem based on CUR matrix decomposition for joint representative drug and discriminative feature selection such that the selected drugs and features can well approximate the original feature spaces and the critical factors discriminative to ADDIs can be properly explored. Different from the existing models that ignore the consistent and unique properties among attributes, a Generative Adversarial Network (GAN) framework is then designed to capture the inter-attribute shared and intra-attribute specific representations of adverse drug pairs for exploiting their consensus and complementary information in ADDI prediction. Meanwhile, MADRL is compatible with any kind of attributes and capable of exploring their respective effects on ADDI prediction. An iterative algorithm based on the alternating direction method of multipliers is developed for optimization. Experiments on publicly available dataset demonstrate the effectiveness of MADRL when compared with eleven baselines and its six variants.

“All mimsy were the borogoves” – a discriminative learning model of morphological knowledge in pseudo-word inflection

ABSTRACT Grammatical knowledge has often been investigated in wug tests, in which participants inflect pseudo-words. It was argued that in inflecting these pseudo-words, speakers apply their knowledge of word formation. However, it remains unclear what exactly this knowledge is and how it is learned. According to one theory, the knowledge is best characterised as abstractions that specify how units are combined. Another theory maintains that it is best characterised by memory-based analogy. In both cases the knowledge is learned by association based on positive evidence alone. In this paper, we model the classification of pseudo-words to Maltese plurals using a shallow neural network trained with an error-driven learning algorithm. We demonstrate that the classifications mirror those of Maltese speakers in a wug test. Our results indicate that speakers rely on gradient knowledge of a relation between the phonetics of whole words and plural classes, which is learned in an error-driven way.

Domain adaptation metric learning method embedded with structural information for person re‐identification in internet of autonomous unmanned vehicles

AbstractInternet of autonomous unmanned vehicles (IAUV) is a global network of sensors, robots, and autonomous vehicles. Person re‐identification (Re‐ID) is an important intelligent transportation application in IAUV, which needs to be solved using artificial intelligence algorithms. In this study, a domain adaptation metric learning method embedded with structural information (called DAML‐ESI) is designed for person Re‐ID in IAUV. Due to the lack of labeling information in the target domain, DAML‐ESI realizes person Re‐ID with the help of the discriminative and structural information of pedestrian images of related domains. DAML‐ESI projects pedestrian images selected from different domains into a common metric space and establishes a discriminative metric learning model, which requires that the positive sample pair be mapped to a point, and the distance distribution of the negative sample pair be mapped to a fixed value. The projection matrix learned by DAML‐ESI is used to eliminate the distribution differences between different domains, and the distance metric is used to ensure that the learned metric learning model has strong discriminative ability in the metric space. To verify the effectiveness of DAML‐ESI, experimental comparisons are conducted on three person Re‐ID datasets, and DAML‐ESI achieves satisfactory recognition performance.

A Cross-Media Advertising Design and Communication Model Based on Feature Subspace Learning.

This paper uses feature subspace learning and cross-media retrieval analysis to construct an advertising design and communication model. To address the problems of the traditional feature subspace learning model and make the samples effectively maintain their local structure and discriminative properties after projection into the feature space, this paper proposes a discriminative feature subspace learning model based on Low-Rank Representation (LRR), which explores the local structure of samples through Low-Rank Representation and uses the representation coefficients as similarity constraints of samples in the projection space so that the projection subspace can better maintain the local nearest-neighbor relationship of samples. Based on the common subspace learning, this paper uses the extreme learning machine method to improve the cross-modal retrieval accuracy, mining deeper data features and maximizing the correlation between different modalities, so that the learned shared subspace is more discriminative; meanwhile, it proposes realizing cross-modal retrieval by the deep convolutional generative adversarial network, using unlabeled samples to further explore the correlation of different modal data and improve the cross-modal performance. The clustering quality of images and audios is corrected in the feature subspace obtained by dimensionality reduction through an optimization algorithm based on similarity transfer. Three active learning strategies are designed to calculate the conditional probability of unannotated samples around user-annotated samples in the correlation feedback process, thus improving the efficiency of cross-media retrieval in the case of limited feedback samples. The experimental results show that the method accurately measures the cross-media relevance and effectively achieves mutual retrieval between image and audio data. Through the study of cross-media advertising design and communication models based on feature subspace learning, it is of positive significance to advance commercial advertising design by guiding designers and artists to better utilize digital media technology for artistic design activities at the level of theoretical research and applied practice.

Estimating the impact of health systems factors on antimicrobial resistance in priority pathogens

Antimicrobial resistance (AMR) is the next big pandemic that threatens humanity. The One Health approach to AMR requires quantification of interactions between health, demographic, socioeconomic, environmental, and geopolitical factors to design interventions. This study is focused on learning health system factors on global AMR. This study analysed longitudinal data (2004-2017) of AMR having 6 33 820 isolates from 70 middle and high-income countries. We integrated AMR data with the Global Burden of Disease (GBD), Governance (WGI), and Finance data sets to find AMR's unbiased and actionable determinants. We chose a Bayesian decision network (BDN) approach within the causal modelling framework to quantify determinants of AMR. Further, we integrated Bayesian networks' global knowledge discovery approach with discriminative machine learning to predict individual-level antibiotic susceptibility in patients. From MAR (multiple antibiotic resistance) scores, we found a non-uniform spread pattern of AMR. Components-level analysis revealed that governance, finance, and disease burden variables strongly correlate with AMR. From the Bayesian network analysis, we found that access to immunization, obstetric care, and government effectiveness are strong, actionable factors in reducing AMR, confirmed by what-if analysis. Finally, our discriminative machine learning models achieved an individual-level AUROC (Area under receiver operating characteristic curve) of 0.94 (SE=0.01) and 0.89 (SE=0.002) to predict Staphylococcus aureus resistance to ceftaroline and oxacillin, respectively. Causal machine learning revealed that immunisation strategies and quality of governance are vital, actionable interventions to reduce AMR.

Deep discriminative learning model with calibrated attention map for the automated diagnosis of diffuse large B-cell lymphoma

Blood cancer is a potentially fatal disease that results from abnormal blood cell production and function. It encompasses a wide range of cancers, including cancers of the bone marrow, blood, and lymphatic systems. The uncontrolled growth of abnormal blood cells interrupts the process of blood cell development. As a result, the blood cell is unable to perform a variety of functions, including fighting infections and preventing severe bleeding. This necessitates early detection and treatment. For detecting hematopoietic malignancies, histopathology is the gold standard. Manual examination of Hematoxylin-Eosin (H&E) stained histopathological images has been a time-consuming and challenging task. As a result, automatic processing of these histopathological images is critical. In distinguishing more complex histopathological images, the discriminative ability of feature representations extracted from Convolutional Neural Networks (CNNs) remains limited. Keeping this in mind, the authors have proposed a unified framework for detecting diffuse large B-cell lymphoma (DLBCL) from histopathological images based on a deep discriminative learning model with calibrated attention map (DDLM-CAM). A two-channel CNN architecture is proposed, that fused the features obtained from input images and attention map feature transformer (AMFT). The influence of inter-class dispersion and within-class variance is reduced by optimizing a new objective function that considers both center and cross-entropy loss. 700 images have been used for training and 300 images for validation out of 1000 sparsely annotated images. The proposed model achieved accuracy, recall, precision, and specificity of 96%, 94.67%, 97.26% and 97.33%, respectively on the validation set. Thus, the proposed framework would be very helpful to the doctors and histopathologists in speeding up the diagnostic process.

Meta-Pixel-Driven Embeddable Discriminative Target and Background Dictionary Pair Learning for Hyperspectral Target Detection

In hyperspectral target detection, the spectral high-dimensionality, variability, and heterogeneity will pose great challenges to the accurate characterizations of the target and background. To alleviate the problems, we propose a Meta-pixel-driven Embeddable Discriminative target and background Dictionary Pair (MEDDP) learning model by combining low-dimensional embeddable subspace projection and the discriminative target and background dictionary pair learning. In MEDDP, the meta-pixel set is built by taking the merits of homogeneous superpixel segmentation and the local manifold affinity structures, which can significantly reduce the influence of spectral variability and find the most typical and informative prototype spectral signature. Afterward, an embeddable discriminative dictionary pair learning model is established to learn a target and background dictionary pair based on the structural incoherent constraint with embeddable subspace projection. The proposed joint learning strategy can reduce the high-dimensional redundant information and simultaneously enhance the discrimination and compactness of the target and background dictionaries. The proposed MEDDP model is solved by an iterative and alternate optimization algorithm and applied with the meta-pixel-level target detection method. Experimental results on four benchmark HSI datasets indicate that the proposed method can consistently yield promising performance in comparison with some state-of-the-art target detectors.