Diagonal Discriminant Analysis Research Articles

Identification of Alzheimer’s disease from central lobe EEG signals utilizing machine learning and residual neural network

Cognitive and behavioral deficits are some of the symptoms of Alzheimer's disease, a neurological disease caused by brain deterioration. Early diagnosis of the disease minimizes the disease's progression. In this way, patients' living standards can be maintained. An experienced specialist will assess the results of grueling diagnostic tests and a costly diagnosis process.This motivation led to the creation of a fully automated computer system, which uses the proposed methods to detect Alzheimer's disease from EEG signals acquired from a minimum number of electrodes: three central lobe electrodes. This work presents the advantages of implementing the proposed system. Two different datasets were presented to evaluate the system’s performance. After preprocessing the raw EEG data, wavelet transforms were applied to calculate statistical properties. First, the paper discusses the performance of different traditional machine learning classifiers: Diagonal Discriminant Analysis, Support Vector Machine (LSVM, RBF), K-Nearest Neighbors, Random Forest, Logistic Regression, and Naïve Bayes. By comparing the classifiers’ performance, it is found that Naïve Bayes and LSVM classifiers yield the highest performance of 96.55% and 95.69% when applied to the first dataset and 96.55% and 94.52% for the second dataset, respectively. “ResNet-50” Convolutional Neural Network classifier was implemented to demonstrate the capability of constructing an effective AD diagnosing system. It yields an accuracy of 97.8261% when applied to the first dataset.This work shows how Deep Learning improves the performance of classification in early medical diagnosis, which will enhance different diseases’ treatment, as “Resnet – 50” convolutional neural network classifier outperformed Naïve Bayes Classifier.

Diagonal Discriminant Analysis With Feature Selection for High-Dimensional Data

We introduce a new method of performing high-dimensional discriminant analysis (DA), which we call multiDA. Starting from multiclass diagonal DA classifiers which avoid the problem of high-dimensional covariance estimation we construct a hybrid model that seamlessly integrates feature selection components. Our feature selection component naturally simplifies to weights which are simple functions of likelihood ratio test statistics allowing natural comparisons with traditional hypothesis testing methods. We provide heuristic arguments suggesting desirable asymptotic properties of our algorithm with regard to feature selection. We compare our method with several other approaches, showing marked improvements in regard to prediction accuracy, interpretability of chosen features, and fast run time. We demonstrate such strengths of our model by showing strong classification performance on publicly available high-dimensional datasets, as well as through multiple simulation studies. We make an R package available implementing our approach. Supplementary materials for this article are available online.

Comparison of different statistical approaches for urinary peptide biomarker detection in the context of coronary artery disease.

BackgroundWhen combined with a clinical outcome variable, the size, complexity and nature of mass-spectrometry proteomics data impose great statistical challenges in the discovery of potential disease-associated biomarkers. The purpose of this study was thus to evaluate the effectiveness of different statistical methods applied for urinary proteomic biomarker discovery and different methods of classifier modelling in respect of the diagnosis of coronary artery disease in 197 study subjects and the prognostication of acute coronary syndromes in 368 study subjects.ResultsComputing the discovery sub-cohorts comprising {scriptscriptstyle raisebox{1ex}{2}!left/ !raisebox{-1ex}{3}right.} of the study subjects based on the Wilcoxon rank sum test, t-score, cat-score, binary discriminant analysis and random forests provided largely different numbers (ranging from 2 to 398) of potential peptide biomarkers. Moreover, these biomarker patterns showed very little overlap limited to fragments of type I and III collagens as the common denominator. However, these differences in biomarker patterns did mostly not translate into significant differently performing diagnostic or prognostic classifiers modelled by support vector machine, diagonal discriminant analysis, linear discriminant analysis, binary discriminant analysis and random forest. This was even true when different biomarker patterns were combined into master-patterns.ConclusionIn conclusion, our study revealed a very considerable dependence of peptide biomarker discovery on statistical computing of urinary peptide profiles while the observed diagnostic and/or prognostic reliability of classifiers was widely independent of the modelling approach. This may however be due to the limited statistical power in classifier testing. Nonetheless, our study showed that urinary proteome analysis has the potential to provide valuable biomarkers for coronary artery disease mirroring especially alterations in the extracellular matrix. It further showed that for a comprehensive discovery of biomarkers and thus of pathological information, the results of different statistical methods may best be combined into a master pattern that then can be used for classifier modelling.Electronic supplementary materialThe online version of this article (doi:10.1186/s12859-016-1390-1) contains supplementary material, which is available to authorized users.

Genomic Landscape Predictive of Minimal Residual Disease (MRD) in Multiple Myeloma (MM)

Progress in the treatment of multiple myeloma (MM) has increased extent and frequency of response, as well as prolonged progression-free (PFS) and overall survival (OS). Today complete remission (CR) rates up to 70% are achieved with new drug combinations. This has lead to development of sensitive next generation sequencing (NGS) -based methods to predict deeper responses that may more accurately predict survival outcomes in MM. Our large recent study has confirmed the clinical impact of achieving MRD- status in MM. Here we are evaluating the genomic alterations that may predict attainment of MRD negative status in MM.MRD status was evaluted in 279 patients from IFM/DFCI 2009 trial. We obtained gene expression by RNA-seq, and copy number profile by cytoScan HD array to evaluate genomic differences between MRD negative and MRD positive groups. We generated copy number data for 175 / 279 patients (72 MRD- and 103 MRD+) with Affymetrix Cytoscan HD array and compared genome wide copy number alterations. We observed statistically significant copy number alterations in chromosome 1p, 2, 4q, 11q, 13, 14 and 20 between MRD- and MRD+ patients. However, the extent of alterations in these regions is limited. The largest difference was on chromosome 11q arm where MRD- patients had 2.2 copies on average and MRD+ had 2.4 (p value < 0.001).Similarly, we generated gene expression profiles with RNAseq for 69 MRD- patients and 92 MRD+ patients to study gene expression alterations that may predict attainment of MRD negative status and to examine possible biological pathways. Although first two component of principle component analysis (PCA) showed that two groups have similar expression profile, we were able to identify 586 differentially expressed genes; 333 of those were up and 253 were down regulated in MRD+ compared to MRD- groups. We found that seven oncogenes (CCND1, CD79B, IDH1, PATZ1, PAX5, POU2AF1, RUNX1) were significantly high in MRD+ and two (CCND2 and MYCN) were high in MRD-. Additional genes that were high in MRD+ samples were enriched in genes regulated by NF-kB in response to TNF, P53 pathway, KRAS signaling and genes down-regulated in response to ultraviolet (UV) radiation. Genes that were high in MRD- compared to MRD+ were also enriched in genes up-regulated by STAT5 in response to IL2 stimulation, p53 pathways and networks, and genes up-regulated in response to ultraviolet (UV) radiation pathways.Finally, we have created a signature to predict MRD+ and MRD- in MM samples from differentially expressed genes. We used 40 genes that has at least 2 fold change difference between MRD+ and MRD- groups as a predictor and we randomly separated 161 RNAseq samples into train (n=99) and test group (n=62). We developed our classifiers with diagonal discriminant analysis and we achieved 0.79 classifier performance on test dataset. Then we tested our signature against 1000 random signature and it was significantly different than random signatures (Figure).In conclusion, we here report a first genomic landscape predictive of minimal residual disease (MRD) in Multiple Myeloma (MM). This analysis will help understand genomic and molecular correlates of achieving minimal residula disease and confirms feasibility of using RNAseq data from diagnosis sample to predict MRD status. The ongoing integration of other genomic correlates such as copy number status as well as alternate splicing may allow further improvement in the performance of prediction. [Display omitted] DisclosuresAnderson:Gilead: Consultancy; acetylon pharmaceuticals: Equity Ownership; Oncocorp: Equity Ownership; Celgene Corporation: Consultancy; BMS: Consultancy; Millennium: Consultancy. Attal:jansen: Honoraria; celgene: Membership on an entity's Board of Directors or advisory committees. Munshi:onyx: Membership on an entity's Board of Directors or advisory committees; celgene: Membership on an entity's Board of Directors or advisory committees; millenium: Membership on an entity's Board of Directors or advisory committees; novartis: Membership on an entity's Board of Directors or advisory committees.

Discriminant analysis with Gaussian graphical tree models

We consider Gaussian graphical tree models in discriminant analysis for two populations. Both the parameters and the structure of the graph are assumed to be unknown. For the estimation of the parameters maximum likelihood is used, and for the estimation of the structure of the tree graph we propose three methods; in these, the function to be optimized is the J-divergence for one and the empirical log-likelihood ratio for the two others. The main contribution of this paper is the introduction of these three computationally efficient methods. We show that the optimization problem of each proposed method is equivalent to one of finding a minimum weight spanning tree, which can be solved efficiently even if the number of variables is large. This property together with the existence of the maximum likelihood estimators for small group sample sizes is the main advantage of the proposed methods. A numerical comparison of the classification performance of discriminant analysis using these methods, as well as three other existing ones, is presented. This comparison is based on the estimated error rates of the corresponding plug-in allocation rules obtained from real and simulated data. Diagonal discriminant analysis is considered as a benchmark, as well as quadratic and linear discriminant analysis whenever the sample size is sufficient. The results show that discriminant analysis with Gaussian tree models, using these methods for selecting the graph structure, is competitive with diagonal discriminant analysis in high-dimensional settings.

Stability of feature selection in classification issues for high-dimensional correlated data

Handling dependence or not in feature selection is still an open question in supervised classification issues where the number of covariates exceeds the number of observations. Some recent papers surprisingly show the superiority of naive Bayes approaches based on an obviously erroneous assumption of independence, whereas others recommend to infer on the dependence structure in order to decorrelate the selection statistics. In the classical linear discriminant analysis (LDA) framework, the present paper first highlights the impact of dependence in terms of instability of feature selection. A second objective is to revisit the above issue using a flexible factor modeling for the covariance. This framework introduces latent components of dependence, conditionally on which a new Bayes consistency is defined. A procedure is then proposed for the joint estimation of the expectation and variance parameters of the model. The present method is compared to recent regularized diagonal discriminant analysis approaches, assuming independence among features, and regularized LDA procedures, both in terms of classification performance and stability of feature selection. The proposed method is implemented in the R package FADA, freely available from the R repository CRAN.

Block-diagonal discriminant analysis and its bias-corrected rules

High-throughput expression profiling allows simultaneous measure of tens of thousands of genes at once. These data have motivated the development of reliable biomarkers for disease subtypes identification and diagnosis. Many methods have been developed in the literature for analyzing these data, such as diagonal discriminant analysis, support vector machines, and k-nearest neighbor methods. The diagonal discriminant methods have been shown to perform well for high-dimensional data with small sample sizes. Despite its popularity, the independence assumption is unlikely to be true in practice. Recently, a gene module based linear discriminant analysis strategy has been proposed by utilizing the correlation among genes in discriminant analysis. However, the approach can be underpowered when the samples of the two classes are unbalanced. In this paper, we propose to correct the biases in the discriminant scores of block-diagonal discriminant analysis. In simulation studies, our proposed method outperforms other approaches in various settings. We also illustrate our proposed discriminant analysis method for analyzing microarray data studies.

Performance and estimation of the true error rate of classification rules built with additional information. An application to a cancer trial

Classification rules that incorporate additional information usually present in discrimination problems are receiving certain attention during the last years as they perform better than the usual rules. Fernández, M. A., C. Rueda and B. Salvador (2006): "Incorporating additional information to normal linear discriminant rules," J. Am. Stat. Assoc., 101, 569-577, proved that these rules have lower total misclassification probability than the usual Fisher's rule. In this paper we consider two issues; on the one hand, we compare these rules with those based on shrinkage estimators of the mean proposed by Tong, T., L. Chen and H. Zhao (2012): "Improved mean estimation and its application to diagonal discriminant analysis," Bioinformatics, 28(4): 531-537. with regard to four criteria: total misclassification probability, area under ROC curve, well-calibratedness and refinement; on the other hand, we consider the estimation of the true error rate, which is a very interesting parameter in applications. We prove results on the apparent error rate of the rules that expose the need of new estimators of their true error rate. We propose four such new estimators. Two of them are defined incorporating the additional information into the leave-one-out-bootstrap. The other two are the corresponding cross-validation after bootstrap versions. We compare these estimators with the usual ones in a simulation study and in a cancer trial application, showing the good behavior of the rules that incorporate additional information and of the new leave-one-out bootstrap estimators of their true error rate.

Improved mean estimation and its application to diagonal discriminant analysis

High-dimensional data such as microarrays have created new challenges to traditional statistical methods. One such example is on class prediction with high-dimension, low-sample size data. Due to the small sample size, the sample mean estimates are usually unreliable. As a consequence, the performance of the class prediction methods using the sample mean may also be unsatisfactory. To obtain more accurate estimation of parameters some statistical methods, such as regularizations through shrinkage, are often desired. In this article, we investigate the family of shrinkage estimators for the mean value under the quadratic loss function. The optimal shrinkage parameter is proposed under the scenario when the sample size is fixed and the dimension is large. We then construct a shrinkage-based diagonal discriminant rule by replacing the sample mean by the proposed shrinkage mean. Finally, we demonstrate via simulation studies and real data analysis that the proposed shrinkage-based rule outperforms its original competitor in a wide range of settings.

Simple and flexible classification of gene expression microarrays via Swirls and Ripples

BackgroundA simple classification rule with few genes and parameters is desirable when applying a classification rule to new data. One popular simple classification rule, diagonal discriminant analysis, yields linear or curved classification boundaries, called Ripples, that are optimal when gene expression levels are normally distributed with the appropriate variance, but may yield poor classification in other situations.ResultsA simple modification of diagonal discriminant analysis yields smooth highly nonlinear classification boundaries, called Swirls, that sometimes outperforms Ripples. In particular, if the data are normally distributed with different variances in each class, Swirls substantially outperforms Ripples when using a pooled variance to reduce the number of parameters. The proposed classification rule for two classes selects either Swirls or Ripples after parsimoniously selecting the number of genes and distance measures. Applications to five cancer microarray data sets identified predictive genes related to the tissue organization theory of carcinogenesis.ConclusionThe parsimonious selection of classifiers coupled with the selection of either Swirls or Ripples provides a good basis for formulating a simple, yet flexible, classification rule. Open source software is available for download.

Comparison of linear discriminant analysis methods for the classification of cancer based on gene expression data

BackgroundMore studies based on gene expression data have been reported in great detail, however, one major challenge for the methodologists is the choice of classification methods. The main purpose of this research was to compare the performance of linear discriminant analysis (LDA) and its modification methods for the classification of cancer based on gene expression data.MethodsThe classification performance of linear discriminant analysis (LDA) and its modification methods was evaluated by applying these methods to six public cancer gene expression datasets. These methods included linear discriminant analysis (LDA), prediction analysis for microarrays (PAM), shrinkage centroid regularized discriminant analysis (SCRDA), shrinkage linear discriminant analysis (SLDA) and shrinkage diagonal discriminant analysis (SDDA). The procedures were performed by software R 2.80.ResultsPAM picked out fewer feature genes than other methods from most datasets except from Brain dataset. For the two methods of shrinkage discriminant analysis, SLDA selected more genes than SDDA from most datasets except from 2-class lung cancer dataset. When comparing SLDA with SCRDA, SLDA selected more genes than SCRDA from 2-class lung cancer, SRBCT and Brain dataset, the result was opposite for the rest datasets. The average test error of LDA modification methods was lower than LDA method.ConclusionsThe classification performance of LDA modification methods was superior to that of traditional LDA with respect to the average error and there was no significant difference between theses modification methods.

Shrinkage‐based Diagonal Discriminant Analysis and Its Applications in High‐Dimensional Data

High-dimensional data such as microarrays have brought us new statistical challenges. For example, using a large number of genes to classify samples based on a small number of microarrays remains a difficult problem. Diagonal discriminant analysis, support vector machines, and k-nearest neighbor have been suggested as among the best methods for small sample size situations, but none was found to be superior to others. In this article, we propose an improved diagonal discriminant approach through shrinkage and regularization of the variances. The performance of our new approach along with the existing methods is studied through simulations and applications to real data. These studies show that the proposed shrinkage-based and regularization diagonal discriminant methods have lower misclassification rates than existing methods in many cases.

SePaCS—a web-based application for classification of seroreactivity profiles

Immunogenic antigen sets possess high potential for minimally invasive disease detection and monitoring. For various diseases, including cancer, appropriate antigen sets have already been detected in blood sera of patients. Typically, a large number of sera from diseased and unaffected persons is screened for the antigens of interest. Sophisticated statistical learning approaches are trained on the resulting data set to classify sera as either tumor or normal sera. We developed a web-based application, called ‘Seroreactivity Profile Classification Service’ (SePaCS) that enables clinical groups to carry out analyzes of training sets and predictions of unclassified seroreactivity profiles with minimal effort. SePaCS provides a broad range of classification methods: four versions of a Naïve Bayes Classifier, Support Vector Machines with a radial basis function kernel, Linear Discriminant Analysis, and Diagonal Discriminant Analysis. The computed results are summarized in a PDF file. We demonstrate the functionality of SePaCS exemplarily for meningioma, a generally benign intracranial tumor. As a second example, we evaluated SePaCS on glioma, a malignant brain tumor. SePaCS is freely available at http://www.bioinf.uni-sb.de/sepacs.

Prediction error estimation: a comparison of resampling methods

In genomic studies, thousands of features are collected on relatively few samples. One of the goals of these studies is to build classifiers to predict the outcome of future observations. There are three inherent steps to this process: feature selection, model selection and prediction assessment. With a focus on prediction assessment, we compare several methods for estimating the 'true' prediction error of a prediction model in the presence of feature selection. For small studies where features are selected from thousands of candidates, the resubstitution and simple split-sample estimates are seriously biased. In these small samples, leave-one-out cross-validation (LOOCV), 10-fold cross-validation (CV) and the .632+ bootstrap have the smallest bias for diagonal discriminant analysis, nearest neighbor and classification trees. LOOCV and 10-fold CV have the smallest bias for linear discriminant analysis. Additionally, LOOCV, 5- and 10-fold CV, and the .632+ bootstrap have the lowest mean square error. The .632+ bootstrap is quite biased in small sample sizes with strong signal-to-noise ratios. Differences in performance among resampling methods are reduced as the number of specimens available increase. A complete compilation of results and R code for simulations and analyses are available in Molinaro et al. (2005) (http://linus.nci.nih.gov/brb/TechReport.htm).