Chance Classification Research Articles

A Baseline Electroencephalography Motor Imagery Brain-Computer Interface System Using Artificial Intelligence and Deep Learning

This paper presents a baseline or reference (single channel, single subject, single trial) electroencephalography (EEG) motor imagery (MI) brain computer interface (BCI) that harnesses deep learning artificial neural networks (ANNs) for brainwave signal classification. The EEG electrode or sensor is placed on the scalp within the frontal lobe of the right hemisphere of the brain and approximately above the motor cortex. Signal classification discriminates among three MI classes, namely, right first closed event, neutral event and left first closed event and the measured accuracy of the deep learning ANN was 83% which significantly outperforms chance classification. The effectiveness of the system is demonstrated by applying it to the navigation of a virtual environment, specifically, immersive 360-degree panoramas in equirectangular projection.

477 Identifying the Neural Correlates of the Triple Code Model in Numerical Cognition Using Stereotactic Electroencephalography in Epilepsy Patients

INTRODUCTION: The capacity to process numerical quantities is an essential human trait. The “Triple Code Model” (TCM) in numerical cognition, theorizes that different neural substrates encode the processing of visual, auditory, and nonsymbolic numerical representations formats. Limited data exists examining the direct electrophysiological activity involved in numerical cognition. METHODS: Thirteen patients with medically refractory epilepsy that underwent sEEG for the localization of seizures performed a passive numerical recognition task. Subjects were presented with a number quantity from 1 to 9 in one of five different representation formats: two were auditory (spoken numbers, sequential beeps) and three were visual (Arabic numeral, written word, assortment of dots). A total of 2,482 electrode contacts were analyzed. Time-frequency spectrograms were reduced with principal component analysis (PCA) and passed into a linear support vector machine (SVM) classification algorithm to identify neural correlates of number processing. RESULTS: The highest classification accuracy in number processing, irrespective of representation format occurred in the bilateral parietal lobes, right lingual cortex, and bilateral putamen. A total of 407, 776, 854, 475, and 513 contacts had significant classification values during Arabic, beep, spoken, written, and dots, respectively. Bilateral superior temporal cortices demonstrated the best classification value for the processing of auditory formats, whereas visual formats (Arabic and written number) showed greatest involvement in the frontal lobe and inferior precentral gyrus. Classification of dots indicated preferential engagement of the left parietal cortex. Spectral analyses revealed that non-gamma frequency bands held greater than chance classification values to characterize format-specific number representations. CONCLUSIONS: Using a linear SVM classifier with PCA, we examined the Triple Code Model in numerical cognition and identified several neural structures with high classification values involved in numerical processing.

Classification of head movements during multi-person conversation

Hearing-aid processing strategies could be greatly improved by real-time knowledge of the users’ listening intentions, information that may be encoded by characteristic head movements during conversational turn-takingwith multiple talkers, and measured by accelerometers in the hearing devices. First, however, it must be determined to what degree head movements are stereotypic across listeners, or alternatively, unique to each individual, and whether an automatic classifier can identify specific movements. Three cohorts of three young, normal-hearing individuals participated in a semi-structured conversation for 50 minutes. Participants wore hearing aids with accelerometers and were video-recorded. Video recordings were then used to track head movements using zFace (Jeni et al. 2017) and manually annotate head movement and communication activities. Accelerometer data was segmented into 1-s windows and labeled by the annotated activity identified in that window. Features based on temporal and frequency characteristics were used to train multi-class models with across-subject or within-subject design. Significantly better than chance classification was observed for all activities using both model designs, with higher classification accuracy for the models trained on the individuals’ own-data. In summary, head movement behaviors can be classified with short-scale time resolution, and individual differences are a key factor for accurate classification.

Set the tone: Trustworthy and dominant novel voices classification using explicit judgement and machine learning techniques.

Prior research has established that valence-trustworthiness and power-dominance are the two main dimensions of voice evaluation at zero-acquaintance. These impressions shape many of our interactions and high-impact decisions, so it is crucial for many domains to understand this dynamic. Yet, the relationship between acoustical properties of novel voices and personality/attitudinal traits attributions remains poorly understood. The fundamental problem of understanding vocal impressions and relative decision-making is linked to the complex nature of the acoustical properties in voices. In order to disentangle this relationship, this study extends the line of research on the acoustical bases of vocal impressions in two ways. First, by attempting to replicate previous finding on the bi-dimensional nature of first impressions: using personality judgements and establishing a correspondence between acoustics and voice-first-impression (VFI) dimensions relative to sex (Study 1). Second (Study 2), by exploring the non-linear relationships between acoustical parameters and VFI by the means of machine learning models. In accordance with literature, a bi-dimensional projection comprising valence-trustworthiness and power-dominance evaluations is found to explain 80% of the VFI. In study 1, brighter (high center of gravity), smoother (low shimmers), and louder (high minimum intensity) voices reflected trustworthiness, while vocal roughness (harmonic to noise-ratio), energy in the high frequencies (Energy3250), pitch (Quantile 1, Quantile 5) and lower range of pitch values reflected dominance. In study 2, above chance classification of vocal profiles was achieved by both Support Vector Machine (77.78%) and Random-Forest (Out-Of-Bag = 36.14) classifiers, generally confirming that machine learning algorithms could predict first impressions from voices. Hence results support a bi-dimensional structure to VFI, emphasize the usefulness of machine learning techniques in understanding vocal impressions, and shed light on the influence of sex on VFI formation.

Detecting word-level stress in continuous speech: A case study of Brazilian Portuguese

This study discusses the detection of primary stress in continuous speech in Brazilian Portuguese (BP) using the West Point corpus (Morgan et al. 2008), and compressed representations of the speech signal (MFCCs, modelled by HMM-GMMs), as implemented in the toolkit Kaldi (Povey et al. 2011). An acoustic model of BP was trained using 5-fold cross validation and tested in three experimental conditions. Fairly high measures of accuracy were achieved in all conditions tested, yielding high MCCs and Kappas, indicating that the results are neither an effect of imbalanced data sets, nor of chance classification. These results, along with metrics obtained for vowels in pre- and posttonic positions indicate (i) that stress in BP is captured fairly well across speakers and genders by representations of the speech signal that encode spectral features and energy information but which do not directly compute duration or F0; (ii) as captured by the models used herein, there is an asymmetry between pretonic and posttonic vowels; (iii) in a preliminary analysis, Unstressed word tokens tend to cluster in prosodically weak positions of the utterance, raising the question of whether stress is consistently realized in these positions; (iv) pending further studies, there is an asymmetry between ultimate, penultimate and antepenultimate words as to how successfully stress is captured by the models used herein.

Locally Linear Embedding and fMRI Feature Selection in Psychiatric Classification.

Background: Functional magnetic resonance imaging (fMRI) provides non-invasive measures of neuronal activity using an endogenous Blood Oxygenation-Level Dependent (BOLD) contrast. This article introduces a nonlinear dimensionality reduction (Locally Linear Embedding) to extract informative measures of the underlying neuronal activity from BOLD time-series. The method is validated using the Leave-One-Out-Cross-Validation (LOOCV) accuracy of classifying psychiatric diagnoses using resting-state and task-related fMRI. Methods: Locally Linear Embedding of BOLD time-series (into each voxel’s respective tensor) was used to optimise feature selection. This uses Gauß’ Principle of Least Constraint to conserve quantities over both space and time. This conservation was assessed using LOOCV to greedily select time points in an incremental fashion on training data that was categorised in terms of psychiatric diagnoses. Findings: The embedded fMRI gave highly diagnostic performances (> 80%) on eleven publicly-available datasets containing healthy controls and patients with either Schizophrenia, Attention-Deficit Hyperactivity Disorder (ADHD), or Autism Spectrum Disorder (ASD). Furthermore, unlike the original fMRI data before or after using Principal Component Analysis (PCA) for artefact reduction, the embedded fMRI furnished significantly better than chance classification (defined as the majority class proportion) on ten of eleven datasets. Interpretation: Locally Linear Embedding appears to be a useful feature extraction procedure that retains important information about patterns of brain activity distinguishing among psychiatric cohorts.

A Machine-Learning Approach to Predicting Smoking Cessation Treatment Outcomes.

Most cigarette smokers want to quit smoking and more than half make an attempt every year, but less than 10% remain abstinent for at least 6 months. Evidence-based tobacco use treatment improves the likelihood of quitting, but more than two-thirds of individuals relapse when provided even the most robust treatments. Identifying for whom treatment is effective will improve the success of our treatments and perhaps identify strategies for improving current approaches. Two cohorts (training: N = 90, validation: N = 71) of cigarette smokers enrolled in group cognitive-behavioral therapy (CBT). Generalized estimating equations were used to identify baseline predictors of outcome, as defined by breath carbon monoxide and urine cotinine. Significant measures were entered as candidate variables to predict quit status. The resulting decision trees were used to predict cessation outcomes in a validation cohort. In the training cohort, the decision trees significantly improved on chance classification of smoking status following treatment and at 6-month follow-up. The first split of all decision trees, which was delay discounting, significantly improved on chance classification rates in both the training and validation cohort. Delay discounting emerged as the single best predictor of group CBT treatment response with an average baseline discount rate of ln(k) = -7.1, correctly predicting smoking status of 80% of participants at posttreatment and 81% of participants at follow-up. This study provides a first step toward personalized care for smoking cessation though future work is needed to identify individuals that are likely to be successful in treatments beyond group CBT. This study provides a first step toward personalized care for smoking cessation. Using a novel machine-learning approach, baseline measures of clinical and executive functioning are used to predict smoking cessation outcomes following group CBT. A decision point is recommended for the single best predictor of treatment outcomes, delay discounting, to inform future research or clinical practice in an effort to better allocate patients to treatments that are likely to work.

Classification based hypothesis testing in neuroscience: Below‐chance level classification rates and overlooked statistical properties of linear parametric classifiers

Multivariate pattern analysis (MVPA) has recently become a popular tool for data analysis. Often, classification accuracy as quantified by correct classification rate (CCR) is used to illustrate the size of the effect under investigation. However, we show that in low sample size (LSS), low effect size (LES) data, which is typical in neuroscience, the distribution of CCRs from cross-validation of linear MVPA is asymmetric and can show classification rates considerably below what would be expected from chance classification. Conversely, the mode of the distribution in these cases is above expected chance levels, leading to a spuriously high number of above chance CCRs. This unexpected distribution has strong implications when using MVPA for hypothesis testing. Our analyses warrant the conclusion that CCRs do not well reflect the size of the effect under investigation. Moreover, the skewness of the null-distribution precludes the use of many standard parametric tests to assess significance of CCRs. We propose that MVPA results should be reported in terms of P values, which are estimated using randomization tests. Also, our results show that cross-validation procedures using a low number of folds, e.g. twofold, are generally more sensitive, even though the average CCRs are often considerably lower than those obtained using a higher number of folds. Hum Brain Mapp 37:1842-1855, 2016. © 2016 Wiley Periodicals, Inc.

Sitting and standing intention can be decoded from scalp EEG recorded prior to movement execution.

Low frequency signals recorded from non-invasive electroencephalography (EEG), in particular movement-related cortical potentials (MRPs), are associated with preparation and execution of movement and thus present a target for use in brain-machine interfaces. We investigated the ability to decode movement intent from delta-band (0.1–4 Hz) EEG recorded immediately before movement execution in healthy volunteers. We used data from epochs starting 1.5 s before movement onset to classify future movements into one of three classes: stand-up, sit-down, or quiet. We assessed classification accuracy in both externally triggered and self-paced paradigms. Movement onset was determined from electromyography (EMG) recordings synchronized with EEG signals. We employed an artifact subspace reconstruction (ASR) algorithm to eliminate high amplitude noise before building our time-embedded EEG features. We applied local Fisher's discriminant analysis to reduce the dimensionality of our spatio-temporal features and subsequently used a Gaussian mixture model classifier for our three class problem. Our results demonstrate significantly better than chance classification accuracy (chance level = 33.3%) for the self-initiated (78.0 ± 2.6%) and triggered (74.7 ± 5.7%) paradigms. Surprisingly, we found no significant difference in classification accuracy between the self-paced and cued paradigms when using the full set of non-peripheral electrodes. However, accuracy was significantly increased for self-paced movements when only electrodes over the primary motor area were used. Overall, this study demonstrates that delta-band EEG recorded immediately before movement carries discriminative information regarding movement type. Our results suggest that EEG-based classifiers could improve lower-limb neuroprostheses and neurorehabilitation techniques by providing earlier detection of movement intent, which could be used in robot-assisted strategies for motor training and recovery of function.

B-10 * CPT-II versus TOVA: Assessing the Diagnostic Power of Continuous Performance Tests

Objective: To determine if there is a significant difference in classification accuracy between Conners' Continuous Performance Test – II (CPT-II) and the Test of Variables of Attention (TOVA) for Attention Deficit Hyperactivity Disorder (ADHD). It was hypothesized the TOVA would better predict a diagnosis of ADHD, and that response time variability would have the strongest relationship with diagnosis. Method: Participants were 50 children referred for neuropsychological evaluation; 30% were female and 70% male. Mean age was 12.59 (SD = 2.87). Participants were administered the CPT-II and TOVA within the same neuropsychological evaluation. Results: The majority of the sample (86%) received a diagnosis of ADHD as a result of the evaluation. Receiver Operating Characteristic (ROC) was used to assess classification accuracy of various aspects of the two tests. Neither standard scores for commission errors nor omission errors of the CPT-II or the TOVA resulted in a ROC significantly above chance. ROCs for response time and response time variability were also near chance for the CPT-II but slightly better for the TOVA (AUCs of .571 and .629, respectively). The TOVA's API produced an ROC with better than chance classification (AUC = .618). When the CPT-II's confidence index was transformed into a continuous variable, the resulting ROC had an area under the curve of .687, which was the highest of any variable examined. Conclusion(s): Given the number of cognitive diagnoses which can be accompanied by attentional difficulties, the poor classification accuracies demonstrated here for ADHD are not unexpected, but the CPT-II's slightly better performance for total score was unanticipated.

Neuroimaging correlates of pathologically defined subtypes of Alzheimer's disease: a case-control study

Three subtypes of Alzheimer's disease (AD) have been pathologically defined on the basis of the distribution of neurofibrillary tangles: typical AD, hippocampal-sparing AD, and limbic-predominant AD. Compared with typical AD, hippocampal-sparing AD has more neurofibrillary tangles in the cortex and fewer in the hippocampus, whereas the opposite pattern is seen in limbic-predominant AD. We aimed to determine whether MRI patterns of atrophy differ between these subtypes and whether structural neuroimaging could be a useful predictor of pathological subtype at autopsy. We identified patients who had been followed up in the Mayo Clinic Alzheimer's Disease Research Center (Rochester, MN, USA) or in the Alzheimer's Disease Patient Registry (Rochester, MN, USA) between 1992 and 2005. To be eligible for inclusion, participants had to have had dementia, AD pathology at autopsy (Braak stage ≥IV and intermediate to high probability of AD), and an ante-mortem MRI. Cases were assigned to one of three pathological subtypes--hippocampal-sparing, limbic-predominant, and typical AD--on the basis of neurofibrillary tangle counts in hippocampus and cortex and ratio of hippocampal to cortical burden, without reference to neuronal loss. Voxel-based morphometry and atlas-based parcellation were used to compare patterns of grey matter loss between groups and with age-matched control individuals. Neuroimaging was obtained at the time of first presentation. To summarise pair-wise group differences, we report the area under the receiver operator characteristic curve (AUROC). Of 177 eligible patients, 125 (71%) were classified as having typical AD, 33 (19%) as having limbic-predominant AD, and 19 (11%) as having hippocampal-sparing AD. Most patients with typical (98 [78%]) and limbic-predominant AD (31 [94%]) initially presented with an amnestic syndrome, but fewer patients with hippocampal-sparing AD (eight [42%]) did. The most severe medial temporal atrophy was recorded in patients with limbic-predominant AD, followed by those with typical disease, and then those with hippocampal-sparing AD. Conversely, the most severe cortical atrophy was noted in patients with hippocampal-sparing AD, followed by those with typical disease, and then limbic-predominant AD. The ratio of hippocampal to cortical volumes allowed the best discrimination between subtypes (p<0·0001; three-way AUROC 0·52 [95% CI 0·47-0·52]; ratio of AUROC to chance classification 3·1 [2·8-3·1]). Patients with typical AD and non-amnesic initial presentation had a significantly higher ratio of hippocampal to cortical volumes (median 0·045 [IQR 0·035-0·056]) than did those with an amnesic presentation (0·041 [0·031-0·057]; p=0·001). Patterns of atrophy on MRI differ across the pathological subtypes of AD. MRI regional volumetric analysis can reliably track the distribution of neurofibrillary tangle pathology and can predict pathological subtype of AD at autopsy. US National Institutes of Health (National Institute on Aging).

STATISTICAL FEATURE-BASED CRAQUELURE CLASSIFICATION

We demonstrate an automatic procedure for extracting features such as directionality of crack patterns, distribution of node distances and segment lengths, fractal dimension, entropy, and crack coverage to aid in automatic classification of painting cracks or craquelures. To test our classifier, we make use of four distinct craquelure patterns, designated by names based on their country of origin, namely: Dutch, Flemish, French or Italian. We report that selecting features based on effect size ratio from the above statistical measures, the standard linear discriminant analysis (LDA) can make predictive classification of the craquelure patterns with a 69.4% accuracy. Effect size ratio simultaneously quantifies the extent of correlation and variance of two statistical sets of data. This test set accuracy is more than two times better than mere chance classification, or the proportional chance criterion, computed to be ΦPCC= 27.61% and also twice the recommended classifier accuracy 1.25 × ΦPCC= 34.4%. We compare the result with the nonlinear method of neural network and we observe no marked improvement in the resulting accuracy. This suggests that the problem at hand with respect to the statistical features extracted is a linear classification problem. The work provides a comprehensive guide on the algorithms that can be used to extract quantitative information of crack patterns.

Examining the incidence of and time to recidivism within the risk contingency framework: A 20-year follow up study.

This study examined the extent to which the contingencies risk framework (consisting of dispositional, historical, contextual, and clinical domains) predicted detected recidivism (i.e., arrest or conviction). Secondary data were extracted on 413 prisoners who underwent a psychiatric evaluation, were assessed on the risk domains, and followed up over 20 years. There were 273 (66.1%) cases of recidivism for serious offenses (e.g., assaults). Criminal career outcomes examined included: years to and the incidence of recidivism. Statistics showed that chance classification of the incidence of recidivism was 21% more accurate for the recursive partitioning than the bilinear model. These results are consistent with the contingencies risk framework, support its use over linear models, and highlight its predictive utility.

Executed and Observed Movements Have Different Distributed Representations in Human aIPS

How similar are the representations of executed and observed hand movements in the human brain? We used functional magnetic resonance imaging (fMRI) and multivariate pattern classification analysis to compare spatial distributions of cortical activity in response to several observed and executed movements. Subjects played the rock-paper-scissors game against a videotaped opponent, freely choosing their movement on each trial and observing the opponent's hand movement after a short delay. The identities of executed movements were correctly classified from fMRI responses in several areas of motor cortex, observed movements were classified from responses in visual cortex, and both observed and executed movements were classified from responses in either left or right anterior intraparietal sulcus (aIPS). We interpret above chance classification as evidence for reproducible, distributed patterns of cortical activity that were unique for execution and/or observation of each movement. Responses in aIPS enabled accurate classification of movement identity within each modality (visual or motor), but did not enable accurate classification across modalities (i.e., decoding observed movements from a classifier trained on executed movements and vice versa). These results support theories regarding the central role of aIPS in the perception and execution of movements. However, the spatial pattern of activity for a particular observed movement was distinctly different from that for the same movement when executed, suggesting that observed and executed movements are mostly represented by distinctly different subpopulations of neurons in aIPS.

Pathway Analysis of Single-Nucleotide Polymorphisms Potentially Associated with Glioblastoma Multiforme Susceptibility Using Random Forests

Glioma is a complex disease that is unlikely to result from the effect of a single gene. Genetic analysis at the pathway level involving multiple genes may be more likely to capture gene-disease associations than analyzing genes one at a time. The current pilot study included 112 Caucasians with glioblastoma multiforme and 112 Caucasian healthy controls frequency matched to cases by age and gender. Subjects were genotyped using a commercially available (ParAllele/Affymetrix) assay panel of 10,177 nonsynonymous coding single-nucleotide polymorphisms (SNP) spanning the genome known at the time the panel was constructed. For this analysis, we selected 10 pathways potentially involved in gliomagenesis that had SNPs represented on the panel. We performed random forests (RF) analyses of SNPs within each pathway group and logistic regression to assess interaction among genes in the one pathway for which the RF prediction error was better than chance and the permutation P < 0.10. Only the DNA repair pathway had a better than chance classification of case-control status with a prediction error of 45.5% and P = 0.09. Three SNPs (rs1047840 of EXO1, rs12450550 of EME1, and rs799917 of BRCA1) of the DNA repair pathway were identified as promising candidates for further replication. In addition, statistically significant interactions (P < 0.05) between rs1047840 of EXO1 and rs799917 or rs1799966 of BRCA1 were observed. Despite less than complete inclusion of genes and SNPs relevant to glioma and a small sample size, RF analysis identified one important biological pathway and several SNPs potentially associated with the development of glioblastoma.

Nonverbal selective reminding test: Efficacy in the assessment of adults with temporal lobe epilepsy

Although the relationship of verbal memory tests to left temporal lobe epilepsy (LTLE) has reportedly been well established, research on the relationship of nonverbal memory to right TLE (RTLE) has yielded mixed findings, thus bringing into question the clinical utility of nonverbal memory assessment. In the present study, we used a selective reminding procedure for nonverbal spatial material to control for procedural factors, such as verbal mediation, which may in part be the basis for these mixed findings. A sample of 39 LTLE patients and 33 RTLE patients were given both a test of verbal selective reminding (VSR) and a test of nonverbal selective reminding (NVSR) as part of a presurgical evaluation. Diagnosis of RTLE and LTLE was rendered after an extensive presurgical evaluation including magnetic resonance imaging, video-electroencephalography, the intracarotid sodium amytal procedure, and intracranial electrodes when necessary. Results obtained from the NVSR allowed better classification of patients than did VSR, yielding 40% above chance classification. More important, given impaired performance, the probability that a patient had RTLE was 0.70. VSR performance did not allow for classification significantly above chance levels. Although combining the resultsfrom the VSR and NVSR resulted in the best classification, the gain was minimal (72% vs. 70% for NVSR alone), particularly because 26 patients could not be classified. Our results support previous findings of a relationship between nonverbal memory performance and RTLE epilepsy but bring into question the unequivocal acceptance of an exclusive relationship of verbal memory to LTLE.

Adolescent Delinquency and Perceptions of Parental Care and Protection: A Case Control Study

This study examined differences in perceptions of parental attitudes and behaviours throughout childhood among 103 pairs (63 male and 40 female) of delinquents and their matched controls. Male delinquents perceived both parents to be less caring and more protective than did nondelinquents. Female delinquents also perceived their parents as overprotective. Discriminant analyses, including main and interaction effects of perceived parental characteristics, showed that perceptions of both mothers and fathers significantly predicted male and female official delinquency status above the chance classification rates. The implications and limitations of the findings are discussed.

Pattern recognition studies of complex chromatographic data sets: Design and analysis of pattern recognition experiments

Chromatographic fingerprinting of complex biological and environmental samples is a active research area with a large and growing literature. Multivariate statistical and pattern recognition techniques can be effective methods for the analysis of such complex data. However, the classification of complex samples on the basis of their chromatographic profiles is complicated by two factors: (1) confounding of the desired group information by experimental variables or other systematic variations, and (2) random or chance classification effects with linear discriminants. Several interesting projects involving these effects and methods for dealing with the effects are discussed. Complex chromatographic data sets often contain information dependent on experimental variables as well as information which differentiates classes. The existence of these types of complicating relationships is an innate part of fingerprint-type data. ADAPT, an interactive computer software system, has the clustering, mapping, and statistical tools necessary to identify and study these effects in realistically large data sets. In one study, pattern recognition analysis of 144 pyrochromatograms from cultured skin fibroblasts was used to differentiate cystic fibrosis carriers from presumed normal donors. Several experimental variables (door gender, chromatographic column, etc.) were observed to contribute to the overall classification process. Notwithstanding these effects, discriminants were developed from the chromatographic peaks that assigned a given pyrochromatogram to its respective class (cystic fibrosis carrier versus normal) largely on the basis of the desired pathological difference. In another study gas chromatographic profiles of cuticular hydrocarbon extracts obtained from 170 red fire at samples were analyzed using pattern recognition methods. Clustering according to the biological variables of social caste and colony was observed. Previously, Monte-Carlo simulation studies have been carried out to assess the probability of chance classification for nonparametric linear discriminants. The level of expected chance classification as a function of the number of observations, the dimensionality, class membership distribution, and covariance structure of the data were examined. These simulation studies established limits on the approaches that can be taken with real data sets so that chance classifications are improbable.

Chance classifications by non‐parametric linear discriminant functions

AbstractIn applications of pattern recognition techniques to problems in chemical fingerprinting, only limited knowledge about the underlying statistical distribution of the data is generally available. This means that non‐parametric methods must be used. Non‐parametric discriminant functions have been used to provide insight into relationships contained within sets of chemical measurements. However, classification based on random or chance separation can be a serious problem. Monte Carlo simulation studies have been carried out to assess the probability of chance classification for non‐parametric linear discriminants. The level of expected chance classification is a function of the number of observations (the number of samples), the dimensionality of the problem (the number of independent variables per observation), class membership distribution and the covariance structure of the data being examined. An approach for assessing the level of significance of classification scores obtained from real training sets will be presented. These simulation studies establish limits on the approaches that can be taken with real data sets so that chance classification are improbable, and provide information necessary for integrating the data analysis into the overall experimental design.

Monte Carlo studies of non‐parametric linear discriminant functions

AbstractClassification rules using non‐parametric linear discriminant functions are often developed from training sets that are not linearly separable. In these situations it is a common practice among inexperienced workers to use many different pattern recognition methods and then select the results that look the best. However, this practice will only increase the risk of spurious results. To document this, we recently carried out a series of Monte Carlo simulation studies to assess the level of chance classification for two different classification algorithms. The level of chance classification for a given dichotomy is found to vary with the choice of the non‐parametric linear discriminant function employed. Although previous workers have indicated that the degree of separation in the data due to chance is only a function of the object‐to‐descriptor ratio, the results of this study suggest otherwise.