Fisher's Discriminant Function Research Articles

Universal non-data-aided small-sample-support adaptive spread-spectrum receivers

We revisit the problem of recovering a spread-spectrum (SS) signal in the presence of unknown correlated SS interference and additive Gaussian or impulsive noise. The general receiver structure is composed of an adaptive chip-based nonlinear pre-processor followed by an adaptive linear filter. We propose two blind algorithms for the adaptation of the parameters of the nonlinear pre-processor that are based on the Fisher discriminant function evaluated at the linear filter output. The first algorithm couples itself with the adaptation of the linear post-processor, while the second algorithm is a decoupled procedure that is independent of the linear post-processor. Both algorithms lead to universal receiver solutions in the sense that they mitigate a wide range of additive noise effects (from purely Gaussian to impulsive) and correlated SS interference. Both algorithms offer superior detection performance when receiver adaptation is carried out with limited input data observations

A compact, high numerical aperture imaging Fourier transform spectrometer and its application

This paper describes a compact imaging Fourier transform spectrometer with high numerical aperture. In comparison with other optical arrangements in which extended interferometer paths are required for the inclusion of dispersion compensation optics, this technique utilizes a rudimentary cubic beam splitter based Michelson interferometer with minimal optical path so that the numerical aperture of the system is maximized. Mathematical modelling is presented showing that the fringe distortions caused by the dispersion in the cubic beam splitter can be entirely removed without any loss of the spectral information. An illustration of the power of the technique is given classifying between different plant foliage performed using a Fisher discriminant function based optimal linear filtering.

Fetal weight, maternal age and height are poor predictors of the need for caesarean section for arrest of labor

Retrospective data on 228 patients was analyzed in order to develop a predictive model of operative delivery, caesarean section for arrest of labor. The ANOVA, discriminant analysis and the Fisher discriminant function of SPSS were used. Birth weight, maternal age and maternal height were statistically significant risk factors, but only 10.9% of caesarean sections could be predicted with these variables. Seven percent of patients who delivered vaginally were predicted as needing a caesarean section for arrest of labor.

Robustness of the Fisher's discriminant function to skew-curved normal distribution

Discriminant analysis is a widely used multivariate technique with Fisher's discriminant analysis (FDA) being its most venerable form. FDA assumes equality of population covariance matrices, but does not require multivariate normality. Nevertheless, the latter is desirable for optimal classification. To test FDA's performance under non-normality caused by skewness the method was assessed with simulation based on a skew-curved normal (SCN) distribution belonging to the family of skew-generalised normal distributions; additionally, effects of sample size and rotation were evaluated. Apparent error rate (APER) was used as the measure of classification performance. The analysis was performed using ANOVA with (transformed) mean APER as the dependent variable. Results show the FDA to be highly robust to skewness introduced into the model via the SCN distributed simulated data.

The discriminating power of wavelets to detect non-Gaussianity in the cosmic microwave background

We investigate the power of wavelet techniques in detecting non-Gaussianity in the cosmic microwave background (CMB). We use the method to discriminate between an inflationary and a cosmic strings model using small simulated patches of the sky. We show the importance of the choice of a good test statistic in order to optimise the discriminating power of the wavelet technique. In particular, we construct the Fisher discriminant function, which combines all the information available in the different wavelet scales. We also compare the performance of different decomposition schemes and wavelet bases. For our case, we find that the Mallat and {\it \`a trous} algorithms are superior to the 2D-tensor wavelets. Using this technique, the inflationary and strings models are clearly distinguished even in the presence of a superposed Gaussian component with twice the rms amplitude of the original cosmic string map.

A theorem on the uncorrelated optimal discriminant vectors

This paper proposes a theorem on the uncorrelated optimal discriminant vectors (UODVs). It is proved that the classical optimal discriminant vectors are equivalent to UODV, which can be used to extract ( L−1) uncorrelated discriminant features for L-class problems without losing any discriminant information in the meaning of Fisher discriminant criterion function. Experiments on Concordia University CENPARMI handwritten numeral database indicate that UODVs are much more powerful than the Foley–Sammon optimal discriminant vectors. It is believed that when the number of training samples is large, the conjugate orthogonal set of discriminant vectors can be much more powerful than the orthogonal set of discriminant vectors.

Rao's score, Neyman's C(α) and Silvey's LM tests: an essay on historical developments and some new results

Abstract Rao's (Proc. Cambridge Philos. Soc. 44 (1948a) 50) seminal paper introduced a fundamental principle of testing based on the score function as an alternative to likelihood ratio and Wald tests. Neyman's (In: Grenander, (Ed.), Probability and Statistics, the Harald Cramer Volume, Almqvist and Wiksell, Uppsala, pp. 213–234) approach, in view of the presence of nuisance parameters, emphasized the generality and attractive features of the score-based tests. Silvey (Ann. Math. Statist. 30 (1959) 389) rediscovered the score test as a Lagrange multiplier test. Breusch and Pagan's (Rev. Econom. Stud. 47 (1980) 239) exposition of the score test in a general framework in the context of econometric modeling resulted in an increased activity on specification testing in econometrics. In this paper we trace these historical developments emphasizing the optimality features of tests based on scores and their usefulness in practical problems in statistics and econometrics. In so doing we give some new results, present easier computation of score-based tests and alternative derivations of some known results. We also discuss a connection between Rao's score test and the seemingly unrelated literature of Fisher's discriminant function, Mahalanobis’ D 2 and Hotelling's T 2 .

Computer vision analysis of somatic embryos of sweet potato [Ipomoea batatas (L.) Lam.] for assessing their ability to convert to plants.

Apical and axial shoot tips of sweet potato were cultured to produce somatic embryos that mature and develop into plants in basal nutrient medium. However, the lack of high regeneration efficiency is an impediment to the use of somatic embryos to produce synthetic seeds. Conversion experiments with mature embryos over a 20-day period revealed that 80-90% of the embryos formed roots but only 40-50% formed shoots. Using computer vision and canonical or Fisher discriminant function (CDA) analysis along with conversion results, it was possible to correctly classify competent embryos 40-50% of the time based on size features, 50-60% of the time based on shape features, and 55-60% of the time based on color features. Non-competent embryos were correctly classified 65-75%, 55-60%, and 70-75% of the time based on size, shape, and color, respectively. These results can be used effectively to identify and select competent embryos for improved regeneration efficiency.

General methodology for simultaneous representation and discrimination of multiple object classes

We address a new general method for linear and nonlinear feature extraction for simultaneous representation and classification. We call this approach the maximum representation and discrimination feature (MRDF) method. We develop a novel nonlinear eigenfeature (NLEF) extraction technique to represent data with closed-form solutions and use it to derive a nonlinear MRDF algorithm. Results of the MRDF method on synthetic databases are shown and compared with results from standard Fukunaga-Koontz transform and Fisher discriminant function methods. The method is also applied to an automated product inspection problem (discrimination) and for classification and pose estimation of two similar objects under 3-D aspect angle variations (representation and discrimination).

Evolution and generalization of a single neurone: I. Single-layer perceptron as seven statistical classifiers

Unlike many other investigations on this topic, the present one considers the non-linear single-layer perceptron (SLP) as a process in which the weights of the perceptron are increasing, and the cost function of the sum of squares is changing gradually. During the backpropagation training, the decision boundary of of SLP becomes identical or close to that of seven statistical classifiers: (1) the Euclidean distance classifier, (2) the regularized linear discriminant analysis, (3) the standard Fisher linear discriminant function, (4) the Fisher linear discriminant function with a pseudoinverse covariance matrix, (5) the generalized Fisher discriminant function, (6) the minimum empirical error classifier, and (7) the maximum margin classifier. In order to obtain a wider range of classifiers, five new complexity-control techniques are proposed: target value control, moving of the learning data centre into the origin of coordinates, zero weight initialization, use of an additional negative weight decay term called “anti-regularization”, and use of an exponentially increasing learning step. Which particular type of classifier will be obtained depends on the data, the cost function to be minimized, the optimization technique and its parameters, and the stopping criteria.

Fog forecasting in Cuba. Neural networks versus discriminant analysis

AbstractTwo methods for fog forecasting in Cuba are compared: use of the classical Fisher discriminant function, and the LVQ algorithm developed by the Laboratory of Computer and Information Sciences, Helsinki University of Technology. The relative number of correct forecasts is over 70% for both, which can be considered a good performance. When the learning sample is large enough and nearly equi‐probabalistic, the LVQ algorithm provides a greater number of correct forecasts than those obtained via the Fisher discriminant function. However, the results attained via the LVQ algorithm are not steady when the learning sample is far from being equi‐probabalistic, because the number of fog cases is much reduced. Until larger samples are available for some regions, it will be necessary to use both methods for fog forecasting in Cuba.

Effect of some modifications in standard discriminant analysis

In this paper, the results of three simulation studies about discrimination are described. The first study investigates the use of James–Stein estimators in Fisher's discriminant function. The second compares the Anderson-Bahadur minimax linear discriminant function with the quadratic discriminant function and the last one deals with the choice of an unweighted between groups sum of squares and sum of products matrix instead if the usual between groups matrix

Comparison of a Neural Network and Traditional Classifier for Machine Vision Inspection of Potatoes

This work addressed the relative strengths and weaknesses of the backpropagation neural network versus the Fisher discriminant function. Their performance was compared for machine vision inspection of greening, shape, and shatter bruise in two potato cultivars. The backpropagation networks number of hidden nodes were varied from zero to eight for each defect type to determine the optimal network classification size. The network was trained and tested five times at each hidden node number and defect type to minimize local minima variation. For greening, the best backpropagation network averaged 74.0% with three hidden nodes while the Fisher method performed with a 70.0% accuracy. The backpropagation method also performed better for shape discrimination with a 73.3% average accuracy at seven hidden layer nodes versus a 68.1% accuracy. The Fisher method performed better for shatter bruise detection with a 76.7% accuracy versus a 56.0% average accuracy at four hidden layer nodes for backpropagation.

A Nonmetric Approach to Linear Discriminant Analysis

Abstract A new nonmetric linear discriminant analysis approach is proposed that is based on the maximization of an index of separation differing from that used by the classical method. The possibility of choosing between Fisher's classical discriminant function and the one proposed here enables us to reduce the number of misclassifications for given data. The method is exemplified on empirical data and various simulations and is compared with the classical linear discriminant analysis.

A Nonmetric Approach to Linear Discriminant Analysis

Abstract A new nonmetric linear discriminant analysis approach is proposed that is based on the maximization of an index of separation differing from that used by the classical method. The possibility of choosing between Fisher's classical discriminant function and the one proposed here enables us to reduce the number of misclassifications for given data. The method is exemplified on empirical data and various simulations and is compared with the classical linear discriminant analysis.

A Multiple-Map Model for Pattern Classification

A characteristic feature of vertebrate sensory cortex (and midbrain) is the existence of multiple two-dimensional map representations. Some workers have considered single-map classification (e.g. Kohonen 1984) but little work has focused on the use of multiple maps. We have constructed a multiple-map classifier, which permits abstraction of the computational properties of a multiple-map architecture. We identify three problems which characterize a multiple-map classifier: classification in two dimensions, mapping from high dimensions to two dimensions, and combination of multiple maps. We demonstrate component solutions to each of the problems, using Parzen-window density estimation in two dimensions, a generalized Fisher discriminant function for dimensionality reduction, and split/merge methods to construct a “tree of maps” for the multiple-map representation. The combination of components is modular and each component could be improved or replaced without affecting the other components. The classifier training procedure requires time linear in the number of training examples; classification time is independent of the number of training examples and requires constant space. Performance of this classifier on Fisher's iris data, Gaussian clusters on a five-dimensional simplex, and digitized speech data is comparable to competing algorithms, such as nearest-neighbor, back-propagation and Gaussian classifiers. This work provides an example of the computational utility of multiple-map representations for classification. It is one step towards the goal of understanding why brain areas such as visual cortex utilize multiple map-like representations of the world.

Applied Multivariate Statistical Analysis.

(NOTE: Each chapter begins with an Introduction, and concludes with Exercises and References.) I. GETTING STARTED. 1. Aspects of Multivariate Analysis. Applications of Multivariate Techniques. The Organization of Data. Data Displays and Pictorial Representations. Distance. Final Comments. 2. Matrix Algebra and Random Vectors. Some Basics of Matrix and Vector Algebra. Positive Definite Matrices. A Square-Root Matrix. Random Vectors and Matrices. Mean Vectors and Covariance Matrices. Matrix Inequalities and Maximization. Supplement 2A Vectors and Matrices: Basic Concepts. 3. Sample Geometry and Random Sampling. The Geometry of the Sample. Random Samples and the Expected Values of the Sample Mean and Covariance Matrix. Generalized Variance. Sample Mean, Covariance, and Correlation as Matrix Operations. Sample Values of Linear Combinations of Variables. 4. The Multivariate Normal Distribution. The Multivariate Normal Density and Its Properties. Sampling from a Multivariate Normal Distribution and Maximum Likelihood Estimation. The Sampling Distribution of 'X and S. Large-Sample Behavior of 'X and S. Assessing the Assumption of Normality. Detecting Outliners and Data Cleaning. Transformations to Near Normality. II. INFERENCES ABOUT MULTIVARIATE MEANS AND LINEAR MODELS. 5. Inferences About a Mean Vector. The Plausibility of ...m0 as a Value for a Normal Population Mean. Hotelling's T 2 and Likelihood Ratio Tests. Confidence Regions and Simultaneous Comparisons of Component Means. Large Sample Inferences about a Population Mean Vector. Multivariate Quality Control Charts. Inferences about Mean Vectors When Some Observations Are Missing. Difficulties Due To Time Dependence in Multivariate Observations. Supplement 5A Simultaneous Confidence Intervals and Ellipses as Shadows of the p-Dimensional Ellipsoids. 6. Comparisons of Several Multivariate Means. Paired Comparisons and a Repeated Measures Design. Comparing Mean Vectors from Two Populations. Comparison of Several Multivariate Population Means (One-Way MANOVA). Simultaneous Confidence Intervals for Treatment Effects. Two-Way Multivariate Analysis of Variance. Profile Analysis. Repealed Measures, Designs, and Growth Curves. Perspectives and a Strategy for Analyzing Multivariate Models. 7. Multivariate Linear Regression Models. The Classical Linear Regression Model. Least Squares Estimation. Inferences About the Regression Model. Inferences from the Estimated Regression Function. Model Checking and Other Aspects of Regression. Multivariate Multiple Regression. The Concept of Linear Regression. Comparing the Two Formulations of the Regression Model. Multiple Regression Models with Time Dependant Errors. Supplement 7A The Distribution of the Likelihood Ratio for the Multivariate Regression Model. III. ANALYSIS OF A COVARIANCE STRUCTURE. 8. Principal Components. Population Principal Components. Summarizing Sample Variation by Principal Components. Graphing the Principal Components. Large-Sample Inferences. Monitoring Quality with Principal Components. Supplement 8A The Geometry of the Sample Principal Component Approximation. 9. Factor Analysis and Inference for Structured Covariance Matrices. The Orthogonal Factor Model. Methods of Estimation. Factor Rotation. Factor Scores. Perspectives and a Strategy for Factor Analysis. Structural Equation Models. Supplement 9A Some Computational Details for Maximum Likelihood Estimation. 10. Canonical Correlation Analysis Canonical Variates and Canonical Correlations. Interpreting the Population Canonical Variables. The Sample Canonical Variates and Sample Canonical Correlations. Additional Sample Descriptive Measures. Large Sample Inferences. IV. CLASSIFICATION AND GROUPING TECHNIQUES. 11. Discrimination and Classification. Separation and Classification for Two Populations. Classifications with Two Multivariate Normal Populations. Evaluating Classification Functions. Fisher's Discriminant Function...nSeparation of Populations. Classification with Several Populations. Fisher's Method for Discriminating among Several Populations. Final Comments. 12. Clustering, Distance Methods and Ordination. Similarity Measures. Hierarchical Clustering Methods. Nonhierarchical Clustering Methods. Multidimensional Scaling. Correspondence Analysis. Biplots for Viewing Sample Units and Variables. Procustes Analysis: A Method for Comparing Configurations. Appendix. Standard Normal Probabilities. Student's t-Distribution Percentage Points. ...c2 Distribution Percentage Points. F-Distribution Percentage Points. F-Distribution Percentage Points (...a = .10). F-Distribution Percentage Points (...a = .05). F-Distribution Percentage Points (...a = .01). Data Index. Subject Index.

Maximum Shannon information content of diagnostic medical testing. Including application to multiple non-independent tests.

The increase in Shannon information available from a diagnostic test associated with grading of the test results into many outcomes, rather than simply positive or negative, was examined to determine its upper limit as the number of test outcomes is increased indefinitely. Numerical methods were employed to find the optimal locations of outcome boundaries when a single normally distributed test variable is classified into 2, 3, 4, 5, 6, 8, 14, or 20 outcome categories. In each case Shannon information was computed for values of prior probability between 0.01 and 0.99 and for distances between the means in diseased and nondiseased populations ranging from 0.5 to 5.0 standard deviations. There is an important improvement in Shannon information as the number of outcomes defined is increased, but the increment in information diminishes rapidly with each additional category. A 20%-30% increment in information may be achieved with three outcomes instead of two. A further important increase in information occurs with four to seven outcomes, but beyond this the increment in inforation is negligible. The findings were similar over a wide range of prior probabilities and distances between the means. The analysis was extended to the case of multiple nonindependent tests by demonstrating their application to a Fisher discriminant function incorporating such tests. It was concluded that for normally distributed test variables: grading of test results significantly improves the information content of both single and multiple tests; the value of information content for 8-20 outcomes represents very nearly the maximum information content of a test; there is little value in using more than five to seven test outcomes; multiple grading should not be neglected for discriminant functions.

Preference structure analysis: A nonmetric approach

A nonmetric procedure is suggested to analyze the structure of preferences among multivariate groups. This data analysis technique enables one to compete with discriminant analysis as a special case as well as with some other more complicated problems. The goal of the article is mainly twofold: 1. 1. to suggest a nonmetric procedure in which goodness of discrimination is higher than or equal to that of Fisher's discriminant function (FDF) (the number of misclassification errors for given data will be less or equal to that obtained by FDF while using the nonmetric procedure); 2. 2. to exemplify the fact that very different discriminant functions could yield the very same number of errors. The latter issue was revealed due to the apparent drawback of non-uniqueness of the discriminant function (solution) obtained by our procedure. Non-uniqueness usually seems to be a drawback, nevertheless it might be an advantage, as exemplified later by real examples. Non-uniqueness is a common property of data analysis methods.

質問紙健康調査票THIによる神経症者,分裂病者などの判別診断のこころみ

One of the general health questionnaires, the Todai Health Index(THI), developed by S. Aoki et al. in 1974, is composed of 130 questions concerning subjective symptom, mental health, personality, health habit and the ones to check the reliability of the answered THI. Twelve scale scores are available from THI which tell the pattern of complaint and personality, and are able to identify psychosomatics from normals by Fisher's discriminant function. The authors applied THI to the several groups of psychiatric outpatient who had been definitely diagnosed by psychiatrists to be neurosis(52patients), schizophrenia(37), depression(16), epilepsia(15)and maina(3), and characteristic patterns were analysed of the 12 scale scores in the respective patient group. Two discriminant functions were obtained and they proved to identify 42 out of 52 clinical neurotics as it and 30 out of 37 clinical schizophrenics as it. The method of forward selection by means of Mahalanobis generalized distance was also applied for the identification of the two patient groups.