Fisher's Linear Discriminant Function Research Articles

The Authentication of Basmati Rice Using near Infrared Spectroscopy

The feasibility of using near infrared transmission spectroscopy to discriminate between Basmati and other long-grain rice samples was investigated. A discriminant rule was derived using the Fisher linear discriminant function calculated from the first few principal component scores of the NIR spectra. The discriminant rule was assessed by cross-validation. Nine Basmati and 53 other rice samples were classified correctly from NIR spectra measured on 200 g bulk samples but 8% of the Basmatis and 14% of the others were misclassified on the basis of spectra of individual grains. Using average spectra from 23 grains resulted in correct classification of 16 Basmatis but a 20% error rate on 100 other samples.

The bias of Fisher's linear discriminant function when the variancies are not equal

Since Fisher's linear discriminant rule (FLDR) is the most widely used classification rule, its behaviour under non-standard situations is of great interest. This paper gives an expansion for the mean of FLDR when the equal variance matrices assumption is violated. The expansion has a particularly simple form for proportional covariance matrices.

Error Rates of Non-Bayes Classification Rules and the Robustness of Fisher's Linear Discriminant Function

Error Rates of Non-Bayes Classification Rules and the Robustness of Fisher's Linear Discriminant Function

Error rates of non-Bayes classification rules and the robustness of Fisher's linear discriminant function

We call a classification procedure non-Bayes if it does not converge to the Bayes classification procedure. An asymptotic expansion is found for the expected error rate of such a classification rule. This is used to compare the estimates of Fisher's linear discriminant rule, F, and the quadratic discriminant rule, Q, under departures from the equal variance matrices assumption. It is found that F is quite robust to departures from the equal variances assumption.

Optimal Sample Allocation for Normal Discrimination and Logistic Regression Under Stratified Sampling

Abstract For two multivariate normal populations with a common covariance matrix and stratified sampling, we consider two methods of estimation—Fisher's linear discriminant function and logistic regression. Intuition suggests that taking half of the observations from each population is a reasonable design choice. Based on minimizing the expected error regret, asymptotic optimal sample allocations are found. The results indicate that the differences in the expected error regret for optimal versus balanced allocation are generally quite small. It is recommended that equal sample sizes for the two populations be used for these problems.

Optimal Sample Allocation for Normal Discrimination and Logistic Regression under Stratified Sampling

Abstract For two multivariate normal populations with a common covariance matrix and stratified sampling, we consider two methods of estimation—Fisher's linear discriminant function and logistic regression. Intuition suggests that taking half of the observations from each population is a reasonable design choice. Based on minimizing the expected error regret, asymptotic optimal sample allocations are found. The results indicate that the differences in the expected error regret for optimal versus balanced allocation are generally quite small. It is recommended that equal sample sizes for the two populations be used for these problems.

On estimation of discriminant coefficients

Estimation of discriminant coefficients arises in linear discriminant analysis when the Fisher's linear discriminant function is viewed as the posterior log-odds. Accordingly, we treat the problem of estimating the discriminant coefficients under the quadratic loss. We obtain a class of improved estimators dominating the ‘unbiased’ estimator, considerably generalizing earlier work of Haff (1986).

Utilization of a best linear discriminant function for designing the binary decision tree†

Abstract This paper describes an accurate and efficient method for supervised classification of multispectral images. First, a simple derivation of a best linear discriminant function (BLD) is presented through geometrical consideration on ellipses with equi-Mahalanobis distance. It is shown that the function satisfies the Minimax criterion, which implies the robustness with regard to prior probabilities. Then, the binary decision tree (BDT) is introduced in order to make the method efficient, where the BLD is utilized as a decision rule. Care is taken to prevent a decline in the classification accuracy during the process of constructing the BDT. In addition to the theoretical evaluation of the processing speed, the actual performance of the proposed method is shown by applying it to multispectral data of Landsat-5. It is shown that the BDT/BLD becomes more efficient than Fisher's linear discriminant function when the number of classes increases.

Error-rate estimation in two-group discriminant analysis using the linear discriminant function

Fisher's linear discriminant function, adapted by Anderson for allocating new observations into one of two existing groups, is considered in this paper. Methods of estimating the misclassification error rates are reviewed and evaluated by Monte Carlo simulations. The investigation is carried out under both ideal (Multivariate Normal data) and non-ideal (Multivariate Binary data) conditions. The assessment is based on the usual mean square error (MSE) criterion and also on a new criterion of optimism. The results show that although there is a common cluster of good estimators for both ideal and non-ideal conditions, the single best estimators vary with respect to the different criteria

A Comparison of Linear Programming and Parametric Approaches to the Two‐Group Discriminant Problem*

Recent simulation‐based studies of linear programming models for discriminant analysis have used the Fisher linear discriminant function as the benchmark for parametric methods. This article reports experimental evidence which suggests that, while some linear programming models may match or even exceed the Fisher approach in classification accuracy, none of the fifteen models tested is as accurate on normally distributed data as the Smith quadratic discriminant function. At the minimum, further testing is warranted with an emphasis on data sets that arise from significantly non‐Gaussian populations.

Modified fisherrsquo;s linear discriminant function with reduction of dimensionality

The use of principal components to reduce the number of dimensions is an optimum procedure for data representation, but may involve the loss of valuable information for discriminant analysis. In this paper a simple approach is proposed to improve the discrimination based on a given number (k) of principal components, without requiring the calculation of additional ones. This is achieved by introducing a metric which is a linear combination of the Mahalanobis distance in the subspace of the first k components and the euclidean distance in its orthogonal complement. This concept, applied to Fisher's linear discriminant function, yields an optimum combination of the two distances mentioned. The empirical performance of this procedure for estimated parameters is investigated by a simulation study. Some suggestions for further extensions of this method to nonlinear discrimination are briefly discussed.

A comparison of a robust mixed-integer approach to existing methods for establishing classification rules for the discriminant problem

A nonparametric mixed-integer programming formulation to solve the classification problem in linear discriminant analysis is proposed. The classification performance of this formulation is compared to the MSD linear programming approach and two commonly used statistical methods, Fisher's linear discriminant function and the quadratic discriminant function. Using real data with highly nonnormal distributions the mixed-integer formulation is shown to outperform any of the other three approaches. To study the performance of the mixed-integer formulation systematically, a Monte Carlo simulation experiment is conducted, sampling from several different distributions. The results show that the mixed-integer formulation outperforms the other three approaches on the training samples, except when the variance-covariances are heterogeneous, in which case the quadratic function classifies better. When holdout samples are used to evaluate the relative performance, the mixed-integer approach classifies well when the data are highly discrete and the variance-covariances are homogeneous, but does not fare as well under other data conditions. Therefore, this study suggests that under certain conditions the mixed-integer approach is an attractive alternative to establish classification methods.

On the balancing of error rates for LP discriminant methods

AbstractThis paper reports the results of an experimental comparison between a linear programming (LP) approach and Fisher's linear discriminant function (FLDF) for solving discriminant analysis problems. In particular, we examine the two‐group, two‐variable problem and the associated error rates for the two groups when each of the models is applied to various sets of simulated data. In contrast to previous research, we concentrate on individual error rates for the two groups, i.e. we count the number of group 1 values classified into group 2 and the number of group 2 values classified into group 1. The results indicate that not only is FLDF better at overall classification but it also provides a much better balance between error rates for the two groups.

Classification performance of mathematical programming techniques in discriminant analysis: Results for small and medium sample sizes

AbstractThe performance on small and medium‐size samples of several techniques to solve the classification problem in discriminant analysis is investigated. The techniques considered are two widely used parametric statistical techniques (Fisher's linear discriminant function and Smith's quadratic function), and a class of recently proposed nonparametric estimation techniques based on mathematical programming (linear and mixed‐integer programming). A simulation study is performed, analyzing the relative performance of the above techniques in the two‐group case, for various small sample sizes, moderate group overlap and across six different data conditions. Training samples as well as validation samples are used to assess the classificatory performance of the techniques. The degree of group overlap and sample sizes selected for analysis in this paper are of interest in practice because they closely reflect conditions of many real data sets. The results of the experiment show that Smith's nonlinear quadratic function tends to be superior on the training samples and validation samples when the variances–covariances across groups are heterogeneous, while the mixed‐integer technique performs best on the training samples when the variances–covariances are equal, and on validation samples with equal variances and discrete uniform independent variables. The mixed‐integer technique and the quadratic discriminant function are also found to be more sensitive than the other techniques to the sample size, giving disproportionally inaccurate results on small samples.

A neural network approach to the classification problem

The task of classifying observations into known groups is a common problem in decision making. A wealth of statistical approaches, commencing with Fisher's linear discriminant function, and including variations to accomodate a variety of modeling assumptions, have been proposed. In addition, nonparametric approaches based on various mathematical programming models have also been proposed as solutions. All of these proposed aolutions have performed well when conditions favorable to the specific model are present. The modeler, therefore, can usually be assured of a good solution to his problem of he chooses a model which fits his situation. In this paper, the performance of a neural network as a classifier is evaluated. It is found that the performance of the neural network is comparable to the best of otheother methods under a wide variety of modeling assumptions. The use of neural networks as classifiers thus relieves the modeler of testing assumptions which would otherwise be critical to the performance of the usual classification techniques.

A floristic classification of the Walpole-Nornalup national park, Western Australia

A floristic classification based on 219 sample sites and 233 species is presented for the Walpole-Nornalup National Park in southwest Western Australia. Twelve community types were derived by the use of clustering and ordination techniques, and are associated with recently devised land-form and soils units. Four of the community types are associated with aeolian dune systems and relate to different ages of dune formation. These are the most species-rich of the community types. Community types on older dune systems show greater floristic richness and less structural variation than those occurring on younger dune systems. Three of the community types occur on hills or ridges, each being associated with swampy terrain on fluvial sediments or granitic base rocks. One of the community types is associated with sites where aeolian sands overly rock or pan. Species analysis grouped species with similar ecological tolerance to environmental gradients, although they may not occur sympatrically. Stepwise discriminant analysis retained 52 of the species when deriving 12 Fisher's linear discriminant functions for allocation of independent sites to the classification. The functions allocated 96% of the original sites to the community type designated by the classification, and 92% of 24 sites sampled immediately adjacent to the original sample sites, to the same community type.

FOUR APPROACHES TO THE CLASSIFICATION PROBLEM IN DISCRIMINANT ANALYSIS: AN EXPERIMENTAL STUDY*

ABSTRACTFour discriminant models were compared in a simulation study: Fisher's linear discriminant function [14], Smith's quadratic discriminant function [34], the logistic discriminant model, and a model based on linear programming [17]. The study was conducted to estimate expected rates of misclassification for these four procedures when observations were sampled from a variety of normal and nonnormal distributions. In contrast to previous research, data were taken from four types of Kurtotic population distributions. The results indicate the four discriminant procedures are robust toward data from many types of distributions. The misclassification rates for both the logistic discriminant model and the formulation based on linear programming consistently decreased as the kurtosis in the data increased. The decreases, however, were of small magnitude. None of these procedures yielded statistically significant lower rates of misclassification under nonnormality. The quadratic discriminant function produced significantly lower error rates when the variances across groups were heterogeneous.

Multiple degree of freedom object recognition using optical relational graph decision nets

Multiple degree of freedom object recognition concerns objects with no stable rest position with all scale, rotation, and aspect distortions possible. We assume that the objects are in a fairly benign background, so that feature extractors are usable. In-plane distortion invariance is provided by use of a polar-log coordinate transform feature space, and out-of-plane distortion invariance is provided by linear discriminant function design. Relational graph decision nets are considered for multiple degree of freedom pattern recognition. The design of Fisher linear discriminant functions and synthetic discriminant functions for use at the nodes of binary and multidecision nets is discussed. Case studies are detailed for two-class and multiclass problems. Simulation results demonstrate the robustness of the processors to quantization of the filter coefficients and to noise.

FEXPAC: a program for linear discriminant classification

A new program for feature extraction and two-class classification utilizing the Fisher linear discriminant function model has been developed. It is intended as a tool in multivariate data analysis, and is especially suited for the design and storage of linear classifiers from training data sets.

Morphological discriminants of Scottish wildcats (Felis silvestris), domestic cats (F. catus) and their hybrids

Skull characters of wildcats (Felis silvestris), classed as ‘old’, ‘recent’ or ‘modern’, depending on collection date, were compared with those of domestic cats (F. catus) and hybrids, using Fisher Linear Discriminant Functions, Principal Component Analyses and Canonical Variates. A key is derived for classification of individual (unknown) skulls. All analyses indicate that ‘recent’ and ‘modern’ wildcats were different from those collected earlier, and implied marked changes in the rate of hydridization during this century. We suggest that the older wildcat populations were a relatively ‘pure’ form but more recent populations contain a significant hybrid component. Although there may be a trend towards re‐establishment of the earlier type, an alternative conclusion is that the ‘pure’ form of wildcat is effectively extinct in Scotland.