Linear Discriminant Analysis Approach Research Articles

Unsupervised real-time constrained linear discriminant analysis to hyperspectral image classification

We have proposed a constrained linear discriminant analysis (CLDA) approach for classifying the remotely sensed hyperspectral images. Its basic idea is to design an optimal linear transformation operator which can maximize the ratio of inter-class to intra-class distance while satisfying the constraint that the different class centers after transformation are aligned along different directions. Its major advantage over the traditional Fisher's linear discriminant analysis is that the classification can be achieved simultaneously with the transformation. The CLDA is a supervised approach, i.e., the class spectral signatures need to be known a priori. But, in practice, these informations may be difficult or even impossible to obtain. So in this paper we will extend the CLDA algorithm into an unsupervised version, where the class spectral signatures are to be directly generated from an unknown image scene. Computer simulation is used to evaluate how well the algorithm performs in terms of finding the pure signatures. We will also discuss how to implement the unsupervised CLDA algorithm in real-time for resolving the critical situations when the immediate data analysis results are required.

A multivariate statistical analysis of the developing human brain in preterm infants

Preterm delivery accounts for 5% of all deliveries and its consequences contribute to significant individual, medical, and social problems. The neuroanatomical substrates of these disorders are not known, but are essential for understanding mechanisms of causation, and developing strategies for intervention. In the recent years, multivariate pattern recognition methods that analyse all voxels simultaneously have been proposed to characterise the neuroanatomical differences between a reference group of magnetic resonance (MR) images and the population under investigation. Most of these techniques have overcome the difficulty of dealing with the inherent high dimensionality of 3D MR brain image data by using pre-processed segmented images or a small number of specific features. However, an intuitive way of mapping the classification results back into the original image domain for further interpretation remains challenging. In this paper, we propose the idea of using Principal Components Analysis (PCA) plus the maximum uncertainty Linear Discriminant Analysis (MLDA) approach to classify and analyse MR brain images that have been aligned with either affine or non-rigid registration techniques. This approach avoids the computation costs intrinsic to commonly used covariance-based optimisation processes for solving small sample size problems, resulting in a simple and efficient implementation for the maximisation and interpretation of the Fisher’s classification results. In order to demonstrate the effectiveness of the approach, we have used a neonatal MR brain data set that contains images of 93 preterm infants at term equivalent age and 20 term controls. Our results indicate that the two-stage linear framework makes clear the statistical differences between the control and preterm samples, showing a classification accuracy of 95.0% and 97.8% for the controls and preterms samples, respectively, using the leave-one-out method. Moreover, it provides a simple and intuitive method of visually analysing the differences between preterm infants at term equivalent age and the control group, such as differences in cerebrospinal fluid spaces, structure of the corpus callosum, and subtle differences in myelination.

Dimensionality reduction of image features using the canonical contextual correlation projection

A linear, discriminative, supervised technique for reducing feature vectors extracted from image data to a lower-dimensional representation is proposed. It is derived from classical linear discriminant analysis (LDA), extending this technique to cases where there is dependency between the output variables, i.e., the class labels, and not only between the input variables. (The latter can readily be dealt with in standard LDA.) The novel method is useful, for example, in supervised segmentation tasks in which high-dimensional feature vectors describe the local structure of the image. The principal idea is that where standard LDA merely takes into account a single class label for every feature vector, the new technique incorporates class labels of its neighborhood in the analysis as well. In this way, the spatial class label configuration in the vicinity of every feature vector is accounted for, resulting in a technique suitable for, e.g. image data. This extended LDA, that takes spatial label context into account, is derived from a formulation of standard LDA in terms of canonical correlation analysis. The novel technique is called the canonical contextual correlation projection (CCCP). An additional drawback of LDA is that it cannot extract more features than the number of classes minus one. In the two-class case this means that only a reduction to one dimension is possible. Our contextual LDA approach can avoid such extreme deterioration of the classification space and retain more than one dimension. The technique is exemplified on a pixel-based medical image segmentation problem in which it is shown that it may give significant improvement in segmentation accuracy.

Study of probabilistic neural networks to classify the active compounds in medicinal plants

Probabilistic neural networks (PNNs) were utilized for the classifications of 102 active compounds from diverse medicinal plants with anticancer activity against human rhinopharyngocele cell line KB. Molecular descriptors calculated from structure alone were used to represent molecular structures. A subset of the calculated descriptors selected using factor correlation analysis and forward stepwise regression was used to construct the prediction models. Linear discriminant analysis (LDA) was also utilized to construct the classification model to compare the results with those obtained by PNNs. The accuracy of the training set, the cross-validation set, and the test set given by PNNs and LDA were 100, 92.3, 90.9% and 71.8, 92.3, 54.5%, respectively, which indicated that the results obtained by PNNs agree well with the experimental values of these compounds and also revealed the superiority of PNNs over LDA approach for the classification of anticancer activities of compounds. The models built in this work would be of potential help in the design of novel and more potent anticancer agents.

Face recognition using direct, weighted linear discriminant analysis and modular subspaces

We present a modular linear discriminant analysis (LDA) approach for face recognition. A set of observers is trained independently on different regions of frontal faces and each observer projects face images to a lower-dimensional subspace. These lower-dimensional subspaces are computed using LDA methods, including a new algorithm that we refer to as direct, weighted LDA or DW-LDA. DW-LDA combines the advantages of two recent LDA enhancements, namely direct LDA (D-LDA) and weighted pairwise Fisher criteria. Each observer performs recognition independently and the results are combined using a simple sum-rule. Experiments compare the proposed approach to other face recognition methods that employ linear dimensionality reduction. These experiments demonstrate that the modular LDA method performs significantly better than other linear subspace methods. The results also show that D-LDA does not necessarily perform better than the well-known principal component analysis followed by LDA approach. This is an important and significant counterpoint to previously published experiments that used smaller databases. Our experiments also indicate that the new DW-LDA algorithm is an improvement over D-LDA.

Implementation of real-time constrained linear discriminant analysis to remote sensing image classification

In this paper, we investigate the practical implementation issues of the real-time constrained linear discriminant analysis (CLDA) approach for remotely sensed image classification. Specifically, two issues are to be resolved: (1) what is the best implementation scheme that yields lowest chip design complexity with comparable classification performance, and (2) how to extend CLDA algorithm for multispectral image classification. Two limitations about data dimensionality have to be relaxed. One is in real-time hyperspectral image classification, where the number of linearly independent pixels received for classification must be larger than the data dimensionality (i.e., the number of spectral bands) in order to generate a non-singular sample correlation matrix R for the classifier, and relaxing this limitation can help to resolve the aforementioned first issue. The other is in multispectral image classification, where the number of classes to be classified cannot be greater than the data dimensionality, and relaxing this limitation can help to resolve the aforementioned second issue. The former can be solved by introducing a pseudo inverse initiate of sample correlation matrix for R - 1 adaptation, and the latter is taken care of by expanding the data dimensionality via the operation of band multiplication. Experiments on classification performance using these modifications are conducted to demonstrate their feasibility. All these investigations lead to a detailed ASIC chip design scheme for the real-time CLDA algorithm suitable to both hyperspectral and multispectral images. The proposed techniques to resolving these two dimensionality limitations are instructive to the real-time implementation of several popular detection and classification approaches in remote sensing image exploitation.

Face recognition using direct, weighted linear discriminant analysis and modular subspaces

We present a modular linear discriminant analysis (LDA) approach for face recognition. A set of observers is trained independently on different regions of frontal faces and each observer projects face images to a lower-dimensional subspace. These lower-dimensional subspaces are computed using LDA methods, including a new algorithm that we refer to as direct, weighted LDA or DW-LDA. DW-LDA combines the advantages of two recent LDA enhancements, namely direct LDA (D-LDA) and weighted pairwise Fisher criteria. Each observer performs recognition independently and the results are combined using a simple sum-rule. Experiments compare the proposed approach to other face recognition methods that employ linear dimensionality reduction. These experiments demonstrate that the modular LDA method performs significantly better than other linear subspace methods. The results also show that D-LDA does not necessarily perform better than the well-known principal component analysis followed by LDA approach. This is an important and significant counterpoint to previously published experiments that used smaller databases. Our experiments also indicate that the new DW-LDA algorithm is an improvement over D-LDA.

Real-time constrained linear discriminant analysis to target detection and classification in hyperspectral imagery

In this paper, we present a constrained linear discriminant analysis (CLDA) approach to hyperspectral image detection and classification as well as its real-time implementation. The basic idea of CLDA is to design an optimal transformation matrix which can maximize the ratio of inter-class distance to intra-class distance while imposing the constraint that different class centers after transformation are along different directions such that different classes can be better separated. The solution turns out to be a constrained version of orthogonal subspace projection (OSP) implemented with a data whitening process. The CLDA approach can be applied to solve both detection and classification problems. In particular, by introducing color for display the classification is achieved with a single classified image where a pre-assigned color is used to display a specified class. The real-time implementation is also developed to meet the requirement of on-line image analysis when the immediate data assessment is critical. The experiments using HYDICE data demonstrate the strength of CLDA approach in discriminating the targets with subtle spectral difference.

Linear models for minimizing misclassification costs in bankruptcy prediction

AbstractThis paper illustrates how a misclassification cost matrix can be incorporated into an evolutionary classification system for bankruptcy prediction. Most classification systems for predicting bankruptcy have attempted to minimize misclassifications. The minimizing misclassification approach assumes that Type I and Type II error costs for misclassifications are equal. There is evidence that these costs are not equal and incorporating costs into the classification systems can lead to better and more desirable results. In this paper, we use the principles of evolution to develop and test a genetic algorithm (GA) based approach that incorporates the asymmetric Type I and Type II error costs. Using simulated and real‐life bankruptcy data, we compare the results of our proposed approach with three linear approaches: statistical linear discriminant analysis (LDA), a goal programming approach, and a GA‐based classification approach that does not incorporate the asymmetric misclassification costs. Our results indicate that the proposed approach, incorporating Type I and Type II error costs, results in lower misclassification costs when compared to LDA and GA approaches that do not incorporate misclassification costs. Copyright © 2001 John Wiley & Sons, Ltd.

Fully automated segmentation of cerebrospinal fluid in computed tomography

A method is presented for automated delineation and measurement of cerebrospinal fluid (CSF) regions in computed tomographic (CT) sections. Regions of skull and scalp are removed by using a linear discriminant analysis approach. Beam-hardening artifact is reduced by subtracting from each section the average radial intensity profile, characterized by a polynomial function. Remaining intensity gradients are suppressed by implementing CSF segmentation with a local thresholding technique based on maximum-entropy principles. CSF fractions from 12 regions of interest (ROIs) were measured in 10 patients with alcoholic Korsakoff syndrome and 9 normal volunteers. The same ROIs were also assessed by an interactive segmentation method, which enabled the operator to compensate for beam-hardening distortions by selecting suitable threshold values for each ROI. Both methods identified the same ROIs as displaying statistically significant differences between the two subject groups. However, interactive segmentation underestimated sulcal CSF by 20–70%, which was confirmed by applying both methods to CT scans of an anthropomorphic phantom. Hence, in contrast to interactive thresholding, unsupervised segmentation relies on firmly fixed criteria that reduce the influence of beam-hardening distortions and provide more objective results.

A linear goal programming model for classification with non-monotone attributes

Classification function analysis concerns separating two or more groups of objects in a data set and allocating new objects to previously defined groups. Usually, a set of attribute weights are estimated and the classification decision of an object is based on the weighted sum of its attribute scores. Statistical linear discriminant analysis, logistic regression, and linear programming approaches to classification problems have been proposed to address this problem. However, monotonicity of the attribute scores with respect to the likelihood of belonging to one specific group is presumed by these approaches. This may not be realistic in many applications. In this paper, a linear goal programming approach with the ability to capture the non-monotonicity of some attribute scores in classification problems is proposed. Classification performances of this approach and other classification approaches are evaluated by a simulation experiment. The results are very encouraging for the proposed approach.

A Nonmetric Approach to Linear Discriminant Analysis

Abstract A new nonmetric linear discriminant analysis approach is proposed that is based on the maximization of an index of separation differing from that used by the classical method. The possibility of choosing between Fisher's classical discriminant function and the one proposed here enables us to reduce the number of misclassifications for given data. The method is exemplified on empirical data and various simulations and is compared with the classical linear discriminant analysis.

A Nonmetric Approach to Linear Discriminant Analysis

Abstract A new nonmetric linear discriminant analysis approach is proposed that is based on the maximization of an index of separation differing from that used by the classical method. The possibility of choosing between Fisher's classical discriminant function and the one proposed here enables us to reduce the number of misclassifications for given data. The method is exemplified on empirical data and various simulations and is compared with the classical linear discriminant analysis.

A Robust Credit Screening Model Using Categorical Data

Motivated by an application in a public utility, the credit screening problem is re-examined from a decision theoretic viewpoint. The relationships between several alternative problem formulations are explored, and compared to the classical linear discriminant analysis (LDA) approach. Several mathematical programming based solution methods are proposed when the data are binary, and an efficient algorithm is developed for the case when the screening function must also have binary weights. Actual results of both the mathematical programming and LDA methods are presented and compared. The resulting mathematical programming rules are effective, robust, and flexible to administer. Practical advantages of the resulting “n out of N” type rules are discussed. These screening rules have been widely implemented by a major public utility and have resulted in substantial benefits to the utility and to the public.