Fisher Discriminant Criterion Research Articles

Discriminant and adaptive extensions to local temporal common spatial patterns

An important issue in brain–computer interfaces based on multi-channel EEG signals is to extract spatial filters. Local temporal common spatial patterns (LTCSP) is a recently developed approach to find spatial filters, which takes temporally local information into account. The formulation of LTCSP, however, is essentially a subspace decomposition technique. In this paper, we extend LTCSP from the aspects of discrimination and adaption. The discriminant extension is based on the Fisher discriminant criterion that considers both the between-class and the within-class scatters. By contrast, LTCSP considers maximizing the between-class scatter only. The adaptive extension uses sparse representation to specify the weights between samples in constructing the temporally local scatter matrices. Experiments on single-trial EEG classification confirm the effectiveness of the proposed methods.

Geometric Distribution Weight Information Modeled Using Radial Basis Function with Fractional Order for Linear Discriminant Analysis Method

Fisher linear discriminant analysis (FLDA) is a classic linear feature extraction and dimensionality reduction approach for face recognition. It is known that geometric distribution weight information of image data plays an important role in machine learning approaches. However, FLDA does not employ the geometric distribution weight information of facial images in the training stage. Hence, its recognition accuracy will be affected. In order to enhance the classification power of FLDA method, this paper utilizes radial basis function (RBF) with fractional order to model the geometric distribution weight information of the training samples and proposes a novel geometric distribution weight information based Fisher discriminant criterion. Subsequently, a geometric distribution weight information based LDA (GLDA) algorithm is developed and successfully applied to face recognition. Two publicly available face databases, namely, ORL and FERET databases, are selected for evaluation. Compared with some LDA-based algorithms, experimental results exhibit that our GLDA approach gives superior performance.

Enhanced fisher discriminant criterion for image recognition

Many previous studies have shown that image recognition can be significantly improved by Fisher linear discriminant analysis (FLDA) technique. However, FLDA ignores the variation of data points from the same class, which characterizes the most important modes of variability of patterns and helps to improve the generalization capability of FLDA. Thus, the performance of FLDA on testing data is not good enough. In this paper, we propose an enhanced fisher discriminant criterion (EFDC). EFDC explicitly considers the intra-class variation and incorporates the intra-class variation into the Fisher discriminant criterion to build a robust and efficient dimensionality reduction function. EFDC obtains a subspace which best detects the discriminant structure and simultaneously preserves the modes of variability of patterns, which will result in stable intraclass representation. Experimental results on four image database show a clear improvement over the results of FLDA-based methods.

Top scoring pairs for feature selection in machine learning and applications to cancer outcome prediction

BackgroundThe widely used k top scoring pair (k-TSP) algorithm is a simple yet powerful parameter-free classifier. It owes its success in many cancer microarray datasets to an effective feature selection algorithm that is based on relative expression ordering of gene pairs. However, its general robustness does not extend to some difficult datasets, such as those involving cancer outcome prediction, which may be due to the relatively simple voting scheme used by the classifier. We believe that the performance can be enhanced by separating its effective feature selection component and combining it with a powerful classifier such as the support vector machine (SVM). More generally the top scoring pairs generated by the k-TSP ranking algorithm can be used as a dimensionally reduced subspace for other machine learning classifiers.ResultsWe developed an approach integrating the k-TSP ranking algorithm (TSP) with other machine learning methods, allowing combination of the computationally efficient, multivariate feature ranking of k-TSP with multivariate classifiers such as SVM. We evaluated this hybrid scheme (k-TSP+SVM) in a range of simulated datasets with known data structures. As compared with other feature selection methods, such as a univariate method similar to Fisher's discriminant criterion (Fisher), or a recursive feature elimination embedded in SVM (RFE), TSP is increasingly more effective than the other two methods as the informative genes become progressively more correlated, which is demonstrated both in terms of the classification performance and the ability to recover true informative genes. We also applied this hybrid scheme to four cancer prognosis datasets, in which k-TSP+SVM outperforms k-TSP classifier in all datasets, and achieves either comparable or superior performance to that using SVM alone. In concurrence with what is observed in simulation, TSP appears to be a better feature selector than Fisher and RFE in some of the cancer datasetsConclusionsThe k-TSP ranking algorithm can be used as a computationally efficient, multivariate filter method for feature selection in machine learning. SVM in combination with k-TSP ranking algorithm outperforms k-TSP and SVM alone in simulated datasets and in some cancer prognosis datasets. Simulation studies suggest that as a feature selector, it is better tuned to certain data characteristics, i.e. correlations among informative genes, which is potentially interesting as an alternative feature ranking method in pathway analysis.

Identifying Concealed Information Using Wavelet Feature Extraction and Support Vector Machine

In this paper, a new approach based on wavelet feature extraction and support vector machine (SVM) is proposed to identify concealed information. Firstly, the wavelet coefficients of event related potential (ERP) in delta, theta, alpha and beta bands are extracted as useful features of brain activity responded to different stimulus information. Next, a Fisher discriminant criterion is applied to reduce the feature vector dimensions. Finally, a SVM classifier is employed to classify the data and the leave-one-out cross validation method is used for accuracy assessment. For the evaluation of the method, 16 subjects went through the designed CIT paradigm and their respective brain signals were recorded. The experimental results show that SVM classifier can effectively differentiate between concealed information and irrelevant information, and it achieves the maximum classification accuracy of 90.63%. The investigation also suggests that the wavelet decomposition coefficient can reflect more comprehensive time-frequency information correlating with deception, which can effectively distinguish concealed information between irrelevant information.

Cluster-fusion recognition method for rivet lines based on Fisher discriminant criterion function

A cluster-fusion recognition method for rivet lines was proposed based on Fisher discriminant criterion function.The edge points were extracted using Canny operator,and processed by morphology in order to get the location of the centroid.Then,any two centroids were combined to construct the collection of line segments which were classified by Fisher criterion.According to the principle that the distance from the segments on one rivet line to origin point is similar,the segment-vectors were fused to fit the rivet line.The experimental results show that its accuracy and robustness are improved,which can meet the requirements of real-time detection.

A Rank-One Update Algorithm for Fast Solving Kernel Foley–Sammon Optimal Discriminant Vectors

Discriminant analysis plays an important role in statistical pattern recognition. A popular method is the Foley-Sammon optimal discriminant vectors (FSODVs) method, which aims to find an optimal set of discriminant vectors that maximize the Fisher discriminant criterion under the orthogonal constraint. The FSODVs method outperforms the classic Fisher linear discriminant analysis (FLDA) method in the sense that it can solve more discriminant vectors for recognition. Kernel Foley-Sammon optimal discriminant vectors (KFSODVs) is a nonlinear extension of FSODVs via the kernel trick. However, the current KFSODVs algorithm may suffer from the heavy computation problem since it involves computing the inverse of matrices when solving each discriminant vector, resulting in a cubic complexity for each discriminant vector. This is costly when the number of discriminant vectors to be computed is large. In this paper, we propose a fast algorithm for solving the KFSODVs, which is based on rank-one update (ROU) of the eigensytems. It only requires a square complexity for each discriminant vector. Moreover, we also generalize our method to efficiently solve a family of optimally constrained generalized Rayleigh quotient (OCGRQ) problems which include many existing dimensionality reduction techniques. We conduct extensive experiments on several real data sets to demonstrate the effectiveness of the proposed algorithms.

Extraction of Combined Features from Global/Local Statistics of Visual Words Using Relevant Operations

This paper presents a combined feature extraction method to improve the performance of bag-of-features image classification. We apply 10 relevant operations to global/local statistics of visual words. Because the pairwise combination of visual words is large, we apply feature selection methods including fisher discriminant criterion and L1-SVM. The effectiveness of the proposed method is confirmed through the experiment.

Heteroscedastic Feature Extraction for Texture Classification

Linear discriminant analysis (LDA) is a well-known feature extraction method in statistical pattern recognition community. The basic idea of LDA is to find a set of optimal discriminant vectors that maximize the Fisher's discriminant criterion. One major problem of the Fisher's criterion is that its discriminant performance largely depends on the class mean differences. Hence, the LDA method may not work well as for the case of the heteroscedastic problem since it can not make use of the discriminant information from the class covariance differences. To this end, in this paper we propose a new discriminant criterion consisting of both class mean and covariance differences to replace the Fisher's criterion. Based on the new discriminant criterion, we propose a heteroscedastic extension method of linear discriminant analysis (namely the HELDA method). We also propose an approximate solution method for HELDA (AHELDA) via matrices joint diagonalization (JD) to reduce the computational complexity. The extensive experiments on texture classification confirm the better classification performance of our method.

Feature extraction using fuzzy inverse FDA

This paper proposes a new method of feature extraction and recognition, namely, the fuzzy inverse Fisher discriminant analysis (FIFDA) based on the inverse Fisher discriminant criterion and fuzzy set theory. In the proposed method, a membership degree matrix is calculated using FKNN, then the membership degree is incorporated into the definition of the between-class scatter matrix and within-class scatter matrix to get the fuzzy between-class scatter matrix and fuzzy within-class scatter matrix. Experimental results on the ORL, FERET face databases and pulse signal database show that the new method outperforms Fisherface, fuzzy Fisherface and inverse Fisher discriminant analysis.

Tensor Rank One Discriminant Analysis—A convergent method for discriminative multilinear subspace selection

This paper proposes Tensor Rank One Discriminant Analysis (TR1DA) in which general tensors are input for pattern classification. TR1DA is based on Differential Scatter Discriminant Criterion (DSDC) and Tensor Rank One Analysis (TR1A). DSDC is a generalization of the Fisher discriminant criterion. It ensures convergence during training stage. TR1A is a method for adapting general tensors as input to DSDC. The benefits of TR1DA include: (1) a natural way of representing data without losing structure information, i.e., the information about the relative positions of pixels or regions; (2) a reduction in the small sample size problem which occurs in conventional discriminant learning because the number of training samples is much less than the dimensionality of the feature space; (3) a better convergence during the training procedure. We use a graph-embedding framework to generalize TR1DA in manifold learning-based feature selection algorithms, such as locally linear embedding, ISOMAP, and the Laplace eigenmap. We also kernelize TR1DA to nonlinear problems. TR1DA is then demonstrated to outperform traditional subspace methods, such as principal component analysis and linear discriminant analysis.

A kernel-based method for pattern extraction in random process signals

In many applications, one is interested to detect certain patterns in random process signals. We consider a class of random process signals which contain sub-similarities at random positions representing the texture of an object. Those repetitive parts may occur in speech, musical pieces and sonar signals. We suggest a warped time-resolved spectrum kernel for extracting the subsequence similarity in time series in general, and as an example in biosonar signals. Having a set of those kernels for similarity extraction in different size of subsequences, we propose a new method to find an optimal linear combination of those kernels. We formulate the optimal kernel selection via maximizing the kernel Fisher discriminant (KFD) criterion and use Mesh Adaptive Direct Search (MADS) method to solve the optimization problem. Our method is used for biosonar landmark classification with promising results.

Effects of phenoterol and histamine on lung function in patients with diabetes mellitus

The purpose of the study was to investigate airway reactivity in patients with diabetes mellitus. We examined 68 patients with diabetes mellitus of 16 to 66 years of age. Of them, 29 patients (20 with diabetes mellitus type 1 and 9 with diabetes mellitus type 2) were tested with phenoterol (the 1st group). The 2nd group patients (12 with diabetes mellitus type 1 and 27 with diabetes mellitus type 2) were examined with histamine challenge test. Bronchodilator test (ΔFEV 1 > 200 ml or > 12 % of the baseline value) was positive in 6 (20.7 %) patients, five of them had diabetes type 1. The histamine challenge test was positive in 9 patients with diabetes type 2 (23.1 %). This was an evidence of significant bronchial hyperreactivity. A qualitative analysis using Fisher's discriminant criterion revealed the following correlations: 1) between bronchial hyperreactivity and a type of diabetes mellitus; 2) between bronchial hyperreactivity and administration of oral antidiabetic drugs. Correlations between bronchial hyperreactivity, body mass index, duration of the disease and late complications of diabetes mellitus were not revealed. In conclusion, the positive bronchodilator test predominated in patients with diabetes mellitus type 1 and bronchial hyperreactivity for histamine was mainly detected in patients with diabetes mellitus type 2. This could be considered as an evidence of different mechanisms of bronchoconstriction in patients with diabetes mellitus types 1 and 2. Generally, susceptibility to bronchoconstriction may be an unfavorable factor for occurrence of nonspecific lung disease in patients with diabetes mellitus.

Optimizing the data-dependent kernel under a unified kernel optimization framework

The kernel functions play a central role in kernel methods, accordingly over the years the optimization of kernel functions has been a promising research area. Ideally Fisher discriminant criteria can be used as an objective function to optimize the kernel function to augment the margin between different classes. Unfortunately, Fisher criteria are optimal only in the case that all the classes are generated from underlying multivariate normal distributions of common covariance matrix but different means and each class is expressed by a single cluster. Due to the assumptions, Fisher criteria obviously are not a suitable choice as a kernel optimization rule in some applications such as the multimodally distributed data. In order to solve this problem, recently many improved discriminant criteria (DC) have been also developed. Therefore, to apply these discriminant criteria to kernel optimization, in this paper based on a data-dependent kernel function we propose a unified kernel optimization framework, which can use any discriminant criteria formulated in a pairwise manner as the objective functions. Under the kernel optimization framework, to employ different discriminant criteria, one has to only change the corresponding affinity matrices without having to resort to any complex derivations in feature space. Experimental results based on some benchmark data demonstrate the efficiency of our method.

Face recognition based on a novel linear discriminant criterion

As an effective technique for feature extraction and pattern classification Fisher linear discriminant (FLD) has been successfully applied in many fields. However, for a task with very high-dimensional data such as face images, conventional FLD technique encounters a fundamental difficulty caused by singular within-class scatter matrix. To avoid the trouble, many improvements on the feature extraction aspect of FLD have been proposed. In contrast, studies on the pattern classification aspect of FLD are quiet few. In this paper, we will focus our attention on the possible improvement on the pattern classification aspect of FLD by presenting a novel linear discriminant criterion called maximum scatter difference (MSD). Theoretical analysis demonstrates that MSD criterion is a generalization of Fisher discriminant criterion, and is the asymptotic form of discriminant criterion: large margin linear projection. The performance of MSD classifier is tested in face recognition. Experiments performed on the ORL, Yale, FERET and AR databases show that MSD classifier can compete with top-performance linear classifiers such as linear support vector machines, and is better than or equivalent to combinations of well known facial feature extraction methods, such as eigenfaces, Fisherfaces, orthogonal complementary space, nullspace, direct linear discriminant analysis, and the nearest neighbor classifier.

Feature selection for image retrieval

A new image retrieval method based on dynamical interesting features selection is proposed. The method uses the Fisher discriminant criterion to dynamically select optimal interesting features from the images and uses them to retrieve the similar images in the database. Experimental results show that it has better retrieval performance than the DPF based and classical Minkowski distance based methods.

Face recognition based on 2D Fisherface approach

Two-dimensional (2D) discrimination analysis using methods such as 2D PCA and Image LDA is of interest in face recognition because it extracts discriminative features faster than one-dimensional (1D) discrimination analysis. However, existing 2D methods generally use more discriminative features and take longer to test than 1D methods. 2D PCA in particular cannot make full use of the Fisher discriminant criterion. Image LDA also has drawbacks in that it cannot perform 2D principal component analysis and discards components with poor discriminative capabilities. In addition, existing 2D methods cannot provide an automatic strategy to choose 2D principal components or discriminant vectors. In this paper, we propose 2D Fisherface, a novel discrimination approach that combines the two-stage “ PCA + LDA ” strategy and 2D discrimination techniques. It can extract face discriminative features by automatically selecting two-dimensional principal components and discriminant vectors. Using the AR database as the test data, it is shown that the proposed approach is faster and more effective than several representative 1D and 2D discrimination methods.

Weighted maximum margin discriminant analysis with kernels

A new kernel-based learning algorithm, called kernel weighted maximum margin discriminant analysis (KWMMDA), is presented in this paper. Different from the previous discriminant analysis algorithms based on the traditional Fisher discriminant criterion, KWMMDA is derived based on a new discriminant criterion, called weighted maximum margin criterion (WMMC). The better performance of KWMMDA is demonstrated by experiments on real data set.

Modified linear discriminant analysis

In this paper, a modified Fisher linear discriminant analysis (FLDA) is proposed and aims to not only overcome the rank limitation of FLDA, that is, at most only finding a discriminant vector for 2-class problem based on Fisher discriminant criterion, but also relax singularity of the within-class scatter matrix and finally improves classification performance of FLDA. Experiments on nine publicly available datasets show that the proposed method has better or comparable performance on all the datasets than FLDA.

A generalized Foley–Sammon transform based on generalized fisher discriminant criterion and its application to face recognition

As the generalization of Fisher discriminant criterion, in this paper, the conception of the generalized Fisher discriminant criterion is presented. On the basis of the generalized Fisher discriminant criterion, the generalized Foley–Sammon transform (GFST) is proposed. The main difference between the GFST and the Foley–Sammon transform (FST) is that the sample set has the minimum within-class scatter and the maximum between-class scatter in the subspace spanned by all discriminant vectors constituting GFST while the sample set has these properties only on the one-dimensional subspace spanned by each discriminant vector constituting FST, that is, the transformed sample set by GFST has the best discriminant ability in global sense while FST has this property only in part sense. To calculate the GFST, an iterative algorithm is proposed, which is proven to converge to the precise solution. The speed and errors of the iterative procedure are also analyzed in detail. Lastly, our method is applied to facial image recognition, and the experimental results show that present method is superior to the existing methods in terms of correct classification rate.