Nonlinear Discriminant Analysis Research Articles

Class-Specific Kernel Fusion of Multiple Descriptors for Face Verification Using Multiscale Binarised Statistical Image Features

This paper addresses face verification in unconstrained settings. For this purpose, first, a nonlinear binary class-specific kernel discriminant analysis classifier (CS-KDA) based on spectral regression kernel discriminant analysis is proposed. By virtue of the two-class formulation, the proposed CS-KDA approach offers a number of desirable properties such as specificity of the transformation for each subject, computational efficiency, simplicity of training, isolation of the enrolment of each client from others and increased speed in probe testing. Using the proposed CS-KDA approach, a regional discriminative face image representation based on a multiscale variant of the binarized statistical image features is proposed next. The proposed component-based representation when coupled with the dense pixel-wise alignments provided by a symmetric MRF matching model reduces the sensitivity to misalignments and pose variations, gauging the similarity more effectively. Finally, the discriminative representation is combined with two other effective image descriptors, namely the multiscale local binary patterns and the multiscale local phase quantization histograms via a kernel fusion approach to further enhance system accuracy. The experimental evaluation of the proposed methodology on challenging databases demonstrates its advantage over other methods.

Kernel Reference Discriminant Analysis

Linear Discriminant Analysis (LDA) and its nonlinear version Kernel Discriminant Analysis (KDA) are well-known and widely used techniques for supervised feature extraction and dimensionality reduction. They determine an optimal discriminant space for (non)linear data projection based on certain assumptions, e.g. on using normal distributions (either on the input or in the kernel space) for each class and employing class representation by the corresponding class mean vectors. However, there might be other vectors that can be used for classes representation, in order to increase class discrimination in the resulted feature space. In this paper, we propose an optimization scheme aiming at the optimal class representation, in terms of Fisher ratio maximization, for nonlinear data projection. Compared to the standard approach, the proposed optimization scheme increases class discrimination in the reduced-dimensionality feature space and achieves higher classification rates in publicly available data sets.

TU-F-18C-09: Mammogram Surveillance Using Texture Analysis for Breast Cancer Patients After Radiation Therapy

Purpose: To study the feasibility of applying cancer risk model established from treated patients to predict the risk of recurrence on follow-up mammography after radiation therapy for both ipsilateral and contralateral breast. Methods: An extensive set of textural feature functions was applied to a set of 196 Mammograms from 50 patients. 56 Mammograms from 28 patients were used as training set, 44 mammograms from 22 patients were used as test set and the rest were used for prediction. Feature functions include Histogram, Gradient, Co-Occurrence Matrix, Run-Length Matrix and Wavelet Energy. An optimum subset of the feature functions was selected by Fisher Coefficient (FO) or Mutual Information (MI) (up to top 10 features) or a method combined FO, MI and Principal Component (FMP) (up to top 30 features). One-Nearest Neighbor (1-NN), Linear Discriminant Analysis (LDA) and Nonlinear Discriminant Analysis (NDA) were utilized to build a risk model of breast cancer from the training set of mammograms at the time of diagnosis. The risk model was then used to predict the risk of recurrence from mammogram taken one year and three years after RT. Results: FPM with NDA has the best classification power in classifying the training set of the mammogram with lesions versus those without lesions. The model of FPM with NDA achieved a true positive (TP) rate of 82% compared to 45.5% of using FO with 1-NN. The best false positive (FP) rates were 0% and 3.6% in contra-lateral breast of 1-year and 3-years after RT, and 10.9% in ipsi-lateral breast of 3-years after RT. Conclusion: Texture analysis offers high dimension to differentiate breast tissue in mammogram. Using NDA to classify mammogram with lesion from mammogram without lesion, it can achieve rather high TP and low FP in the surveillance of mammogram for patient with conservative surgery combined RT.

Characterization of local complex structures in a recurrence plot to improve nonlinear dynamic discriminant analysis

Structures in recurrence plots (RPs), preserving the rich information of nonlinear invariants and trajectory characteristics, have been increasingly analyzed in dynamic discrimination studies. The conventional analysis of RPs is mainly focused on quantifying the overall diagonal and vertical line structures through a method, called recurrence quantification analysis (RQA). This study extensively explores the information in RPs by quantifying local complex RP structures. To do this, an approach was developed to analyze the combination of three major RQA variables: determinism, laminarity, and recurrence rate (DLR) in a metawindow moving over a RP. It was then evaluated in two experiments discriminating (1) ideal nonlinear dynamic series emulated from the Lorenz system with different control parameters and (2) data sets of human heart rate regulations with normal sinus rhythms (n = 18) and congestive heart failure (n = 29). Finally, the DLR was compared with seven major RQA variables in terms of discriminatory power, measured by standardized mean difference (DSMD). In the two experiments, DLR resulted in the highest discriminatory power with DSMD = 2.53 and 0.98, respectively, which were 7.41 and 2.09 times the best performance from RQA. The study also revealed that the optimal RP structures for the discriminations were neither typical diagonal structures nor vertical structures. These findings indicate that local complex RP structures contain some rich information unexploited by RQA. Therefore, future research to extensively analyze complex RP structures would potentially improve the effectiveness of the RP analysis in dynamic discrimination studies.

Clustering seismic activities using linear and nonlinear discriminant analysis

Identification and classification of different seismo-tectonic events with similar characteristics in a region of interest is one of the most important subjects in seismic hazard studies. In this study, linear and nonlinear discriminant analyses have been applied to classify seismic events in the vicinity of Istanbul. The vertical components of the digital velocity seismograms are used for seismic events with magnitude (Md) between 1.8 and 3.0 that occurred between 2001 and 2004. Two, time dependent parameters, complexity and S/P peak amplitude ratio are selected as predictands. Linear, quadratic, diaglinear and diagquadratic discriminant functions are investigated. Accuracy of methods with an additional adjusted quadratic models are 96.6%, 96.6%, 95.5%, 96.6%, and 97.6%, respectively with a various misclassified rate for each class. The performances of models are justified with cross validation and resubstitution error. Although all models remarkably well performed, adjusted quadratic function achieved the best success rate with just 4 misclassified events out of 179, even better compared to complex methods such as, self organizing method, k-means, Gaussion mixture models that applied to same dataset in literature.

Using global maps to predict the risk of dengue in Europe

This article attempts to quantify the risk to Europe of dengue, following the arrival and spread there of one of dengue's vector species Aedes (Stegomyia) albopictus. A global risk map for dengue is presented, based on a global database of the occurrence of this disease, derived from electronic literature searches. Remotely sensed satellite data (from NASA's MODIS series), interpolated meteorological data, predicted distribution maps of dengue's two main vector species, Aedes aegypti and Aedes albopictus, a digital elevation surface and human population density data were all used as potential predictor variables in a non-linear discriminant analysis modelling framework. One hundred bootstrap models were produced by randomly sub-sampling three different training sets for dengue fever, severe dengue (i.e. dengue haemorrhagic fever, DHF) and all-dengue, and output predictions were averaged to produce a single global risk map for each type of dengue. This paper concentrates on the all-dengue models. Key predictor variables were various thermal data layers, including both day- and night-time Land Surface Temperature, human population density, and a variety of rainfall variables. The relative importance of each may be shown visually using rainbow files and quantitatively using a ranking system. Vegetation Index variables (a common proxy for humidity or saturation deficit) were rarely chosen in the models. The kappa index of agreement indicated an excellent (dengue haemorrhagic fever, Cohen's kappa=0.79±0.028, AUC=0.96±0.007) or good fit of the top ten models in each series to the data (Cohen's kappa=0.73±0.018, AUC=0.94±0.007 for dengue fever and 0.74±0.017, AUC=0.95±0.005 for all dengue). The global risk map predicts widespread dengue risk in SE Asia and India, in Central America and parts of coastal South America, but in relatively few regions of Africa. In many cases these are less extensive predictions than those of other published dengue risk maps and arise because of the key importance of high human population density for the all-dengue risk maps produced here. Three published dengue risk maps are compared using the Fleiss kappa index, and are shown to have only fair agreement globally (Fleiss kappa=0.377). Regionally the maps show greater (but still only moderate) agreement in SE Asia (Fleiss kappa=0.566), fair agreement in the Americas (Fleiss kappa=0.325) and only slight agreement in Africa (Fleiss kappa=0.095). The global dengue risk maps show that very few areas of rural Europe are presently suitable for dengue, but several major cities appear to be at some degree of risk, probably due to a combination of thermal conditions and high human population density, the top two variables in many models. Mahalanobis distance images were produced of Europe and the southern United States showing the distance in environmental rather than geographical space of each site from any site where dengue currently occurs. Parts of Europe are quite similar in Mahalanobis distance terms to parts of the southern United States, where dengue occurred in the recent past and which remain environmentally suitable for it. High standards of living rather than a changed environmental suitability keep dengue out of the USA. The threat of dengue to Europe at present is considered to be low but sufficiently uncertain to warrant monitoring in those areas of greatest predicted environmental suitability, especially in northern Italy and parts of Austria, Slovenia and Croatia, Bosnia and Herzegovina, Serbia and Montenegro, Albania, Greece, south-eastern France, Germany and Switzerland, and in smaller regions elsewhere.

Mapping Environmental Suitability for Malaria Transmission, Greece

During 2009–2012, Greece experienced a resurgence of domestic malaria transmission. To help guide malaria response efforts, we used spatial modeling to characterize environmental signatures of areas suitable for transmission. Nonlinear discriminant analysis indicated that sea-level altitude and land-surface temperature parameters are predictive in this regard.

Photon level chemical classification using digital compressive detection

A key bottleneck to high-speed chemical analysis, including hyperspectral imaging and monitoring of dynamic chemical processes, is the time required to collect and analyze hyperspectral data. Here we describe, both theoretically and experimentally, a means of greatly speeding up the collection of such data using a new digital compressive detection strategy. Our results demonstrate that detecting as few as ∼10 Raman scattered photons (in as little time as ∼30μs) can be sufficient to positively distinguish chemical species. This is achieved by measuring the Raman scattered light intensity transmitted through programmable binary optical filters designed to minimize the error in the chemical classification (or concentration) variables of interest. The theoretical results are implemented and validated using a digital compressive detection instrument that incorporates a 785nm diode excitation laser, digital micromirror spatial light modulator, and photon counting photodiode detector. Samples consisting of pairs of liquids with different degrees of spectral overlap (including benzene/acetone and n-heptane/n-octane) are used to illustrate how the accuracy of the present digital compressive detection method depends on the correlation coefficients of the corresponding spectra. Comparisons of measured and predicted chemical classification score plots, as well as linear and non-linear discriminant analyses, demonstrate that this digital compressive detection strategy is Poisson photon noise limited and outperforms total least squares-based compressive detection with analog filters.

Optimal subset-division based discrimination and its kernelization for face and palmprint recognition

Discriminant analysis is effective in extracting discriminative features and reducing dimensionality. In this paper, we propose an optimal subset-division based discrimination (OSDD) approach to enhance the classification performance of discriminant analysis technique. OSDD first divides the sample set into several subsets by using an improved stability criterion and K-means algorithm. We separately calculate the optimal discriminant vectors from each subset. Then we construct the projection transformation by combining the discriminant vectors derived from all subsets. Furthermore, we provide a nonlinear extension of OSDD, that is, the optimal subset-division based kernel discrimination (OSKD) approach. It employs the kernel K-means algorithm to divide the sample set in the kernel space and obtains the nonlinear projection transformation. The proposed approaches are applied to face and palmprint recognition, and are examined using the AR and FERET face databases and the PolyU palmprint database. The experimental results demonstrate that the proposed approaches outperform several related linear and nonlinear discriminant analysis methods.

Feature space locality constraint for kernel based nonlinear discriminant analysis

Subspace learning is an important approach in pattern recognition. Nonlinear discriminant analysis (NDA), due to its capability of describing nonlinear manifold structure of samples, is considered to be more powerful to undertake classification tasks in image related problems. In kernel based NDA representation, there are three spaces involved, i.e., original data space, implicitly mapped high dimension feature space and the target low dimension subspace. Existing methods mainly focus on the information in original data space to find the most discriminant low dimension subspace. The implicit high dimension feature space plays a role that connects the original space and the target subspace to realize the nonlinear dimension reduction, but the sample geometric structure information in feature space is not involved. In this work, we try to utilize and explore this information. Specifically, the locality information of samples in feature space is modeled and integrated into the traditional kernel based NDA methods. In this way, both the sample distributions in original data space and the mapped high dimension feature space are modeled and more information is expected to be explored to improve the discriminative ability of the subspace. Two algorithms, named FSLC-KDA and FSLC-KSR, are presented. Extensive experiments on ORL, Extended-YaleB, PIE, Multi-PIE and FRGC databases validate the efficacy of the proposed method.

NONLINEAR SUBPROFILE SPACE FOR RADAR HRRP RECOGNITION

In this paper, a novel approach, namely nonlinear subprofile space (NSS), is proposed for radar target recognition using high-resolution range profile (HRRP). First, the HRRP samples are mapped into a high-dimensional feature space using nonlinear mapping. Second, the nonlinear features, namely nonlinear subprofiles, are extracted by nonlinear discriminant analysis. Then, for each class, the nonlinear subprofile space is formed using all the training nonlinear subprofiles of class. Finally, the minimum hyperplane distance classifier (MHDC) is used for classification. The aim of NSS method is to represent the feature area of target using nonlinear subprofile space, and effectively measure the distance between the test HRRP and feature area via minimum hyperplane distance (MHD) metric. The experimental results of measured data show that the proposed method has better performance of recognition than KPCA and KFDA.

Scar tissue classification using nonlinear optical microscopy and discriminant analysis

This paper addresses the scar tissue maturation process that occurs stepwise, and calls for reliable classification. The structure of collagen imaged by nonlinear optical microscopy (NLOM) in post-burn hypertrophic and mature scar, as well as in normal skin, appeared to distinguish these maturation steps. However, it was a discrimination analysis, demonstrated here, that automated and quantified the scar tissue maturation process. The achieved scar classification accuracy was as high as 96%. The combination of NLOM and discrimination analysis is believed to be instrumental in gaining insight into the scar formation, for express diagnosis of scar and surgery planning.

Kernel discriminant analysis for regression problems

In this paper, we propose a nonlinear feature extraction method for regression problems to reduce the dimensionality of the input space. Previously, a feature extraction method LDAr, a regressional version of the linear discriminant analysis, was proposed. In this paper, LDAr is generalized to a nonlinear discriminant analysis by using the so-called kernel trick. The basic idea is to map the input space into a high-dimensional feature space where the variables are nonlinear transformations of input variables. Then we try to maximize the ratio of distances of samples with large differences in the target value and those with small differences in the target value in the feature space. It is well known that the distribution of face images, under a perceivable variation in translation, rotation, and scaling, is highly nonlinear and the face alignment problem is a complex regression problem. We have applied the proposed method to various regression problems including face alignment problems and achieved better performances than those of conventional linear feature extraction methods.

A Respiratory Sensitization Study by a New Quantitative Structure-Activity Relationships (QSAR)

New respiratory sensitization positive/negative prediction models with discriminant functions were generated and parameter analyses were discussed on the basis of QSAR technology. Samples used in this research were selected from the list of European Chemical Bureau (ECB): R42, R42/43 for positive samples (respiratory sensitizers) and from the classification results of the Japanese Inter-ministerial Committee for negative respiratory sensitizers (controls). A total of 214 compounds (61 positive sensitizers and 153 negative sensitizers) were used in this study. Parameters were generated from 2-D and 3-D structures of compound. All of the approximately 800 parameters generated were reduced to 12 parameter set by feature selection. Various linear and non-linear discriminant analysis methods were applied using the parameter set. All data analyses were performed using ADMEWORKS/ ModelBuilder software. Perfect classification ratios (100%) were achieved using Iterative Least Squares (ILS) and AdaBoost. The highest prediction ratio of 97.2% by leave-one-out cross-validation was achieved with Support Vector Machine (SVM). This model is applicable to initial prediction of respiratory sensitization.

Gaussian Based Classification with Application to the Iris Data Set

AbstractThe paper describes a novel method to group data points into clusters. Our method is derived from the assumption of Gaussian distribution of the points. We apply the clustering method to the well-known Iris flower data set and we suggest a nonlinear discriminant analysis of the data for a more sophisticated classification. To the best knowledge of the authors this paper shows the best classification result for the Iris data so far.

Improved sub‐cellular resolution via simultaneous analysis of organelle proteomics data across varied experimental conditions

Spatial organisation of proteins according to their function plays an important role in the specificity of their molecular interactions. Emerging proteomics methods seek to assign proteins to sub-cellular locations by partial separation of organelles and computational analysis of protein abundance distributions among partially separated fractions. Such methods permit simultaneous analysis of unpurified organelles and promise proteome-wide localisation in scenarios wherein perturbation may prompt dynamic re-distribution. Resolving organelles that display similar behavior during a protocol designed to provide partial enrichment represents a possible shortcoming. We employ the Localisation of Organelle Proteins by Isotope Tagging (LOPIT) organelle proteomics platform to demonstrate that combining information from distinct separations of the same material can improve organelle resolution and assignment of proteins to sub-cellular locations. Two previously published experiments, whose distinct gradients are alone unable to fully resolve six known protein-organelle groupings, are subjected to a rigorous analysis to assess protein-organelle association via a contemporary pattern recognition algorithm. Upon straightforward combination of single-gradient data, we observe significant improvement in protein-organelle association via both a non-linear support vector machine algorithm and partial least-squares discriminant analysis. The outcome yields suggestions for further improvements to present organelle proteomics platforms, and a robust analytical methodology via which to associate proteins with sub-cellular organelles.

Generalized nonlinear discriminant analysis and its small sample size problems

This paper develops a generalized nonlinear discriminant analysis (GNDA) method and deals with its small sample size (SSS) problems. GNDA is a nonlinear extension of linear discriminant analysis (LDA), while kernel Fisher discriminant analysis (KFDA) can be regarded as a special case of GNDA. In LDA, an under sample problem or a small sample size problem occurs when the sample size is less than the sample dimensionality, which will result in the singularity of the within-class scatter matrix. Due to a high-dimensional nonlinear mapping in GNDA, small sample size problems arise rather frequently. To tackle this issue, this research presents five different schemes for GNDA to solve the SSS problems. Experimental results on real-world data sets show that these schemes for GNDA are very effective in tackling small sample size problems.

MRI Texture Analysis in Multiple Sclerosis: Toward a Clinical Analysis Protocol

Magnetic resonance imaging (MRI)-based texture analysis has been shown to be effective in classifying multiple sclerosis lesions. Regarding the clinical use of texture analysis in multiple sclerosis, our intention was to show which parts of the analysis are sensitive to slight changes in textural data acquisition and which steps tolerate interference. The MRI datasets of 38 multiple sclerosis patients were used in this study. Three imaging sequences were compared in quantitative analyses, including a comparison of anatomical levels of interest, variance between sequential slices and two methods of region of interest drawing. We focused on the classification of white matter and multiple sclerosis lesions in determining the discriminatory power of textural parameters. Analyses were run with MaZda software for texture analysis, and statistical tests were performed for raw parameters. MRI texture analysis based on statistical, autoregressive-model and wavelet-derived texture parameters provided an excellent distinction between the image regions corresponding to multiple sclerosis plaques and white matter or normal-appearing white matter with high accuracy (nonlinear discriminant analysis 96%-100%). There were no significant differences in the classification results between imaging sequences or between anatomical levels. Standardized regions of interest were tolerant of changes within an anatomical level when intra-tissue variance was tested. The MRI texture analysis protocol with fixed imaging sequence and anatomical levels of interest shows promise as a robust quantitative clinical means for evaluating multiple sclerosis lesions.

Effect of slice thickness on brain magnetic resonance image texture analysis

BackgroundThe accuracy of texture analysis in clinical evaluation of magnetic resonance images depends considerably on imaging arrangements and various image quality parameters. In this paper, we study the effect of slice thickness on brain tissue texture analysis using a statistical approach and classification of T1-weighted images of clinically confirmed multiple sclerosis patients.MethodsWe averaged the intensities of three consecutive 1-mm slices to simulate 3-mm slices. Two hundred sixty-four texture parameters were calculated for both the original and the averaged slices. Wilcoxon's signed ranks test was used to find differences between the regions of interest representing white matter and multiple sclerosis plaques. Linear and nonlinear discriminant analyses were applied with several separate training and test sets to determine the actual classification accuracy.ResultsOnly moderate differences in distributions of the texture parameter value for 1-mm and simulated 3-mm-thick slices were found. Our study also showed that white matter areas are well separable from multiple sclerosis plaques even if the slice thickness differs between training and test sets.ConclusionsThree-millimeter-thick magnetic resonance image slices acquired with a 1.5 T clinical magnetic resonance scanner seem to be sufficient for texture analysis of multiple sclerosis plaques and white matter tissue.

Fuzzy linear and nonlinear discriminant analysis algorithms for face recognition

This paper presents two novels algorithms based on fuzzy logic and discriminant analysis for face recognition. The first one is a fuzzy extension of a linear discriminant analysis algorithm namely LDA/QR and the second one is a fuzzy extension of kernel scatter-difference based discriminant analysis (KSDA) algorithm. They can deal with small sample size and nonlinear problems which degrade the performance of face recognition system. Experimental results on the ORL and the extended Yale B face databases show that the two proposed approaches, using fuzzy logic, achieves a better performance in face recognition compared with their original versions.