Nonparametric Discriminant Analysis Research Articles

Shaped patch based nonparametric discriminant analysis for hyperspectral image classification through the CNN model

ABSTRACT Convolutional neural network (CNN) has shown superior classification results for hyperspectral image classification. Due to high dimensionality of the hyperspectral image, usually a feature reduction method is applied to the hyperspectral data before feeding it to the CNN model. Principal component analysis is the most widely used approach to this end. However, it has two main disadvantages. It uses the mean square error measure and ignores the class separability. Moreover, it is a pixel-based method, which does not utilize the spatial information. To deal with these weaknesses, the patch-based nonparametric discriminant analysis method is proposed for hyperspectral image feature reduction in this work. However, due to the complex and heterogeneous nature of the hyperspectral images, the use of fixed square patches is not so appropriate. To deal with this difficulty, a shaping approach is introduced for the generation of adaptive patches with consistent shapes in each local region. The input patches of the CNN model are also weighted according to the spectral similarity and class boundaries information. Moreover, a simple residual CNN model is suggested for the classification of the extracted shaped input patches. The efficiency of the proposed framework is assessed in different cases using various evaluation measures on six real hyperspectral images.

Properties of Flare-imminent versus Flare-quiet Active Regions from the Chromosphere through the Corona. II. Nonparametric Discriminant Analysis Results from the NWRA Classification Infrastructure (NCI)

A large sample of active-region-targeted time-series images from the Solar Dynamics Observatory/Atmospheric Imaging Assembly (AIA), the AIA Active Region Patch database (Paper I) is used to investigate whether parameters describing the coronal, transition region, and chromospheric emission can differentiate a region that will imminently produce a solar flare from one that will not. Parameterizations based on moment analysis of direct and running-difference images provide for physically interpretable results from nonparametric discriminant analysis. Across four event definitions including both 24 hr and 6 hr validity periods, 160 image-based parameters capture the general state of the atmosphere, rapid brightness changes, and longer-term intensity evolution. We find top Brier Skill Scores in the 0.07–0.33 range, True Skill Statistics in the 0.68–0.82 range (both depending on event definition), and Receiver Operating Characteristic Skill Scores above 0.8. Total emission can perform notably, as can steeply increasing or decreasing brightness, although mean brightness measures do not, demonstrating the well-known active-region size/flare productivity relation. Once a region is flare productive, the active-region coronal plasma appears to stay hot. The 94 Å filter data provide the most parameters with discriminating power, with indications that it benefits from sampling multiple physical regimes. In particular, classification success using higher-order moments of running-difference images indicate a propensity for flare-imminent regions to display short-lived small-scale brightening events. Parameters describing the evolution of the corona can provide flare-imminent indicators, but at no preference over “static” parameters. Finally, all parameters and NPDA-derived probabilities are available to the community for additional research.

Properties of Flare-imminent versus Flare-quiet Active Regions from the Chromosphere through the Corona. I. Introduction of the AIA Active Region Patches (AARPs)

We begin here a series of papers examining the chromospheric and coronal properties of solar active regions. This first paper describes an extensive data set of images from the Atmospheric Imaging Assembly on the Solar Dynamics Observatory curated for large-sample analysis of this topic. Based on (and constructed to coordinate with) the “Active Region Patches” as identified by the pipeline data analysis system for the Helioseismic and Magnetic Imager on the same mission (the “HARPs”), the “AIA Active Region Patches” (AARPs), described herein, comprise an unbiased multiwavelength set of FITS files downsampled spatially only by way of HARP-centered patch extractions (full spatial sampling is retained), and downsampled in the temporal domain but still able to describe both short-lived kinematics and longer-term trends. The AARPs database enables physics-informed parameterization and analysis using nonparametric discriminant analysis in Paper II of this series, and is validated for analysis using differential emission measure techniques. The AARP data set presently covers mid-2010 through 2018 December, is ≈9 TB in size, and is available through the Solar Data Analysis Center.

Nonparametric discriminant analysis with network structures in predictor

ABSTRACT Multiclassification, known as classification for multi-label responses, has been an important problem in supervised learning and has attracted our attention. Discriminant analysis (DA) is a popular method to deal with multiclassification. With the increasing availability of complex data, it becomes more challenging to analyse them. One of the important features in complex data is the network structure, which is ubiquitous in high-dimensional data because of strong or weak correlations among variables. In addition, in the framework of DA, an assumption of normal distributions is imposed on the predictors, but it is usually invalid in applications. To relax the normality assumption, we propose a nonparametric discriminant function to address multiclassification. In addition, to incorporate the network structure and improve the accuracy of classification, we develop three different network-based surrogate predictors to replace conventional predictors. The key features of the proposed method include the incorporation of network structures in predictors and allowance of predictors to follow exponential family distributions. Finally, numerical studies, including simulation and real data analysis, are conducted to assess the performance of the proposed method.

TX-001HR is associated with a clinically meaningful effect on severity of moderate to severe vasomotor symptoms in the REPLENISH trial.

Objective:The aim of the study was to evaluate the clinically meaningful effect of oral TX-001HR (17β-estradiol [E2]/progesterone [P4]) capsules on hot flushes severity (vasomotor symptoms [VMS] severity scale) using the patient-reported Clinical Global Impression (CGI).Methods:REPLENISH (NCT01942668) was a phase 3, randomized, double-blind, placebo-controlled, multicenter trial that evaluated TX-001HR in postmenopausal women (40-65 y) with a uterus. Those with frequent moderate to severe hot flushes (≥7/d or ≥50/wk) were randomized in a VMS substudy to daily E2/P4 (1/100, 0.5/100, 0.5/50, or 0.25/50 mg/mg), or placebo. Patients rated VMS severity from 1 (mild) to 3 (severe) and symptom improvements with the CGI. CGI results were an anchor in a nonparametric discriminant analysis to define clinically important differences (CIDs) and minimal CID in VMS severity at weeks 4 and 12.Results:In the VMS substudy (n = 726), determined CID and minimal CID severity thresholds were reductions of 0.525 and 0.350 points at week 4, respectively, and 0.775 and 0.225 points at week 12. Significantly more women taking the two highest E2/P4 doses (1/100 and 0.5/100) versus placebo met CID severity thresholds at weeks 4 (40% and 44% vs 17%; P < 0.05) and 12 (56% and 48% vs 29%; P < 0.05).Conclusion:REPLENISH trial data demonstrated that E2/P4 1/100 and 0.5/100 provided clinically meaningful improvements in hot flushes severity in postmenopausal women. In conjunction with previously demonstrated clinically meaningful VMS frequency improvements, these data support oral E2/P4 1/100 and 0.5/100 for postmenopausal women with a uterus seeking treatment for moderate to severe VMS.

A novel incremental Kernel Nonparametric SVM model (iKN-SVM) for data classification: An application to face detection

In this paper, we propose a novel incremental classifier to overcome problems associated with batch techniques, along with issues related to data spread that Kernel Support Vector Machines (KSVM) may encounter. Basically, we present a Kernel SVM-based model that learns incrementally, as new data is available over time, in order to handle dynamic and large data effectively and reduce the computational time. The proposed model deals with the data spread issues by introducing near-global variations, from the scatter matrices of the Kernel Nonparametric Discriminant Analysis (KNDA), into the optimization problem of incremental KSVM, while considering local characteristics of the data provided by KSVM. Besides, our model has a quadratic convex optimization problem with one global solution. Furthermore, an extensive comparison of the model with other state-of-the-art incremental and batch algorithms on various datasets, has been carried out, in order to show its advantages and effectiveness for classification tasks. Moreover, an evaluation of the proposed method on face detection is provided.

Variational nonparametric discriminant analysis

Variable selection and classification are common objectives in the analysis of high-dimensional data. Most such methods make distributional assumptions that may not be compatible with the diverse families of distributions data can take. A novel Bayesian nonparametric discriminant analysis model that performs both variable selection and classification within a seamless framework is proposed. Pólya tree priors are assigned to the unknown group-conditional distributions to account for their uncertainty, and allow prior beliefs about the distributions to be incorporated simply as hyperparameters. The adoption of collapsed variational Bayes inference in combination with a chain of functional approximations led to an algorithm with low computational cost. The resultant decision rules carry heuristic interpretations and are related to an existing two-sample Bayesian nonparametric hypothesis test. By an application to some simulated and publicly available real datasets, the proposed method exhibits good performance when compared to current state-of-the-art approaches.

US adults with unmet mental health treatment needs: Profiling and underlying causes using machine learning techniques

According to the Substance Abuse and Mental Health Services Administration (USA), an estimated 9.7 million US adults did not receive mental health treatment in 2016. Based on 2015 and 2016 data reported in the National Survey on Drug Use and Health, hierarchical optimal discriminant analysis reveals that several socio-demographic groups have compelling unmet mental health treatment needs with 94.47% prediction accuracy. Non-parametric hierarchical optimal discriminant analysis was appropriate for this study, especially with little a-priori knowledge about relationships among variables examined. The results are further enhanced by detailed cross-tabulation analyses which show how unmet mental health treatment needs differ among various sociodemographic groups. The underlying reasons are also examined and reported. These results and analyses provide useful information for policymakers and decision makers to offer predictions and take appropriate actions for the purpose of intervention and/or prevention.

A bilevel framework for joint optimization of session compensation and classification for speaker identification

The i-vector framework based system is one of the most popular systems in speaker identification (SID). In this system, session compensation is usually employed first and then the classifier. For any session-compensated representation of i-vector, there is a corresponding identification result, so that both the stages are related. However, in current SID systems, session compensation and classifier are usually optimized independently. An incomplete knowledge about the session compensation to the identification task may lead to involving uncertainties. In this paper, we propose a bilevel framework to jointly optimize session compensation and classifier to enhance the relationship between the two stages. In this framework, we use the sparse coding (SC) to obtain the session-compensated feature by learning an overcomplete dictionary, and employ the softmax classifier and support vector machine (SVM) in classifying respectively. Moreover, we present a joint optimization of the dictionary and classifier parameters under a discriminative criterion for classifier with conditions for SC. In addition, the proposed methods are evaluated on the King-ASR-010, VoxCeleb and RSR2015 databases. Compared with typical session compensation techniques, such as linear discriminant analysis (LDA) and nonparametric discriminant analysis (NDA), our methods can be more robust to complex session variability. Moreover, compared with the typical classifiers in i-vector framework, i.e. the cosine distance scoring (CDS) and probabilistic linear discriminant analysis (PLDA), our methods can be more suitable for SID (multiclass task).

A global manifold margin learning method for data feature extraction and classification

This paper presents a global manifold margin learning approach for data feature extraction or dimensionality reduction, which is named locally linear representation manifold margin (LLRMM). Provided that points locating on one manifold are of the same class and those residing on the corresponding manifolds are varied labeled, LLRMM is desired to identify different manifolds, respectively. In the proposed LLRMM, it firstly constructs both a between-manifold graph and a within-manifold graph. In the between-manifold graph, for any point, its k nearest neighbors and itself must belong to different manifolds. However, any node and its neighborhood points should be on the same manifold in the within-manifold graph. Then we use the minimum locally linear representation trick to reconstruct any node with their corresponding k nearest neighbors in both graphs, from which a between-manifold graph scatter and a within-manifold graph scatter can be reasoned, followed by a novel global model of manifold margin. At last, a projection will be explored to map the original data into a low dimensional subspace with the maximum manifold margin. Experiments on some widely used face data sets including AR, CMU PIE, Yale, YaleB and LFW have been carried out, where the performance of the proposed LLRMM outperforms those of some other methods such as kernel principal component analysis (KPCA), non-parametric discriminant analysis (NDA), reconstructive discriminant analysis (RDA), discriminant multiple manifold learning (DMML) and large margin nearest neighbor (LMNN).

Molecular activity prediction by means of supervised subspace projection based ensembles of classifiers

This paper proposes a method for molecular activity prediction in QSAR studies using ensembles of classifiers constructed by means of two supervised subspace projection methods, namely nonparametric discriminant analysis (NDA) and hybrid discriminant analysis (HDA). We studied the performance of the proposed ensembles compared to classical ensemble methods using four molecular datasets and eight different models for the representation of the molecular structure. Using several measures and statistical tests for classifier comparison, we observe that our proposal improves the classification results with respect to classical ensemble methods. Therefore, we show that ensembles constructed using supervised subspace projections offer an effective way of creating classifiers in cheminformatics.

An ensemble approach for in silico prediction of Ames mutagenicity

In this paper, we evaluate three learning algorithms based on supervised projections for molecular activity prediction. Using an approach based on supervised projections of the input space to construct ensembles of classifiers, three algorithms were tested. We constructed the projections by considering only instances that were misclassified by a previous classifier using the hidden layer of an Artificial Neural Network. We applied a supervised linear projection of the input space using a Nonparametric Discriminant Analysis method. Finally, we projected onto a subspace that minimizes the weighted error for each step. Using these three methods to construct ensembles of classifiers for the in silico prediction of Ames mutagenicity, we demonstrated the improved behavior of our proposal compared to classical methods.

Modelling and compensation for language mismatch in speaker verification

Language mismatch represents one of the more difficult challenges in achieving effective speaker verification in naturalistic audio streams. The portion of bi-lingual speakers worldwide continues to grow making speaker verification for speech technology more difficult. In this study, three specific methods are proposed to address this issue. Experiments are conducted on the PRISM (Promoting Robustness in Speaker Modeling) evaluation-set. We first show that adding small amounts of multi-lingual seed data to the Probabilistic Linear Discriminant Analysis (PLDA) development set, leads to a significant relative improvement of +17.96% in system Equal Error Rate (EER). Second, we compute the eigendirections that represent the distribution of multi-lingual data added to PLDA. We show that by adding these new eigendirections as part of the Linear Discriminant Analysis (LDA), and then minimizing them to directly compensate for language mismatch, further performance gains for speaker verification are achieved. By combining both multi-lingual PLDA and this minimization step with the new set of eigendirections, we obtain a +26.03% relative improvement in EER. In practical scenarios, it is highly unlikely that multi-lingual seed data representing the languages present in the test-set would be available. Hence, in the third phase, we address such scenarios, by proposing a method for Locally Weighted Linear Discriminant Analysis (LWLDA). In this third method, we reformulate the LDA equations to incorporate a local affine transform that weighs the same speaker samples. This method effectively preserves the local intrinsic information represented by the multimodal structure of the within-speaker scatter matrix, thereby helping to improve the class discriminating ability of LDA. It also helps in extending the ability of LDA to transform the speaker i-Vectors to dimensions that are greater than the total number of speaker classes. Using LWLDA, a relative improvement of +8.54% is obtained in system EER. LWLDA provides even more gain when multi-lingual seed data is available, and improves the system peformance by relative +26.03% in terms of EER. We also compare LWLDA to the recently proposed Nearest Neighbor Non-Parametric Discriminant Analysis (NDA). We show that not only is LWLDA better than NDA in terms of system performance but is also computationally less expensive. Comparative studies on DARPA Robust Automatic Transcription of Speech (RATS) corpus also show that LWLDA consistently outperforms NDA and LDA on different evaluation conditions. Our solutions offer new directions for addressing a challenging problem which has received limited attention in the speaker recognition community.

Multi-View Nonparametric Discriminant Analysis for Image Retrieval and Recognition

A novel multi-view nonparametric discriminant analysis method is proposed for the application of cross-modal image retrieval and zero-shot recognition. We exploit the class boundary structure and discrepancy information of the available views in order to formulate an optimization criterion, which is automatically adjusted to the multi-view class structures. The proposed method allows for multiple projection directions, by relaxing the Gaussian distribution assumption of related methods. The experiments demonstrate that the proposed method can achieve superior results comparing to several existing methods.

Factors Predicting Length of Hospital Stay and Extended Care Facility Admission After Hindfoot Arthrodesis Procedures

Delayed identification of patients requiring admission to extended care facilities (ECFs) can lead to greater healthcare costs through an increased length of hospital stay (LOHS). Previous studies of hip and knee arthroplasty identified factors associated with a likely discharge to an ECF. These issues have not been extensively studied for major hindfoot procedures. We conducted a retrospective review of 198 cases treated during a 3-year period to identify the risk factors for an extended LOHS and ECF admission after ankle arthrodesis, triple arthrodesis, pantalar arthrodesis, and subtalar arthrodesis. The primary outcomes were LOHS and ECF admission. The independent predictors included age, sex, body mass index, housing status, American Society of Anesthesiologists class, diabetes and/or diabetic neuropathy, health insurance, fixation type, and perioperative infection. Stepwise multiple regression analysis was used to determine which variables were related to a longer LOHS. Nonparametric discriminant function analysis was used to identify the preoperative factors that best predicted ECF admission. A longer LOHS was significantly related to postoperative ECF admission, Centers for Medicare and Medicaid Services (CMS) insurance, diabetic neuropathy, external fixation, and infection. ECF admission was required for 34 of 198 patients (17.2%). Discriminant analysis found that older age, living alone, external fixation, and CMS insurance predicted a greater probability of ECF admission. The function accurately classified 94% of ECF admissions and 80% of non-ECF admission patients. ECF admission and CMS insurance extended the LOHS, likely owing to the administrative process of arranging an ECF discharge. If externally validated, the function we have derived could provide preoperative identification of likely ECF discharge candidates and reduce costs by shortening the LOHS.

Local kernel nonparametric discriminant analysis for adaptive extraction of complex structures

Abstract The linear discriminant analysis (LDA) is one of popular means for linear feature extraction. It usually performs well when the global data structure is consistent with the local data structure. Other frequently-used approaches of feature extraction usually require linear, independence, or large sample condition. However, in real world applications, these assumptions are not always satisfied or cannot be tested. In this paper, we introduce an adaptive method, local kernel nonparametric discriminant analysis (LKNDA), which integrates conventional discriminant analysis with nonparametric statistics. LKNDA is adept in identifying both complex nonlinear structures and the ad hoc rule. Six simulation cases demonstrate that LKNDA have both parametric and nonparametric algorithm advantages and higher classification accuracy. Quartic unilateral kernel function may provide better robustness of prediction than other functions. LKNDA gives an alternative solution for discriminant cases of complex nonlinear feature extraction or unknown feature extraction. At last, the application of LKNDA in the complex feature extraction of financial market activities is proposed.

Nonparametric kernel sparse representation-based classifier

Sparse representation-based classifier (SRC) and kernel sparse representation-based classifier (KSRC) are founded on combining pattern recognition and compressive sensing methods and provide acceptable results in many machine learning problems. Nevertheless, these classifiers suffer from some shortcomings. For instance, SRC's accuracy drops against samples from same directional classes or KSRC's output declines when data is not normally distributed in kernel space. This paper introduces nonparametric kernel sparse representation-based classifier (NKSRC) as a generalized framework for SRC and KSRC. First, it applies kernel on samples to overcome data directionality and then employs nonparametric discriminant analysis (NDA) to reduce data dimensionality in kernel space alleviating concern about data distribution type. The experimental results of NKSRC demonstrate its superiority over SRC and KSRC–LDA and its equal or superior performance with respect to KSRC–PCA on some synthetic, four well-known face recognition and several UCI datasets.

A Comparison of Classifiers for Solar Energetic Events

AbstractWe compare the results of using a Random Forest Classifier with the results of using Nonparametric Discriminant Analysis to classify whether a filament channel (in the case of a filament eruption) or an active region (in the case of a flare) is about to produce an event. A large number of descriptors are considered in each case, but it is found that only a small number are needed in order to get most of the improvement in performance over always predicting the majority class. There is little difference in performance between the two classifiers, and neither results in substantial improvements over simply predicting the majority class.

Effects of algal food quality on free amino acid metabolism of Daphnia

Zooplankton frequently face dietary limitation caused by imbalances between their nutrient requirements and the nutrient content of their phytoplankton food. While insufficient nutrition is acknowledged to affect growth and reproduction of zooplankton, the effect of dietary limitation on other aspects of their metabolism remains poorly studied. One understudied biochemical response associated with nutrient stress is changes to the free amino acid (FAA) composition of body tissues resulting from changes in dietary supply. We examined changes in FAA composition in two cladoceran species (Daphnia magna and Daphnia pulex) in response to stress created by ecologically relevant food conditions: low food quantities, high algal carbon: nitrogen (C:N) ratios, and high algal C: phosphorus (P) ratios. We measured 14 FAA after extracting them from lyophilized whole Daphnia grown under these contrasting algal diets. Generally, low food quantity and low algal P increased body FAA content in both daphnid species while many essential FAA decreased due to N-limitation. While significant responses to diet were seen for each daphnid species, the nature of these responses depended on the type of nutritional stress, the FAA examined, and on the species. Non-parametric discriminant analysis exhibited a high predictive ability to classify unknown zooplankton into the correct diet category. The observed changes in body FAA profiles of Daphnia illustrate strong effects that algal food quality can have on the biochemistry of planktonic consumers. Furthermore these results demonstrate body FAA composition of zooplankton may provide an important source of information about zooplankton nutrient limitation especially combined with other nutrient-dependent molecular responses.

Point Context: An Effective Shape Descriptor for RST-Invariant Trajectory Recognition

Motion trajectory recognition is important for characterizing the moving property of an object. The speed and accuracy of trajectory recognition rely on a compact and discriminative feature representation, and the situations of varying rotation, scaling and translation has to be specially considered. In this paper we propose a novel feature extraction method for trajectories. Firstly a trajectory is represented by a proposed point context, which is a rotation-scale-translation (RST) invariant shape descriptor with a flexible tradeoff between computational complexity and discrimination, yet we prove that it is a complete shape descriptor. Secondly, the shape context is nonlinearly mapped to a subspace by kernel nonparametric discriminant analysis (KNDA) to get a compact feature representation, and thus a trajectory is projected to a single point in a low-dimensional feature space. Experimental results show that, the proposed trajectory feature shows encouraging improvement than state-of-art methods.