Regularized Discriminant Analysis Research Articles

Sex assessment using the radius bone in a French sample when applying various statistical models.

Sex estimation of skeletal remains is one of the most important tasks in forensic anthropology. The radius bone is useful to develop standard guidelines for sex estimation across various populations and is an alternative when coxal or femoral bones are not available.The aim of the present study was to assess the sexual dimorphism from radius measurements in a French sample and compare the predictive accuracy of several modelling techniques, using both classical statistical methods and machine learning algorithms.A total of 78 left radii (36 males and 42 females) were used in this study. Sixteen measurements were made. The modelling techniques included a linear discriminant analysis (LDA), flexible discriminant analysis (FDA), regularised discriminant analysis (RDA), penalised logistic regression (PLR), random forests (RF) and support vector machines (SVM).The different statistical models showed an accuracy of classification that is greater than 94%. After selection of variables, the accuracies increased to 97%. The measurements made at the proximal part of the radius (sagittal and transversal diameters of the head, and sagittal diameter of the neck), at distal part (maximum width of the distal epiphysis) and of the entire bone (maximum length) stand out among the various models.The present study suggests that the radius bone constitutes a valid alternative for sex estimation of skeletal remains with comparable classification accuracies to the pelvis or femur and that the non-classical statistical models may provide a novel approach to sex estimation from the radius bone. However, the extrapolation of the current results cannot be made without caution because our sample was composed of very aged individuals.

Comparison of Logistic Regression and Discriminant Analysis for Classification of Multicollinearity Data

The objective of this study is to concentrate on the classification method of the logistic regression and the discriminant analysis by using the simulation dataset and the liver patients as the actual data. These datasets are used the binary dependent variable depending on the correlated independent variables or called multicollinearity data. The standard classification method is logistic regression, which uses the logit function’s probability to conduct the dichotomous dependent variable. The iteration process can be solved to estimate logit function parameters and explain the relationship between a dependent binary variable and independent variables. Discriminant analysis is a powerful classification based on linear discriminant analysis (LDA), quadratic discriminant analysis (QDA), and regularized discriminant analysis (RDA). These methods consider the decision boundaries by building a classifier model on the multivariate normal distribution. LDA defines the standard covariance matrix, but QDA has an individual covariance matrix. RDA extends from QDA by setting the regularized parameter to estimate the covariance matrix. In the case of the simulation study, the independent variables are generated by defining the constant correlation on the multivariate normal distribution that made the multicollinearity problem. Then the binary response variable can be approximated from the logit function. For application to actual data, we expressed the classification of type liver and non-liver patients as the dependent variables and obtained patient personal information on the nine independent variables. The highest average percentage of accuracy determines the performance of these methods. The results have shown that the logistic regression was successful when using small independent variables, but the RDA performed when using large independent variables.

Comparison of the effectiveness of different machine learning algorithms in predicting new fractures after PKP for osteoporotic vertebral compression fractures

BackgroundThe use of machine learning has the potential to estimate the probability of a second classification event more accurately than traditional statistical methods, and few previous studies on predicting new fractures after osteoporotic vertebral compression fractures (OVCFs) have focussed on this point. The aim of this study was to explore whether several different machine learning models could produce better predictions than logistic regression models and to select an optimal model.MethodsA retrospective analysis of 529 patients who underwent percutaneous kyphoplasty (PKP) for OVCFs at our institution between June 2017 and June 2020 was performed. The patient data were used to create machine learning (including decision trees (DT), random forests (RF), support vector machines (SVM), gradient boosting machines (GBM), neural networks (NNET), and regularized discriminant analysis (RDA)) and logistic regression models (LR) to estimate the probability of new fractures occurring after surgery. The dataset was divided into a training set (75%) and a test set (25%), and machine learning models were built in the training set after ten cross-validations, after which each model was evaluated in the test set, and model performance was assessed by comparing the area under the curve (AUC) of each model.ResultsAmong the six machine learning algorithms, except that the AUC of DT [0.775 (95% CI 0.728–0.822)] was lower than that of LR [0.831 (95% CI 0.783–0.878)], RA [0.953 (95% CI 0.927–0.980)], GBM [0.941 (95% CI 0.911–0.971)], SVM [0.869 (95% CI 0.827–0.910), NNET [0.869 (95% CI 0.826–0.912)], and RDA [0.890 (95% CI 0.851–0.929)] were all better than LR.ConclusionsFor prediction of the probability of new fracture after PKP, machine learning algorithms outperformed logistic regression, with random forest having the strongest predictive power.Graphical

The Application of Visible and Near-Infrared Spectroscopy Combined with Chemometrics in Classification of Dried Herbs

The fast differentiation and classification of herb samples are complicated processes due to the presence of many various chemical compounds. Traditionally, separation techniques have been employed for the identification and quantification of compounds present in different plant matrices, but they are tedious, time-consuming and destructive. Thus, a non-targeted approach would be specifically advantageous for this purpose. In the present study, spectroscopy in the visible and near-infrared range and pattern recognition techniques, including the principal component analysis (PCA), linear discriminant analysis (LDA), quadratic discriminant analysis (QDA), regularized discriminant analysis (RDA), super k-nearest neighbor (SKNN) and support vector machine (SVM) techniques, were applied to develop classification models that enabled the discrimination of various commercial dried herbs, including mint, linden, nettle, sage and chamomile. The classification error rates in the validation data were below 10% for all the classification methods, except for SKNN. The results obtained confirm that spectroscopy and pattern recognition methods constitute a good non-destructive tool for the rapid identification of herb species that can be used in routine quality control by the pharmaceutical industry, as well as herbal suppliers, to avoid mislabeling.

Quadratic discriminant analysis by projection

Discriminant analysis, including linear discriminant analysis (LDA) and quadratic discriminant analysis (QDA), is a popular approach to classification problems. It is well known that LDA is suboptimal to analyze heteroscedastic data, for which QDA would be an ideal tool. However, QDA is less helpful when the number of features in a data set is moderate or high, and LDA and its variants often perform better due to their robustness against dimensionality. In this work, we introduce a new dimension reduction and classification method based on QDA. In particular, we define and estimate the optimal one-dimensional (1D) subspace for QDA, which is a novel hybrid approach to discriminant analysis. The new method can handle data heteroscedasticity with number of parameters equal to that of LDA. Therefore, it is more stable than the standard QDA and works well for data in moderate dimensions. We show an estimation consistency property of our method, and compare it with LDA, QDA, regularized discriminant analysis (RDA) and a few other competitors by simulated and real data examples.

Coupled Regularized Sample Covariance Matrix Estimator for Multiple Classes

The estimation of covariance matrices of multiple classes with limited training data is a difficult problem. The sample covariance matrix (SCM) is known to perform poorly when the number of variables is large compared to the available number of samples. In order to reduce the mean squared error (MSE) of the SCM, regularized (shrinkage) SCM estimators are often used. In this work, we consider regularized SCM (RSCM) estimators for multiclass problems that couple together two different target matrices for regularization: the pooled (average) SCM of the classes and the scaled identity matrix. Regularization toward the pooled SCM is beneficial when the population covariances are similar, whereas regularization toward the identity matrix guarantees that the estimators are positive definite. We derive the MSE optimal tuning parameters for the estimators as well as propose a method for their estimation under the assumption that the class populations follow (unspecified) elliptical distributions with finite fourth-order moments. The MSE performance of the proposed coupled RSCMs are evaluated with simulations and in a regularized discriminant analysis (RDA) classification set-up on real data. The results based on three different real data sets indicate comparable performance to cross-validation but with a significant speed-up in computation time.

Determination of Camelina Oil Sterol Composition and Its Application for Authenticity Studies

Camelina oil has a high sterol concentration and is rather expensive compared to other vegetable oils. Because of its higher price, it is often adulterated by the addition of other, cheaper oils. This study was performed to validate a method for sterol determination in camelina oil, enabling the detection of camelina oil adulteration. Sterol levels in camelina oil samples were determined by gas chromatography after saponification and solid phase extraction. The method was validated, and the results proved that the chosen method is specific and selective, repeatable and accurate. The quantitatively assessed average contents of sterols in camelina oil samples of Slovenian origin were 21.4 mg 100 g-1 for brassicasterol, 153.6 mg 100 g-1 for campesterol, 3.9 mg 100 g-1 for stigmasterol, and 447.0 mg 100 g-1 for b-sitosterol. Results of camelina oil authenticity studies regarding botanical origin, performed by Principal Component Analysis (PCA) and Regularized Discriminant Analysis (RDA) enabled us to differentiate 100 % camelina oils from camelina oils adulterated with 10 %-40 % added sunflower, rapeseed or soya oil.

Combined Regularized Discriminant Analysis and Swarm Intelligence Techniques for Gait Recognition.

In the gait recognition problem, most studies are devoted to developing gait descriptors rather than introducing new classification methods. This paper proposes hybrid methods that combine regularized discriminant analysis (RDA) and swarm intelligence techniques for gait recognition. The purpose of this study is to develop strategies that will achieve better gait recognition results than those achieved by classical classification methods. In our approach, particle swarm optimization (PSO), grey wolf optimization (GWO), and whale optimization algorithm (WOA) are used. These techniques tune the observation weights and hyperparameters of the RDA method to minimize the objective function. The experiments conducted on the GPJATK dataset proved the validity of the proposed concept.

RADT-14. COMPARISON OF MACHINE LEARNING ALGORITHMS IN DISTINGUISHING RADIATION NECROSIS FROM PROGRESSION OF BRAIN METASTASES TREATED WITH STEREOTACTIC RADIOSURGERY (SRS)

Abstract PURPOSE To test the effectiveness of machine learning algorithms in distinguishing radiation necrosis (RN) from tumor progression (TP) using MRI radiomic features. METHODS Brain metastases were treated with SRS to a median dose of 18Gy. Lesions that showed evidence of progression on follow-up MRI were sampled surgically, and diagnoses confirmed by histopathology. Cases from 2 institutions were combined and randomly assigned for training (70%) and testing (30%). T1 post-contrast (T1c) and T2 fluid attenuated inversion recovery (T2 FLAIR) MRI were used for radiomic feature extraction (50 features each). Three subsets of radiomic features were obtained and tested: Signature #1 included 10 previously published features that correlated with diagnosis on T test; signature #2 and #3 included 5 and 12 features obtained through recursive elimination using random forest (RF) and support vector machine (SVM), respectively. Supervised machine learning models were trained using RF, SVM (radial kernel) and regularized discriminant analysis (RDA) algorithms based on all three radiomics signatures. Receiver operator characteristics (ROC) were compared between signatures and algorithms. RESULTS A total of 135 individual lesions (37 RN and 98 TP) were included. Signature #3 demonstrated the highest area under the curve in the training set (average AUC=0.98, vs 0.95 and 0.92 for signature #1 and #2), as well as the testing set (average AUC=0.83, vs 0.74 and 0.79 for signature #1 and #2). RF and SVM demonstrated similar performance in both training (average AUC 0.99-1) and testing datasets (average AUC 0.79-0.80) among all three signatures. Both RF and SVM were superior to RDA in performance (average training AUC 0.83, testing AUC 0.77). The greatest sensitivity (83%) and specificity (100%) in the testing set were achieved using signature #3 and SVM. CONCLUSION RF and SVM are effective in distinguishing RN from TP in a multi-institution dataset using radiomic signatures.

PENERAPAN ALGORITMA REGULARIZED DISCRIMINANT ANALYSIS UNTUK KLASIFIKASI KANKER PARU

Lung cancer is a type of cancer that starts from the lungs with abnormal cell growth. Lung cancer is a type of cancer with the largest number of sufferers. In this research, Regularized Discriminant Analysis (RDA) analysis is used to classify the types of lung cancer that exist. Regularized Discriminant Analysis (RDA) is a generalization of Linear Discriminant Analysis (LDA) and Quadratic Discriminant Analysis (QDA). If the alpha parameter is set to 1, this operator performs an LDA. Likewise if the alpha parameter is set to 0, this operator performs QDA. Discriminant analysis is used to determine which variables distinguish between two or more groups that occur naturally. With an accuracy value of 60%, it can be ascertained the type of lung cancer suffered by patients for ease of care and treatment.

EEG intentions recognition in dynamic complex object control task by functional brain networks and regularized discriminant analysis

Most of the tasks involved in neurological rehabilitation are generally constrained. However the complexity of experimental paradigms with constraints for acquiring electroencephalography (EEG) are relatively low. In order to improve the level of arousal in the brain, we propose an energy-constrained dynamic and complex experimental paradigm. In this novel experimental paradigm, EEG signals collected under the novel paradigm have different motor intentions. In this study, we combine the phase synchronization method and common spatial pattern (CSP) to build functional brain networks of subjects. Based on this method, we investigate the correlation of the global and local features of the functional brain networks and use regularized discriminant analysis (RDA) to recognize features. The accuracy obtained by the proposed method for EEG intention recognition can reach 93.47% (p < 0.01). The results show that the proposed method can decode EEG intention with high recognition performance during the manipulation of complex objects, which lays a foundation for the study of neurological rehabilitation.

Quantitative Airborne Inventories in Dense Tropical Forest Using Imaging Spectroscopy

Tropical forests have exceptional floristic diversity, but their characterization remains incomplete, in part due to the resource intensity of in-situ assessments. Remote sensing technologies can provide valuable, cost-effective, large-scale insights. This study investigates the combined use of airborne LiDAR and imaging spectroscopy to map tree species at landscape scale in French Guiana. Binary classifiers were developed for each of 20 species using linear discriminant analysis (LDA), regularized discriminant analysis (RDA) and logistic regression (LR). Complementing visible and near infrared (VNIR) spectral bands with short wave infrared (SWIR) bands improved the mean average classification accuracy of the target species from 56.1% to 79.6%. Increasing the number of non-focal species decreased the success rate of target species identification. Classification performance was not significantly affected by impurity rates (confusion between assigned classes) in the non-focal class (up to 5% of bias), provided that an adequate criterion was used for adjusting threshold probability assignment. A limited number of crowns (30 crowns) in each species class was sufficient to retrieve correct labels effectively. Overall canopy area of target species was strongly correlated to their basal area over 118 ha at 1.5 ha resolution, indicating that operational application of the method is a realistic prospect (R2 = 0.75 for six major commercial tree species).

Comparison of the performance of multiclass classifiers in chemical data: Addressing the problem of overfitting with the permutation test

The objective of this work was to apply different pattern recognition techniques in datasets—i.e., the Glass Identification Dataset and the Wine Quality Dataset—commonly used as a chemometric study of cases. In this paper, three types of different classification models were used. The first type was based on discriminant analysis and other linear classification models such as Linear Discriminant Analysis (LDA), Regularized Discriminant Analysis (RDA), Mixture Discriminant Analysis (MDA), and Partial Least Squares Discriminant Analysis (PLS-DA). The second type was based on nonlinear classification models such as Artificial Neural Networks (ANN), Support Vector Machine (SVM) with a radial kernel function, k-Nearest Neighbors (k-NN), Naive Bayes (NB), and Learning Vector Quantization (LVQ). The last type was based on classification trees and rule-based models such as Classification and Regression Tree (CART), Bagging, Random Forest (RF), C5.0, and Generalized Boosted Machine (GBM). The obtained results outperformed the classification concerning works previously published in the literature. The computational experiments show that the LVQ was the one method able to classify all three datasets correctly. The permutation tests were applied to evaluate the occurrences of the overfitting problem. The results showed that the overfitting problem was absent, which was confirmed by the pairwise Wilcoxon signed-rank test.

Transfer Learning for a Multimodal Hybrid EEG-fTCD Brain–Computer Interface

Humans can transfer knowledge previously acquired from a specific task to new and unknown ones. Recently, transfer learning (TL) has been extensively used in brain–computer interface (BCI) research to reduce the training/calibration requirements. BCI systems have been designed to provide alternative communication or control access through computers to individuals with limited speech and physical abilities (LSPA). These systems generally require a calibration session in order to train the BCI before each usage. Such a calibration session may be burdensome for the individuals with LSPA. In this article, we introduce a multimodal hybrid BCI based on electroencephalography (EEG) and functional transcranial Doppler ultrasound (fTCD) and present a TL approach to reduce the calibration requirements. In the hybrid BCI, EEG, and fTCD are used simultaneously to measure the electrical brain activity and cerebral blood velocity, respectively, in response to motor imagery (MI) tasks. Using the data we collected from ten healthy individuals, we perform dimensionality reduction utilizing regularized discriminant analysis (RDA). Using the scores from RDA, we learn class conditional probabilistic distributions for each individual. We use these class conditional distributions to perform TL across different participants. More specifically, in order to reduce the calibration requirements for each individual, we choose the recorded data from other individuals to augment the training data for that specific individual. We choose the data for augmentation based on the probabilistic similarities between the class conditional distributions. For the final classification, we use the RDA scores after TL as features input to three different classifiers: quadratic discriminant analysis (QDA), linear discriminant analysis (LDA), and support vector machines (SVMs). Using our experimental data, we show that TL decreases the calibration requirements up to $87.5\%$ . Also by comparing SVM, LDA, and QDA, we observe that the SVM provides the best classification performance.

Chemometric methods for classification of clonal varieties of green coffee using Raman spectroscopy and direct sample analysis

This work presents the development of methods for classification of clonal varieties of coffee using chemometrics coupled to Raman spectroscopy and direct sample analysis. Spectra were collected directly from each bean in triplicate, and the homogeneity of the sample’s surface was studied. The spectral range between 1200 and 1800 cm−1 is related to organic groups that are relevant for the discrimination of coffee. The main peaks were observed at about 1600 cm−1, 1630 cm−1, 1120 cm−1, and 1200 cm−1. All collected spectra were baseline-aligned, then the preprocessing mean centering (MC) or multiplicative scatter correction (MSC) were used. The original data matrix X was replaced by the PCA-scores matrix T before applying the classification methods. Linear discriminant analysis (LDA), mixture discriminant analysis (MDA), quadratic discriminant analysis (QDA), regularized discriminant analysis (RDA), partial least squares-discriminant analysis with Bayesian inference (PLS–DA), and soft independent modeling of class analogies (SIMCA) methods were applied and compared.Multiplicative scatter correction provided more accurate results when compared to MC, which may be attributed to the direct analysis of solid samples that requires correction for radiation scattering. Using MSC, the LDA correctly classified 98.7% of the samples, while MDA, RDA, QDA, PLS–DA, and SIMCA corrected classified 100% of the samples. On the other hand, using MC, correct classification of the samples was 62.7% for LDA, 70.7% for MDA, 62.7% for RDA, 62.7% for QDA, 61.3% for PLS–DA, and 97.3% for SIMCA.

Tea types classification with data fusion of UV-Vis, synchronous fluorescence and NIR spectroscopies and chemometric analysis.

The potential of selected spectroscopic methods - UV-Vis, synchronous fluorescence and NIR as well a data fusion of the measurements by these methods - for the classification of tea samples with respect to the production process was examined. Four classification methods - Linear Discriminant Analysis (LDA), Quadratic Discriminant Analysis (QDA), Regularized Discriminant Analysis (RDA) and Support Vector Machine (SVM) - were used to analyze spectroscopic data. PCA analysis was applied prior to classification methods to reduce multidimensionality of the data. Classification error rates were used to evaluate the performance of these methods in the classification of tea samples. The results indicate that black, green, white, yellow, dark, and oolong teas, which are produced by different methods, are characterized by different UV-Vis, fluorescence, and NIR spectra. The lowest error rates in the calibration and validation data sets for individual spectroscopies and data fusion models were obtained with the use of the QDA and SVM methods, and did not exceed 3.3% and 0.0%, respectively. The lowest classification error rates in the validation data sets for individual spectroscopies were obtained with the use of RDA (12,8%), SVM (6,7%), and QDA (2,7%), for the UV-Vis, SF, and NIR spectroscopies, respectively. NIR spectroscopy combined with QDA outperformed other individual spectroscopic methods. Very low classification errors in the validation data sets - below 3% - were obtained for all the data fusion data sets (SF + UV-Vis, SF + NIR, NIR + UV-Vis combined with the SVM method). The results show that UV-Vis, fluorescence and near infrared spectroscopies may complement each other, giving lower errors for the classification of tea types.

M-Estimation-Based Subspace Learning for Brain Computer Interfaces

Many problems in signal processing, statistical learning, and data science can be posed as the problem of learning lower dimensional representation of the data. Particularly, we consider the brain computer interface (BCI) application where electroencephalography (EEG) data are used to determine user's intent to type letters through stimulation with rapid serial visual presentations (RSVP). Such a typing system requires dimensionality reduction and classification abilities. The former is achieved through principal component analysis (PCA), whereas the latter involves regularized discriminant analysis (RDA). Remarkably, EEG recordings are prone to contain user-produced artifacts, such as eye blinks, jaw contractions, or scalp movements. In this article, we present a methodology to fully robustify the aforementioned BCI application, enhancing its suitability in challenging recording scenarios. We consider a solution for robust PCA (RPCA) of fully observed data with outlying samples based on M-estimation theory, as well as a similar methodology for robust RDA (RRDA). Our proposed algorithm iteratively yields robust mean vector and covariance matrix estimates, and then applies eigenanalysis on the estimated robust covariance matrix. The result is a principled way of dealing with outliers in the data that is simple, yet effective in delivering remarkable classification performance. The methodology is validated with real EEG data and compared to the state-of-the-art RSVP Keyboard, with a thorough experimental setup being recorded and described.

Fatty acid composition and δ13C isotopic ratio characterisation of pumpkin seed oil

This study aimed to verify the authenticity and geographical origin of pumpkin seed oil using chemical analysis by gas chromatography (GC) and isotopic characterisation by gas chromatography combustion stable isotope ratio mass spectrometry (GC/C/IRMS) combined with chemometric analysis. Thirty-eight pumpkin seed samples from various parts of the world were collected and authentic oils were prepared. Pumpkin seed oils are highly unsaturated, oleic acid varies from 26.8 to 43.6% and the content of linoleic acid is between 37.2 and 54.9%. The average δ13C values of the main fatty acids are −29.2±1.2‰, −30.3±1.6‰, −27.9±1.7‰ and −28.1±1.7‰ for C16:0, C18:0, C18:1, C18:2, respectively. To determine adulteration, rapeseed, sunflower and soybean oil, were added to pumpkin seed oil in varying percentages (1–10%). A 100% correct classification of both geographical and botanical origin was achieved based on the composition and δ13C values of fatty acids. Principal component analysis (PCA) and regularised discriminant analysis (RDA) analysis gave comparable results.

Harvest locations of goose barnacles can be successfully discriminated using trace elemental signatures.

European Union regulations state that consumers must be rightfully informed about the provenance of fishery products to prevent fraudulent practices. However, mislabeling of the geographical origin is a common practice. It is therefore paramount to develop forensic methods that allow all players involved in the supply chain to accurately trace the origin of seafood. In this study, trace elemental signatures (TES) of the goose barnacle Pollicipes pollicipes, collected from ten sites along the Portuguese coast, were employed to discriminate individual’s origin. Barium (Ba), boron (B), cadmium (Cd), chromium (Cr), lithium (Li), magnesium (Mg), manganese (Mn), phosphorous (P), lead (Pb), strontium (Sr) and zinc (Zn) - were quantified using Inductively Coupled Plasma−Mass Spectrometry (ICP-MS). Significant differences were recorded among locations for all elements. A regularized discriminant analysis (RDA) revealed that 83% of all individuals were correctly assigned. This study shows TES can be a reliable tool to confirm the geographic origin of goose barnacles at fine spatial resolution. Although additional studies are required to ascertain the reliability of TES on cooked specimens and the temporal stability of the signature, the approach holds great promise for the management of goose barnacles fisheries, enforcement of conservation policies and assurance in accurate labeling.

Application of Interactive Regularized Discriminant Analysis to Wine Data

Testing the possibility of determining the geographical origin (country) of wines on the base of chemico-analytical parameters was the aim of the European project ”Establishing of a wine data bank for analytical parameters for wines from Third countries (G6RD-CT-2001-00646-WINE DB)” supported by the European Commission. Therefore a data base containing 400 samples of commercial and authentic wines from Hungary, Czech Republic, Romania and South Africa was created. For each of those samples around 100 analytical parameters, among them rare earth elements and isotopic ratios were measured.Besides other multivariate statistical methods of discrimination and classification the method of regularized discriminant analysis (RDA) was used to distinguish the wines of the different countries on the base of a minimal number of the most important parameters. A MATLAB-program, developed by Vandev (2004) which allows an interactive stepwise discriminant model building on the base of an optimal choice of the “nonlinearity” parameter alpha was used. This program will be described shortly and models for commercial wines with corresponding classification and prediction error rates will be given.As a result of using RDA it was possible to reduce the number of analytical parameters to the eight to infer the geographical origin of these commercial wines.