Linear Discriminant Analysis Method Research Articles

Joint Sparse Local Linear Discriminant Analysis for Feature Dimensionality Reduction of Hyperspectral Images

Although linear discriminant analysis (LDA)-based subspace learning has been widely applied to hyperspectral image (HSI) classification, the existing LDA-based subspace learning methods exhibit several limitations: (1) They are often sensitive to noise and demonstrate weak robustness; (2) these methods ignore the local information inherent in data; and (3) the number of extracted features is restricted by the number of classes. To address these drawbacks, this paper proposes a novel joint sparse local linear discriminant analysis (JSLLDA) method by integrating embedding regression and locality-preserving regularization into the LDA model for feature dimensionality reduction of HSIs. In JSLLDA, a row-sparse projection matrix can be learned, to uncover the joint sparse structure information of data by imposing a L2,1-norm constraint. The L2,1-norm is also employed to measure the embedding regression reconstruction error, thereby mitigating the effects of noise and occlusions. A locality preservation term is incorporated to fully leverage the local geometric structural information of the data, enhancing the discriminability of the learned projection. Furthermore, an orthogonal matrix is introduced to alleviate the limitation on the number of acquired features. Finally, extensive experiments conducted on three hyperspectral image (HSI) datasets demonstrated that the performance of JSLLDA surpassed that of some related state-of-the-art dimensionality reduction methods.

Comparison of dimensionality reduction methods on hyperspectral images for the identification of heathlands and mires

Hyperspectral data and machine learning offer great potential for identifying valuable open ecosystems. Due to the large volume of data, preprocessing of hyperspectral images must involve dimensionality reduction. The main goal of this study was to test the effectiveness of various types of feature reduction (feature selection and feature extraction) when performing classification using the Random Forest algorithm. A comparison was conducted between two ecosystems - heathlands and mires protected as Natura 2000 habitats. Two transformations of feature extraction were chosen, namely Minimum Noise Fraction (MNF) and Principal Component Analysis (PCA), while Linear Discriminant Analysis (LDA) was used as a feature selection method. It was proven that irrespective of the class type, accuracy is higher with the feature extraction method (mean F1 accuracy of 0.922) than with feature selection (mean F1 accuracy of 0.787). At the same time, no significant differences in accuracies were found between the MNF and PCA methods. Although LDA resulted in lower accuracies (0.816 for heathland and 0.750 for mires), the method could also be used due to relatively high F1 values. The effectiveness of the LDA method for feature reduction in open ecosystem identification was confirmed for the first time for open natural vegetation.

I-vector and variability compensation techniques for mobile phone recognition

Mobile phone recognition consists of trying to identify the mobile phone brand or model, which is very important in forensic analysis. In this paper, we exploit the audio recordings to realize this task. Audio data conveys many information, one of them is related to the source of records, i.e., is a specific audio was captured by a given mobile phone. In order to separate between this particular information and the rest of audio content, we introduce the I-vector technique. Additionally, Linear Discriminant Analysis (LDA) and Within-Class Covariance Normalization (WCCN) methods were used in different ways to compensate the speaker and channel variability. We evaluate the performance of our proposed system by correct rate recognition based on Gaussian Probabilistic Linear Discriminant Analysis (GPLDA) and Cosine Similarity Score (CSS) methods. To train the mobile phone models and do tests, two datasets were prepared: the public MOBIPHONE database and a novel local database. Experiments done with different configurations demonstrate the effectiveness of I-vectors models with the combination of LDA – WCCN compensation techniques and CSS as a similarity measurement method. A correct recognition rates of 97.23 %, 98.91 %, and 99.45 % were obtained for MOBIPHONE, the local Same Environment (SE) and Different Environment (DE) databases, respectively.

Conventional Machine Learning and Ensemble Learning Techniques in Cardiovascular Disease Prediction and Analysis

Cardiovascular diseases, which significantly affect the heart and blood vessels, are one of the leading causes of death worldwide. Early diagnosis and treatment of these diseases, which cause approximately 19.1 million deaths, are essential. Many problems, such as coronary artery disease, blood vessel disease, irregular heartbeat, heart muscle disease, heart valve problems, and congenital heart defects, are included in this disease definition. Today, researchers in the field of cardiovascular disease are using approaches based on diagnosis-oriented machine learning. In this study, feature extraction is performed for the detection of cardiovascular disease, and classification processes are performed with a Support Vector Machine, Naive Bayes, Decision Tree, K-Nearest Neighbor, Bagging Classifier, Random Forest, Gradient Boosting, Logistic Regression, AdaBoost, Linear Discriminant Analysis and Artificial Neural Networks methods. A total of 918 observations from Cleveland, Hungarian Institute of Cardiology, University Hospitals of Switzerland, and Zurich, VA Medical Center were included in the study. Principal Component Analysis, a dimensionality reduction method, was used to reduce the number of features in the dataset. In the experimental findings, feature increase with artificial variables was also performed and used in the classifiers in addition to feature reduction. Support Vector Machines, Decision Trees, Grid Search Cross Validation, and existing various Bagging and Boosting techniques have been used to improve algorithm performance in disease classification. Gaussian Naïve Bayes was the highest-performing algorithm among the compared methods, with 91.0% accuracy on a weighted average basis as a result of a 3.0% improvement.

Fingerprint Pattern Feature Extraction for Loop Fingerprint Pattern Identification by Zhang-Suen and Stentiford Thinning Method

Every fingerprint that a human possesses must have a fingerprint pattern and must be unique. Each pattern has a ridge pattern that will not change as long as there is no change due to an accident or injury. This ridge pattern can be used as biometric recognition. Based on the ridge pattern owned, the fingerprint pattern is divided into 4 patterns, including the tented arch pattern, the whorl pattern, the loop pattern (ulnar loop and radial loop). The stage of fingerprint pattern recognition begins by looking for minutiae (termination and bifurcation) points on the fingerprint using the Crossing Number (CN) method. Before searching for CN, the fingerprint image must be processed using the Otsu and Zhang-Suen or Stentiford Thinning methods. Thinning is used to facilitate the extraction of minutiae in fingerprint images. The last stage in obtaining the minutiae point is classified by the Linear Discriminant Analysis (LDA) method. The results obtained from the test data were 30 images, 24 images could be classified correctly, and 4 images could not be recognized. The accuracy rate of the fingerprint pattern identification system is 80%.

A novel linear discriminant analysis based on alternate ratio sum minimization

Linear discriminant analysis (LDA) and its variants are popular supervised dimension reduction methods, which have been widely used to handle high-dimensional data. Since most of previous LDA methods are developed based on the trace ratio (TR) criterion, they usually obtain the low-dimensional data feature with weak discriminative ability, due to the projections with small variance. The ratio sum (RS) criterion was developed to alleviate this drawback. However, the conventional ratio sum is formulated to maximize the arithmetic mean of items which suffers from the domination of the largest objectives and might deteriorate the recognition accuracy in practical applications. In this paper, a novel ratio sum minimization based linear discriminant analysis (RSM-LDA) method is proposed in this paper. Specifically, a new ratio sum minimization (RSM) criterion is developed, which is based on the properties of the harmonic mean and effectively avoids the dominance problem to discover more discriminative features of the data. However, obtaining a closed solution for the RSM-LDA problem is challenging. For this purpose, three optimization methods are used to solve the optimization problem of RSM-LDA, and the inherent relationships between the methods are discussed. Experimental results demonstrate that the proposed RSM-LDA method has better performance in classification tasks on several datasets, when compared with some comparison methods based dimensionality reduction methods. In addition, combined with other experimental results, the effectiveness of RSM-LDA was verified.

Instant Sign Language Recognition by WAR Strategy Algorithm Based Tuned Machine Learning

Sign language serves as the primary means of communication utilized by individuals with hearing and speech disabilities. However, the comprehension of sign language by those without disabilities poses a significant challenge, resulting in a notable disparity in communication across society. Despite the utilization of numerous effective Machine learning techniques, there remains a minor compromise between accuracy rate and computing time when it comes to sign language recognition. A novel sign language recognition system is presented in this paper with an exceptionally accurate and expeditious, which is developed upon the recently devised metaheuristic WAR Strategy optimization algorithm. Following the preprocessing, both of spatial and temporal features has been extracted using the Linear Discriminant Analysis (LDA) and Gray-level cooccurrence matrix (GLCM) methods. Afterward, the WAR Strategy optimization algorithm has been adopted in two procedures, first in optimizing the extracted set of features, and second to fine-tune the hyperparameters of six standard machine learning models in order to achieve precise and efficient sign language recognition. The proposed system was assessed on sign language datasets of different languages (American, Arabic, and Malaysian) containing numerous variations. The proposed system attained a recognition accuracy ranging from 93.11% to 100% by employing multiple optimized machine learning classifiers and training time of 0.038–10.48 s. As demonstrated by the experimental outcomes, the proposed system is exceptionally efficient regarding time, complexity, generalization, and accuracy.

Identification of adulterants in açaí pulp samples using laser-induced fluorescence spectroscopy combined with linear discriminant analysis (LDA) method: A demonstration of the potential applicability of the technique

Identification of adulterants in açaí pulp samples using laser-induced fluorescence spectroscopy combined with linear discriminant analysis (LDA) method: A demonstration of the potential applicability of the technique

Improving flood susceptibility forecasting with an innovative deep learning approach

Floods are often regarded as one of the natural disasters that cause the greatest damage, properly it is notoriously challenging to effectively estimate and forecast flooding. Loss of lives and properties are damaged due to flood disaster and that also affects the basic needs of people to live, property damage, agricultural and animal loss, failure of infrastructure facilities, and deterioration of health due to waterborne infections. To overcome this problem, we proposed an advanced water wave optimized adaptive generative adversarial network (AWWO-AGAN) method. The purpose of our proposed AWWO-AGAN method is to predict the flood susceptibility in order to rescue people from flood disaster. The research collected rainfall regions dataset in India for flood susceptibility and the collected data is preprocessed using Continuous Wavelet Transform (CWT) to clean the unnecessary data. Linear Discriminant Analysis (LDA) method is employed to extract the data features. The simulation findings of the result use a Python tool. The findings demonstrated that the suggested framework performed better than other traditional methods. The metrics used to assess the performance of proposed method in terms of accuracy (96%), Precision (93.25%), sensitivity (94%) and specificity (97%).The parameter of specificity provides a high prediction level of flood susceptibility in suspected areas and the results would be beneficial for real-time flood predictions.

Photovoltaic Power Generation Related Physical Quantity Mining Method Based On Historical Time Series Data Analysis

Abstract The impact of distributed photovoltaic grid-connected on distribution network security, power quality and system stability cannot be ignored. In order to better cope with the uncertainty and output of new energy output and understand the characteristics of distributed new energy power output, it is necessary to predict photovoltaic power. The historical time series data of photovoltaic power is large in dimension and quantity. If the data output is not carried out, a large amount of redundant data will affect the accuracy of photovoltaic prediction. Therefore, this paper proposes a feature selection method based on MIC most mutual information coefficient, which filters out the most relevant data of photovoltaic power generation from the original feature variables, and then performs the feature dimension reduction method of linear discriminant analysis (LDA) to map the high-dimensional data to a lower dimension space. Finally, using the prediction simulation example, using the long and short time neural network (LSTM) prediction comparison, the feature dimension reduction method of MIC feature selection and LDA effectively improves the accuracy of photovoltaic power generation prediction.

Automated collective variable discovery for MFSD2A transporter from molecular dynamics simulations

Biomolecules often exhibit complex free energy landscapes in which long-lived metastable states are separated by large energy barriers. Overcoming these barriers to robustly sample transitions between the metastable states with classical molecular dynamics (MD) simulations presents a challenge. To circumvent this issue, collective variable (CV)-based enhanced sampling MD approaches are often employed. Traditional CV selection relies on intuition and prior knowledge of the system. This approach introduces bias, which can lead to incomplete mechanistic insights. Thus, automated CV detection is desired to gain a deeper understanding of the system/process. Analysis of MD data with various machine-learning algorithms, such as principal component analysis (PCA), support vector machine, and linear discriminant analysis (LDA) based approaches have been implemented for automated CV detection. However, their performance has not been systematically evaluated on structurally and mechanistically complex biological systems. Here, we applied these methods to MD simulations of the MFSD2A (Major Facilitator Superfamily Domain 2A) lysolipid transporter in multiple functionally relevant metastable states with the goal of identifying optimal CVs that would structurally discriminate these states. Specific emphasis was on the automated detection and interpretive power of LDA-based CVs. We found that LDA methods, which included a novel gradient descent-based multiclass harmonic variant, termed GDHLDA, we developed here, outperform PCA in class separation, exhibiting remarkable consistency in extracting CVs critical for distinguishing metastable states. Furthermore, the identified CVs included features previously associated with conformational transitions in MFSD2A. Specifically, conformational shifts in transmembrane helix 7 and in residue Y294 on this helix emerged as critical features discriminating the metastable states in MFSD2A. This highlights the effectiveness of LDA-based approaches in automatically extracting from MD trajectories CVs of functional relevance that can be used to drive biased MD simulations to efficiently sample conformational transitions in the molecular system.

Linear discriminant analysis with trimmed and difference distribution modeling

Linear discriminant analysis (LDA) has been widely used to extract features to solve various machine learning problems. However, the current LDA methods can be confused by highly dispersed samples. To address this problem, we develop a new LDA method based on a new expression of the theoretical error in classification. The method first combines different class distributions and then constructs the projection matrix on the basis of the statistics of the combined distribution. This is very different from conventional methods that estimate the statistics from individual classes first. The statistics are then combined to construct the projection matrix. The merit of the proposed approach is that it selects discriminant pairs of the combined distribution and discards the scatter structure caused by the dispersed samples. The statistics of this trimmed distribution are less sensitive to the dispersed samples and produce a robust projection matrix. The pairs are selected by a trimming model. The solution is obtained by the alternating direction method of multipliers (ADMM). The proposed LDA method is compared with 17 other LDA methods over 13 datasets belonging to the UC Irvine (UCI) repository, the knowledge extraction based on evolutionary learning (KEEL) repository, and publicly available image websites with different numbers of training samples. The experimental results indicate that the proposed method outperforms the existing methods in many cases.

High dimensional discriminant rules with shrinkage estimators of the covariance matrix and mean vector

Linear discriminant analysis (LDA) is a typical method for classification problems with large dimensions and small samples. There are various types of LDA methods that are based on the different types of estimators for the covariance matrices and mean vectors. In this paper, we consider shrinkage methods based on a non-parametric approach. For the precision matrix, methods based on the sparsity structure or data splitting are examined. Regarding the estimation of mean vectors, Non-parametric Empirical Bayes (NPEB) methods and Non-parametric Maximum Likelihood Estimation (NPMLE) methods, also known as f-modeling and g-modeling, respectively, are adopted. The performance of linear discriminant rules based on combined estimation strategies of the covariance matrix and mean vectors are analyzed in this study. Particularly, the study presents a theoretical result on the performance of the NPEB method and compares it with previous studies. Simulation studies with various covariance matrices and mean vector structures are conducted to evaluate the methods discussed in this paper. Furthermore, real data examples such as gene expressions and EEG data are also presented.

Comparison of Matrix Decomposition in Null Space-Based LDA Method

Problems with small sample sizes and high dimensionality are common in pattern recognition. Almost all machine learning algorithms degrade in high-dimensional data, so that singularities in the scatter matrices, the main problem of the Linear Discriminant Analysis (LDA) technique, might result. A null space-based LDA (NLDA) has been conceived to address the singularity issue. NLDA aims to maximize the distance between classes in the null space of the within-class scatter matrix. In the first research, the NLDA method was performed by computing the eigenvalue decomposition and singular value decomposition (SVD). This research led to several new implementations of the NLDA method that use other matrix decompositions. The new implementations include NLDA using Cholesky decomposition and NLDA using QR decomposition. This paper compares the performance of three NLDA methods that use different matrix decompositions, namely, SVD, Cholesky decomposition, and QR decomposition. Two sets of data were used in the experiments that used three different NLDA algorithms. To determine the performance of the NLDA methods, the classification accuracy of the three methods was measured using the confusion matrix. The results show that the NLDA method using SVD has the best performance when compared to the other two methods, achieving a precision of 77.8% accuracy for the Colon dataset and a precision of 98.8% accuracy for the TKI-resistance dataset.

Modified wild Horse Herd Optimization based Bhattacharyya error constraint (BEC) based L2-norm Linear discriminant analysis (LDA) method for the sentiment analysis

People's opinions are analyzed via sentiment analysis in all fields. Reviews and tweets, among other formats, are used to express opinions. Irony, sarcasm, and other difficult-to-discern hidden meanings can occasionally be found in viewpoints. Artificial intelligence must be used to examine the sentiments as a result. We suggest a unique Bhattacharyya error constraint (BEC) based L2-norm linear discriminant analysis (LDA) because some of the earlier efforts lack optimization. There are some overfitting and class disparity issues in this. To address this, we used a brand-new method called Modified Wild Horse Herd Optimization (MHHO). The experiment is run to evaluate the performance of the suggested strategy and to compare it to other approaches already in use. We have used performance measures for comparison, and the results demonstrate that the suggested method successfully assesses the sentiment from the acquired dataset.

Towards robust and sparse linear discriminant analysis for image classification

Linear discriminant analysis (LDA) is a popular dimensionality reduction technique that has been widely used in pattern recognition. However, there exist a large number of redundant features and corrupted noise in real-world applications, which makes the performance of existing LDA methods degrade and thus leads to a decrease in classification accuracy. To address the above issues, we propose a novel robust and sparse LDA formulation dubbed RSLDA+. The key idea is introducing the mixed sparse regularization, i.e., ℓ0-norm plus ℓ2,0-norm, for feature representation and enforce ℓ0-norm for noise reduction. Furthermore, an optimization algorithm based on the alternating direction method of multipliers (ADMM) is developed in combination with hard thresholding operators. Extensive experiments on six common image datasets verify that the proposed RSLDA+ outperforms state-of-the-art LDA variants in classification accuracy. In addition, the ablation, robustness, convergence, stability, and sparsity are analyzed in detail. The results suggest that the proposed RSLDA+ provides an effective and robust method for image classification.

Prediction of technological properties of wheat flour by combination of UV-VIS-NIR spectroscopy and multivariate analysis methods

Over the last decades, optical spectroscopy methods that do not require complex sample preparation have been widely used to identify and control the composition of food products. In the present study, the possibility of using UV-VIS-NIR spectroscopy combined with multivariate analysis for grading wheat flour into groups differing in technological properties was analyzed. UV-VIS-NIR spectra contain information on the combination and intensity of absorption bands assigned to functional groups of the composition components and determining the technological properties of wheat flour. The database of UV-VIS-NIR spectra of wheat flour samples differing by technological properties was formed into three groups: the first group — wheat flour samples with good baking properties, the second group — with reduced baking properties, the third group — with low baking properties. The visible range of UV-VIS-NIR diffuse reflectance spectrum was used to calculate the color coordinates in the CIE colorimetric system L*a*b*. The greatest difference among the groups in the color coordinates of the samples was found for the coordinate b*, which is associated with the different content of coloring pigments. The spectra database was used to build a classification model for grading wheat flour into quality groups using a combination of principal component analysis and linear discriminant analysis (PCA-LDA) methods. The achieved results indicate that the classification model built on the training sample is able to distinguish wheat flour spectra by quality groups with an accuracy of 96.49%. The effective- ness of the model is verified using a test set of spectra of wheat flour samples. The present study confirms that the combina- tion of UV-VIS-NIR spectroscopy in conjunction with the PCA-LDA method has significant potential for determining a quality group of wheat flour based on technological properties.

Support vector machine with discriminative low‐rank embedding

AbstractSupport vector machine (SVM) is a binary classifier widely used in machine learning. However, neglecting the latent data structure in previous SVM can limit the performance of SVM and its extensions. To address this issue, the authors propose a novel SVM with discriminative low‐rank embedding (LRSVM) that finds a discriminative latent low‐rank subspace more suitable for SVM classification. The extension models of LRSVM are introduced by imposing different orthogonality constraints to prevent computational inaccuracies. A detailed derivation of the authors’ iterative algorithms are given that is essentially for solving the SVM on the low‐rank subspace. Additionally, some theorems and properties of the proposed models are presented by the authors. It is worth mentioning that the subproblems of the proposed algorithms are equivalent to the standard or the weighted linear discriminant analysis (LDA) problems. This indicates that the projection subspaces obtained by the authors’ algorithms are more suitable for SVM classification compared to those from the LDA method. The convergence analysis for the authors proposed algorithms are also provided. Furthermore, the authors conduct experiments on various machine learning data sets to evaluate the algorithms. The experiment results show that the authors’ algorithms perform significantly better than other algorithms, which indicates their superior abilities on classification tasks.

On the Equivalence of Linear Discriminant Analysis and Least Squares Regression.

Studying the relationship between linear discriminant analysis (LDA) and least squares regression (LSR) is of great theoretical and practical significance. It is well-known that the two-class LDA is equivalent to an LSR problem, and directly casting multiclass LDA as an LSR problem, however, becomes more challenging. Recent study reveals that the equivalence between multiclass LDA and LSR can be established based on a special class indicator matrix, but under a mild condition which may not hold under the scenarios with low-dimensional or oversampled data. In this article, we show that the equivalence between multiclass LDA and LSR can be established based on arbitrary linearly independent class indicator vectors and without any condition. In addition, we show that LDA is also equivalent to a constrained LSR based on the data-dependent indicator vectors. It can be concluded that under exactly the same mild condition, such two regressions are both equivalent to the null space LDA method. Illuminated by the equivalence of LDA and LSR, we propose a direct LDA classifier to replace the conventional framework of LDA plus extra classifier. Extensive experiments well validate the above theoretic analysis.

"Susceptibility MRI in Brain Nuclei Improves Early Diagnosis of Parkinson’s Disease Using Linear Discriminant Analysis Method"

"Susceptibility MRI in Brain Nuclei Improves Early Diagnosis of Parkinson’s Disease Using Linear Discriminant Analysis Method"