Fisher Discriminant Analysis Research Articles

A Review of Statistical-Based Fault Detection and Diagnosis with Probabilistic Models

As industrial processes grow increasingly complex, fault identification becomes challenging, and even minor errors can significantly impact both productivity and system safety. Fault detection and diagnosis (FDD) has emerged as a crucial strategy for maintaining system reliability and safety through condition monitoring and abnormality recovery to manage this challenge. Statistical-based FDD methods that rely on large-scale process data and their features have been developed for detecting faults. This paper overviews recent investigations and developments in statistical-based FDD methods, focusing on probabilistic models. The theoretical background of these models is presented, including Bayesian learning and maximum likelihood. We then discuss various techniques and methodologies, e.g., probabilistic principal component analysis (PPCA), probabilistic partial least squares (PPLS), probabilistic independent component analysis (PICA), probabilistic canonical correlation analysis (PCCA), and probabilistic Fisher discriminant analysis (PFDA). Several test statistics are analyzed to evaluate the discussed methods. In industrial processes, these methods require complex matrix operation and cost computational load. Finally, we discuss the current challenges and future trends in FDD.

CDPNet: a radiomic feature learning method with epigenetic application to estimating MGMT promoter methylation status in glioblastoma.

Radiomics has been widely recognized for its effectiveness in decoding tumor phenotypes through the extraction of quantitative imaging features. However, the robustness of radiomic methods to estimate clinically relevant biomarkers non-invasively remains largely untested. In this study, we propose Cascaded Data Processing Network (CDPNet), a radiomic feature learning method to predict tumor molecular status from medical images. We apply CDPNet to an epigenetic case, specifically targeting the estimation of O6-methylguanine-DNA-methyltransferase (MGMT) promoter methylation from Magnetic Resonance Imaging (MRI) scans of glioblastoma patients. CDPNet has three components: 1) Principal Component Analysis (PCA), 2) Fisher Linear Discriminant (FLD), and 3) a combination of hashing and blockwise histograms. The outlined architectural framework capitalizes on PCA to reconstruct input image patches, followed by FLD to extract discriminative filter banks, and finally using binary hashing and blockwise histogram module for indexing, pooling, and feature generation. To validate the effectiveness of CDPNet, we conducted an exhaustive evaluation on a comprehensive retrospective cohort comprising 484 IDH-wildtype glioblastoma patients with pre-operative multi-parametric MRI scans (T1, T1-Gd, T2, and T2-FLAIR). The prediction of MGMT promoter methylation status was cast as a binary classification problem. The developed model underwent rigorous training via 10-fold cross-validation on a discovery cohort of 446 patients. Subsequently, the model's performance was evaluated on a distinct and previously unseen replication cohort of 38 patients. Our method achieved an accuracy of 70.11% and an area under the curve of 0.71 (95% CI: 0.65 - 0.74).

Trace the origin of yak meat in Xizang based on stable isotope combined with multivariate statistics

In this study, the feasibility of tracing the origin of yak meat in Xizang Autonomous Region based on stable isotope combined with multivariable statistics was researched. The δ13C, δ15N, δ2H and δ18O in yak meat were determined by stable isotope ratio mass spectrometry, and the data were analyzed by analysis of variance, fisher discriminant analysis (FDA), back propagation (BP) neural network and orthogonal partial least squares discrimination analysis (OPLS-DA). The results showed that the δ13C, δ15N, δ2H and δ18O had significant differences among different origins (P < 0.05). The overall original correct discrimination rate of fisher discriminant analysis was 89.7 %, and the correct discrimination rate of cross validation was 88.2 %. The correct classification rate of BP neural network based on training set was 93.38 %, and the correct classification rate of BP neural network based on test set was 89.83 %. The OPLS-DA model interpretation rate parameter R2Y was 0.67, the model prediction rate parameter Q2 was 0.409, which could distinguish yak meat from seven different producing areas in Xizang Autonomous Region. The results showed that the origin of yak meat in Xizang Autonomous Region can be traced based on stable isotope combined with multivariate statistics.

Discriminative sparse subspace learning with manifold regularization

Common subspace learning methods only utilize local or global structure in feature extraction, and cannot obtain the global optimal discriminative projection matrix. For this reason, this paper proposes a discriminative sparse subspace learning method based on the manifold regularization framework (DSSL-MR), which introduces the graph Laplacian matrix that reflects the intrinsic geometric structure of the sample as a penalty term. DSSL-MR simultaneously uses both sub-manifold and multi-manifold information of samples for obtaining optimal projection to enhance the discriminability of different classes in subspace. DSSL-MR uses the sparse property of the L2,1-norm to constrain the projection matrix, which can eliminate redundant features and select features that are significant for classification. It is a linear supervised method, which belongs to the Fisher discriminant analysis framework. Experimental results on multiple real-world datasets show that the algorithm is very effective in classification and has high recognition rates.

Hybrid learning based on Fisher linear discriminant

Hybrid learning is an excellent method that combines the global information of data with the local information of data. Different from the known hybrid learning algorithms, in this paper we propose a new hybrid learning strategy for Fisher Linear Discriminant (FLD) and introduce novel hybrid learning based on FLD (HL-FLD). The main idea of HL-FLD is to obtain the global structural information by FLD firstly, and then use the obtained global information to divide the given data locally in a more detailed way. To study systematically the proposed HL-FLD, we not only establish the generalization bound of HL-FLD and prove that the proposed HL-FLD algorithm is consistent, but also present some discussions on HL-FLD. Since splitting the blocks of a given data is a hyperparameter, we also improve HL-FLD and introduce another new self-adaptive hybrid learning based on FLD (SHL-FLD). The experimental researches for benchmark repository confirm that the proposed two algorithms have better performance in terms of misclassification rates and total time.

ILFDA Model: An Online Soft Measurement Method Using Improved Local Fisher Discriminant Analysis

With the advancement of soft measurement, just-in-time learning (JITL) has become a widely adopted framework for online soft-sensing modeling in industrial processes. However, traditional JITL model approaches often rely on simple similarity measures like Euclidean distance, resulting in the underutilization of labeled data. This paper proposes a supervised, improved local Fisher discriminant analysis method based on a JITL framework and local Fisher discriminant analysis (LFDA) to improve data utilization efficiency. In particular, by incorporating the indirect correlation information matrix, this method integrates the inter-class and intra-class dispersion matrix, overcoming the limitation of the LFDA algorithm that only captures direct data correlations. We select two different carbon depositions in the Methanol-to-Olefin reaction system for comparative experiments and use the root mean squared error (RMSE) and R-square (R2) to evaluate the effectiveness of the proposed method. Fitting results show that two kinds of carbon depositions were better than the control model, namely the RMSE of the model were 0.1431 and 0.1513, R2 were 0.8952 and 0.8707.

Flow regime classification using various dimensionality reduction methods and AutoML

Accurate identification of flow regimes is paramount in several industries, especially in chemical and hydrocarbon sectors. This paper describes a comprehensive data-driven workflow for flow regime identification. The workflow encompasses: i) the collection of dynamic pressure signals using an experimentally verified numerical two-phase flow model for three different flow regimes: stratified, slug and annular flow, ii) feature extraction from pressure signals using Discrete Wavelet Transformation (DWT), iii) Evaluation and testing of 12 different Dimensionality Reduction (DR) techniques, iv) the application of an AutoML framework for automated Machine Learning classifier selection among K-Nearest Neighbors, Artificial Neural Networks, Support Vector Machines, Gradient Boosting, Random Forest, and Logistic Regression, with hyper-parameter tuning. Kernel Fisher Discriminant Analysis (KFDA) is the best DR technique, exhibiting superior goodness of clustering, while KNN proved to be the top classifier with an accuracy of 92.5 % and excellent repeatability. The combination of DWT, KFDA and KNN was used to produce a virtual flow regime map. The proposed workflow represents a significant step forward in automating flow regime identification and enhancing the interpretability of ML classifiers, allowing its application to opaque pipes fitted with pressure sensors for achieving flow assurance and automatic monitoring of two-phase flow in various process industries.

A Robust Semi-Supervised Fisher Discriminant Analysis for Industrial Fault Classification

A Robust Semi-Supervised Fisher Discriminant Analysis for Industrial Fault Classification

Analysis of Quality Differences in Radix Dipsaci before and after Processing with Salt Based on Quantitative Control of HPLC Multi-Indicator Components Combined with Chemometrics.

Radix Dipsaci (RD) is the dry root of the Dipsacus asper Wall. ex DC., which is commonly used for tonifying the kidney and strengthening bone. The purpose of this study was to analyze the difference between raw and salt-processed RD from the chemical composition comprehensively. The fingerprints of raw and salt-processed RD were established by HPLC-DAD to determine the contents of loganin (LN), asperosaponin VI (AVI), caffeic acid (CaA), dipsanoside A (DA), dipsanoside B (DB), chlorogenic acid (CA), loganic acid (LA), isochlorogenic acid A (IA), isochlorogenic acid B (IB), and isochlorogenic acid C (IC). The results showed that after processing with salt, the components with increased contents were LA, CaA, DA, and AVI, and the components with decreased contents were CA, LN, IB, IA, IC, and DB. Then, the chemometric methods such as principal component analysis (PCA) and fisher discriminant analysis (FDA) were used to evaluate the quality of raw and salt-processed RD. In the classification of raw and salt-processed RD, the order of importance of each chemical component was LA > DB > IA > IC > IB > LN > CA > DA > AVI > CaA. These integrated methods successfully assessed the quality of raw and salt-processed RD, which will provide guidance for the development of RD as a clinical medication.

Comparative Study on the Effects of Four Plant Protein Sources on the Liver and Intestinal Health of Largemouth Bass, Micropterus salmoides.

The effects of plant protein sources (PPSs) on the health of the liver and intestine of the largemouth bass, Micropterus salmoides, were compared to verify the potential damaging effects of dietary fiber (DF). A diet containing 55% fish meal (FM) was used as the control. The test diets contained 25% soybean meal (SBM), rapeseed meal (RSM), cottonseed meal, or peanut meal, and the FM content was decreased to 30%. The protein and lipid contents of these five diets were balanced by casein and oil. Fish were raised for 8 weeks. The fish fed the diet containing PPS showed a trend of decreasing growth and apparent digestibility coefficients. The contents of total bile acid, lipid, and collagen in the liver were increased, and the mRNA expression levels of genes encoding inflammatory factors and enzymes involved in de novo fatty acid synthesis and bile acid synthesis were upregulated. Both the lipid and collagen contents in the liver were positively correlated with the DF content in the diet significantly. Morphology and histology showed reduced liver size, hepatic steatosis, and fibrosis in fish fed diets containing PPS. The lowest hepatosomatic index was observed in fish fed the SBM diet, and the most severe damage was observed in fish fed the RSM diet. No obvious histological abnormalities were observed in the hindgut. The bile acid profile in the liver could be used to distinguish the types of PPS very well by Fisher discriminant analysis. These results indicated that 25% of each of the four PPSs in the diet exceeded the tolerance range of largemouth bass and caused liver damage, which might be mediated by bile acid. DF in PPS might be an important agent contributing to liver damage.

Landslide susceptibility analysis in the Bhilangana Basin (India) using GIS-based machine learning methods

Landslides are frequent natural hazards in mountainous regions, and harshly upset people's lives and livelihoods. In the present study, we have carried out an analysis of seven GIS-based machine-learning techniques; and asses their performance for landslide susceptibility mapping (LSM) in the Bhilangana Basin, Garhwal Himalaya. A landslide inventory consisting of 423 polygons was prepared using repeated field investigations, and multi-dated satellite images for the periods between 2000 and 2022. The landslide dataset was classified into two groups: training (70%) and test dataset (30%), and 12 predictive variables were used for the LSM. The methods used to produce LSM are boosted regression tree (BRT), Fisher discriminant analysis (FDA), generalized linear model (GLM), multivariate adaptive regression splines (MARS), model-architect analysis (MDA), random forest (RF) and support vector machine (SVM). The sensitivity and performance of these models to predict landslide susceptible areas were carried out using the area under the curve (AUC) method. The RF model (AUC = 0.988) has given the highest precision indicating the best performance. Though MARS (0.974), SVM (0.965) and MDA (0.952) models have also performed adequately for the LSM (all have AUC values above 0.95), however, it is recommended that the RF model is highly suitable for LSM in the mountainous region.

Identification of core carcinogenic elements based on the age-standardized mortality rate of lung cancer in Xuanwei Formation coal in China.

In this study, the core carcinogenic elements in Xuanwei Formation coal were identified. Thirty-one samples were collected based on the age-standardized mortality rate (ASMR) of lung cancer; Si, V, Cr, Co, Ni, As, Mo, Cd, Sb, Pb, and rare earth elements and yttrium (REYs) were analyzed and compared; multivariate statistical analyses (CA, PCA, and FDA) were performed; and comprehensive identification was carried out by combining multivariate statistical analyses with toxicology and mineralogy. The final results indicated that (1) the high-concentration Si, Ni, V, Cr, Co, and Cd in coal may have some potential carcinogenic risk. (2) The concentrations of Cr, Ni, As, Mo, Cd, and Pb meet the zoning characteristics of the ASMR, while the Si concentration is not completely consistent. (3) The REY distribution pattern in Longtan Formation coal is lower than that in Xuanwei Formation coal, indicating that the materials of these elements in coal are different. (5) The heatmap divides the sampling sites into two clusters and subtypes in accordance with carcinogenic zoning based on the ASMR. (6) PC1, PC2, and PC3 explain 62.629% of the total variance, identifying Co, Ni, As, Cd, Mo, Cr, and V. (7) Fisher discriminant analysis identifies Ni, Si, Cd, As, and Co based on the discriminant function. (8) Comprehensive identification reveals that Ni is the primary carcinogenic element, followed by Co, Cd, and Si in combination with toxicology. (9) The paragenesis of Si (nanoquartz), Ni, Co, and Cd is an interesting finding. In other words, carcinogenic elements Ni, Co, Cd, and Si and their paragenetic properties should receive more attention.

Test

Image steganalysis ranges from detecting the presence of covert information in an image (passive steganalysis) to extraction of the information from the stego image (active steganalysis). One of the steps in active steganalysis is determining the stego algorithm used to produce the stego image. In this paper, a low dimensional combination of textural features is adapted for steganalysis. Also a novel blind statistical steganalyser to determine the spatial domain Least Significant Bit (LSB) based algorithms using one against one multi class classification is proposed. The proposed steganalyser is a multi class ensemble Fisher Linear Discriminant (FLD) classifier that uses novel low dimensional textural features for steganalysis. The performed experiments on the Bossbase database for 5 different LSB based algorithms for 8 different payloads show that the results are much better than the state of art steganalyser

Low Dimensional Multi Class Steganalysis of Spatial LSB based Stego Images Using Textural Features

Image steganalysis ranges from detecting the presence of covert information in an image (passive steganalysis) to extraction of the information from the stego image (active steganalysis). One of the steps in active steganalysis is determining the stego algorithm used to produce the stego image. In this paper, a low dimensional combination of textural features is adapted for steganalysis. Also a novel blind statistical steganalyser to determine the spatial domain Least Significant Bit (LSB) based algorithms using one against one multi class classification is proposed. The proposed steganalyser is a multiclass ensemble Fisher Linear Discriminant (FLD) classifier that uses novel low dimensional textural features for steganalysis. The performed experiments on the Bossbase database for 5 different LSB based algorithms for 8 different payloads show that the results are much better than the state of art steganalyser.

Identification of reservoir types in deep carbonates based on mixed-kernel machine learning using geophysical logging data

Identification of reservoir types in deep carbonates has always been a great challenge due to complex logging responses caused by the heterogeneous scale and distribution of storage spaces. Traditional cross-plot analysis and empirical formula methods for identifying reservoir types using geophysical logging data have high uncertainty and low efficiency, which cannot accurately reflect the nonlinear relationship between reservoir types and logging data. Recently, the kernel Fisher discriminant analysis (KFD), a kernel-based machine learning technique, attracts attention in many fields because of its strong nonlinear processing ability. However, the overall performance of KFD model may be limited as a single kernel function cannot simultaneously extrapolate and interpolate well, especially for highly complex data cases. To address this issue, in this study, a mixed kernel Fisher discriminant analysis (MKFD) model was established and applied to identify reservoir types of the deep Sinian carbonates in central Sichuan Basin, China. The MKFD model was trained and tested with 453 datasets from 7 coring wells, utilizing GR, CAL, DEN, AC, CNL and RT logs as input variables. The particle swarm optimization (PSO) was adopted for hyper-parameter optimization of MKFD model. To evaluate the model performance, prediction results of MKFD were compared with those of basic-kernel based KFD, RF and SVM models. Subsequently, the built MKFD model was applied in a blind well test, and a variable importance analysis was conducted. The comparison and blind test results demonstrated that MKFD outperformed traditional KFD, RF and SVM in the identification of reservoir types, which provided higher accuracy and stronger generalization. The MKFD can therefore be a reliable method for identifying reservoir types of deep carbonates.

Two-phase flow regime identification using multi-method feature extraction and explainable kernel Fisher discriminant analysis

PurposeIdentifying the flow regime is a prerequisite for accurately modeling two-phase flow. This paper aims to introduce a comprehensive data-driven workflow for flow regime identification.Design/methodology/approachA numerical two-phase flow model was validated against experimental data and was used to generate dynamic pressure signals for three different flow regimes. First, four distinct methods were used for feature extraction: discrete wavelet transform (DWT), empirical mode decomposition, power spectral density and the time series analysis method. Kernel Fisher discriminant analysis (KFDA) was used to simultaneously perform dimensionality reduction and machine learning (ML) classification for each set of features. Finally, the Shapley additive explanations (SHAP) method was applied to make the workflow explainable.FindingsThe results highlighted that the DWT + KFDA method exhibited the highest testing and training accuracy at 95.2% and 88.8%, respectively. Results also include a virtual flow regime map to facilitate the visualization of features in two dimension. Finally, SHAP analysis showed that minimum and maximum values extracted at the fourth and second signal decomposition levels of DWT are the best flow-distinguishing features.Practical implicationsThis workflow can be applied to opaque pipes fitted with pressure sensors to achieve flow assurance and automatic monitoring of two-phase flow occurring in many process industries.Originality/valueThis paper presents a novel flow regime identification method by fusing dynamic pressure measurements with ML techniques. The authors’ novel DWT + KFDA method demonstrates superior performance for flow regime identification with explainability.

Discriminative manifold domain adaptation for cross-domain fault diagnosis of rotating machineries

Domain adaptation aims to solve the issue of distribution shift. Manifold learning uses Riemann metrics as nonlinear distance to overcome the limitation of Euclidean metric in domain adaptation. However, it ignores the feature distortion and relationship change caused by the feature mapping framework. Class overlap leads to degradation of classifier performance. Thus, this study proposes a new discriminative manifold domain adaptation (DMDA) framework to solve these problems. DMDA can align manifold structures and domain distributions through discriminative manifold alignment (DMA) and joint distribution alignment, respectively. The DMA method is developed via a new pseudo label voting mechanism, the local fisher discriminant analysis, and the geodesic flow kernel. It can achieve the manifold structure alignment and learn the discriminative features. Moreover, a structural risk minimization classifier is constructed to achieve Laplacian regularization and align the joint distributions of the source domain and target domain. It also implements the fault classification task. The experimental results on bearing and planetary gearbox datasets prove that DMDA has better fault transfer diagnosis performance than the typical domain adaptation methods based on machine learning.

Dynamic Feature Extraction-Based Quadratic Discriminant Analysis for Industrial Process Fault Classification and Diagnosis.

This paper introduces a novel method for enhancing fault classification and diagnosis in dynamic nonlinear processes. The method focuses on dynamic feature extraction within multivariate time series data and utilizes dynamic reconstruction errors to augment the feature set. A fault classification procedure is then developed, using the weighted maximum scatter difference (WMSD) dimensionality reduction criterion and quadratic discriminant analysis (QDA) classifier. This method addresses the challenge of high-dimensional, sample-limited fault classification, offering early diagnosis capabilities for online samples with smaller amplitudes than the training set. Validation is conducted using a cold rolling mill simulation model, with performance compared to classical methods like linear discriminant analysis (LDA) and kernel Fisher discriminant analysis (KFD). The results demonstrate the superiority of the proposed method for reliable industrial process monitoring and fault diagnosis.

A Multi-Layer Classifier Model XR-KS of Human Activity Recognition for the Problem of Similar Human Activity

Sensor-based human activity recognition is now well developed, but there are still many challenges, such as insufficient accuracy in the identification of similar activities. To overcome this issue, we collect data during similar human activities using three-axis acceleration and gyroscope sensors. We developed a model capable of classifying similar activities of human behavior, and the effectiveness and generalization capabilities of this model are evaluated. Based on the standardization and normalization of data, we consider the inherent similarities of human activity behaviors by introducing the multi-layer classifier model. The first layer of the proposed model is a random forest model based on the XGBoost feature selection algorithm. In the second layer of this model, similar human activities are extracted by applying the kernel Fisher discriminant analysis (KFDA) with feature mapping. Then, the support vector machine (SVM) model is applied to classify similar human activities. Our model is experimentally evaluated, and it is also applied to four benchmark datasets: UCI DSA, UCI HAR, WISDM, and IM-WSHA. The experimental results demonstrate that the proposed approach achieves recognition accuracies of 97.69%, 97.92%, 98.12%, and 90.6%, indicating excellent recognition performance. Additionally, we performed K-fold cross-validation on the random forest model and utilized ROC curves for the SVM classifier to assess the model’s generalization ability. The results indicate that our multi-layer classifier model exhibits robust generalization capabilities.

Exponential Local Fisher Discriminant Analysis with Sparse Variables Selection: A Novel Fault Diagnosis Scheme for Industry Application

Local Fisher discriminant analysis (LFDA) has been widely applied to dimensionality reduction and fault classification fields. However, it often suffers from small sample size (SSS) problem and incorporates all process variables without emphasizing the key faulty ones, thus leading to degraded fault diagnosis performance and poor model interpretability. To this end, this paper develops the sparse variables selection based exponential local Fisher discriminant analysis (SELFDA) model, which can overcome the two limitations of basic LFDA concurrently. First, the responsible faulty variables are identified automatically through the least absolute shrinkage and selection operator, and the current optimization problem are subsequently recast as an iterative convex optimization problem and solved by the minimization-maximization method. After that, the matrix exponential strategy is implemented on LFDA, it can essentially overcome the SSS problem by ensuring that the within-class scatter matrix is always full-rank, thus more practical in real industrial practices, and the margin between different categories is enlarged due to the distance diffusion mapping, which is benefit for the enhancement of classification accuracy. Finally, the Tennessee Eastman process and a real-world diesel working process are employed to validate the proposed SELFDA method, experimental results prove that the SELFDA framework is more excellent than the other approaches.