Kernel Local Fisher Discriminant Analysis Research Articles

A novel imbalanced fault diagnosis method integrated KLFDA with improved cost‐sensitive learning ANBSVM

AbstractFault diagnosis, as an important approach to ensure the safety and stability of industrial processes, has been widely studied in recent years. During the running process, it is noted that the normal data are always much more than the fault data, which demonstrates imbalanced characteristics and leads to a negative effect on the overall accuracy of fault diagnosis. Targeting the problem, a novel imbalanced fault diagnosis method integrated kernel local Fisher discriminant analysis (KLFDA) with improved adaptive near‐Bayesian support vector machine (ANBSVM) is proposed in this paper. First, KLFDA is used to extract the non‐linear features while maintaining the local spatial structure of the data by introducing flow pattern learning. Second, considering the imbalance characteristics of the data, the data set is divided into a majority class (normal data) and a minority class (fault data). The density distributions of the two classes in their overlapping region are characterized by the proportional function of variance. Third, by minimizing the Bayesian error under the proportion function, the weight factors are adaptively obtained and then introduced into the objective function of the support vector machine (SVM). Namely, a cost sensitivity‐based ANBSVM classifier for fault diagnosis is constructed. Finally, by the simulation experiment on the Tennessee Eastman (TE) process, the comparison results show that the proposed ANBSVM‐based fault diagnosis method makes progress in the performance of fault diagnosis with higher diagnostic accuracy and F1 score.

A multi‐fault diagnosis method based on improved SMOTE for class‐imbalanced data

AbstractWith the development of industrial processes, how to effectively diagnose the faults in an increasingly complex production process has attracted widespread attention. It is worth noting that there may be multiple types of faults in the actual industrial process, and there is an extreme class imbalance between the normal samples and the fault samples. Therefore, it is of practical significance to carry out research on the multi‐fault diagnosis method for class‐imbalanced data. In this paper, a multi‐fault diagnosis method based on improved synthetic minority sampling technology (SMOTE) is proposed. First, aiming at the class imbalance, an improved SMOTE algorithm based on Mahalanobis distance (Mahalanobis distance‐based SMOTE [MSMOTE]) is proposed for oversampling. As the Euclidean distance in the traditional SMOTE algorithm does not consider the coupling relationship between features, the Mahalanobis distance is introduced, which is not dependent on the scale and eliminates the influence of different dimensions. Second, in order to better obtain the global and local information of the sample, the kernel local Fisher discriminant analysis (KLFDA) algorithm is used for feature extraction. Third, a multi‐fault diagnosis model based on the AdaBoost.M2 classifier is constructed in which the decision tree is introduced as the weak classifier. The Adaboost.M2 algorithm integrates multiple decision trees by setting the sample weight, the label weight, and the classifier weight, which effectively improve the classification accuracy by only using the decision tree. Finally, the Tennessee Eastman process is used to conduct case studies. For the comparison results, the proposed multi‐fault diagnosis method based on improved SMOTE has higher accuracy and F1‐Score.

KLFDAPC: a supervised machine learning approach for spatial genetic structure analysis.

Geographic patterns of human genetic variation provide important insights into human evolution and disease. A commonly used tool to detect and describe them is principal component analysis (PCA) or the supervised linear discriminant analysis of principal components (DAPC). However, genetic features produced from both approaches could fail to correctly characterize population structure for complex scenarios involving admixture. In this study, we introduce Kernel Local Fisher Discriminant Analysis of Principal Components (KLFDAPC), a supervised non-linear approach for inferring individual geographic genetic structure that could rectify the limitations of these approaches by preserving the multimodal space of samples. We tested the power of KLFDAPC to infer population structure and to predict individual geographic origin using neural networks. Simulation results showed that KLFDAPC has higher discriminatory power than PCA and DAPC. The application of our method to empirical European and East Asian genome-wide genetic datasets indicated that the first two reduced features of KLFDAPC correctly recapitulated the geography of individuals and significantly improved the accuracy of predicting individual geographic origin when compared to PCA and DAPC. Therefore, KLFDAPC can be useful for geographic ancestry inference, design of genome scans and correction for spatial stratification in GWAS that link genes to adaptation or disease susceptibility.

Research on complex attribute big data classification based on iterative fuzzy clustering algorithm

In order to overcome the low classification accuracy of traditional methods, this paper proposes a new classification method of complex attribute big data based on iterative fuzzy clustering algorithm. Firstly, principal component analysis and kernel local Fisher discriminant analysis were used to reduce dimensionality of complex attribute big data. Then, the Bloom Filter data structure is introduced to eliminate the redundancy of the complex attribute big data after dimensionality reduction. Secondly, the redundant complex attribute big data is classified in parallel by iterative fuzzy clustering algorithm, so as to complete the complex attribute big data classification. Finally, the simulation results show that the accuracy, the normalized mutual information index and the Richter’s index of the proposed method are close to 1, the classification accuracy is high, and the RDV value is low, which indicates that the proposed method has high classification effectiveness and fast convergence speed.

A Novel Three-stage Feature Fusion Methodology and its Application in Degradation State Identification for Hydraulic Pumps

Abstract The performance of feature is essential to the degradation state identification for hydraulic pumps. The initial feature set extracted from the vibration signal of the hydraulic pump is often high-dimensional and contains redundant information, which undermines the effectiveness of the feature set. The novel three-stage feature fusion scheme proposed in this paper aims to enhance the performance of the original features extracted from the vibration signal. First, sparse local Fisher discriminant analysis (SLFDA) performs intra-set fusion within the two original feature sets, respectively. SLFDA has a good effect on samples with intra-class multimodality, and the feature set fused by it has obvious multivariate normal distribution characteristics, which is conducive to the next fusion. Second, our modified intra-class correlation analysis (MICA) is used to fuse two feature sets in the second stage. MICA is a CCA (Canonical correlation analysis) -based method. A new class matrix is used to modify the covariance matrix between two feature sets, which allows MICA to conveniently inherit the discriminating structure while fusing features. Finally, we propose a feature selection algorithm based on kernel local Fisher discriminant analysis (KLFDA) and kernel canonical correlation analysis (KCCA) to select the desired features. This algorithm based on Max-Relevance and Min-Redundancy (mRMR) framework solves the problem that CCA cannot properly evaluate the correlation between features and the class variable, as well as accurately evaluates the correlation among features. Based on the experimental data, the proposed method is compared with several popular methods, and the feature fusion methods used in some previous studies related to the fault diagnosis of rotating machinery are compared with it as well. The results show that the fusion effectiveness of our method is better than other methods, which obtains higher recognition accuracy.

Bearing defect diagnosis based on semi-supervised kernel Local Fisher Discriminant Analysis using pseudo labels

In bearings defect diagnosis applications, information fusion has been widely used to improve identification accuracy for different types of faults, which may lead to high-dimensionality and information redundancy of the data and thus degenerate the classification performance. Therefore, it is a major challenge for machinery fault diagnosis to extract optimal features from high-dimensional and redundant data for classification. In addition, in order to guarantee the performance of fault diagnosis, conventional supervised methods usually require a large amount of labeled data available for learning. However, it is extremely difficult, costly and time-consuming to collect faulty labeled samples with class information, especially for expensive and critical machines, which often results in only a few labeled data available with a large amount of unlabeled data redundant. In this paper, we propose a novel bearing defect diagnosis model based on semi-supervised kernel local Fisher Discriminant Analysis (SSKLFDA) using pseudo labels, which can effectively extract optimal features for classification and simultaneously utilize unlabeled data for regularizing the supervised dimensionality reduction. The proposed SSKLFDA first adopts Density Peak Clustering technique to generate pseudo cluster labels for the labeled and unlabeled data and then regularizes the between-class scatter and within-class scatter according to two corresponding regularization strategies associated with the generated pseudo cluster labels. This regularization can further improve the discriminant performance of the extracted features and also make it suitable for the cases with the multimodality and noises. In order to accommodate for non-linear feature extraction, the kernel version of the proposed method is also provided with the introduction of kernel trick. The experimental results under different feature dimensions, numbers of labeled data, and subsequent classifiers scenarios demonstrate that the proposed SSKLFDA based bearings fault diagnosis model achieves higher classification performance than other existing dimensionality reduction methods-based models.

Multishot person reidentification using joint group sparse representation

Sparse representation (SR) has shown great potential in multishot person reidentification. However, conventional SR methods usually encode each probe image individually, which ignores the prior knowledge shared by similar probe images with the same identity. Considering that probe images with the same identity are similar and can be complementary to each other, we propose a joint group sparse representation (JGSR) method to simultaneously encode a set of probe person images with the same identity. Additionally, we consider the fact that prior knowledge is also contained in the set of images with the same identity in the gallery. In JGSR, probe images with the same identity are coded on the same dictionary atoms and shared similar coefficients. Furthermore, the person reidentification performance is also improved by incorporating JGSR with k-means clustering and kernel local Fisher discriminant analysis into a unified framework. Extensive experiments on three publicly available iLIDS-VID, PRID 2011, and SAIVT-SoftBio multishot benchmark datasets were conducted and demonstrated the superior performance of the JGSR method in comparison with current state-of-the-art methods.

Re-ranking pedestrian re-identification with multiple Metrics

Pedestrian re-identification (re-ID) is a video surveillance technology for specific pedestrians in non-overlapping multi-camera scenes. However, due to the influence of dramatic changes in perspectives and pedestrian occasions, it is still a huge challenge to find a stable, reliable algorithm in high accuracy rate. In this paper, to increase the robustness and performance of re-ID, we proposed a re-ID method by re-ranking the refined re-ID results (i.e. initial lists) gotten from the kernel-Local Fisher Discriminant Analysis (kLFDA) and Marginal Fisher Analysis (MFA) metrics, which can improve the probability of the correct target on the initial result lists and also enhance the robustness. During the process of re-ranking, in order to distinguish pedestrians in high similarity, a rigorous distance constraint model named Perspective Distance Model (PDM) is designed to further reduce the intra-class variations and increase the distance of inter-class variations. By using the PDM, the concise results gotten from the kLFDA and MFA metrics are re-ranked in order to further recognize different individuals in high similarity and improve the re-ID rate. Experimental results on seven challenging re-ID datasets (VIPeR, CUHK01, Prid2011, iLIDS, CUHK03, Market-1501and DukeReID) show that the performance of proposed method is high and effective.

A Novel Indoor Positioning System Using Kernel Local Discriminant Analysis in Internet‐of‐Things

WLAN based localization is a key technique of location‐based services (LBS) indoors. However, the indoor environment is complex; received signal strength (RSS) is highly uncertain, multimodal, and nonlinear. The traditional location estimation methods fail to provide fair estimation accuracy under the said environment. We proposed a novel indoor positioning system that considers the nonlinear discriminative feature extraction of RSS using kernel local Fisher discriminant analysis (KLFDA). KLFDA extracts location features in a well‐preserved kernelized space. In the new kernel featured space, nonlinear RSS features are characterized effectively. Along with handling of nonlinearity, KLFDA also copes well with the multimodality in the RSS data. By performing KLFDA, the discriminating information contained in RSS is reorganized and maximally extracted. Prior to feature extraction, we performed outlier detection on RSS data to remove any anomalies present in the data. Experimental results show that the proposed approach obtains higher positioning accuracy by extracting maximal discriminate location features and discarding outlying information present in the RSS data.

Bearing Defect Classification Based on Individual Wavelet Local Fisher Discriminant Analysis with Particle Swarm Optimization

In order to enhance the performance of bearing defect classification, feature extraction and dimensionality reduction have become important. In order to extract the effective features, wavelet kernel local fisher discriminant analysis (WKLFDA) is first proposed; herein, a new wavelet kernel function is proposed to construct the kernel function of LFDA. In order to automatically select the parameters of WKLFDA, a particle swarm optimization (PSO) algorithm is employed, yielding a new PSO-WKLFDA. When compared with the other state-of-the-art methods, the proposed PSO-WKLFDA yields better performance. However, the use of a single global transformation of PSO-WKLFDA for the multiclass task does not provide excellent classification accuracy due to the fact that the projected data still significantly overlap with each other in the projected subspace. In order to enhance the performance of bearing defect classification, a novel method is then proposed by transforming the multiclass task into all possible binary classification tasks using a one-against-one (OAO) strategy. Then, individual PSO-WKLFDA (I-PSO-WKLFDA) is used for extracting effective features of each binary class. The extracted effective features of each binary class are input to a support vector machine (SVM) classifier. Finally, a decision fusion mechanism is employed to merge the classification results from each SVM classifier to identify the bearing condition. Simulation results using synthetic data and experimental results using different bearing fault types show that the proposed method is well suited and effective for bearing defect classification.

Wavelet Kernel Local Fisher Discriminant Analysis With Particle Swarm Optimization Algorithm for Bearing Defect Classification

Feature extraction and dimensionality reduction (DR) are necessary and helpful preprocessing steps for bearing defect classification. Linear local Fisher discriminant analysis (LFDA) has recently been developed as a popular method for feature extraction and DR. However, the linear method tends to give undesired results if the samples between classes are nonlinearly separated in the input space. To enhance the performance of LFDA in bearing defect classification, a new feature extraction and DR algorithm based on wavelet kernel LFDA (WKLFDA) is presented in this paper. Herein, a new wavelet kernel function is proposed to construct the kernel function of LFDA. To seek the optimal parameters for WKLFDA, particle swarm optimization (PSO) is used; as a result, a new PSO-WKLFDA algorithm is proposed. The experimental results for the synthetic data and measured vibration bearing data show that the proposed WKLFDA and PSO-WKLFDA outperform other state-of-the-art algorithms.

Locality Preserving Composite Kernel Feature Extraction for Multi-Source Geospatial Image Analysis

Multi-source data, either from different sensors or disparate features extracted from the same sensor, are valuable for geospatial image analysis due to their potential for providing complementary features. In this paper, a composite-kernel-based feature extraction method is proposed for multi-source remote sensing data classification. Features from different sources are first fused via a weighted composite kernel mapping, and then projected to a lower-dimensional subspace in which kernel local Fisher discriminant analysis (KLFDA) is used to extract the most discriminative information. We hypothesize that after such a projection, multi-source data would have better class separability between classes, and an efficient linear classification model–multinomial logistic regression (MLR) would be suitable for classification. The efficacy of the proposed method is demonstrated via experiments using two different sets of multi-source geospatial data. For feature fusion, the raw spectral data and extended multi-attribute profiles (EMAPs) derived from the hyperspectral image are used as a testbed for multi-source image analysis. The second multi-source testbed used for validation involves sensor fusion, in which the hyperspectral and light detection and ranging (LiDAR) data are utilized. Experimental results show that composite kernel local Fisher’s discriminant analysis when combined with MLR based classifier (CKLFDA-MLR) is very effective at feature extraction and classification of multi-source geospatial images.

Manifold Adaptive Kernel Local Fisher Discriminant Analysis for Face Recognition

To efficiently cope with the high dimensionalities and complex nonlinear variations of face images in face recognition task, a novel manifold adaptive kernel local Fisher discriminant analysis algorithm is proposed in this paper. The core idea of this algorithm is as follows: First, the local manifold structure of the face image is modeled by a nearest neighbor graph. Then, an original input kernel function is deformed with respect to the local manifold structure. Finally, the resulting manifold adaptive kernel function is incorporated into the kernel local Fisher discriminant analysis(LFDA) method, which leads to the manifold adaptive kernel LFDA(MAKL) algorithm for face recognition. Experimental results on three popular face databases show that the proposed algorithm performs much better than other related algorithms.

Multiple kernel local Fisher discriminant analysis for face recognition

Dimensionality reduction algorithms, which aim to obtain low-dimensional feature representation with enhanced discrimination power, have attracted great attention for face recognition. Local Fisher discriminant analysis (LFDA) is a recently developed linear dimensionality reduction algorithm. It has been shown that LFDA is a strong analyzer of high-dimensional data. However, LFDA is a linear method, and this makes it difficult to describe the complex nonlinearity of face images. In addition, LFDA only focuses on using a single data descriptor to depict the whole face image dataset, while lacks a systematic way of integrating multiple image features for dimensionality reduction. To enhance the performance of LFDA in face recognition, a new algorithm termed multiple kernel local Fisher discriminant analysis (MKLFDA) is proposed in this paper. MKLFDA produces nonlinear discriminant features via kernel theory, and considers multiple image features with multiple base kernels. Experimental results on three face databases demonstrate the effectiveness of the proposed algorithm.

Kernel local Fisher discriminant analysis based manifold-regularized SVM model for financial distress predictions

Support vector machines (SVM) have demonstrated excellent performance in numerous areas of pattern recognitions. However, traditional SVM does not make efficient use of both labeled training data and unlabeled testing data. Moreover, high dimensional and nonlinear distributed data generally degrade the performance of a classifier due to the curse of dimensionality in financial distress (or bankruptcy) predictions. To address these problems, this study proposes a novel hybrid classifier which integrates Kernel local Fisher discriminant analysis (KLFDA) with a manifold-regularized SVM (MR-SVM). KLFDA is employed to find an optimal projection which maximizes the margin between data points from different classes at each local area of data manifold, while MR-SVM data-dependently warps the structure of feature space to reflect the underlying geometry of the data manifold. Compared with other dimensionality reduction methods and conventional classifiers, the hybrid classifier performs best.

Locality-Preserving Discriminant Analysis in Kernel-Induced Feature Spaces for Hyperspectral Image Classification

Linear discriminant analysis (LDA) has been widely applied for hyperspectral image (HSI) analysis as a popular method for feature extraction and dimensionality reduction. Linear methods such as LDA work well for unimodal Gaussian class-conditional distributions. However, when data samples between classes are nonlinearly separated in the input space, linear methods such as LDA are expected to fail. The kernel discriminant analysis (KDA) attempts to address this issue by mapping data in the input space onto a subspace such that Fisher's ratio in an intermediate (higher-dimensional) kernel-induced space is maximized. In recent studies with HSI data, KDA has been shown to outperform LDA, particularly when the data distributions are non-Gaussian and multimodal, such as when pixels represent target classes severely mixed with background classes. In this letter, a modified KDA algorithm, i.e., kernel local Fisher discriminant analysis (KLFDA), is studied for HSI analysis. Unlike KDA, KLFDA imposes an additional constraint on the mapping-it ensures that neighboring points in the input space stay close-by in the projected subspace and vice versa. Classification experiments with a challenging HSI task demonstrate that this approach outperforms current state-of-the-art HSI-classification methods.

Robust Appearance-Based Method for Head Pose Estimation

PDF HTML阅读 XML下载 导出引用 引用提醒 一种基于图像表观的鲁棒姿态估计方法 DOI: 作者: 作者单位: 作者简介: 通讯作者: 中图分类号: 基金项目: Supported by the National Natural Science Foundation of China under Grant No.60673091 (国家自然科学基金); the NationalHigh-Tech Research and Development Plan of China under Grant Nos.2006AA01Z122, 2007AA01Z163 (国家高技术研究发展计划(863)); the 100 Talents Program of the CAS (中国科学院百人计划); the ISVISION Technology Co. Ltd. (上海银晨智能识别科技有限公司) Robust Appearance-Based Method for Head Pose Estimation Author: Affiliation: Fund Project: 摘要 | 图/表 | 访问统计 | 参考文献 | 相似文献 | 引证文献 | 资源附件 | 文章评论 摘要:提出一种利用图像的表观特征进行头部姿态估计的方法.该方法首先使用了一维Gabor 滤波器对头部图像进行特征提取,然后对提取得到的一维Gabor 特征进一步使用了基于核函数的局部费舍尔判别分析方法增强特征的判别能力.与传统二维Gabor 特征相比,一维Gabor 特征除了在计算速度和存储空间上具有明显的优势以外,更与姿态紧密相关.而基于核函数的局部费舍尔判别分析方法,能够解决姿态问题中存在的非线性问题和多模态问题.大量的实验结果表明,该算法对于姿态估计问题是有效的.特别需要指出的是,该算法具有良好的推广能力 Abstract:This paper proposes a new pose estimation method based on the appearance of 2D head image. First, the 1D Gabor filters are used to extract the features on the raw images. Compared with the traditional 2D Gaborrepresents, the 1D Gabor represents are more closely related to the head pose, while the advantages of computation and storage are obvious. Second, for the extracted features, a new method, named kernel local fisher discriminant analysis, is applied to eliminate the multimodal problem, while at the same time enhance the discrimination ability.Experimental results show that the proposed method is effective for pose estimation. It must be pointed out that the generalizability of the proposed method is illustrated by the impressive performance when the training dataset and the testing dataset are heterogeneous. 参考文献 相似文献 引证文献

Dimensionality reduction for visualization of normal and pathological speech data

For an adequate analysis of pathological speech signals, a sizeable number of parameters is required, such as those related to jitter, shimmer and noise content. Often this kind of high-dimensional signal representation is difficult to understand, even for expert voice therapists and physicians. Data visualization of a high-dimensional dataset can provide a useful first step in its exploratory data analysis, facilitating an understanding about its underlying structure. In the present paper, eight dimensionality reduction techniques, both classical and recent, are compared on speech data containing normal and pathological speech. A qualitative analysis of their dimensionality reduction capabilities is presented. The transformed data are also quantitatively evaluated, using classifiers, and it is found that it may be advantageous to perform the classification process on the transformed data, rather than on the original. These qualitative and quantitative analyses allow us to conclude that a nonlinear, supervised method, called kernel local Fisher discriminant analysis is superior for dimensionality reduction in the actual context.

Screening for cancer of the cervix.

<h3>Abstract</h3> Geographic patterns of human genetic variation provide important insights into human evolution and disease. A commonly used tool to detect geographic patterns from genetic data is principal components analysis (PCA) or the supervised linear discriminant analysis of principal components (DAPC). However, genetic features produced from both approaches could fail to correctly characterize population structure for complex scenarios involving admixture. In this study, we introduce Kernel Local Fisher Discriminant Analysis of Principal Components (KLFDAPC), a supervised nonlinear approach for inferring individual geographic genetic structure that could rectify the limitations of these approaches by preserving the multimodal space of samples. We tested the power of KLFDAPC to infer population structure and to predict individual geographic origin using neural networks. Simulation results showed that KLFDAPC significantly improved the population separability compared with PCA and DAPC. The application to POPRES and CONVERGE datasets indicated that the first two reduced features of KLFDAPC correctly recapitulated the geography of individuals, and significantly improved the accuracy of predicting individual geographic origin when compared to PCA and DAPC. Therefore, KLFDAPC can be useful for geographic ancestry inference, design of genome scans and correction for spatial stratification in GWAS that link genes to adaptation or disease susceptibility.