Semi-supervised Local Fisher Discriminant Analysis Research Articles

Fault diagnosis of rolling bearing based on hierarchical discrete entropy and semi-supervised local Fisher discriminant analysis

Fault diagnosis of rolling bearing based on hierarchical discrete entropy and semi-supervised local Fisher discriminant analysis

Multi-Scale Dense Graph Attention Network for Hyperspectral Classification

In recent years, numerous deep learning-based methods have gained increasing attention in hyperspectral classification, particularly the Graph Neural Network, which exhibits superior capabilities in structural description. However, a single graph structure is not suitable for hyperspectral feature representation. Therefore, we propose a novel Multiple-Scale graph network structure, known as the Multi-Scale Dense Graph Attention network for hyperspectral classification. Firstly, semi-supervised local Fisher discriminant analysis and superpixel segmentation were employed for dimensionality reduction and multi-scale graph construction, respectively. Secondly, Spectral-Spatial convolution is applied to extract shallow features from the image. Subsequently, an improved graph self-attention network is sequentially applied to each scale graph, and the different scale graphs are densely connected through spatial feature alignment modules, designed using twice matrix multiplication. Finally, the combined pixel-level feature map from multiple graph spaces is derived, and Spectral-Spatial convolution is employed to fuse the abundant feature maps for hyperspectral classification. Experimental results on various hyperspectral datasets demonstrate the superiority of our MSDesGATnet over many state-of-the-art methods. The code is available at https://github.com/l7170/MSDesGAT.git.

Knock Detection Based on Recursive Variational Mode Decomposition and Multilevel Semi-Supervised Local Fisher Discriminant Analysis

Knock is an abnormal combustion phenomenon in gasoline engines. Strong knocks will reduce the efficiency and durability of engine, while with slight knocks engines can run on a high-efficiency state. It is necessary to detect knock and control the state of knock in order to improve the thermal efficiency of engine. This paper proposes a novel approach for detecting engine knocks in various intensities based on vibration signal of engine block using variational mode decomposition (VMD) and semi-supervised local fisher discriminant analysis (SELF). Since the quadratic penalty of recursive VMD has a great influence on decomposition results, the approach establishes the connection between the quadratic penalty and the stop condition by analyzing a large amount of data and quantifies the relationship by polynomial fitting, which reduces the complexity and subjectivity of recursive VMD. A multilevel SELF is developed for solving the problem that SELFs sometimes may not find ideal embedding space under large scale dimensionality reduction. This method adopts multi embedding spaces, with gradually decreasing dimension, to reduce the dimensionality of original data by a series of small steps. Verifications show the proposed approach can achieve high classification accuracy in knock detection and is able to identify the intensity of knock. This research contributes to the field of engine abnormality detection and can be implemented on vibration-based faults diagnosis area.

Semi-supervised dimensionality reduction of hyperspectral imagery using pseudo-labels

Dimensionality reduction has been proven to be efficient in preparing high dimensional data for various tasks in machine learning. As supervised dimensionality reduction methods such as Fisher discriminant analysis (FDA) and local Fisher discriminant analysis (LFDA) tend to suffer from overfitting when only a small number of labeled samples are available, the abundant unlabeled samples could be helpful in finding a better embedding space. However, applying discriminant analysis on unlabeled data is challenging since we do not have labels for unlabeled data. In this paper, we propose a semi-supervised Semi-Supervised Local Fisher Discriminant Analysis (SSLFDA) using pseudo labels, aiming to perform discriminant analysis on both labeled and unlabeled samples. SSLFDA makes use of pseudo labels, learned from the Dirichlet process mixture model (DPMM) based clustering algorithm, to enable local Fisher discriminant analysis on unlabeled data. In addition, a kernel extension of SSLFDA is derived for non-linear dimensionality reduction. We present experimental results with real hyperspectral data to show that our method provides better classification performance compared to other existing dimensionality reduction methods.

DETECTING DISEASED IMAGES BY USING SEMI – SUPERVISED LEARNING

The goal of semi-supervised image segmentation is to obtain the segmentation from a partially labeled image. By utilizing the image manifold structure in labeled and unlabeled pixels, semi-supervised methods propagates the user labeling to the unlabeled data, thus minimizing the need for user labeling. Several semi-supervised learning methods have been proposed in the literature. In this paper, we consider the delinquent of segmentation of large collections of images and the classification of images by allied diseases. We are detecting diseased images by the process of segmentation and classification. The segmentation used in this paper has two advantages. First, user can specify what they want by highly controlling the segmentation. Another is, at initial stage this model requires only minimum tuning of model parameters. Once initial tuning is done, the setup can be used to automatically segment a large collection of images that are distinct but share similar features. And for classification of diseases, a manifold learning method, called parameter-free semi-supervised local Fisher discriminant analysis is used. This method preserves the global structure of unlabeled samples in addition to separating labeled samples in different classes from each other. The semi-supervised method has an analytic form of the globally optimal solution, which can be computed efficiently by Eigen decomposition. Espousal experiments on various collections of biological images suggest that the proposed model is effective for segmentation with classification and is computationally efficient.

High-dimensional semi-supervised learning via a fusion-refinement procedure

This paper develops a sufficient dimension reduction (SDR) approach for the high-dimensional semi-supervised learning (SSL) problem. In the proposed technique, we first modify the fusion-refinement (FR) procedure, which was proposed in [1], to extract the essential features for a lower-dimensional representation. We then apply an SSL algorithm (e.g., the low density separation (LDS)) in the lower-dimensional feature space to tackle the SSL problem. Numerical experiments are conducted on some widely-used data sets. We carry out a comparison between the proposed procedure and some recently proposed semi-supervised learning approaches (including greedy gradient Max-Cut (GGMC), semi-supervised extreme learning machines (SS-ELM)) and dimension reduction procedures (such as the semi-supervised local Fisher discriminant analysis (SELF), the trace ratio based flexible semi-supervised discriminant analysis (TR-FSDA), and trace ratio relevance feedback (TRRF)). In extensive numerical simulations, the new technique outperforms its competitors in many cases, demonstrating its effectiveness.

Semi-Supervised Local Fisher Discriminant Analysis Based on Reconstruction Probability Class

Fisher discriminant analysis (FDA) is a classic supervised dimensionality reduction method in statistical pattern recognition. FDA can maximize the scatter between different classes, while minimizing the scatter within each class. As it only utilizes the labeled data and ignores the unlabeled data in the analysis process of FDA, it cannot be used to solve the unsupervised learning problems. Its performance is also very poor in dealing with semi-supervised learning problems in some cases. Recently, several semi-supervised learning methods as an extension of FDA have proposed. Most of these methods solve the semi-supervised problem by using a tradeoff parameter that evaluates the ratio of the supervised and unsupervised methods. In this paper, we propose a general semi-supervised dimensionality learning idea for the partially labeled data, namely the reconstruction probability class of labeled and unlabeled data. Based on the probability class optimizes Fisher criterion function, we propose a novel Semi-Supervised Local Fisher Discriminant Analysis (S2LFDA) method. Experimental results on real-world datasets demonstrate its effectiveness compared to the existing similar correlation methods.

Sparse dimensionality reduction of hyperspectral image based on semi-supervised local Fisher discriminant analysis

This paper presents a novel sparse dimensionality reduction method of hyperspectral image based on semi-supervised local Fisher discriminant analysis (SELF). The proposed method is designed to be especially effective for dealing with the out-of-sample extrapolation to realize advantageous complementarities between SELF and sparsity preserving projections (SPP). Compared to SELF and SPP, the method proposed herein offers highly discriminative ability and produces an explicit nonlinear feature mapping for the out-of-sample extrapolation. This is due to the fact that the proposed method can get an explicit feature mapping for dimensionality reduction and improve the classification performance of classifiers by performing dimensionality reduction. Experimental analysis on the sparsity and efficacy of low dimensional outputs shows that, sparse dimensionality reduction based on SELF can yield good classification results and interpretability in the field of hyperspectral remote sensing.

Fault Diagnosis Method Based on Orthogonal Semi-supervised Local Fisher Discriminant Analysis

Fault diagnosis method based on orthogonal semi-supervised local Fisher discriminant analysis(OSELF) is proposed,aiming to solve the problem of inadequate number of labeled fault samples and high dimensionality of the feature set.A new dimensionality reduction method named OSELF is proposed combining orthogonal iteration algorithm with semi-supervised local Fisher discriminant analysis(SELF),which can effectively utilize the fault information supported by the labeled and unlabeled fault samples to embed the fault samples into the low-dimensional subspace without the over-fitted problem.The basis vectors of the orthogonal projection matrix are statically uncorrelated,and the discriminations of the obtained low-dimensional fault feature vectors are improved.Then the low-dimensional fault samples are fed into coarse to fine k nearest neighbor classifier(CFKNNC) to recognize the fault type.The proposed method integrated the advantages of OSELF in dimension reduction and CFKNNC in pattern recognition and effectively improved the accuracy of fault diagnosis.The validity of the proposed method is verified by the instance of the fault diagnosis of a gearbox.

Gene Classification Using Parameter-Free Semi-Supervised Manifold Learning

A new manifold learning method, called parameter-free semi-supervised local Fisher discriminant analysis (pSELF), is proposed to map the gene expression data into a low-dimensional space for tumor classification. Motivated by the fact that semi-supervised and parameter-free are two desirable and promising characteristics for dimension reduction, a new difference-based optimization objective function with unlabeled samples has been designed. The proposed method preserves the global structure of unlabeled samples in addition to separating labeled samples in different classes from each other. The semi-supervised method has an analytic form of the globally optimal solution, which can be computed efficiently by eigen decomposition. Experimental results on synthetic data and SRBCT, DLBCL, and Brain Tumor gene expression data sets demonstrate the effectiveness of the proposed method.

Self-adaptive local Fisher discriminant analysis for semi-supervised image recognition

In this paper, we present a self-adaptive semi-supervised dimension reduction framework for image class recognition. Compared with the Semi-supervised Local Fisher Discriminant Analysis (SELF), whose classification performance is significantly influenced by some parameters - Neighbour size k of every labelled sample, Affinity Matrix A = {A

Gene expression data classification based on improved semi-supervised local Fisher discriminant analysis

A new manifold learning method, called improved semi-supervised local fisher discriminant analysis (iSELF), for gene expression data classification is proposed. Motivated by the fact that semi-supervised and parameter-free are two desirable and promising characteristics for dimension reduction, a new difference-based optimization objective function with unlabeled samples has been designed. The proposed method preserves the global structure of unlabeled samples in addition to separating labeled samples in different classes from each other. The semi-supervised method has an analytic form of the globally optimal solution and it can be computed based on Eigen decompositions. Experiments on synthetic data and SRBCT, DLBCL and brain tumor gene expression datasets are performed to test and evaluate the proposed method. The experimental results and comparisons demonstrate the effectiveness of the proposed method.

Enhanced semi-supervised local Fisher discriminant analysis for face recognition

An improved manifold learning method, called enhanced semi-supervised local Fisher discriminant analysis (ESELF), for face recognition is proposed. Motivated by the fact that statistically uncorrelated and parameter-free are two desirable and promising characteristics for dimension reduction, a new difference-based optimization objective function with unlabeled samples has been designed. The proposed method preserves the manifold structure of labeled and unlabeled samples in addition to separating labeled samples in different classes from each other. The semi-supervised method has an analytic form of the globally optimal solution and it can be computed based on eigen decomposition. Experiments on synthetic data and AT&T, Yale and CMU PIE face databases are performed to test and evaluate the proposed algorithm. The experimental results and comparisons demonstrate the effectiveness of the proposed method.

Semi-supervised local Fisher discriminant analysis for dimensionality reduction

When only a small number of labeled samples are available, supervised dimensionality reduction methods tend to perform poorly because of overfitting. In such cases, unlabeled samples could be useful in improving the performance. In this paper, we propose a semi-supervised dimensionality reduction method which preserves the global structure of unlabeled samples in addition to separating labeled samples in different classes from each other. The proposed method, which we call SEmi-supervised Local Fisher discriminant analysis (SELF), has an analytic form of the globally optimal solution and it can be computed based on eigen-decomposition. We show the usefulness of SELF through experiments with benchmark and real-world document classification datasets.

On Computational Issues of Semi-Supervised Local Fisher Discriminant Analysis

Dimensionality reduction is one of the important preprocessing steps in practical pattern recognition. SEmi-supervised Local Fisher discriminant analysis (SELF) — which is a semi-supervised and local extension of Fisher discriminant analysis — was shown to work excellently in experiments. However, when data dimensionality is very high, a naive use of SELF is prohibitive due to high computational costs and large memory requirement. In this paper, we introduce computational tricks for making SELF applicable to large-scale problems.