Supervised Locally Linear Embedding Research Articles

Supervised locally linear embedding for magnetic resonance imaging based Alzheimer's disease classification

In order to solve the problem of early classification of Alzheimer's disease (AD), the conventional linear feature extraction algorithm is difficult to extract the most discriminative information from the high-dimensional features to effectively classify unlabeled samples. Therefore, in order to reduce the redundant features and improve the recognition accuracy, this paper used the supervised locally linear embedding (SLLE) algorithm to transform multivariate data of regional brain volume and cortical thickness to a locally linear space with fewer dimensions. The 412 individuals were collected from the Alzheimer's Disease Neuroimaging Initiative (ADNI) including stable mild cognitive impairment (sMCI, n = 93), amnestic mild cognitive impairment (aMCI, n = 96), AD ( n = 86) and cognitive normal controls (CN, n = 137). The SLLE algorithm used in this paper is to calculate the nearest neighbors of each sample point by adding the distance correction term, and the locally linear reconstruction weight matrix was obtained from its nearest neighbors, then the low dimensional mapping of the high dimensional data can be calculated. In order to verify the validity of SLLE in the task of classification, the feature extraction algorithms such as principal component analysis (PCA), Neighborhood MinMax Projection (NMMP), locally linear mapping (LLE) and SLLE were respectively combined with support vector machines (SVM) classifier to obtain the accuracy of classification of CN and sMCI, CN and aMCI, CN and AD, sMCI and aMCI, sMCI and AD, and aMCI and AD, respectively. Experimental results showed that our method had improvements (accuracy/sensitivity/specificity: 65.16%/63.33%/67.62%) on the classification of sMCI and aMCI by comparing with the combination algorithm of LLE and SVM (accuracy/sensitivity/specificity: 64.08%/66.14%/62.77%) and SVM (accuracy/sensitivity/specificity: 57.25%/56.28%/58.08%). In detail the accuracy of the combination algorithm of SLLE and SVM is 1.08% higher than the combination algorithm of LLE and SVM, and 7.91% higher than SVM. Thus, the combination of SLLE and SVM is more effective in the early diagnosis of Alzheimer's disease.

Hidden feature extraction for unstructured agricultural environment based on supervised kernel locally linear embedding modeling

An online hidden feature extraction algorithm is proposed for unknown and unstructured agricultural environments based on a supervised kernel locally linear embedding (SKLLE) algorithm. Firstly, an online obtaining method for scene training samples is given to obtain original feature data. Secondly, Bayesian estimation of the a posteriori probability of a cluster center is performed. Thirdly, nonlinear kernel mapping function construction is employed to map the original feature data to hyper-high-dimensional kernel space. Fourthly, the automatic determination of hidden feature dimensions is performed using a local manifold learning algorithm. Then, a low-level manifold computation in hidden space is completed. Finally, long-range scene perception is realized using a 1-NN classifier. Experiments are conducted to show the effectiveness and the influence of parameter selection for the proposed algorithm. The kernel principal component analysis (KPCA), locally linear embedding (LLE), and supervised locally linear embedding (SLLE) methods are compared under the same experimental unstructured agricultural environment scene. Test results show that the proposed algorithm is more suitable for unstructured agricultural environments than other existing methods, and that the computational load is significantly reduced.

Object‐Based Classification Method for PolSAR Images with Improved Scattering Powers and Contextual Features

This paper proposes a new object-based classification method for Polarimetric synthetic aperture radar (PolSAR) images, which considers scattering powers from an improved model-based polarimetric decomposition approach, as well as the spatial and textural features.With the decomposition, the scattering ambiguities between oriented buildings and vegetation are reduced. Furthermore, various contextual features are extracted from the object and incorporated into the K-nearest neighbors (k-NN) based classification. To reduce the feature redundancy, a new Supervised locally linear embedding (S-LLE) dimensionality reduction method is introduced to map the high dimensional polarimetric signatures into the most compact low-dimensional structure for classification. Experimental results with Airborne synthetic aperture rada (AIRSAR) C-band PolSAR image demonstrate the superior performance to other methods.

Recognition of Control Chart Pattern Using Improved Supervised Locally Linear Embedding and Support Vector Machine

Control chart patterns (CCPs) have been widely utilized for machining process control, the effective recognition of abnormal CCPs can significantly narrow the set of possible assignable causes with shortening the diagnostic process to improve the intelligence of quality monitoring. This paper proposes a method for control chart classification based on improved supervised locally linear embedding (SLLE) and support vector machine (SVM). The present work extracts 12 dimensional statistical feature and shape feature of the control chart, reduces the dimensionality of high dimensional feature set by SLLE, and estimates the neighborhood size and embedding dimension with normalized cuts (Ncut) criterion. Genetic algorithm (GA) is utilized to optimize SVM classifier by searching the best values of the SVM parameters. According to the data set generated by Monte Carlo simulation, the simulation result and performance are analyzed and compared the recognition accuracy with fore-and-aft dimensionality reduction, different descending dimension algorithms and various classifiers. The result demonstrates the proposed approach can recognize CCPs effectively.

Correlation coefficient based supervised locally linear embedding for pulmonary nodule recognition

Background and objectiveDimensionality reduction techniques are developed to suppress the negative effects of high dimensional feature space of lung CT images on classification performance in computer aided detection (CAD) systems for pulmonary nodule detection. MethodsAn improved supervised locally linear embedding (SLLE) algorithm is proposed based on the concept of correlation coefficient. The Spearman's rank correlation coefficient is introduced to adjust the distance metric in the SLLE algorithm to ensure that more suitable neighborhood points could be identified, and thus to enhance the discriminating power of embedded data. The proposed Spearman's rank correlation coefficient based SLLE (SC2SLLE) is implemented and validated in our pilot CAD system using a clinical dataset collected from the publicly available lung image database consortium and image database resource initiative (LICD-IDRI). Particularly, a representative CAD system for solitary pulmonary nodule detection is designed and implemented. After a sequential medical image processing steps, 64 nodules and 140 non-nodules are extracted, and 34 representative features are calculated. The SC2SLLE, as well as SLLE and LLE algorithm, are applied to reduce the dimensionality. Several quantitative measurements are also used to evaluate and compare the performances. ResultsUsing a 5-fold cross-validation methodology, the proposed algorithm achieves 87.65% accuracy, 79.23% sensitivity, 91.43% specificity, and 8.57% false positive rate, on average. Experimental results indicate that the proposed algorithm outperforms the original locally linear embedding and SLLE coupled with the support vector machine (SVM) classifier. ConclusionsBased on the preliminary results from a limited number of nodules in our dataset, this study demonstrates the great potential to improve the performance of a CAD system for nodule detection using the proposed SC2SLLE.

Supervised locally linear embedding algorithm based on orthogonal matching pursuit

Supervised locally linear embedding (SLLE) has been proposed for classification tasks. SLLE can take full use of the label information and select neighbours only in the same class. However, SLLE uses the least squares (LSs) method for solving a set of linear equations to obtain linear representation coefficients, which relates to the inverse of a matrix. If the matrix is singular, the solution to the set of linear equations does not exist. Additionally, if the size of neighbourhood is not appropriate, some further neighbours along the manifold would be selected. To remedy those, this study deals with SLLE based on orthogonal matching pursuit (SLLE-OMP) by introducing OMP into SLLE. In SLLE-OMP, LS is replaced by OMP and OMP can reselect new neighbours from old ones. Experimental results on some real-world datasets show that SLLE-OMP can achieve better classification performance compared with SLLE.

Multi-Target Recognition of Internal and External Defects of Potato by Semi-Transmission Hyperspectral Imaging and Manifold Learning Algorithm

The present paper put forward a non-destructive detection method which combines semi-transmission hyperspectral imaging technology with manifold learning dimension reduction algorithm and least squares support vector machine (LSSVM) to recognize internal and external defects in potatoes simultaneously. Three hundred fifteen potatoes were bought in farmers market as research object, and semi-transmission hyperspectral image acquisition system was constructed to acquire the hyperspectral images of normal external defects (bud and green rind) and internal defect (hollow heart) potatoes. In order to conform to the actual production, defect part is randomly put right, side and back to the acquisition probe when the hyperspectral images of external defects potatoes are acquired. The average spectrums (390-1,040 nm) were extracted from the region of interests for spectral preprocessing. Then three kinds of manifold learning algorithm were respectively utilized to reduce the dimension of spectrum data, including supervised locally linear embedding (SLLE), locally linear embedding (LLE) and isometric mapping (ISOMAP), the low-dimensional data gotten by manifold learning algorithms is used as model input, Error Correcting Output Code (ECOC) and LSSVM were combined to develop the multi-target classification model. By comparing and analyzing results of the three models, we concluded that SLLE is the optimal manifold learning dimension reduction algorithm, and the SLLE-LSSVM model is determined to get the best recognition rate for recognizing internal and external defects potatoes. For test set data, the single recognition rate of normal, bud, green rind and hollow heart potato reached 96.83%, 86.96%, 86.96% and 95% respectively, and he hybrid recognition rate was 93.02%. The results indicate that combining the semi-transmission hyperspectral imaging technology with SLLE-LSSVM is a feasible qualitative analytical method which can simultaneously recognize the internal and external defects potatoes and also provide technical reference for rapid on-line non-destructive detecting of the internal and external defects potatoes.

Facial Expression Recognition Based on Gabor Features Combined with Fast PCA and SLLE

Gabor filter bank can effectively extract the facial expression characteristic information, but the characteristic dimension is too high and needed dimension reduction. If directly use supervised locally linear embedding (SLLE) to reduce dimension. The algorithm needs large memory, calculate for a long time. In order to solve this problem, this article uses fast principal component analysis (FastPCA) to reduce dimension firstly, keeping the basic information in the expression without missing. Use SLLE processing for further dimension reduction and make a sample to distinguish the different expression more apparent. Finally use support vector machine to classify, doing experiments performed on the JAFFE database indicate the efficiency of the proposed method.

A NEW DATA MINING APPROACH FOR GEAR CRACK LEVEL IDENTIFICATION BASED ON MANIFOLD LEARNING

Gear crack is a common damage model in the gear mechanisms, and an unexpected serious crack may break the transmission system down, leading to significant economic losses. Efficient incipient fault detection and diagnosis are therefore critical to machinery normal running. One of the key points of the fault diagnosis is feature extraction and selection. Literature review indicates that only limited research considered the nonlinear property of the feature space by the use of manifold learning algorithms in the field of mechanic fault diagnosis, and nonlinear feature extraction for gear crack detection are scarce. This paper reports a novel data mining method based on the empirical mode decomposition (EMD) and supervised locally linear embedding (SLLE) applied to gear crack level identification. The EMD was used to decompose the vibration signals into a number of intrinsic mode functions (IMFs) for feature extraction, whilst the SLLE for nonlinear feature selection. The experimental vibration data acquired from the gear fault test-bed were processed for feature reduction and extraction using the proposed method. Study results show that the sensitive characteristics between different gear crack severity vibration signals can be revealed effectively by EMD-SLLE. The energy distribution and the statistic features of IMFs vary with the change of the gear operation conditions, and the most distinguished features can be extracted by nonlinear method of SLLE. In addition, the performance of feature extraction of SLLE is better than that of the linear method of principal component analysis (PCA).DOI: http://dx.doi.org/10.5755/j01.mech.18.1.1276

Facial expression recognition using local binary patterns and discriminant kernel locally linear embedding

Given the nonlinear manifold structure of facial images, a new kernel-based supervised manifold learning algorithm based on locally linear embedding (LLE), called discriminant kernel locally linear embedding (DKLLE), is proposed for facial expression recognition. The proposed DKLLE aims to nonlinearly extract the discriminant information by maximizing the interclass scatter while minimizing the intraclass scatter in a reproducing kernel Hilbert space. DKLLE is compared with LLE, supervised locally linear embedding (SLLE), principal component analysis (PCA), linear discriminant analysis (LDA), kernel principal component analysis (KPCA), and kernel linear discriminant analysis (KLDA). Experimental results on two benchmarking facial expression databases, i.e., the JAFFE database and the Cohn-Kanade database, demonstrate the effectiveness and promising performance of DKLLE.

Dimensionality reduction-based spoken emotion recognition

To improve effectively the performance on spoken emotion recognition, it is needed to perform nonlinear dimensionality reduction for speech data lying on a nonlinear manifold embedded in a high-dimensional acoustic space. In this paper, a new supervised manifold learning algorithm for nonlinear dimensionality reduction, called modified supervised locally linear embedding algorithm (MSLLE) is proposed for spoken emotion recognition. MSLLE aims at enlarging the interclass distance while shrinking the intraclass distance in an effort to promote the discriminating power and generalization ability of low-dimensional embedded data representations. To compare the performance of MSLLE, not only three unsupervised dimensionality reduction methods, i.e., principal component analysis (PCA), locally linear embedding (LLE) and isometric mapping (Isomap), but also five supervised dimensionality reduction methods, i.e., linear discriminant analysis (LDA), supervised locally linear embedding (SLLE), local Fisher discriminant analysis (LFDA), neighborhood component analysis (NCA) and maximally collapsing metric learning (MCML), are used to perform dimensionality reduction on spoken emotion recognition tasks. Experimental results on two emotional speech databases, i.e. the spontaneous Chinese database and the acted Berlin database, confirm the validity and promising performance of the proposed method.

Feature extraction based on supervised locally linear embedding for classification of hyperspectral images

Hyperspectral image has high spectral dimension,vast data and altitudinal interband redundancy,which brings problems to image classification.To effectively reduce dimensionality and improve classification precision,a new extraction method of nonlinear manifold learning feature based on Supervised Local Linear Embedding(SLLE) for classification of hyperspectral image was proposed in this paper.A data point's k Nearest Neighbours(NN) were found by using new distance function which was proposed according to prior class-label information.Because the intra-class distance is smaller than inter-class distance,classification is easy for SLLE algorithm.The experimental results on hyperspectral datasets and UCI data set demonstrate the effectiveness of the presented method.

Supervised locally linear embedding projection (SLLEP) for machinery fault diagnosis

Following the intuition that the measured signal samples usually distribute on or near the nonlinear low-dimensional manifolds embedded in the high-dimensional signal space, this paper proposes a new machinery fault diagnosis approach based on supervised locally linear embedding projection (SLLEP). The approach first performs the recently proposed manifold learning algorithm supervised locally linear embedding (SLLE) on the high-dimensional fault signal samples to learn the intrinsic embedded multiple manifold features corresponding to different fault modes, and map them into a low-dimensional embedded space to achieve fault feature extraction. For dealing with the new fault sample, the approach then applies local linear regression to find the projection that best approximates the implicit mapping from high-dimensional samples to the embedding. Finally fault classification is carried out in the embedded manifold space. The ball bearing data and rotor bed data are both used to validate the proposed approach. The results show that the proposed approach obviously improves the fault classification performance and outperform the other traditional approaches.

Autonomic failure prediction based on manifold learning for large-scale distributed systems

This article investigates autonomic failure prediction in large-scale distributed systems with nonlinear dimensionality reduction to automatically extract failure features. Most existing methods for failure prediction focus on building prediction models or heuristic rules by discovering failure patterns, but the process of feature extraction before failure patterns recognition is rarely considered due to the increasing complexity of modern distributed systems. In this work, a novel performance-centric approach to automate failure prediction is proposed based on manifold learning (ML). In addition, the ML algorithm named supervised locally linear embedding (SLLE) is applied to achieve feature extraction. To generalize the dimensionality reduction mapping, the nonlinear mapping approximation and optimization solution is also proposed. In experimental work a file transfer test bed with fault injection is developed which can gather multilevel performance metrics transparently. Based on the runtime monitoring of these metrics, the SLLE method can automatically predict more than 50% of the central processing unit (CPU) and memory failures, and around 70% of the network failure.

Supervised locally linear embedding with probability-based distance for classification

We present a novel dimension reduction method for classification based on probability-based distance and the technique of locally linear embedding (LLE). Logistic Discrimination (LD) is adopted for estimating the probability distribution as well as for classification on the reduced data. Different from the supervised locally linear embedding (SLLE) that is only used for the dimension reduction of training data, our probability-based locally linear embedding (PLLE) can be applied on both training and testing data. Five microarray data sets in high-dimensional spaces, the IRIS data, and a real set of handwritten digits are experimented. The numerical results show the proposed methodology performs better, compared with the LD classifiers applied on the lower-dimensional embedding coordinates computed by LLE or SLLE.

A facial expression recognition system based on supervised locally linear embedding

In this paper, a facial expression recognition system based on supervised locally linear embedding (SLLE) is introduced. The system consists of three modules: face detection, feature extraction with SLLE and classification. In face detection module, two independent characteristics, skin color characteristic and motion characteristic are used to detect face region, and a trained SVM is used to verify candidate regions. In feature extraction module, SLLE, a supervised learning algorithm that can compute low dimensional, neighborhood-preserving embeddings of high dimensional data is used to reduce data dimension and extract features. In classification module, minimum-distance classifier is used to recognize different expressions. The experiments show that the proposed method is superior to PCA-based method.

SLLE for predicting membrane protein types

Introduction of the concept of pseudo amino acid composition (PROTEINS: Structure, Function, and Genetics 43 (2001) 246; Erratum: ibid. 44 (2001) 60) has made it possible to incorporate a considerable amount of sequence-order effects by representing a protein sample in terms of a set of discrete numbers, and hence can significantly enhance the prediction quality of membrane protein type. As a continuous effort along such a line, the Supervised Locally Linear Embedding (SLLE) technique for nonlinear dimensionality reduction is introduced (Science 22 (2000) 2323). The advantage of using SLLE is that it can reduce the operational space by extracting the essential features from the high-dimensional pseudo amino acid composition space, and that the cluster-tolerant capacity can be increased accordingly. As a consequence by combining these two approaches, high success rates have been observed during the tests of self-consistency, jackknife and independent data set, respectively, by using the simplest nearest neighbour classifier. The current approach represents a new strategy to deal with the problems of protein attribute prediction, and hence may become a useful vehicle in the area of bioinformatics and proteomics.