Hessian Locally Linear Embedding Research Articles

A Study on Dimensionality Reduction and Parameters for Hyperspectral Imagery Based on Manifold Learning.

With the rapid advancement of remote-sensing technology, the spectral information obtained from hyperspectral remote-sensing imagery has become increasingly rich, facilitating detailed spectral analysis of Earth's surface objects. However, the abundance of spectral information presents certain challenges for data processing, such as the "curse of dimensionality" leading to the "Hughes phenomenon", "strong correlation" due to high resolution, and "nonlinear characteristics" caused by varying surface reflectances. Consequently, dimensionality reduction of hyperspectral data emerges as a critical task. This paper begins by elucidating the principles and processes of hyperspectral image dimensionality reduction based on manifold theory and learning methods, in light of the nonlinear structures and features present in hyperspectral remote-sensing data, and formulates a dimensionality reduction process based on manifold learning. Subsequently, this study explores the capabilities of feature extraction and low-dimensional embedding for hyperspectral imagery using manifold learning approaches, including principal components analysis (PCA), multidimensional scaling (MDS), and linear discriminant analysis (LDA) for linear methods; and isometric mapping (Isomap), locally linear embedding (LLE), Laplacian eigenmaps (LE), Hessian locally linear embedding (HLLE), local tangent space alignment (LTSA), and maximum variance unfolding (MVU) for nonlinear methods, based on the Indian Pines hyperspectral dataset and Pavia University dataset. Furthermore, the paper investigates the optimal neighborhood computation time and overall algorithm runtime for feature extraction in hyperspectral imagery, varying by the choice of neighborhood k and intrinsic dimensionality d values across different manifold learning methods. Based on the outcomes of feature extraction, the study examines the classification experiments of various manifold learning methods, comparing and analyzing the variations in classification accuracy and Kappa coefficient with different selections of neighborhood k and intrinsic dimensionality d values. Building on this, the impact of selecting different bandwidths t for the Gaussian kernel in the LE method and different Lagrange multipliers λ for the MVU method on classification accuracy, given varying choices of neighborhood k and intrinsic dimensionality d, is explored. Through these experiments, the paper investigates the capability and effectiveness of different manifold learning methods in feature extraction and dimensionality reduction within hyperspectral imagery, as influenced by the selection of neighborhood k and intrinsic dimensionality d values, identifying the optimal neighborhood k and intrinsic dimensionality d value for each method. A comparison of classification accuracies reveals that the LTSA method yields superior classification results compared to other manifold learning approaches. The study demonstrates the advantages of manifold learning methods in processing hyperspectral image data, providing an experimental reference for subsequent research on hyperspectral image dimensionality reduction using manifold learning methods.

SVMWPE-HLLE Based Fault Diagnosis Approach for Rolling Bearing

Multi-scale permutation entropy (MPE) is an analytical method describing the complexity of time series, which has been applied to the fault diagnosis of rolling bearings. To solve the problems of MPE in coarse-grained process and permutation entropy calculation, multi-scale weighted permutation entropy based on sliding variance (SVMWPE) was proposed in this paper. By analyzing WGN signal and 1/f noise signal, the parameter selections for SVMWPE were studied, and the stability and superiority were investigated by comparing SVMWPE with MPE, MWPE, and SVMPE. The high-dimensional matrix obtained by MPE feature extraction to pattern recognition was solved by introducing Hessian local linear embedding (HLLE) dimension reduction method, and the feature extraction method based on SVMWPE-HLLE was proposed. The clustering effect was studied by comparing SVMWPE-HLLE with SVMWPE-LLE through the analysis of three simulation signals. Fault diagnosis method for rolling bearing was proposed by combining SVMWPE-HLLE with extreme learning machine (ELM), which was applied to two experimental cases of rolling bearings for analysis. The experimental results showed that the proposed method can realize intelligent diagnosis of different fault types and degrees of rolling bearing, and the fault recognition rate of the proposed method was higher than other methods.

A new key performance indicator oriented industrial process monitoring and operating performance assessment method based on improved Hessian locally linear embedding

ABSTRACT The industrial process monitoring and operating performance assessment techniques are of great significance to ensure the safety and efficiency of the production and to improve the comprehensive economic benefits for the modern enterprises. In this paper, a new key performance indicator (KPI) oriented nonlinear process monitoring and operating performance assessment method is proposed based on the improved Hessian locally linear embedding (HLLE), in view of the problems of strong nonlinearity, high dimension and information redundancy in actual industrial process data. Firstly, in order to characterise the similarities of samples in both temporal and spatial dimensions, a new measurement, based on Finite Markov theory, is defined to replace the Euclidean distance in traditional HLLE. Secondly, by mining the relationships between process variables and the key performance indicator, the KPI oriented feature extraction method is developed. On this basis, the monitoring statistics is constructed and the corresponding control limit is determined for the real-time fault detection. After that, a new operating performance assessment approach based on sliding window Kullback–Leibler divergence is put forward to facilitate maintenance or adjustments. Finally, the proposed method is applied to the hot strip mill process, and the results show the effectiveness.

Hessian Locally Linear Embedding of PMU Data for Efficient Fault Detection in Power Systems

This article develops a computationally efficient fault-detection method for power systems, by exploiting phasor measurement unit (PMU) data and Hessian locally linear embedding (HLLE) technique. First, via HLLE technique, high-dimensional PMU data are transformed into low-dimensional embedding coordinates, which effectively captures the fluctuations of PMU data. Next, based on the feature space of embedding coordinates, T-squared statistic is employed for online fault detection. The method is evaluated on IEEE 39-bus system and a real-world system, exhibiting decent fault-detection performance as well as a considerably lower computational complexity when compared with existent methods.

Nonlinear Feature Extraction Through Manifold Learning in an Electronic Tongue Classification Task.

A nonlinear feature extraction-based approach using manifold learning algorithms is developed in order to improve the classification accuracy in an electronic tongue sensor array. The developed signal processing methodology is composed of four stages: data unfolding, scaling, feature extraction, and classification. This study aims to compare seven manifold learning algorithms: Isomap, Laplacian Eigenmaps, Locally Linear Embedding (LLE), modified LLE, Hessian LLE, Local Tangent Space Alignment (LTSA), and t-Distributed Stochastic Neighbor Embedding (t-SNE) to find the best classification accuracy in a multifrequency large-amplitude pulse voltammetry electronic tongue. A sensitivity study of the parameters of each manifold learning algorithm is also included. A data set of seven different aqueous matrices is used to validate the proposed data processing methodology. A leave-one-out cross validation was employed in 63 samples. The best accuracy () was obtained when the methodology uses Mean-Centered Group Scaling (MCGS) for data normalization, the t-SNE algorithm for feature extraction, and k-nearest neighbors (kNN) as classifier.

On the Equivalence of HLLE and LTSA.

Among the representative algorithms of manifold learning, Hessian locally linear embedding (HLLE) and local tangent space alignment (LTSA) algorithms haven been regarded as two different algorithms. However, in practice, the effects of these two algorithms are very similar and LTSA performs better than HLLE in some applications. This paper tries to account for this phenomenon from a mathematical point of view. There are only two differences between HLLE and LTSA. First, LTSA includes a data point into its neighborhood, while HLLE does not. Second, HLLE and LTSA use different methods to align the local coordinates of manifold. In this paper, we show that, the first difference between HLLE and LTSA is not essential. However, from the viewpoint of data utilization, LTSA does better than HLLE in the neighborhood construction. This may account for why LTSA can perform better than HLLE in some applications. As for the second difference between HLLE and LTSA, we first prove that, the alignment equations used by HLLE and LTSA are exactly the same. Second, we prove that, although HLLE and LTSA uses different methods to solve the alignment equation, their solutions are exactly the same, provided that HLLE adopts the same method as LTSA to construct the neighborhoods. Based on these arguments, we claim that HLLE and LTSA are equivalent to each other. This conclusion can also be verified experimentally by using manifold learning MATLAB demo (MANI), a widely-used experimental platform of manifold learning. When testing HLLE on MANI, if HLLE adopts the same method as LTSA to construct the neighborhoods, the experimental results presented by MANI will be the same as those of LTSA.

Robust Hessian Locally Linear Embedding Techniques for High-Dimensional Data

Recently manifold learning has received extensive interest in the community of pattern recognition. Despite their appealing properties, most manifold learning algorithms are not robust in practical applications. In this paper, we address this problem in the context of the Hessian locally linear embedding (HLLE) algorithm and propose a more robust method, called RHLLE, which aims to be robust against both outliers and noise in the data. Specifically, we first propose a fast outlier detection method for high-dimensional datasets. Then, we employ a local smoothing method to reduce noise. Furthermore, we reformulate the original HLLE algorithm by using the truncation function from differentiable manifolds. In the reformulated framework, we explicitly introduce a weighted global functional to further reduce the undesirable effect of outliers and noise on the embedding result. Experiments on synthetic as well as real datasets demonstrate the effectiveness of our proposed algorithm.

Triangular mesh parameterization with trimmed surfaces

Given a 2-manifold triangular mesh \(M \subset {\mathbb {R}}^3\), with border, a parameterization of \(M\) is a FACE or trimmed surface \(F=\{S,L_0,\ldots ,L_m\}\). \(F\) is a connected subset or region of a parametric surface \(S\), bounded by a set of LOOPs \(L_0,\ldots ,L_m\) such that each \(L_i \subset S\) is a closed 1-manifold having no intersection with the other \(L_j\) LOOPs. The parametric surface \(S\) is a statistical fit of the mesh \(M\). \(L_0\) is the outermost LOOP bounding \(F\) and \(L_i\) is the LOOP of the i-th hole in \(F\) (if any). The problem of parameterizing triangular meshes is relevant for reverse engineering, tool path planning, feature detection, re-design, etc. State-of-art mesh procedures parameterize a rectangular mesh \(M\). To improve such procedures, we report here the implementation of an algorithm which parameterizes meshes \(M\) presenting holes and concavities. We synthesize a parametric surface \(S \subset {\mathbb {R}}^3\) which approximates a superset of the mesh \(M\). Then, we compute a set of LOOPs trimming \(S\), and therefore completing the FACE \(F=\{S,L_0,\ldots ,L_m\}\). Our algorithm gives satisfactory results for \(M\) having low Gaussian curvature (i.e., \(M\) being quasi-developable or developable). This assumption is a reasonable one, since \(M\) is the product of manifold segmentation pre-processing. Our algorithm computes: (1) a manifold learning mapping \(\phi : M \rightarrow U \subset {\mathbb {R}}^2\), (2) an inverse mapping \(S: W \subset {\mathbb {R}}^2 \rightarrow {\mathbb {R}}^3\), with \(W\) being a rectangular grid containing and surpassing \(U\). To compute \(\phi \) we test IsoMap, Laplacian Eigenmaps and Hessian local linear embedding (best results with HLLE). For the back mapping (NURBS) \(S\) the crucial step is to find a control polyhedron \(P\), which is an extrapolation of \(M\). We calculate \(P\) by extrapolating radial basis functions that interpolate points inside \(\phi (M)\). We successfully test our implementation with several datasets presenting concavities, holes, and are extremely non-developable. Ongoing work is being devoted to manifold segmentation which facilitates mesh parameterization.

Simulation Research of Feature Extraction Method Based on Nonlinear Manifold Learning

This paper deals with the study of Locally Linear Embedding (LLE) and Hessian LLE nonlinear feature extraction for high dimensional data dimension reduction. LLE and Hessian LLE algorithm which reveals the characteristics of nonlinear manifold learning were analyzed. LLE and Hessian LLE algorithm simulation research was studied through different kinds of sample for dimensionality reduction. LLE and Hessian LLE algorithm’s classification performance was compared in accordance with MDS. The simulation experimental results show that LLE and Hessian LLE are very effective feature extraction method for nonlinear manifold learning.

New Intelligent Computer Intrusion Detection Method Using Hessian Local Linear Embedding and Multi-Kernel Support Vector Machine

Computer networks frequently collapse under the destructive intrusions. It is crucial to detection hidden intrusions to protect the computer networks. However, a computer intrusion often distributes high dimensional characteristic signals, which increases the difficulty of intrusion detection. Literature review indicates that limited work has been done to address the nonlinear dimension reduction problem in computer intrusion detection. Hence, this study has proposed a new intrusion detection method based on the Hessian Local Linear Embedding (HLLE) and multi-kernel Support Vector Machine (SVM). The HLLE was firstly used to reduce the dimension of the original intrusion date in a nonlinear manner. Then the SVM with multiply kernels was employed to detect the intrusions. A real computer network experimental system has been established to evaluate the proposed method. Four typical intrusions have been tested. The test results show high effectiveness of the new detection method. In addition, the new method has been compared with the single-kernel SVM with Local Linear Embedding (LLE) or Principal Component Analysis (PCA). The comparison results demonstrate that the proposed HLLE plus multi-kernel SVM can provide the best computer intrusion detection rate of 97.1%.

Manifold Learning by Graduated Optimization

We present an algorithm for manifold learning called manifold sculpting , which utilizes graduated optimization to seek an accurate manifold embedding. An empirical analysis across a wide range of manifold problems indicates that manifold sculpting yields more accurate results than a number of existing algorithms, including Isomap, locally linear embedding (LLE), Hessian LLE (HLLE), and landmark maximum variance unfolding (L-MVU), and is significantly more efficient than HLLE and L-MVU. Manifold sculpting also has the ability to benefit from prior knowledge about expected results.

Distributed Localization Based on Hessian Local Linear Embedding Algorithm in Wireless Sensor Networks

Distributed Localization Based on Hessian Local Linear Embedding Algorithm in Wireless Sensor Networks