Linear Local Tangent Space Alignment Research Articles

An improved transfer learning approach based on geodesic flow kernel for multiphase batch process soft sensor modeling

For multiphase batch process, the characteristics of process data under various batches differ. Consequently, the soft sensor model built for a particular working condition is inapplicable to other working conditions. Besides, each batch can be divided into several phases whose characteristics are probably different. To address the above problems, a soft sensor model based on phase division and transfer learning strategy is proposed. First, transfer learning strategy is adopted to construct a soft sensor model applicable to various working conditions. Specifically, geodesic flow kernel based on linear local tangent space alignment (LLTSA-GFK) algorithm is designed. By projecting process data to the common manifold subspace, the distribution difference of data between various batches is reduced and the performance of the soft sensor model is enhanced. In addition, sequence-based fuzzy clustering and just-in-time learning (JITL) are adopted to solve the multistage characteristic for batch process. The root-mean-square error ( RMSE), coefficient of determination [Formula: see text], and mean absolute error ( MAE) are adopted to compare the conventional soft sensing approach (i.e., partial least-square regression based on JITL, support vector regression, and back propagation neural network) with the proposed approach. The superiority of the proposed model is verified by a fed-batch penicillin fermentation process.

Voice disorder classification using convolutional neural network based on deep transfer learning

Voice disorders are very common in the global population. Many researchers have conducted research on the identification and classification of voice disorders based on machine learning. As a data-driven algorithm, machine learning requires a large number of samples for training. However, due to the sensitivity and particularity of medical data, it is difficult to obtain sufficient samples for model learning. To address this challenge, this paper proposes a pretrained OpenL3-SVM transfer learning framework for the automatic recognition of multi-class voice disorders. The framework combines a pre-trained convolutional neural network, OpenL3, and a support vector machine (SVM) classifier. The Mel spectrum of the given voice signal is first extracted and then input into the OpenL3 network to obtain high-level feature embedding. Considering the effects of redundant and negative high-dimensional features, model overfitting easily occurs. Therefore, linear local tangent space alignment (LLTSA) is used for feature dimension reduction. Finally, the obtained dimensionality reduction features are used to train the SVM for voice disorder classification. Fivefold cross-validation is used to verify the classification performance of the OpenL3-SVM. The experimental results show that OpenL3-SVM can effectively classify voice disorders automatically, and its performance exceeds that of the existing methods. With continuous improvements in research, it is expected to be considered as auxiliary diagnostic tool for physicians in the future.

Linear local tangent space alignment with autoencoder

Linear local tangent space alignment (LLTSA) is a classical dimensionality reduction method based on manifold. However, LLTSA and all its variants only consider the one-way mapping from high-dimensional space to low-dimensional space. The projected low-dimensional data may not accurately and effectively “represent” the original samples. This paper proposes a novel LLTSA method based on the linear autoencoder called LLTSA-AE (LLTSA with Autoencoder). The proposed LLTSA-AE is divided into two stages. The conventional process of LLTSA is viewed as the encoding stage, and the additional and important decoding stage is used to reconstruct the original data. Thus, LLTSA-AE makes the low-dimensional embedding data “represent” the original data more accurately and effectively. LLTSA-AE gets the recognition rates of 85.10, 67.45, 75.40 and 86.67% on handwritten Alphadigits, FERET, Georgia Tech. and Yale datasets, which are 9.4, 14.03, 7.35 and 12.39% higher than that of the original LLTSA respectively. Compared with some improved methods of LLTSA, it also obtains better performance. For example, on Handwritten Alphadigits dataset, compared with ALLTSA, OLLTSA, PLLTSA and WLLTSA, the recognition rates of LLTSA-AE are improved by 4.77, 3.96, 7.8 and 8.6% respectively. It shows that LLTSA-AE is an effective dimensionality reduction method.

Weighted Linear Local Tangent Space Alignment via Geometrically Inspired Weighted PCA for Fault Detection

Principal component analysis (PCA) is widely adopted in local tangent space alignment to estimate local tangent spaces. These estimates are only accurate when uniformly distributed data lies in or is close to linear subspaces. In practice, such conditions are rarely satisfied. Therefore, this approach fails to reveal manifold intrinsic features, resulting in degraded fault detection accuracy. Considering the drawbacks, weighted linear local tangent space alignment (WLLTSA), a manifold learning method is put forward. First, weighted PCA is adopted to provide local tangent space estimates. The parameter selection criterion for the weight matrix is established by taking the context of geometric preservation into account. Second, global low dimensional coordinates are formed by aligning local coordinates with global feature space. Finally, the fault detection model is developed, and kernel density estimation is utilized to approximate confidence bounds for <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathrm{T}^{\mathrm{2}}$</tex-math></inline-formula> and SPE statistics. Simulation results are presented to illustrate the superior feature extraction and fault detection performance of WLLTSA.

Fault diagnosis model based on adaptive generalised morphological filtering and LLTSA-ELM

It is difficult for a single feature to contain all the information needed to describe the running state of the equipment. Though multi-features can contain more information about running state, the redundancy between high-dimension features can easily reduce the accuracy of the classifier. Aimed at that, a fault diagnosis method for rolling bearings combining adaptive generalised morphological filter, linear local tangent space alignment and extreme learning machine (LLTSA-ELM) is proposed. Firstly, the rolling bearing vibration signals are filtered by an adaptive generalised morphological filter. Secondly, the multi-domain features are extracted from filtered signal to construct high-dimensional features set of bearing. Thirdly, the dimension of high-dimensional features is reduced by maximum likelihood estimation (MLE) and LLTSA. Finally, the bearing condition monitoring model based on ELM is constructed by the reduced dimension features, and then is used to analyse and diagnose the running state of bearing. Experiments show that this method can effectively filter noise, reduce the redundancy caused by high-dimensional features, and improve the recognition accuracy of the running state of bearings.

Blind false data injection attacks in smart grids subject to measurement outliers

ABSTRACT False data injection attacks (FDIAs) can manipulate measurement data from Supervisory Control and Data Acquisition (SCADA) system and threat state estimation in smart grids. Blind FDIAs (BFDIAs) enhance traditional FDIAs, which eliminate the limitation of grasping measurement Jacobian matrix H in advance, but when there are outliers in measurement data, attack performance is degraded. In this paper, improved BFDIAs are proposed. In off-line phase, low-dimensional measurement matrix without outliers calculated by Linear Local Tangent Space Alignment algorithm (LLTSA) is sent into Continuous Deep Belief Network (CDBN) as training data to learn their probability distribution. In on-line phase, real-time low-dimensional measurement matrix with outliers are sent into the trained model as inputs, and outputs are reconstructed by the probability distribution in off-line phase, which eliminates the influence of outliers indirectly. Simulations are implemented on PJM 5-bus and IEEE 14-bus systems to verify the performance of proposed strategy compared with PCA-based BFDIAs.

Linear Local Tangent Space Alignment with Autoencoder

Linear Local Tangent Space Alignment with Autoencoder

Modified Deep Autoencoder Driven by Multisource Parameters for Fault Transfer Prognosis of Aeroengine

The existing fault prognosis techniques of aeroengine mostly focus on a single monitoring parameter under stable condition, and have low adaptability to new prognosis scenes. To boost the fault prognosis capability cross aeroengines, modified deep autoencoder (MDAE) driven by multi-source parameters is proposed in this article. First, the sensitive multi-source parameters are selected and fused using linear local tangent space alignment to define a fused health index (FHI) to characterize performance degradation of aeroengine. Second, MDAE model is constructed with adaptive Morlet wavelet to flexibly establish accurate mapping hidden in the FHI under analysis. Third, parameter transfer learning is used to provide good initial parameters for enabling the constructed MDAE to have cross-domain fault prognosis capability. The proposed method is used to analyze both the simulation multisource performance degradation parameters of aeroengines (system level) and experiment run-to-failure bearing datasets (component level). The results confirm the feasibility of the proposed method in cross-domain fault prognosis of aeroengines, which outperforms the existing methods.

Fault diagnosis of high-speed train suspension systems using multiscale permutation entropy and linear local tangent space alignment

Online health monitoring of railway vehicle suspension systems is of critical importance to guarantee train running safety. The currently reported works on vehicle suspension health monitoring mainly adopt model-based approaches. However, detailed parameters of the vehicle suspension systems, usually, are complicated to be acquired. Moreover, an accurate model cannot be easily obtained due to the nonlinearities of vehicle components and the complexity of suspension systems. Considering the limitations of the model-based approaches, a data-driven method, which combines multiscale permutation entropy and linear local tangent space alignment (MPE-LLTSA), is proposed to diagnose the faults of vehicle suspension systems. To demonstrate the effectiveness and advantages of this method, an MBS model of the China CRH3 train is built to generate the bogie frame’s lateral acceleration containing various secondary suspension faults, and the simulated signals are then introduced to evaluate the MPE-LLTSA method. The evaluation results show that the proposed data-driven approach can accurately identify different types of suspension faults. Finally, the MPE-LLTSA method is further validated using the tracking data of the CRH3 train running on a high-speed railway line in China, and the test results show that the proposed method has the potential to be applied to the field of railway engineering.

Rolling bearing fault diagnosis based on adaptive smooth ITD and MF-DFA method

To effectively utilize a feature set to further improve fault diagnosis of a rolling bearing vibration signal, a method based on multi-fractal detrended fluctuation analysis (MF-DFA) and smooth intrinsic time-scale decomposition (SITD) was proposed. The vibration signal was decomposed into several proper rotation components by applying this new SITD method to overcome noise effects, preserve the effective signal, and improve the signal-to-noise ratio. Wavelet analysis was embedded in iteration procedures of intrinsic time-scale decomposition (ITD). For better results, an adaptive threshold function was used for signal recovery from noisy proper rotation components in the wavelet domain. Additionally, MF-DFA was used to reveal the multi-fractality present in the instantaneous amplitude of the proper rotation components. Finally, linear local tangent space alignment was applied for feature dimension reduction and to obtain fault characteristics of different types, further improving identification accuracy. The performance of the proposed method is determined to be superior to that of the ITD-MF-DFA method.

Small-amplitude hunting diagnosis method for high-speed trains based on the bogie frame’s lateral–longitudinal–vertical data fusion, independent mode function reconstruction and linear local tangent space alignment

Hunting stability is an important factor that influences the running safety of high-speed trains. Most of the existing hunting monitoring methods monitor only the standard hunting. The small-amplitude hunting, however, not only affects ride comfort but also aggravates the wheel–rail wear. Therefore, to extend the service life of wheels and rails and to improve the ride comfort, it is extremely important to monitor the small-amplitude hunting. Hunting motion is a coupled movement of lateral and yaw displacements of the wheelset. When the bogie is in an unstable state, instability will occur not only in the lateral side but also in the longitudinal and vertical sides of the bogie. To improve the robustness of the small-amplitude hunting monitoring methods, this study proposes an idea of the bogie frame’s lateral–longitudinal–vertical data fusion. In addition, the small-amplitude hunting signals have strong nonlinear characteristics, and their frequency and amplitude are unstable. Using only the amplitude or frequency to detect the small-amplitude hunting has obvious shortcomings. Therefore, a new feature extraction method based on the independent mode function reconstruction and linear local tangent space alignment (IMFR-LLTSA) is proposed. This method has been tested with three simulated signals. Finally, a method of combining the bogie frame’s lateral–longitudinal–vertical data fusion and IMFR-LLTSA is used to identify the small-amplitude hunting of high-speed trains. This method has been validated using the data of the CRH380a high-speed train running on the Shanghai–Hangzhou line, monitored by the authors’ research group. The results show that this method is superior to the single lateral diagnosis method.

Study on Planetary Gear Degradation State Recognition Method Based on the Features With Multiple Perspectives and LLTSA

Planetary gear is an important part of the transmission system of large electromechanical equipment. Therefore, it is very important to monitor the degradation of the state of the planetary gear. A method for the degradation state recognition of planetary gear based on the features with multiple perspectives and linear local tangent space alignment (LLTSA) algorithm is presented. First, the time domain features of the original vibration signal are extracted, which have the statistical properties and global significance. Then, the detailed features which pay more attention to the detailed information of the vibration signal are extracted on the basis of improved complete ensemble empirical mode decomposition with adaptive noise, and all those features constitute high dimensional original features. In order to solve the problems of information redundancy and interference features, the original features are processed by LLTSA, and the extraction of low dimensional sensitive features can be achieved. Finally, the optimized support vector machine is studied to recognize the low dimensional sensitive features. The result shows that the proposed method can recognize different degradation states of planetary gear accurately and effectively.

Non-destructive identification of maize haploid seeds using nonlinear analysis method based on their near-infrared spectra

The identification of maize haploid seeds is a significant process in genetic research and modern maize breeding. Adopting near-infrared spectroscopy technology to distinguish haploid seeds from hybrid seeds has the advantages of being non-destructive, rapid and low cost. However, due to the influence of light, temperature, humidity, near-infrared intensity, instrument and dynamic change of seed activity, the near-infrared spectra of maize seeds showed high dimensional nonlinear characteristics. In this study, to make full use of the class label information, a nonlinear feature analytical method for haploid maize seeds identification based on Supervised Virtual Sample Kernel Locality Preserving Projection (SVSKLPP) has been proposed. The experimental results showed that the nonlinear identification model SVSKLPP achieved strong classification performance to identify the maize haploid seeds. Moreover, compared with the linear feature extraction methods Principal Component Analysis, Orthogonal Linear Discriminant Analysis, Locality Preserving Projection, Supervised Virtual Sample Locality Preserving Projection and nonlinear feature extraction methods Kernel Locality Preserving Projection, Isomap, Locally Linear Embedding, Laplacian Eigenmaps and Local Tangent Space Alignment, the SVSKLPP model achieved a better performance. The average accuracy, sensitivity and specificity using method SVSKLPP were 97.1%, 98.8% and 95.4% respectively, and it also had high robustness. The overall results show that the SVSKLPP-NIR methodology was efficient in accurately identifying haploid maize seeds, thus demonstrating its capabilities for application in haploid breeding for crop variety improvement.

Spot image ablated by femtosecond laser segmentation and feature clustering after dimension reduction reconstruction

When monocrystalline silicon material is ablated by femtosecond laser, plasma derived luminescence also occurs. By collecting plasma spot image and analyzing the feature of spot image, which can be used for the classification of laser ablation power and the parameters optimization of ablation process. Considering that the contrast between edge and background area of plasma spot is not obvious and the signal to noise ratio is lower, we analyzes the segmentation efficiency for dim spot target by comparing the traditional Otsu (maximum class square error method) and PCA(Principal Component Analysis). The experimental study shows that the extracted multiple geometric features of spot image segmented by PCA method are more consistent; however, the ablation transition zone and part halo are segmented into target by traditional Otsu, which leads to the larger dispersion among extracted geometric features of spot, and it is also not beneficial for feature analysis and recognition of ablation process parameters. Further, the brightness of spot image under different ablation power was extracted, combining with pixel area, perimeter, long and short axis and long and short axis ratio of spot image, a six-dimensional feature matrix is built to classify the laser ablation power. Because of the large amount of data collected plasma spot sequence image, the manifold learning algorithm is used to reduce the dimension of feature matrix. Each spot image with 10mW, 20mW and 50mW ablation power is selected respectively 100 frames to build the six-dimensional feature matrix, three manifold learning algorithms are used in contrast to realize matrix dimensionality reduction and scatter plot reconstruction, then the feature distributions of spot image ablated by three kinds of ablation power are observed in three-dimensional space and two-dimensional plane, we find that the clustering effect for decreased dimension feature points using LPP(Locality preserving projections) algorithm and LLTSA(Linear Local Tangent Space Alignment) are better, they can be used to classify the spot image under different ablation power.

A spectral-spatial method based on low-rank and sparse matrix decomposition for hyperspectral anomaly detection

ABSTRACTRecently, some methods based on low-rank and sparse matrix decomposition (LRASMD) have been developed to improve the performance of hyperspectral anomaly detection (AD). However, these methods mainly take advantage of the spectral information in hyperspectral imagery (HSI), and ignore the spatial information. This article proposes an LRASMD-based spectral-spatial (LS-SS) method for hyperspectral AD. First, the Go Decomposition (GoDec) algorithm is employed to solve the low-rank background component and the sparse anomaly component. Next, the sparse component is explored to calculate the spectral sparsity divergence index (SDI). Based on spectral SDI, the detection result in the spectral domain and the reliable background points, which are employed as training data to construct the background manifold by linear local tangent space alignment (LLTSA), can also be obtained. Then, based on the background manifold and the transformation matrix, the low-dimensional manifold of the whole data is computed by linear mapping. After that, the kernel collaborative representation detector (KCRD) is used in the low-dimensional manifold of the whole data for the spatial SDI. Finally, SS SDI is computed for the final detection result. The theoretical analysis and experimental results demonstrate that the proposed LS-SS can achieve better performance when compared with the comparison algorithms.

On the Equivalence of HLLE and LTSA.

Among the representative algorithms of manifold learning, Hessian locally linear embedding (HLLE) and local tangent space alignment (LTSA) algorithms haven been regarded as two different algorithms. However, in practice, the effects of these two algorithms are very similar and LTSA performs better than HLLE in some applications. This paper tries to account for this phenomenon from a mathematical point of view. There are only two differences between HLLE and LTSA. First, LTSA includes a data point into its neighborhood, while HLLE does not. Second, HLLE and LTSA use different methods to align the local coordinates of manifold. In this paper, we show that, the first difference between HLLE and LTSA is not essential. However, from the viewpoint of data utilization, LTSA does better than HLLE in the neighborhood construction. This may account for why LTSA can perform better than HLLE in some applications. As for the second difference between HLLE and LTSA, we first prove that, the alignment equations used by HLLE and LTSA are exactly the same. Second, we prove that, although HLLE and LTSA uses different methods to solve the alignment equation, their solutions are exactly the same, provided that HLLE adopts the same method as LTSA to construct the neighborhoods. Based on these arguments, we claim that HLLE and LTSA are equivalent to each other. This conclusion can also be verified experimentally by using manifold learning MATLAB demo (MANI), a widely-used experimental platform of manifold learning. When testing HLLE on MANI, if HLLE adopts the same method as LTSA to construct the neighborhoods, the experimental results presented by MANI will be the same as those of LTSA.

Improved linear local tangent space alignment and its application to pattern recognition

Considering the drawbacks of the linear local tangent space alignment, a Semi-Supervised Neighbourhood Adaptive Linear Local Tangent Space Alignment (SSNA-LLTSA) is proposed. The distance metric combining the Cosine similarity and the Euclidean distance is used in the algorithm instead of the Euclidean distance, and the algorithm realises the semi-supervised learning and neighbourhood adaptive adjustment by integrating some of the known category information and the method of Parzen window density estimation into the dimensionality reduction process. The simulation experiment of UCI standard data sets and the pattern recognition example of hydraulic pump show that the redefined distance metric has better performance than the Euclidean distance, and SSNA-LLTSA can overcome the defect that LLTSA is unsupervised. Meanwhile, the capability of neighbourhood adaptive adjustment makes the algorithm find the low-dimensional manifold of the data sets more effectively, which can further improve the accuracy of pattern recognition.

Hyperspectral anomaly detection based on spectral–spatial background joint sparse representation

ABSTRACTIn recent years, some algorithms based on sparse representation have been proposed to improve the detection performance for hyperspectral anomaly detection. Among these algorithms, the background joint sparse representation (BJSR) algorithm adaptively selects the most representative background bases for the local region and can obtain satisfactory results. However, BJSR mainly considers spectral characteristics of hyperspectral image. In this paper, we propose a BJSR-based spectral–spatial method. BJSR is first employed to process the original hyperspectral image in spectral domain. Then, linear local tangent space alignment (LLTSA) is used to obtain the low-dimensional manifold of the hyperspectral image. Next, spatial BJSR is used to process the low-dimensional manifold obtained by LLTSA. Finally, the proposed algorithm combines spectral BJSR with spatial BJSR to detect the anomaly targets. The experimental results demonstrate that the proposed algorithm can achieve a better performance when compared with the comparison algorithms.

Improved linear local tangent space alignment and its application to pattern recognition

Improved linear local tangent space alignment and its application to pattern recognition

Pedestal looseness extent recognition method for rotating machinery based on vibration sensitive time-frequency feature and manifold learning

To realize automation and high accuracy of pedestal looseness extent recognition for rotating machinery, a novel pedestal looseness extent recognition method for rotating machinery based on vibration sensitive time-frequency feature and manifold learning dimension reduction is proposed. Firstly, the pedestal looseness extent of rotating machinery is characterized by vibration signal of rotating machinery and its spectrum, then the time-frequency features are extracted from vibration signal to construct the origin looseness extent feature set. Secondly, the algorithm of looseness sensitivity index is designed to filter out the non-sensitive feature and poor sensitivity feature from the origin looseness extent feature set, avoiding the interference of non-sensitive and poor sensitivity feature. The sensitive features are selected to construct the looseness extent sensitive feature set, which has stronger characterization capabilities than the origin looseness extent feature set. Moreover, an effective manifold learning method called linear local tangent space alignment (LLTSA) is introduced to compress the looseness extent sensitive feature set into the low-dimensional looseness extent sensitive feature set. Finally, the low-dimensional looseness extent sensitive feature set is inputted into weight K nearest neighbor classifier (WKNNC) to recognize the different pedestal looseness extents of rotating machinery, the WKNNC’s recognition accuracy is more stable compared with that of a k nearest neighbor classification (KNNC). At the same time, the pedestal looseness extent recognition of rotating machinery is realized. The feasibility and validity of the present method are verified by successful pedestal looseness extent recognition application in a rotating machinery.