Nonlinear Dimensionality Reduction Method Research Articles

A Study of Speech Feature Extraction Based on Manifold Learning

Manifold learning is a nonlinear data dimension reduction method. It can look for the essence of things from the observed phenomena, and find the inherent law of data. Traditional MFCC feature will lead a slower learning speed on account of it has high dimension and useless noise. Therefore, a speech feature extraction method based on manifold learning is proposed. Firstly, we use the manifold learning dimension reduction algorithm for the dimension reduction of Mel features and then for vowels classification. In order to further demonstrate the effectiveness of manifold learning feature in speech recognition, we propose a fusion speech feature extraction method and apply it to the identification of Chinese isolated words. Experiments prove that the fusion feature extraction method has achieved a better result than that of traditional MFCC feature extraction method.

Probabilistic ancestry maps: a method to assess and visualize population substructures in genetics

BackgroundPrincipal component analysis (PCA) is a standard method to correct for population stratification in ancestry-specific genome-wide association studies (GWASs) and is used to cluster individuals by ancestry. Using the 1000 genomes project data, we examine how non-linear dimensionality reduction methods such as t-distributed stochastic neighbor embedding (t-SNE) or generative topographic mapping (GTM) can be used to provide improved ancestry maps by accounting for a higher percentage of explained variance in ancestry, and how they can help to estimate the number of principal components necessary to account for population stratification. GTM generates posterior probabilities of class membership which can be used to assess the probability of an individual to belong to a given population - as opposed to t-SNE, GTM can be used for both clustering and classification.ResultsPCA only partially identifies population clusters and does not separate most populations within a given continent, such as Japanese and Han Chinese in East Asia, or Mende and Yoruba in Africa. t-SNE and GTM, taking into account more data variance, can identify more fine-grained population clusters. GTM can be used to build probabilistic classification models, and is as efficient as support vector machine (SVM) for classifying 1000 Genomes Project populations.ConclusionThe main interest of probabilistic GTM maps is to attain two objectives with only one map: provide a better visualization that separates populations efficiently, and infer genetic ancestry for individuals or populations. This paper is a first application of GTM for ancestry classification models. Our code (https://github.com/hagax8/ancestry_viz) and interactive visualizations (https://lovingscience.com/ancestries) are available online.

Optimal Landmark Point Selection Using Clustering for Manifold Modeling and Data Classification

As data volume and dimensions continue to grow, effective and efficient methods are needed to obtain the low dimensional features of the data that describe its true structure. Most nonlinear dimensionality reduction methods (NLDR) utilize the Euclidean distance between the data points to form a general idea of the data manifold structure. Isomap uses the geodesic distance between data points and then uses classical multidimensional scaling(cMDS) to obtain low dimensional features. As the data size increases Isomap becomes complex. To overcome this disadvantage, Landmark Isomap (L-Isomap) uses selected data points called landmark points and finds the geodesic distance from these points to all other non-landmark points. Traditionally, landmark points are randomly selected without considering any statistical property of the data manifold. We contend that the quality of the features extracted is dependent on the selection of the landmark points. In applications such as data classification, the net accuracy is dependent on the quality of the features selected, and hence landmark points selection might play a crucial role. In this paper, we propose a clustering approach to obtain the landmark points. These new points are now used to represent the data, and Fisher’s linear discriminants are used for classification. The proposed method is tested with different datasets to verify the efficacy of the approach.

Research on P2P Credit Risk Assessment Model Based on RBM Feature Extraction—Take SME Customers as an Example

This paper combines the nonlinear dimensionality reduction method, and the Restricted Boltzmann machine (RBM algorithm), to assess the credit risk of P2P borrowers. After screening and processing many big data indicators, the most representative indicators are selected to build the P2P customer credit risk assessment model. In addition, after comparing the advantages and disadvantages of linear dimensionality reduction algorithm and nonlinear dimensionality reduction algorithm, this paper establishes a P2P enterprise customer credit risk assessment model based on RBM feature extraction combined with contrast divergence theory. It is concluded that the effect of RBM is better than that of PCA when the same model is selected. The Logistic model performs best in the three models when the same data feature extraction method is selected.

Ugtm: A Python Package for Data Modeling and Visualization Using Generative Topographic Mapping

ugtm is a Python package that implements generative topographic mapping (GTM), a dimensionality reduction algorithm by Bishop, Svensén and Williams. Because of its probabilistic framework, GTM can also be used to build classification and regression models, and is an attractive alternative to t-distributed neighbour embedding (t-SNE) or other non-linear dimensionality reduction methods. The package is compatible with scikit-learn, and includes a GTM transformer (eGTM), a GTM classifier (eGTC) and a GTM regressor (eGTR). The input and output of these functions are numpy arrays. The package implements supplementary functions for GTM visualization and kernel GTM (kGTM). The code is under MIT license and available on GitHub (<a href="https://github.com/hagax8/ugtm" target="_blank">https://github.com/hagax8/ugtm</a>). For installation instructions and documentation, cf. <a href="https://ugtm.readthedocs.io/" target="_blank">https://ugtm.readthedocs.io</a>. <strong>Funding statement:</strong> HG acknowledges funding from the US National Institute of Mental Health (PGC3: U01 MH109528).

Data Visualization using Linear and Non-linear Dimensionality Reduction Methods

Data Visualization using Linear and Non-linear Dimensionality Reduction Methods

A survey on Laplacian eigenmaps based manifold learning methods

As a well-known nonlinear dimensionality reduction method, Laplacian Eigenmaps (LE) aims to find low dimensional representations of the original high dimensional data by preserving the local geometry between them. LE has attracted great attentions because of its capability of offering useful results on a broader range of manifolds. However, when applying it to some real-world data, several limitations have been exposed such as uneven data sampling, out-of-sample problem, small sample size, discriminant feature extraction and selection, etc. In order to overcome these problems, a large number of extensions to LE have been made. So in this paper, we make a systematical survey on these extended versions of LE. Firstly, we divide these LE based dimensionality reduction approaches into several subtypes according to different motivations to address the issues existed in the original LE. Then we successively discuss them from strategies, advantages or disadvantages to performance evaluations. At last, the future works are also suggested after some conclusions are drawn.

Nonlinear dimensionality reduction of hyperspectral images based on spectral angles and exploiting the spatial context

I proposed a nonlinear method for the dimensionality reduction of hyperspectral images is in this paper. A special feature of the proposed method is the use of spectral angles in the initial hyperspectral space as a dissimilarity measure between pixels of an image, as well as taking into account the spatial context of the hyperspectral image pixels. I used a well-known hyperspectral image dataset in the experiments. The experiments showed the advantage of the developed method over the basic nonlinear dimensionality reduction methods and the linear principal component analysis technique.

DimReader: Axis lines that explain non-linear projections.

Non-linear dimensionality reduction (NDR) methods such as LLE and t-SNE are popular with visualization researchers and experienced data analysts, but present serious problems of interpretation. In this paper, we present DimReader, a technique that recovers readable axes from such techniques. DimReader is based on analyzing infinitesimal perturbations of the dataset with respect to variables of interest. The perturbations define exactly how we want to change each point in the original dataset and we measure the effect that these changes have on the projection. The recovered axes are in direct analogy with the axis lines (grid lines) of traditional scatterplots. We also present methods for discovering perturbations on the input data that change the projection the most. The calculation of the perturbations is efficient and easily integrated into programs written in modern programming languages. We present results of DimReader on a variety of NDR methods and datasets both synthetic and real-life, and show how it can be used to compare different NDR methods. Finally, we discuss limitations of our proposal and situations where further research is needed.

Incremental LLE Based on Back Propagation Neural Network

Locally Linear Embedding (LLE) algorithm is one of promising NonLinear Dimensionality Reduction (NLDR) method for feature extraction. Like most NLDR algorithms, LLE operates in a batch or off-line mode, in other words, for newly coming samples, the old data augmented by the new samples must be completely recalculated by LLE algorithm, which is computationally intensive. Back propagation neural network (BP) is a nonlinear mapping method, and it can learn all the information of a dataset, further, when BP is trained well, it is effective to predict new data. Hence, in this paper, BP is combined with LLE (BPLLE) to deal with out-of-sample data. Four synthetic datasets and two real datasets are given to demonstrate that BPLLE is more valid than several classical incremental LLE algorithms.

Multistate multivariate statistical process control

AbstractFor high‐dimensional, autocorrelated, nonlinear, and nonstationary data, adaptive‐dynamic principal component analysis (AD‐PCA) has been shown to do as well or better than nonlinear dimension reduction methods in flagging outliers. In some engineered systems, designed features can create a known multistate scheme among multiple autocorrelated, nonlinear, and nonstationary processes, and incorporating this additional known information into AD‐PCA can further improve it. In simulations with one of three types of faults introduced, we compare accounting for the states versus ignoring them. We find that multistate AD‐PCA reduces the proportion of false alarms and reduces the average time to fault detection. Conversely, we also investigate the impact of assuming multiple states when only one exists, and find that as long as the number of observations is sufficient, this misspecification is not detrimental. We then apply multistate AD‐PCA to real‐world data collected from a decentralized wastewater treatment system during in control and out of control conditions. Multistate AD‐PCA flags a strong system fault earlier and more consistently than its single‐state competitor. Furthermore, accounting for the physical switching system does not increase the number of false alarms when the process is in control and may ultimately assist with fault attribution.

Land Cover Classification Using Integrated Spectral, Temporal, and Spatial Features Derived from Remotely Sensed Images

Obtaining accurate and timely land cover information is an important topic in many remote sensing applications. Using satellite image time series data should achieve high-accuracy land cover classification. However, most satellite image time-series classification methods do not fully exploit the available data for mining the effective features to identify different land cover types. Therefore, a classification method that can take full advantage of the rich information provided by time-series data to improve the accuracy of land cover classification is needed. In this paper, a novel method for time-series land cover classification using spectral, temporal, and spatial information at an annual scale was introduced. Based on all the available data from time-series remote sensing images, a refined nonlinear dimensionality reduction method was used to extract the spectral and temporal features, and a modified graph segmentation method was used to extract the spatial features. The proposed classification method was applied in three study areas with land cover complexity, including Illinois, South Dakota, and Texas. All the Landsat time series data in 2014 were used, and different study areas have different amounts of invalid data. A series of comparative experiments were conducted on the annual time-series images using training data generated from Cropland Data Layer. The results demonstrated higher overall and per-class classification accuracies and kappa index values using the proposed spectral-temporal-spatial method compared to spectral-temporal classification methods. We also discuss the implications of this study and possibilities for future applications and developments of the method.

Study of feature extraction method of multi-information source for continuous casting process parameters

The control of the roll gap of the segment is one of the key links to ensure the quality of cast billet. In this paper, the big data in traditional continuous casting production operations is studied through in-depth experimental comparative analysis of linear and nonlinear dimension reduction method. The method is suitable for continuous casting to obtain the data of the dimension reduction. The method of principal component analysis is improved by using standardized data increment method. A faster and more efficient method of dimension reduction is obtained when the unrelated data, training time and reconstruction error are removed. Actual data simulation results show that this method is more efficient and suitable for continuous casting than any other dimension reduction method.

Стохастическая роевая кластеризация в задачах автоматизированной обработки данных, представленных на естественном языке

Consider natural language data processing technology based on non-linear dimensionality reduction method which takes into account the discriminating power of the solution found for given values of the categorical variable associated with each observation. Stochastic optimization method known as the “Particle swarm optimization” is proposed to found characteristics that ensure the best separation of observations in terms of a given quality functional. The basis for evaluating the quality of the solution lies in the purity of the clusters obtained with the k-means method, or with using self-organizing Kohonen feature maps.

Extreme Learning Machines for VISualization+R: Mastering Visualization with Target Variables

The current paper presents an improvement of the Extreme Learning Machines for VISualization (ELMVIS+) nonlinear dimensionality reduction method. In this improved method, called ELMVIS+R, it is proposed to apply the originally unsupervised ELMVIS+ method for the regression problems, using target values to improve visualization results. It has been shown in previous work that the approach of adding supervised component for classification problems indeed allows to obtain better visualization results. To verify this assumption for regression problems, a set of experiments on several different datasets was performed. The newly proposed method was compared to the ELMVIS+ method and, in most cases, outperformed the original algorithm. Results, presented in this article, prove the general idea that using supervised components (target values) with nonlinear dimensionality reduction method like ELMVIS+ can improve both visual properties and overall accuracy.

Manifold Learning by Curved Cosine Mapping

In the field of pattern recognition, data analysis, and machine learning, data points are usually modeled as high-dimensional vectors. Due to the curse-of-dimensionality, it is non-trivial to efficiently process the orginal data directly. Given the unique properties of nonlinear dimensionality reduction techniques, nonlinear learning methods are widely adopted to reduce the dimension of data. However, existing nonlinear learning methods fail in many real applications because of the too-strict requirements (for real data) or the difficulty in parameters tuning. Therefore, in this paper, we investigate the manifold learning methods which belong to the family of nonlinear dimensionality reduction methods. Specifically, we proposed a new manifold learning principle for dimensionality reduction named Curved Cosine Mapping (CCM). Based on the law of cosines in Euclidean space, CCM applies a brand new mapping pattern to manifold learning. In CCM, the nonlinear geometric relationships are obtained by utlizing the law of cosines, and then quantified as the dimensionality-reduced features. Compared with the existing approaches, the model has weaker theoretical assumptions over the input data. Moreover, to further reduce the computation cost, an optimized version of CCM is developed. Finally, we conduct extensive experiments over both artificial and real-world datasets to demonstrate the performance of proposed techniques.

Deep linear autoencoder and patch clustering-based unified one-dimensional coding of image and video

This paper proposes a unified one-dimensional (1-D) coding framework of image and video, which depends on deep learning neural network and image patch clustering. First, an improved K-means clustering algorithm for image patches is employed to obtain the compact inputs of deep artificial neural network. Second, for the purpose of best reconstructing original image patches, deep linear autoencoder (DLA), a linear version of the classical deep nonlinear autoencoder, is introduced to achieve the 1-D representation of image blocks. Under the circumstances of 1-D representation, DLA is capable of attaining zero reconstruction error, which is impossible for the classical nonlinear dimensionality reduction methods. Third, a unified 1-D coding infrastructure for image, intraframe, interframe, multiview video, three-dimensional (3-D) video, and multiview 3-D video is built by incorporating different categories of videos into the inputs of patch clustering algorithm. Finally, it is shown in the results of simulation experiments that the proposed methods can simultaneously gain higher compression ratio and peak signal-to-noise ratio than those of the state-of-the-art methods in the situation of low bitrate transmission.

Spectral regression based marginal Fisher analysis dimensionality reduction algorithm

Abstract Traditional nonlinear dimensionality reduction methods, such as multiple kernel dimensionality reduction and nonlinear spectral regression (SR), are generally regarded as extended versions of linear discriminant analysis (LDA) in the supervised case. As is well known, LDA has the restrictive assumption that the data of each class is of a Gaussian distribution. Thus, the performance of these methods will be degraded if such an assumption is not hold. Although some methods based on marginal Fisher analysis are proposed to overcome the drawback of LDA, they have to solve the problem of dense metrics generalized eigenvalue decomposition, which is very time-consuming. To address these issues, in this paper, marginal Fisher analysis criterion based on extreme learning machine (ELM) is proposed to improve spectral regression and kernel marginal Fisher analysis. It is proved that the proposed marginal Fisher analysis is a special case of traditional kernel marginal Fisher analysis. Based on the proposed criterion, a novel supervised dimensionality reduction algorithm is presented by virtue of ELM and spectral regression. Experimental results on benchmark datasets validate that the proposed algorithm outperforms the state-of-the-art nonlinear dimensionality reduction methods in supervised scenarios.

Spectral Affine‐Kernel Embeddings

AbstractIn this paper, we propose a controllable embedding method for high‐ and low‐dimensional geometry processing through sparse matrix eigenanalysis. Our approach is equally suitable to perform non‐linear dimensionality reduction on big data, or to offer non‐linear shape editing of 3D meshes and pointsets. At the core of our approach is the construction of a multi‐Laplacian quadratic form that is assembled from local operators whose kernels only contain locally‐affine functions. Minimizing this quadratic form provides an embedding that best preserves all relative coordinates of points within their local neighborhoods. We demonstrate the improvements that our approach brings over existing nonlinear dimensionality reduction methods on a number of datasets, and formulate the first eigen‐based as‐rigid‐as‐possible shape deformation technique by applying our affine‐kernel embedding approach to 3D data augmented with user‐imposed constraints on select vertices.

Optimized Kernel Minimum Noise Fraction Transformation for Hyperspectral Image Classification

This paper presents an optimized kernel minimum noise fraction transformation (OKMNF) for feature extraction of hyperspectral imagery. The proposed approach is based on the kernel minimum noise fraction (KMNF) transformation, which is a nonlinear dimensionality reduction method. KMNF can map the original data into a higher dimensional feature space and provide a small number of quality features for classification and some other post processing. Noise estimation is an important component in KMNF. It is often estimated based on a strong relationship between adjacent pixels. However, hyperspectral images have limited spatial resolution and usually have a large number of mixed pixels, which make the spatial information less reliable for noise estimation. It is the main reason that KMNF generally shows unstable performance in feature extraction for classification. To overcome this problem, this paper exploits the use of a more accurate noise estimation method to improve KMNF. We propose two new noise estimation methods accurately. Moreover, we also propose a framework to improve noise estimation, where both spectral and spatial de-correlation are exploited. Experimental results, conducted using a variety of hyperspectral images, indicate that the proposed OKMNF is superior to some other related dimensionality reduction methods in most cases. Compared to the conventional KMNF, the proposed OKMNF benefits significant improvements in overall classification accuracy.