Original High-dimensional Data Research Articles

Hyperspectral Image Mixed Denoising Using Difference Continuity-Regularized Nonlocal Tensor Subspace Low-Rank Learning

With the rapid advancement of spectrometers, the imaging range of the electromagnetic spectrum starts growing narrower. The reduction of electromagnetic wave energy received in a single wavelength range leads more complex noise into the generated hyperspectral image (HSI), thus causing a severe cripple in the accuracy of subsequent applications. The requirement for the HSI mixed denoising algorithm’s accuracy is further lifted. To address this challenge, in this letter, we propose a novel difference continuity-regularized nonlocal tensor subspace low-rank learning (named DNTSLR) method for HSI mixed denoising. Technically, the original high-dimensional HSI data was first projected into a low-dimensional subspace spanned by a spectral difference continuous basis instead of an orthogonal basis, so the data continuity of the restored HSI spectrum and tensor low-rankness was guaranteed. Then, a cube matching strategy was employed to stack the nonlocal tensor patches from the projected coefficient tensor, and a shrinkage algorithm was used to approximate the low-rank coefficient tensor. Eventually, the subspace low-rank learning algorithm was designed to alternately separate the noise tensor and restore the latent clean low-rank HSI tensor. Extensive experiments on multiple open datasets validate that the proposed method realizes the state-of-the-art denoising accuracy for HSI.

Polynomial approximation to manifold learning

Manifold learning plays an important role in nonlinear dimensionality reduction. But many manifold learning algorithms cannot offer an explicit expression for dealing with the problem of out-of-sample (or new data). In recent, many improved algorithms introduce a fixed function to the object function of manifold learning for learning this expression. In manifold learning, the relationship between the high-dimensional data and its low-dimensional representation is a local homeomorphic mapping. Therefore, these improved algorithms actually change or damage the intrinsic structure of manifold learning, as well as not manifold learning. In this paper, a novel manifold learning based on polynomial approximation (PAML) is proposed, which learns the polynomial approximation of manifold learning by using the dimensionality reduction results of manifold learning and the original high-dimensional data. In particular, we establish a polynomial representation of high-dimensional data with Kronecker product, and learns an optimal transformation matrix with this polynomial representation. This matrix gives an explicit and optimal nonlinear mapping between the high-dimensional data and its low-dimensional representation, and can be directly used for solving the problem of new data. Compare with using the fixed linear or nonlinear relationship instead of the manifold relationship, our proposed method actually learns the polynomial optimal approximation of manifold learning, without changing the object function of manifold learning (i.e., keeping the intrinsic structure of manifold learning). We implement experiments over eight data sets with the advanced algorithms published in recent years to demonstrate the benefits of our algorithm.

Image classification method of cashmere and wool based on the multi-feature selection and random forest method

Cashmere and wool play an important role in the wool industry and textile industry, and suitable features are the key to identifying them. To obtain effective features and improve the accuracy of cashmere and wool classification, the multi-feature selection and random forest method is used to express in this article. Firstly, the gray-gradient co-occurrence matrix model is used for texture feature extraction to construct the original high-dimensional feature data set; secondly, considering that the original feature data set contains a large number of invalid and redundant features, the feature selection algorithm combining correlation analysis and principal component analysis–weight coefficient evaluation is used to obtain important features, independent features, and principal component sensitive features to complement each other; last but not least, the optimized random forest model analyzes the results. The results show that the combination of multi-feature selection subsets and random forest makes the classification accuracy of cashmere and wool more reliable, and the accuracy fluctuates around 90%.

Deep reconstruction of 1D ISOMAP representations

This paper proposes a deep learning priors-based data reconstruction method of 1D isometric feature mapping (ISOMAP) representations. ISOMAP is a classical algorithm of nonlinear dimensionality reduction (NLDR) or manifold leaning (ML), which is devoted to questing for the low dimensional structure of high dimensional data. The reconstruction of ISOMAP representations, or the inverse problem of ISOMAP, reestablishes the high dimensional data from its low dimensional ISOMAP representations, and owns a bright future in data representation, generation, compression and visualization. Due to the fact that the dimension of ISOMAP representations is far less than that of the original high dimensional data, the reconstruction of ISOMAP representations is ill-posed or undetermined. Hence, the residual learning of deep convolutional neural network (CNN) is employed to boost reconstruction performance, via achieving the priors between the low-quality result of general ISOMAP reconstruction method and its residual relative to the original data. In the situation of 1D representations, it is evaluated by the experimental results that the proposed method outbalances the state-of-the-art methods, such as nearest neighbor (NN), discrete cosine transformation (DCT) and sparse representation (SR), in reconstruction performance of video data. In summary, the proposed method is suitable for low-bitrate and high-performance applications of data reconstruction.

Dynamic neural orthogonal mapping for fault detection

Dynamic principal component analysis (DPCA) and its nonlinear extension, dynamic kernel principal component analysis (DKPCA), are widely used in the monitoring of dynamic multivariate processes. In traditional DPCA and DKPCA, extended vectors through concatenating current process data point and a certain number of previous process data points are utilized for feature extraction. The dynamic relations among different variables are fixed in the extended vectors, i.e. the adoption of the dynamic information is not adaptively learned from raw process data. Although DKPCA utilizes a kernel function to handle dynamic and (or) nonlinear information, the prefixed kernel function and the associated parameters cannot be most effective for characterizing the dynamic relations among different process variables. To address these problems, this paper proposes a novel nonlinear dynamic method, called dynamic neural orthogonal mapping (DNOM), which consists of data dynamic extension, a nonlinear feedforward neural network, and an orthogonal mapping matrix. Through backpropagation and Eigen decomposition (ED) technique, DNOM can be optimized to extract key low-dimensional features from original high-dimensional data. The advantages of DNOM are demonstrated by both theoretical analysis and extensive experimental results on the Tennessee Eastman (TE) benchmark process. The results on the TE benchmark process show the superiority of DNOM in terms of missed detection rate and false alarm rate. The source codes of DNOM can be found in https://github.com/htz-ecust/DNOM .

Multi-level adaptive neuro-fuzzy inference system-based reconstruction of 1D ISOMAP representations

This paper addresses the inverse problem of isometric feature mapping (ISOMAP) via multi-level adaptive neuro-fuzzy inference system (ML-ANFIS). ISOMAP is a conventional nonlinear dimensionality reduction (NLDR) method, which prospects for low dimensional interior structure embedded in high dimensional data space. The inverse problem of ISOMAP reconstructs the original high dimensional data from the related low dimensional ISOMAP representations and holds promising applications in data representations, generation, compression and visualization. Because the reconstruction of 1D ISOMAP representations is ill-posed and undetermined, ML-ANFIS is wielded to augment the recovery quality of general ISOMAP reconstruction algorithm. By linearly combing inputs with nonlinear weights as output, ML-ANFIS can efficiently achieve the latent nonlinear relationship between the low-performance result of general ISOMAP reconstruction algorithm and its original data. The membership functions and fuzzy rules of ML-ANFIS can be automatically learned by gradient descent method as deep learning. It is demonstrated by the experimental results that, in the situation of 1D representations, the proposed method is superior to the state-of-the-art methods, such as nearest neighbor (NN), discrete cosine transformation (DCT), sparse representations (SR) and classical ANFIS algorithms, in the reconstruction performance of video data.

Kernel‐based low‐rank tensorized multiview spectral clustering

Multiview spectral clustering aims to separate data into different clusters efficiently by the use of multiview information. Many studies learn the affinity matrix from the original high-dimensional data, whose noise goes against the clustering results. Besides, some methods based on self-representation subspace clustering have a high time complexity. In this paper, we propose a simple, yet effective, and efficient method named Kernel-based Low-rank Tensorized Multiview Spectral Clustering (KLTMSC) to address these issues. Instead of using the original data to get the affinity matrix, KLTMSC learns the affinity matrix from kernel representation of the high-dimensional data to reduce the noisy information. Furthermore, to be robust to noise, the low-rank tensor is learned in the process of exploring the high-order correlations between data. Experiments on real-world data sets show that our method not only yields better results but also is quite time-saving compared with other state-of-the-art models.

Development of an efficient data-driven method to estimate the hydraulic properties of aquifers from groundwater level fluctuation pattern features

A method to develop a data-driven model able to estimate hydraulic properties based on groundwater level (GWL) fluctuation patterns is proposed. In particular, a preprocessing method using a denoising autoencoder (DAE) is incorporated into the proposed method to improve the performance of the developed method. DAE is applied to prepare the input variable of the estimation model by extracting informative low-dimensional features from the original high-dimensional GWL data. Before applying the proposed DAE to this study, the reliability of applying the DAE is validated. First, the ability to reduce the noise of GWL data is validated by observing that an average of 71% of the noise is reduced. Additionally, the performances of the extracted principal characteristics of the GWL data is confirmed by reasonable matches in the extracted features to the corresponding hydraulics of the aquifers. In this case, both synthetic data and actual data acquired over South Korea are applied. Based on the validated DAE results, models to estimate two types of hydraulic properties are constructed. The estimation performances of the models are quantitatively validated using the correlation coefficient between the estimated and actual hydraulic properties. Overall, the constructed models for k and α/n show an appropriate estimation accuracy with a high correlation coefficient between the actual result and estimate (0.8663 and 0.7207, respectively). Therefore, using the proposed method, the hydraulic properties of an un-informed aquifer can be effectively inferred given GWL data without conducting field experiments (e.g., pumping tests). The proposed method is promising for efficient evaluations of the physical hydraulics of un-informed aquifers and, therefore, can be used as an effective tool to manage groundwater resources.

Dual subspace discriminative projection learning

In this paper, we propose a dual subspace discriminative projection learning (DSDPL) framework for multi-category image classification. Our approach reflects the notion that images are composed of class-shared information, class-specific information, and sparse noise. Unlike traditional subspace learning methods, DSDPL serves to decompose original high dimensional data, via learned projection matrices, into class-shared and class-specific subspaces. The learned projection matrices are jointly constrained with l2,1 sparse norm and LDA terms while the reconstructive properties of DSDPL reduce information loss, leading to greater stability within low dimensional subspaces. Regression-based terms are also included to facilitate a more robust classification approach, using extracted class-specific features for better classification. Our approach is examined on five different datasets for face, object and scene classifications. Experimental results demonstrate not only the superiority and versatility of DSDPL over current benchmark approaches, but also a more robust classification approach with low sample size training data.

Quantum locally linear embedding for nonlinear dimensionality reduction

Reducing the dimension of nonlinear data is crucial in data processing and visualization. The locally linear embedding algorithm (LLE) is specifically a representative nonlinear dimensionality reduction method with maintaining well the original manifold structure. In this paper, we present two implementations of the quantum locally linear embedding (QLLE) algorithm to perform the nonlinear dimensionality reduction on quantum devices. One implementation, the linear-algebra-based QLLE algorithm, utilizes quantum linear algebra subroutines to reduce the dimension of the given data. The other implementation, the variational quantum locally linear embedding (VQLLE) algorithm, utilizes a variational hybrid quantum-classical procedure to acquire the low-dimensional data. The classical LLE algorithm requires polynomial time complexity of N, where N is the global number of the original high-dimensional data. Compared with the classical LLE, the linear-algebra-based QLLE achieves quadratic speedup in the number and dimension of the given data. The VQLLE can be implemented on the near-term quantum devices in two different designs. In addition, the numerical experiments are presented to demonstrate that the two implementations in our work can achieve the procedure of locally linear embedding.

Multi-view projected clustering with graph learning

Graph based multi-view learning is well known due to its effectiveness and good clustering performance. However, most existing methods directly construct graph from original high-dimensional data which always contain redundancy, noise and outlying entries in real applications, resulting in unreliable and inaccurate graph. Moreover, they do not effectively select some useful features which are important for graph learning and clustering. To solve these limits, we propose a novel model that combines dimensionality reduction, manifold structure learning and feature selection into a framework. We map high-dimensional data into low-dimensional space to reduce the complexity of the algorithm and reduce the effect of noise and redundance. Therefore, we can adaptively learn a more accurate graph. Further more, ℓ21-norm regularization is adopted to adaptively select some important features which help improve clustering performance. Finally, an efficiently algorithm is proposed to solve the optimal solution. Extensive experimental results on some benchmark datasets demonstrate the superiority of the proposed method.

Towards Accurate Prediction for High-Dimensional and Highly-Variable Cloud Workloads with Deep Learning

Resource provisioning for cloud computing necessitates the adaptive and accurate prediction of cloud workloads. However, the existing methods cannot effectively predict the high-dimensional and highly-variable cloud workloads. This results in resource wasting and inability to satisfy service level agreements (SLAs). Since recurrent neural network (RNN) is naturally suitable for sequential data analysis, it has been recently used to tackle the problem of workload prediction. However, RNN often performs poorly on learning long-term memory dependencies, and thus cannot make the accurate prediction of workloads. To address these important challenges, we propose a deep Learning based Prediction Algorithm for cloud Workloads (L-PAW). First, a top-sparse auto-encoder (TSA) is designed to effectively extract the essential representations of workloads from the original high-dimensional workload data. Next, we integrate TSA and gated recurrent unit (GRU) block into RNN to achieve the adaptive and accurate prediction for highly-variable workloads. Using real-world workload traces from Google and Alibaba cloud data centers and the DUX-based cluster, extensive experiments are conducted to demonstrate the effectiveness and adaptability of the L-PAW for different types of workloads with various prediction lengths. Moreover, the performance results show that the L-PAW achieves superior prediction accuracy compared to the classic RNN-based and other workload prediction methods for high-dimensional and highly-variable real-world cloud workloads.

Layer-constrained variational autoencoding kernel density estimation model for anomaly detection

Unsupervised techniques typically rely on the probability density distribution of the data to detect anomalies, where objects with low probability density are considered to be abnormal. However, modeling the density distribution of high dimensional data is known to be hard, making the problem of detecting anomalies from high-dimensional data challenging. The state-of-the-art methods solve this problem by first applying dimension reduction techniques to the data and then detecting anomalies in the low dimensional space. Unfortunately, the low dimensional space does not necessarily preserve the density distribution of the original high dimensional data. This jeopardizes the effectiveness of anomaly detection. In this work, we propose a novel high dimensional anomaly detection method called LAKE. The key idea of LAKE is to unify the representation learning capacity of layer-constrained variational autoencoder with the density estimation power of kernel density estimation (KDE). Then a probability density distribution of the high dimensional data can be learned, which is able to effectively separate the anomalies out. LAKE successfully consolidates the merits of the two worlds, namely layer-constrained variational autoencoder and KDE by using a probability density-aware strategy in the training process of the autoencoder. Extensive experiments on six public benchmark datasets demonstrate that our method significantly outperforms the state-of-the-art methods in detecting anomalies and achieves up to 37% improvement in F1 score.

Adaptive Local Linear Discriminant Analysis

Dimensionality reduction plays a significant role in high-dimensional data processing, and Linear Discriminant Analysis (LDA) is a widely used supervised dimensionality reduction approach. However, a major drawback of LDA is that it is incapable of extracting the local structure information, which is crucial for handling multimodal data. In this article, we propose a novel supervised dimensionality reduction method named Adaptive Local Linear Discriminant Analysis (ALLDA), which adaptively learns a k -nearest neighbors graph from data themselves to extract the local connectivity of data. Furthermore, the original high-dimensional data usually contains noisy and redundant features, which has a negative impact on the evaluation of neighborships and degrades the subsequent classification performance. To address this issue, our method learns the similarity matrix and updates the subspace simultaneously so that the neighborships can be evaluated in the optimal subspaces where the noises have been removed. Through the optimal graph embedding, the underlying sub-manifolds of data in intra-class can be extracted precisely. Meanwhile, an efficient iterative optimization algorithm is proposed to solve the minimization problem. Promising experimental results on synthetic and real-world datasets are provided to evaluate the effectiveness of proposed method.

Linear regression classification steered discriminative projection for dimension reduction

Because of the simplicity and effectiveness of linear regression classification (LRC), LRC is widely applied into image classification. However, it processes the original high-dimensional data directly. It is well known that the original data usually contains a lot of redundant information or noise, which will reduce the performance of LRC algorithm and increase its running cost. At the same time, it usually suffers from out of sample problem. In order to overcome the weaknesses of LRC, a novel dimension reduction algorithm termed linear regression classification steered discriminative projection (LRC-DP) is presented by combining LRC with discriminative projection. LRC-DP not only fits LRC well, but also seeks a linear projection, in which the ratio of between-class reconstruction errors to within-class reconstruction errors is maximized in the transformation space. The proposed LRC-DP can learn a robust low-dimensional projection subspace from the original sample images in high-dimension space. In order to validate the performance of LRC-DP algorithm, extensive experiments are conducted on several public image databases. Experimental results reveal that the LRC-DP algorithm is feasible and effective.

Bayesian compressive principal component analysis

Principal component analysis (PCA) is a widely used method for multivariate data analysis that projects the original high-dimensional data onto a low-dimensional subspace with maximum variance. However, in practice, we would be more likely to obtain a few compressed sensing (CS) measurements than the complete high-dimensional data due to the high cost of data acquisition and storage. In this paper, we propose a novel Bayesian algorithm for learning the solutions of PCA for the original data just from these CS measurements. To this end, we utilize a generative latent variable model incorporated with a structure prior to model both sparsity of the original data and effective dimensionality of the latent space. The proposed algorithm enjoys two important advantages: 1) The effective dimensionality of the latent space can be determined automatically with no need to be pre-specified; 2) The sparsity modeling makes us unnecessary to employ multiple measurement matrices to maintain the original data space but a single one, thus being storage efficient. Experimental results on synthetic and real-world datasets show that the proposed algorithm can accurately learn the solutions of PCA for the original data, which can in turn be applied in reconstruction task with favorable results.

Dimensionality Reduction of Tensors Based on Local Homeomorphism and Global Subspace Projection Distance Minimum

Tensors are powerful tools for representing and processing multidimensional data. In this paper, an algorithm of dimension reduction for multidimensional data is proposed. We consider both the global and local structures of the data to fully extract important features. Multiplying a tensor by a matrix in mode-n can change the size of a certain dimension of the tensor. Therefore, for the global structure, we use a matrix to span a subspace to minimize the distance between the tensor data and its projection on the subspace for each mode. For local structure, we construct a dimensionality reduction algorithm for tensor data based on local homeomorphism to keep the continuous dependency relationship of the original high-dimensional tensor data unchanged after dimensionality reduced in each locality. The proposed algorithm achieves dimensionality reduction by combining the global subspace projection distance minimum and local homeomorphism, which realizes global variance maximization and maintains the local non-linear geometric structure at the same time. The proposed algorithm performs good feasibility while compared with other advanced algorithms in classification and clustering experiments.

Classification and Identification of Industrial Gases Based on Electronic Nose Technology.

Rapid detection and identification of industrial gases is a challenging problem. They have a complex composition and different specifications. This paper presents a method based on the kernel discriminant analysis (KDA) algorithm to identify industrial gases. The smell prints of four typical industrial gases were collected by an electronic nose. The extracted features of the collected gases were employed for gas identification using different classification algorithms, including principal component analysis (PCA), linear discriminant analysis (LDA), PCA + LDA, and KDA. In order to obtain better classification results, we reduced the dimensions of the original high-dimensional data, and chose a good classifier. The KDA algorithm provided a high classification accuracy of 100% by selecting the offset of the kernel function c = 10 and the degree of freedom d = 5. It was found that this accuracy was 4.17% higher than the one obtained using PCA. In the case of standard deviation, the KDA algorithm has the highest recognition rate and the least time consumption.

Fuzzy associative memories with autoencoding mechanisms

Associative memories constructed and operating in the presence of big data offer an effective way to realize association mechanisms aimed at storing and recalling items. In this study, we develop a logic-driven model of two-level fuzzy associative memories augmented by autoencoding processing. It is composed of two functional modules. The first module of this architecture implements an efficient dimensionality reduction of the original high dimensional data with the use of an autoencoder. This helps achieve storing and completing the recall realized by a logic-oriented associative memory which constitutes the second module of the architecture. The optimization of the association matrices studied in the paper involves both gradient-based learning mechanisms and the algorithms of population-based optimization, i.e., particle swarm optimization (PSO) and differential evolution (DE). A suite of experimental studies is presented to quantify the performance of the proposed approach. Comparative studies are also conducted to show and quantify the advantages of the mechanisms of associative recall and storage augmented by the autoencoding process.

Semi-Supervised Graph Regularized Deep NMF With Bi-Orthogonal Constraints for Data Representation.

Semi-supervised non-negative matrix factorization (NMF) exploits the strengths of NMF in effectively learning local information contained in data and is also able to achieve effective learning when only a small fraction of data is labeled. NMF is particularly useful for dimensionality reduction of high-dimensional data. However, the mapping between the low-dimensional representation, learned by semi-supervised NMF, and the original high-dimensional data contains complex hierarchical and structural information, which is hard to extract by using only single-layer clustering methods. Therefore, in this article, we propose a new deep learning method, called semi-supervised graph regularized deep NMF with bi-orthogonal constraints (SGDNMF). SGDNMF learns a representation from the hidden layers of a deep network for clustering, which contains varied and unknown attributes. Bi-orthogonal constraints on two factor matrices are introduced into our SGDNMF model, which can make the solution unique and improve clustering performance. This improves the effect of dimensionality reduction because it only requires a small fraction of data to be labeled. In addition, SGDNMF incorporates dual-hypergraph Laplacian regularization, which can reinforce high-order relationships in both data and feature spaces and fully retain the intrinsic geometric structure of the original data. This article presents the details of the SGDNMF algorithm, including the objective function and the iterative updating rules. Empirical experiments on four different data sets demonstrate state-of-the-art performance of SGDNMF in comparison with six other prominent algorithms.