Effective Dimension Reduction Research Articles

Detection of false data injection attack in power information physical system based on SVM–GAB algorithm

Cyber physical power system (CPPS) is highly dependent on information and communication technology, which makes it vulnerable to network attacks. Among them, false data injection attack (FDIA) is not easy to be found by traditional bad data detection methods, and becomes one of the main threats to the safe operation of power systems. However, the high complexity, large amount of data and transient characteristics of CPPS put forward higher requirements for the accuracy and efficiency of FDIA detection method. Therefore, in view of the characteristics of CPPS, this paper proposes SVM–GAB (Support Vector Machines–Gentle​ Adaboost) algorithm to effectively detect FDIA. Through the effective dimension reduction and classification of the measured data, the real-time and high-precision detection of FDIA is realized. This algorithm is compared with mainstream detection algorithms in IEEE-14 and IEEE-39 standard systems. The results show that the false alarm rate of this algorithm is reduced by at least 25% compared with the traditional detection algorithm, and the accuracy and real-time performance of the proposed detection algorithm are verified by experiments.

Fusion of effective dimension reduction and discriminative dictionary learning for high-dimensional classification

High-dimensional classification is a hot topic of pattern recognition. However, its performance has much room for improvement due to environment uncertainties and other factors. For example, redundant information from high dimensional data, illumination, and occlusion may lead to the inferior classification performance. Additionally, some potentially useful features of high-dimensional data are not fully captured. To address these problems, we propose a fusion of effective dimension reduction and discriminative dictionary learning for high-dimensional classification. A novel dimension reduction approach is employed to map training samples into a low-dimensional space by using the distance and label information. Then in the low-dimensional subspace, we integrate atomic Fisher constraint and local constraint of coding coefficients into the dictionary learning. Thus the discriminative atoms are effectively extracted. Finally, experimental results prove the efficacy of the proposed model. Besides, the comparative experiments with AlexNet and VGG19 prove the superiority of our method over some deep learning approaches.

Trajectory dimensionality reduction and hyperparameter settings of DBSCAN for trajectory clustering

The density-based spatial clustering of application with noise (DBSCAN) algorithm has good robustness and is widely employed to cluster vehicle trajectories for vehicle movement pattern recognition. However, the distance or similarity between two trajectories varies from tens to hundreds of thousands, and there is no effective method for determining the values of the hyperparameters eps and MinPts of DBSCAN. In addition, with increasing sizes of trajectory datasets, some trajectory clustering methods that directly analyse points and line segments incur large computational costs and time overhead. To solve these two dilemmas, the authors propose an effective trajectory dimensionality reduction method and a DBSCAN hyperparameter initial value setting method. The trajectory dimensionality reduction algorithm processes trajectories with different lengths into the same dimensionality (the same number of feature points). The reserved points preserve the spatial and temporal information of these trajectories as much as possible. The DBSCAN hyperparameter initial value setting algorithm obtains the effective initial values of eps and MinPts for facilitating subsequent adjustments. Finally, we validate these proposed methods on two trajectory datasets collected from two real-world scenes, and the experimental results are promising and effective.

Sparse minimum average variance estimation through signal extraction approach to multivariate regression

In this paper, a new sparse method called (MAVE-SiER) is proposed, to introduce MAVE-SiER, we combined the effective sufficient dimension reduction method MAVE with the sparse method Signal extraction approach to multivariate regression (SiER). MAVE-SiER has the benefit of expanding the Signal extraction method to multivariate regression (SiER) to nonlinear and multi-dimensional regression. MAVE-SiER also allows MAVE to deal with problems which the predictors are highly correlated. MAVE-SiER may estimate dimensions exhaustively while concurrently choosing useful variables. Simulation studies confirmed MAVE-SiER performance.

Learning Interaction Variables and Kernels from Observations of Agent-Based Systems

Dynamical systems across many disciplines are modeled as interacting particles or agents, with interaction rules that depend on a very small number of variables (e.g. pairwise distances, pairwise differences of phases, etc...), functions of the state of pairs of agents. Yet, these interaction rules can generate self-organized dynamics, with complex emergent behaviors (clustering, flocking, swarming, etc.). We propose a learning technique that, given observations of states and velocities along trajectories of the agents, yields both the variables upon which the interaction kernel depends and the interaction kernel itself, in a nonparametric fashion. This yields an effective dimension reduction which avoids the curse of dimensionality from the high-dimensional observation data (states and velocities of all the agents). We demonstrate the learning capability of our method to a variety of first-order interacting systems.

A Divisive Hierarchical Clustering Approach to Hyperspectral Band Selection

Band selection, considered as an effective dimensionality reduction technique for hyperspectral imagery (HSI), has become a hot topic for decades. Although various clustering-based methods have been applied to band selection, only a few studies explored the hierarchical structure among different spectral bands. And with regard to conventional hierarchical clustering, implemented in an agglomerative manner, both efficiency and accuracy of band selection still remain to rise. Moreover, the noise sensitivity is a defect inherent in the procedure of clustering. To address these issues, we propose a divisive hierarchical clustering approach (DHCA) to hyperspectral band selection. Inspired by divisive analysis, DHCA is designed to obtain any number of band subsets, which captures the intrinsic hierarchy of hyperspectral bands simultaneously. By introducing the local density into average dissimilarity, it can suppress the outliers clustering separately. Also, given the order of the spectrum, channel interval makes the similarity more rational among bands. Finally, we select a representative band in each cluster from the information viewpoint to ensure the band subset with a high quality. Extensive experiments on three real public HSI datasets fully validate the superiority of the proposed method against state-of-the-art competitors.

Examining the Size of the Latent Space of Convolutional Variational Autoencoders Trained With Spectral Topographic Maps of EEG Frequency Bands

Dimensionality reduction and the automatic learning of key features from electroencephalographic (EEG) signals have always been challenging tasks. Variational autoencoders (VAEs) have been used for EEG data generation and augmentation, denoising, and automatic feature extraction. However, investigations of the optimal shape of their latent space have been neglected. This research tried to understand the minimal size of the latent space of convolutional VAEs, trained with spectral topographic EEG head-maps of different frequency bands, that leads to the maximum reconstruction capacity of the input and maximum utility for classification tasks. Head-maps are generated employing a sliding window technique with a 125ms shift. Person-specific convolutional VAEs are trained to learn latent spaces of varying dimensions while a dense neural network is trained to investigate their utility on a classification task. The empirical results suggest that when VAEs are deployed on spectral topographic maps with shape 32 <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">x</i> 32, deployed for 32 electrodes from 2 seconds cerebral activity, they were capable of reducing the input up to almost 99%, with a latent space of 28 means and standard deviations. This did not compromise the salient information, as confirmed by a structural similarity index, and mean squared error between the input and reconstructed maps. Additionally, along the 28 means maximized the utility of latent spaces in the classification task, with an average 0.93% accuracy. This study contributes to the body of knowledge by offering a pipeline for effective dimensionality reduction of EEG data by employing convolutional variational autoencoders.

Unsupervised classification of voltammetric data beyond principal component analysis.

In this study, we evaluate different apoproaches to unsupervised classification of cyclic voltammetric data, including Principal Component Analysis (PCA), t-distributed Stochastic Neighbour Embedding (t-SNE), Uniform Manifold Approximation and Projection (UMAP) as well as neural networks. To this end, we exploit a form of transfer learning, based on feature extraction in an image recognition network, VGG-16, in combination with PCA, t-SNE or UMAP. Overall, we find that t-SNE performs best when applied directly to numerical data (noise-free case) or to features (in the presence of noise), followed by UMAP and then PCA.

Wavelet-based estimation in a semiparametric regression model

In this paper, we introduce a wavelet-based method for estimating the effective dimension reduction (EDR) space in the semiparametric regression model introduced by Li [Sliced inverse regression for dimension reduction, J. Amer. Statist. Assoc. 86 (1991) 316–327]. This method is obtained by using linear wavelet estimators of the density and regression functions that are involved in the covariance matrix of conditional expectation whose eigenvectors span the EDR space. Then, consistency of the proposed estimators is proved. A simulation study that allows one to evaluate the performance of the proposal with comparison to existing methods is presented.

Riemannian proximal stochastic gradient descent for sparse 2DPCA

Sparse two-dimensional principal component analysis (2DPCA) is an effective dimensionality reduction and data processing technique, which has been successfully applied in face recognition. Sparse 2DPCA is based on 2D image matrices instead of 1D vectors so it can well exploit the structural information of image pixels. The sparse 2DPCA model is regarded as a non-convex and non-smooth composite optimization problem. The manifold proximal gradient algorithm (ManPG) that is efficient and robust can globally converge to an optimal point in solving this kind of problem on the Stiefel manifold, such as sparse PCA. ManPG uses the proximal gradient method for the non-smooth term of this problem, which yields a convex subproblem on the tangent space. By solving this subproblem, a descent direction on the tangent space is obtained. Then a new update point on the manifold is obtained via retraction. However, in solving large-scale problems, the computational complexity of ManPG is quite high since ManPG requires calculating the full gradient in each iteration. In order to reduce the computational complexity of each iteration, a more computationally efficient Riemannian proximal stochastic gradient descent algorithm (RPSGD) is proposed, which utilizes the stochastic gradient instead of the full gradient in ManPG. We prove that the iteration complexity of the proposed RPSGD is O(1/ϵ4) while it maintains the sublinear convergence for the large-scale sparse 2DPCA. The experimental results on face recognition and handwritten digits classification data sets show RPSGD has the advantages of small-time overhead, high test accuracy, good reconstructed image quality, and relatively sparse solution in solving the large-scale sparse 2DPCA.

A discriminant kernel entropy-based framework for feature representation learning

The intelligent multimedia processing community has developed increasing interest in kernel entropy component analysis (KECA) due to its abilities in effective data transformation and dimensionality reduction. However, since only the unsupervised structural information of Renyi entropy from the given data set is utilized, KECA alone is incapable of generating high quality discriminant features. Aiming to develop a new and generic approach for feature representation learning, this paper proposes a discriminant kernel entropy-based framework, which integrates KECA and a complete discriminant strategy (consisting of regular and irregular discriminant information), to explore discriminant feature representations from the given data set. The framework is realized and further optimized to generate a more powerful discriminant descriptor for feature representation learning, leading to improved performance. Since the joint utilization of kernel entropy estimation and the complete discriminant strategy is able to reveal the distribution and semantic information of the given data, the proposed framework opens up a new front for discriminant feature representation learning via information theoretic learning (ITL). To demonstrate the generic nature and effectiveness of the proposed framework, experiments are conducted on two different data sources; the visual data source (e.g., University of California Irvine (UCI) database, Olivetti Research Lab (ORL) database, Caltech 256 database) and the audio data source (Ryerson Multimedia Lab (RML) audio emotion database). The results show this framework yields superior performance over other methods on the data sets evaluated for feature representation learning.

Generalized Tensor Decomposition With Features on Multiple Modes

Higher-order tensors have received increased attention across science and engineering. While most tensor decomposition methods are developed for a single tensor observation, scientific studies often collect side information, in the form of node features and interactions thereof, together with the tensor data. Such data problems are common in neuroimaging, network analysis, and spatial-temporal modeling. Identifying the relationship between a high-dimensional tensor and side information is important yet challenging. Here, we develop a tensor decomposition method that incorporates multiple feature matrices as side information. Unlike unsupervised tensor decomposition, our supervised decomposition captures the effective dimension reduction of the data tensor confined to feature space of interest. An efficient alternating optimization algorithm with provable spectral initialization is further developed. Our proposal handles a broad range of data types, including continuous, count, and binary observations. We apply the method to diffusion tensor imaging data from human connectome project and multi-relational political network data. We identify the key global connectivity pattern and pinpoint the local regions that are associated with available features. The package and data used are available at https://CRAN.R-project.org/package=tensorregress. Supplementary materials for this article are available online.

Deep learning method for identifying the minimal representations and nonlinear mode decomposition of fluid flows

We propose a deep learning method to learn the minimal representations of fluid flows. It uses the deep variational autoencoder (VAE) to decouple the independent representations for fluid flows. We apply this method to several simple flows and show that the network successfully identifies the independent and interpretable representations. It shows that the proposed method can extract the physically suggestive information. We further employ the VAE network to improve the mode decomposing autoencoder framework. It decomposes the cylinder flow fields into two independent ordered states. The cylinder flow at different Reynolds numbers and time can be described as the composition of the two decomposed fields. The present results suggest that the proposed network can be used as an effective nonlinear dimensionality reduction tool for flow fields.

Fault Detection for Pitch System of Wind Turbine-Driven Doubly Fed Based on IHHO-LightGBM

To address the issue of a large calculation and difficult optimization for the traditional fault detection of a wind turbine-based pitch control system, a fault detection model, based on LightGBM by the improved Harris Hawks optimization algorithm (light gradient boosting machine by the improved Harris Hawks optimization, IHHO-LightGBM) for the wind turbine-based pitch control system, is proposed in this article. Firstly, a trigonometric function model is introduced by IHHO to update the prey escape energy, to balance the global exploration ability and local development ability of the algorithm. In this model, the fault detection false alarm rate is used as the fitness function, and the two parameters are used as the optimization objects of the improved Harris Hawks optimization algorithm, to optimize the parameters, so as to achieve the global optimal parameters to improve the performance of the fault detection model. Three different fault data of the pitch control system in actual operations of domestic wind farms are used as the experimental data, the Pearson correlation analysis method is introduced, and the wind turbine power output is taken as the main state parameter, to analyze the correlation degree of all the characteristic variables of the data and screen the important characteristic variables out, so as to achieve the effective dimensionality reduction process of the data, by using the feature selection method. Three established fault detection models are selected and compared with the proposed method, to verify its feasibility. The experimental data indicate that compared with other algorithms, the fault detecting ability of the proposed model is improved in all aspects, and the false alarm rate and false negative rate are lower.

Dimensionality reduction and classification for hyperspectral image based on robust supervised ISOMAP

ABSTRACT This study contributes to classification and recognition of hyperspectral image in industrial sorting, mineral identification, geological exploration, environmental survey, and other industrial production fields. The finding eliminates the redundancy of hyperspectral data and improves the classification accuracy of hyperspectral image. Hyperspectral remote sensing data are inherently nonlinear, and traditional linear dimensionality reduction methods cannot effectively reveal the nonlinear structure contained in the data. Manifold learning is a nonlinear dimensionality reduction method which has been developed rapidly in recent years. The intrinsic structure of the data is preserved by Isometric Feature Mapping (ISOMAP), but it is sensitive to noise and is not conducive to final classification. Prior studies neglected a higher dimensional manifold real structure in hyperspectral data dimensionality reduction due to lack of consideration credibility information and class information of samples in calculating manifold distance. This study proposes a Robust Supervised ISOMAP method. Data sample credibility mode is constructed, and a data similarity measure named triple geodesic distance is defined by introducing samples’ credibility information, class information, and neighborhood information. Triple geodesic distance fitting is proven. Multidimensional scaling analysis (MDS) and generalized regression neural network are used to construct the embedded coordinates of training samples and test samples. The results show that the proposed method has higher anti-noise performance than the traditional one. Also, the runtime is feasible, and can effectively improve the classification precision of the hyperspectral images, in comparison to traditional ISOMAP methods. The conclusion is that Robust Supervised ISOMAP is an effective nonlinear dimensionality reduction method for hyperspectral remote sensing image.

Effective dimension reduction with mode transformations: Simulating two-dimensional fermionic condensed matter systems with matrix-product states

Tensor network methods have progressed from variational techniques based on matrix-product states able to compute properties of one-dimensional condensed-matter lattice models into methods rooted in more elaborate states such as projected entangled pair states aimed at simulating the physics of two-dimensional models. In this work, we advocate the paradigm that for two-dimensional fermionic models, matrix-product states are still applicable to significantly higher accuracy levels than direct embeddings into one-dimensional systems allow for. To do so, we exploit schemes of fermionic mode transformations and overcome the prejudice that one-dimensional embeddings need to be local. This approach takes the insight seriously that the suitable exploitation of both the manifold of matrix-product states and the unitary manifold of mode transformations can more accurately capture the natural correlation structure. By demonstrating the residual low levels of entanglement in emerging modes, we show that matrix-product states can describe ground states strikingly well. The power of the approach is exemplified by investigating a phase transition of spin-less fermions for lattice sizes up to 10x10.

Variable Selection for the Spatial Autoregressive Model with Autoregressive Disturbances

Along with the rapid development of the geographic information system, high-dimensional spatial heterogeneous data has emerged bringing theoretical and computational challenges to statistical modeling and analysis. As a result, effective dimensionality reduction and spatial effect recognition has become very important. This paper focuses on variable selection in the spatial autoregressive model with autoregressive disturbances (SARAR) which contains a more comprehensive spatial effect. The variable selection procedure is presented by using the so-called penalized quasi-likelihood approach. Under suitable regular conditions, we obtain the rate of convergence and the asymptotic normality of the estimators. The theoretical results ensure that the proposed method can effectively identify spatial effects of dependent variables, find spatial heterogeneity in error terms, reduce the dimension, and estimate unknown parameters simultaneously. Based on step-by-step transformation, a feasible iterative algorithm is developed to realize spatial effect identification, variable selection, and parameter estimation. In the setting of finite samples, Monte Carlo studies and real data analysis demonstrate that the proposed penalized method performs well and is consistent with the theoretical results.

An Autoencoder-Based Deep Learning Approach for Load Identification in Structural Dynamics.

In civil engineering, different machine learning algorithms have been adopted to process the huge amount of data continuously acquired through sensor networks and solve inverse problems. Challenging issues linked to structural health monitoring or load identification are currently related to big data, consisting of structural vibration recordings shaped as a multivariate time series. Any algorithm should therefore allow an effective dimensionality reduction, retaining the informative content of data and inferring correlations within and across the time series. Within this framework, we propose a time series AutoEncoder (AE) employing inception modules and residual learning for the encoding and the decoding parts, and an extremely reduced latent representation specifically tailored to tackle load identification tasks. We discuss the choice of the dimensionality of this latent representation, considering the sources of variability in the recordings and the inverse-forward nature of the AE. To help setting the aforementioned dimensionality, the false nearest neighbor heuristics is also exploited. The reported numerical results, related to shear buildings excited by dynamic loadings, highlight the signal reconstruction capacity of the proposed AE, and the capability to accomplish the load identification task.

Dimension reduction in spatial regression with kernel SAVE method

We consider the smoothed version of sliced average variance estimation (SAVE) dimension reduction method for dealing with spatially dependent data that are observations of a strongly mixing random field. We propose kernel estimators for the interest matrix and the effective dimension reduction (EDR) space, and show their consistency.

Classification of SSVEP-based BCIs using Genetic Algorithm

Brain computer interface (BCI) systems have been regarded as a new way of communication for humans. In this research, common methods such as wavelet transform are applied in order to extract features. However, genetic algorithm (GA), as an evolutionary method, is used to select features. Finally, classification was done using the two approaches support vector machine (SVM) and Bayesian method. Five features were selected and the accuracy of Bayesian classification was measured to be 80% with dimension reduction. Ultimately, the classification accuracy reached 90.4% using SVM classifier. The results of the study indicate a better feature selection and the effective dimension reduction of these features, as well as a higher percentage of classification accuracy in comparison with other studies.