Eigendecomposition Of Matrix Research Articles

Classical vs. neural network-based PCA approaches for lossy image compression: Similarities and differences

The paper describes three lossy data compression techniques based on the principal component analysis (PCA), which are compared using the image compression task. The presented approach uses both classical PCA method based on the eigen-decomposition of the image data covariance matrix and two different neural network structures. The first neural structure is a two-layer feed-forward network with supervised learning acting as the so-called autoencoder, and the second one is a single-layered network with an unsupervised learning method commonly known as the generalized Hebbian algorithm. For each compression method mentioned above, the influence of the number of image segments (frames) and the number of eigenvalues/eigenvectors on the compression ratio and the image quality are examined using three different gray-scale test images. The paper also addresses a vital implementation issue regarding selecting appropriate data types to represent the compressed data. The paper’s main conclusion is that the classical PCA method outperforms its neural counterparts, both in terms of the decompressed image quality and the time required to perform the compression procedure. The positive aspect of using neural networks as a tool for PCA-based lossy data compression is that they do not require calculating the correlation matrix explicitly and thus can be used in online data acquisition schemes.

General spatial photonic Ising machine based on the interaction matrix eigendecomposition method

The spatial photonic Ising machine has achieved remarkable advancements in solving combinatorial optimization problems. However, it still remains a huge challenge to flexibly map an arbitrary problem to the Ising model. In this paper, we propose a general spatial photonic Ising machine based on the interaction matrix eigendecomposition method. The arbitrary interaction matrix can be configured in the two-dimensional Fourier transformation based spatial photonic Ising model by using values generated by matrix eigendecomposition. The error in the structural representation of the Hamiltonian decreases substantially with the growing number of eigenvalues utilized to form the Ising machine. In combination with the optimization algorithm, as low as ∼65% of the eigenvalues are required by intensity modulation to guarantee the best probability of optimal solution for a 20-vertex graph Max-cut problem, and this percentage decreases to below ∼20% for near-zero probability. The 4-spin experiments and error analysis demonstrate the Hamiltonian linear mapping and ergodic optimization. Our work provides a viable approach for spatial photonic Ising machines to solve arbitrary combinatorial optimization problems with the help of the multi-dimensional optical property.

Comments on “Outsourcing Eigen-Decomposition and Singular Value Decomposition of Large Matrix to a Public Cloud”

Comments on “Outsourcing Eigen-Decomposition and Singular Value Decomposition of Large Matrix to a Public Cloud”

Effect of detector placement on joint estimation in X-ray fluorescence emission tomography.

Imaging the spatial distribution of low concentrations of metal is a growing problem of interest with applications in medical and material sciences. X-ray fluorescence emission tomography (XFET) is an emerging metal mapping imaging modality with potential sensitivity improvements and practical advantages over other methods. However, XFET detector placement must first be optimized to ensure accurate metal density quantification and adequate spatial resolution. In this work, we first use singular value decomposition of the imaging model and eigendecomposition of the object-specific Fisher information matrix to study how detector arrangement affects spatial resolution and feature preservation. We then perform joint image reconstructions of a numerical gold phantom. For this phantom, we show that two parallel detectors provide metal quantification with similar accuracy to four detectors, despite the resulting anisotropic spatial resolution in the attenuation map estimate. Two orthogonal detectors provide improved spatial resolution along one axis, but underestimate the metal concentration in distant regions. Therefore, this work demonstrates the minor effect of using fewer, but strategically placed, detectors in the case where detector placement is restricted. This work is a critical investigation into the limitations and capabilities of XFET prior to its translation to preclinical and benchtop uses.

Handling Missing Data via Max-Entropy Regularized Graph Autoencoder

Graph neural networks (GNNs) are popular weapons for modeling relational data. Existing GNNs are not specified for attribute-incomplete graphs, making missing attribute imputation a burning issue. Until recently, many works notice that GNNs are coupled with spectral concentration, which means the spectrum obtained by GNNs concentrates on a local part in spectral domain, e.g., low-frequency due to oversmoothing issue. As a consequence, GNNs may be seriously flawed for reconstructing graph attributes as graph spectral concentration tends to cause a low imputation precision. In this work, we present a regularized graph autoencoder for graph attribute imputation, named MEGAE, which aims at mitigating spectral concentration problem by maximizing the graph spectral entropy. Notably, we first present the method for estimating graph spectral entropy without the eigen-decomposition of Laplacian matrix and provide the theoretical upper error bound. A maximum entropy regularization then acts in the latent space, which directly increases the graph spectral entropy. Extensive experiments show that MEGAE outperforms all the other state-of-the-art imputation methods on a variety of benchmark datasets.

Dimension decomposition algorithm for multiple source localization using uniform circular array

A dimension decomposition (DIDE) method for multiple incoherent source localization using uniform circular array (UCA) is proposed. Due to the fact that the far-field signal can be considered as the state where the range parameter of the near-field signal is infinite, the algorithm for the near-field source localization is also suitable for estimating the direction of arrival (DOA) of far-field signals. By decomposing the first and second exponent term of the steering vector, the three-dimensional (3-D) parameter is transformed into two-dimensional (2-D) and one-dimensional (1-D) parameter estimation. First, by partitioning the received data, we exploit propagator to acquire the noise subspace. Next, the objective function is established and partial derivative is applied to acquire the spatial spectrum of 2-D DOA. At last, the estimated 2-D DOA is utilized to calculate the phase of the decomposed vector, and the least squares (LS) is performed to acquire the range parameters. In comparison to the existing algorithms, the proposed DIDE algorithm requires neither the eigendecomposition of covariance matrix nor the search process of range spatial spectrum, which can achieve satisfactory localization and reduce computational complexity. Simulations are implemented to illustrate the advantages of the proposed DIDE method. Moreover, simulations demonstrate that the proposed DIDE method can also classify the mixed far-field and near-field signals.

Tensorized Bipartite Graph Learning for Multi-View Clustering.

Despite the impressive clustering performance and efficiency in characterizing both the relationship between the data and cluster structure, most existing graph-based multi-view clustering methods still have the following drawbacks. They suffer from the expensive time burden due to both the construction of graphs and eigen-decomposition of Laplacian matrix. Moreover, none of them simultaneously considers the similarity of inter-view and similarity of intra-view. In this article, we propose a variance-based de-correlation anchor selection strategy for bipartite construction. The selected anchors not only cover the whole classes but also characterize the intrinsic structure of data. Following that, we present a tensorized bipartite graph learning for multi-view clustering (TBGL). Specifically, TBGL exploits the similarity of inter-view by minimizing the tensor Schatten p-norm, which well exploits both the spatial structure and complementary information embedded in the bipartite graphs of views. We exploit the similarity of intra-view by using the [Formula: see text]-norm minimization regularization and connectivity constraint on each bipartite graph. So the learned graph not only well encodes discriminative information but also has the exact connected components which directly indicates the clusters of data. Moreover, we solve TBGL by an efficient algorithm which is time-economical and has good convergence. Extensive experimental results demonstrate that TBGL is superior to the state-of-the-art methods. Codes and datasets are available: https://github.com/xdweixia/TBGL-MVC.

Generated Image Editing Method Based on Global-Local Jacobi Disentanglement for Machine Learning.

Accurate semantic editing of the generated images is extremely important for machine learning and sample enhancement of big data. Aiming at the problem of semantic entanglement in generated image latent space of the StyleGAN2 network, we proposed a generated image editing method based on global-local Jacobi disentanglement. In terms of global disentanglement, we extract the weight matrix of the style layer in the pre-trained StyleGAN2 network; obtain the semantic attribute direction vector by using the weight matrix eigen decomposition method; finally, utilize this direction vector as the initialization vector for the Jacobi orthogonal regularization search algorithm. Our method improves the speed of the Jacobi orthogonal regularization search algorithm with the proportion of effective semantic attribute editing directions. In terms of local disentanglement, we design a local contrast regularized loss function to relax the semantic association local area and non-local area and utilize the Jacobi orthogonal regularization search algorithm to obtain a more accurate semantic attribute editing direction based on the local area prior MASK. The experimental results show that the proposed method achieves SOTA in semantic attribute disentangled metrics and can discover more accurate editing directions compared with the mainstream unsupervised generated image editing methods.

Numerical Tolerance for Spectral Decompositions of Random Matrices and Applications to Network Inference

We precisely quantify the impact of statistical error in the quality of a numerical approximation to a random matrix eigendecomposition, and under mild conditions, we use this to introduce an optimal numerical tolerance for residual error in spectral decompositions of random matrices. We demonstrate that terminating an eigendecomposition algorithm when the numerical error and statistical error are of the same order results in computational savings with no loss of accuracy. We illustrate the practical consequences of our stopping criterion with an analysis of simulated and real networks. Our theoretical results and real-data examples establish that the tradeoff between statistical and numerical error is of significant importance for subsequent inference.

Phase fMRI defines brain resting-state functional hubs within central and posterior regions.

From a brain functional connectivity (FC) matrix, we can identify the hub nodes by a new method of eigencentrality mapping, which not only counts for one node's centrality but also all other nodes' centrality values through correlation connections in an eigenvector of the FC matrix. For the resting-state functional MRI (fMRI) data (complex-valued EPI images in nature), both magnitude and phase images are useful for brain FC analysis. We herein report on brain functional hubness analysis by constructing the FC matrix from phase fMRI data and identifying the hub nodes by eigencentrality mapping. In our study, we collected a cohort of 160 complex-valued fMRI dataset (consisting of magnitude and phase in pairs), and performed independent component analysis (ICA), FC matrix calculation (in size of 50 × 50) and FC matrix eigen decomposition; thereby obtained the 50-node eigencentrality values in the eigenvector associated with the largest eigenvalue. We also compared the hub structures inferred from FC matrices under different thresholding. Alternatively, we obtained the geometric hubs among p value the 50 nodes involved in the FC matrix through the use of harmonic centrality metric. Our results showed that phase fMRI data analysis defines the resting-state brain functional hubs primarily in the central region (subcortex) and the posterior region (parieto-occipital lobes and cerebella). The brain central hubness was supported by the geometric central hubness, which, however, is distinct from the magnitude-inferred hubness in brain superior region (frontal and parietal lobes). Our findings pose a new understanding of (or a debate over) brain functional connectivity architecture.

An Improved Method for Phase Triangulation Algorithm Based on the Coherence Matrix Eigen-Decomposition in Time-Series SAR Interferometry

Time-series SAR interferometry, which combines permanent scatterers (PSs) and distributed scatterers (DSs), has been strongly developed in recent years. Unlike PS, DS corresponds to a natural target whose neighboring pixels share similar reflectivity values. The selection of DS is relevant to the goodness-of-fit value, the estimation of which is based on all possible combined interferometric phases and fails to avoid the adverse effect of low-quality phases. This paper used eigen-decomposition of coherence matrix that was constructed based on the identified homogeneous pixels to perform phase optimization, and then only the interferometric phases with low noise and clear fringes are utilized to measure the goodness-of-fit. 30 Sentinel-1A images were applied to test the improved method of land subsidence monitoring in Beijing, China. The deformation results of different methods were cross verified and the area statistics of the study area were carried out. The results show that the maximum subsidence monitored by the improved method is located in Jinzhan Town with a maximum rate of −109 mm/yr. This improved method extracts more measurement points with accuracy ensured, which is proved to be an effective way to provide the high spatial density of deformation measurements. The land subsidence in this area is mainly caused by the excessive exploitation of groundwater resources. This research provides support for the prevention and control work of relevant departments.

Spectral analysis modal methods (SAMMs) using non-time-resolved PIV

We present spectral analysis modal methods (SAMMs) to perform POD in the frequency domain using non-time-resolved particle image velocity (PIV) data combined with unsteady surface pressure measurements. In particular, time-resolved unsteady surface pressure measurements are synchronized with non-time-resolved planar PIV measurements acquired at 15 Hz in a Mach 0.6 cavity flow. Leveraging the spectral linear stochastic estimation (LSE) method of Tinney et al. (Exp Fluids 41:763–775, 2006), we first estimate the cross-correlations between the velocity field and the unsteady pressure sensors via sequential time shifts, followed by a Fast Fourier transform to obtain the pressure–velocity cross spectral density matrix. This leads to a linear multiple-input/multiple-output (MIMO) model that determines the optimal transfer functions between the input cavity wall pressure and the output velocity field. Two variants of SAMMs are developed and applied. The first, termed “SAMM-SPOD”, combines the MIMO model with the SPOD algorithm of Towne et al. (J Fluid Mech, https://doi.org/10.1017/jfm.2018.283, 2018). The second, called “SAMM-RR”, adds independent sources and uses a sorted eigendecomposition of the input pressure cross-spectral matrix to enable an efficient reduced-rank eigendecomposition of the velocity cross-spectral matrix. In both cases, the resulting rank-1 POD eigenvalues associated with the Rossiter frequencies exhibit very good agreement with those obtained using independent time-resolved PIV measurements. The results demonstrate that SAMMs provide a methodology to perform space-time POD without requiring a high-speed PIV system, while avoiding potential pitfalls associated with traditional time-domain LSE.

Intelligent Time-Scale Operator-Splitting Integration for Chemical Reaction Systems.

The wide range of time scales in chemical reaction systems has become an important problem in reactive flow simulations. This work proposes an intelligent time-scale operator-splitting (OS) chemistry integration method, which is effective in reduction of numerical stiffness and model complexity. Different from most existing publications, a pretrained backpropagation neural network is used to identify the slow and fast reactions and detect the sources of model stiffness on the fly, which replaces the expensive eigendecomposition of Jacobian matrix. With the fast-slow decomposition, the chemical source term can be represented as the sum of a stiff part and a nonstiff part. A stable time-scale OS integration is performed to solve the stiff chemical ordinary differential equations, which balances the computational cost with accuracy. In the simulation, a favorable comparison of the proposed integration method with the existing ODE solvers, such as implicit Euler, explicit Euler, and Runge-Kutta, is included to show its effectiveness and merits.

Incremental learning of an abnormal behavior detection algorithm based on principal components

In this paper, we propose an incremental learning scheme for the abnormal behavior detection algorithm based on principal component. The results obtained on a UCSD dataset and our experimental videos at a different number of training samples show that error rates are similar to conventional learning. Moreover, the proposed scheme allows the incremental learning time to be significantly reduced in comparison with a method based on matrix eigendecomposition.

Direction of arrival estimation with antenna arrays based on fuzzy cerebellar model articulation controller neural network

Direction of arrival (DOA) estimation has been a challenging problem in many applications such as wireless communication, radar, sonar, and navigation. However, it is difficult to improve the angle resolution and reduce the computational complexity of super-resolution methods. To solve these problems, the DOA estimation is viewed as a mapping problem, which can be modeled using a suitable artificial neural network trained with input-output pairs. This article presents the use of a fuzzy cerebellar model articulation controller (FCMAC) neural network for the DOA estimation under a linear antenna array. The FCMAC neural network is a special feedforward neural network based on local approximation that can be adapted to solve the multidimensional nonlinear fitting problem. A new preprocessing scheme has been used in both training and test phase. It use magnitude and phase angles instead of the real and imaginary parts of the array covariance matrix to be the input of neural network. The proposed method avoids complex matrix eigen-decomposition, such as multiple signal classification, and offers fast computation rate. The performance of FCMAC neural network is compared with the conventional subspace methods and the radial basis function neural network in the cases of noisy environment and coherent signal. Simulation results indicate that FCMAC neural network produces up to 61% lower error, 60% higher angle resolution, and 99% lower calculation time than other three methods, which indicates the superior performance of the proposed DOA estimation method under coherent signals and different noise levels.

Multilevel Strategies Improve History Matching of Complex Reservoir Models

This article, written by JPT Technology Editor Chris Carpenter, contains highlights of paper SPE 193895, “Multilevel Strategies and Geological Parameterizations for History Matching Complex Reservoir Models,” by Yimin Liu, SPE, and Louis J. Durlofsky, SPE, Stanford University, prepared for the 2019 SPE Reservoir Simulation Conference, Galveston, Texas, 10–11 April. The paper has not been peer reviewed. The complete paper explores the use of multilevel derivative-free optimization for history matching, with model properties described using principal component analysis (PCA) -based parameterization techniques. The parameterizations applied are optimization-based PCA (O-PCA) and convolutional-neural-network-based PCA (CNN-PCA). Mesh adaptive direct search (MADS), a pattern search method that parallelizes naturally, is used for the optimizations required to generate posterior models. The use of PCA-based parameterization reduces considerably the number of variables that must be determined during history matching, but the optimization problem can still be computationally demanding. The multilevel strategy addresses this issue by reducing the number of simulations that must be performed at each MADS iteration. History-matching results demonstrate that substantial uncertainty reduction is achieved in all cases considered and that the multilevel strategy is effective in reducing the number of simulations required. Overview Parameterization has two key advantages in history matching: Far fewer parameters must be determined, which simplifies greatly the minimization in some cases. Posterior (history-matched) geomodels will be consistent with the geological scenario or training image for which the parameterization was constructed. PCA is the foundation for many geological-dimension-reduction parameterization methods. PCA is based on the eigen-decomposition of the prior geomodel covariance matrix and, thus, honors only two-point spatial statistics. History-matching algorithms largely can be classified as either optimization-based or ensemble-based procedures. Optimization-based algorithms generate one history-matched model at a time, while ensemble-based algorithms update a large set of models simultaneously. The resulting ensemble of posterior models is used to quantify uncertainty. In this study, only optimization-based history matching is considered, though the parameterizations applied also can be used with ensemble-based methods. Optimization-based history-matching algorithms can be classified further into gradient-based and derivative-free optimization (DFO) methods. Gradient-based algorithms often are combined with adjoint procedures that provide the required gradient efficiently. Unlike adjoint-based procedures, DFO algorithms (which are considered in this work) are nonintrusive. They treat the reservoir simulator as a so-called “black box,” and, as a result, they are much easier to implement and can be applied with any simulator. In this study, MADS, a pattern-search DFO, is applied for history matching. In pattern-search (or stencil-based) algorithms, the number of simulations required at each iteration scales with the number of parameters to be determined. Because the latter typically is much smaller than the number of grid-blocks in the model, geological parameterization leads to large computational savings when a pattern search, or other DFO, is used for history matching. The performance of DFO for history matching with non-Gaussian models, a primary topic in the complete paper, has not been studied extensively. The authors devise a multilevel strategy, applicable for PCA-based parameterization methods, to further reduce the computational cost of history matching using DFO. A multilevel strategy lends itself naturally to PCA-based parameterizations because the PCA representation can be ordered from the most-important to the least-important principal component. Thus, PCA parameters can be determined sequentially, level by level, in groups, rather than all at once.

PrivateGraph: Privacy-Preserving Spectral Analysis of Encrypted Graphs in the Cloud

Big graphs, such as user interactions in social networks and customer rating matrices in collaborative filters, possess great values for both businesses and research. They are not only big but often keep evolving, which requires a large amount of computing resources to maintain. With the wide deployment of public cloud resources, owners of big graphs may want to use cloud resources to obtain storage and computation scalability. However, privacy and ownership of the graphs in the cloud has become a major concern. In this paper, we study privacy-preserving algorithms for one of the important graph analysis techniques—graph spectral analysis for outsourced graph in the cloud. The core operation: eigendecomposition of large matrix is also important to many data mining algorithms. We consider a cloud-centric framework with three collaborative parties: data contributors, data owner, and cloud provider. Graphs are represented as matrices such as adjacency matrix and Laplacian matrix, the elements of which are encrypted and submitted by distributed contributors. The data owner then interacts with the cloud-side programs to conduct spectral analysis, while protecting data privacy from the honest-but-curious cloud provider. For a $N\times N$N×N graph matrix, we aim to design algorithms with the cloud handling expensive storage and computation in $O(N^2)$O(N2) complexity, while data owner and data contributors’ algorithms take only $O(N)$O(N). To achieve this goal, we develop the privacy-preserving versions of the two approximate eigendecomposition algorithms: the Lanczos algorithm and the Nystrom algorithm, considering two encryption methods: additive homomorphic encryption (AHE) methods and somewhat homomorphic encryption (SHE) methods. Both dense and sparse matrices are studied, while sparse matrices also involve a differentially private data submission protocol to allow the trade-off between data sparsity and privacy. Experimental results show that the Nystro algorithm with sparse encoding can dramatically reduce data owners’ costs. SHE-based methods have lower computational time while AHE-based methods have lower communication/storage costs.

An autoencoder-based spectral clustering algorithm

Spectral clustering algorithm suffers from high computational complexity due to the eigen decomposition of Laplacian matrix and large similarity matrix for large-scale datasets. Some researches explore the possibility of deep learning in spectral clustering and propose to replace the eigen decomposition with autoencoder. K-means clustering is generally used to obtain clustering results on the embedding representation, which can improve efficiency but further increase memory consumption. An efficient spectral algorithm based on stacked autoencoder is proposed to solve this issue. In this paper, we select the representative data points as landmarks and use the similarity of landmarks with all data points as the input of autoencoder instead of similarity matrix of the whole datasets. To further refine clustering result, we combine learning the embedding representation and performing clustering. Clustering loss is used to update the parameters of autoencoder and cluster centers simultaneously. The reconstruction loss is also included to prevent the distortion of embedding space and preserve the local structure of data. Experiments on several large-scale datasets validate the effectiveness of the proposed method.

Incremental commute time and its online applications

Commute time is a robust measure on graphs based on random walks. It has been successfully applied in many application domains including personalized search, collaborative filtering and network intrusion detection. However, the computation of the commute time is expensive since it involves the eigen decomposition of the graph Laplacian matrix. There has been effort to approximate the commute time but they only work in an offline mode. In this work, an accurate and efficient approximation for computing the commute time is proposed in an incremental fashion in order to facilitate online applications. Using the incremental commutime, we design an online anomaly detection application where the commute time of each new arriving data point to any point in the current graph can be estimated in constant time. The proposed approach shows its high accuracy and efficiency in synthetic and real datasets for online applications. It takes only 8 milliseconds on average to detect anomalies online on the DBLP graph which has more than 600,000 nodes and 2 millions edges. We also discuss the use of incremental commute time for other online applications such as classification, graph ranking and clustering.

Weighted structure preservation and redundancy minimization for feature selection

The recent literature indicates that structure preserving is of great importance for feature selection and many existing selection criteria essentially work in this way. In this paper, we argue that the Eigen value decomposition of global pair wise similarity matrix should be weighted, and the redundancy among the features should be minimized. In order to show this, we propose a weighted structure preservation and features redundancy minimization framework for feature selection. In this framework, the Eigen vector obtained by the Eigen decomposition of global pair wise similarity matrix is weighted by the corresponding Eigen value, and the cosine distance between two features together with the L2,1 norm of these two features are used to evaluate the degree of redundancy between these two features. A comprehensive experimental study is then conducted in order to compare our feature selection algorithms with many state-of-the art ones in supervised learning scenarios. The conducted experiments validate the effectiveness of our feature selection.