Computation Of Eigenvectors Research Articles

Solution of nonlinear eigenvalues for the viscoelastic damped cylindrical shell considering the frequency dependence of viscoelastic materials

The dynamic model of the cylindrical shell partially attached with a viscoelastic free damping layer is established by a semi-analytical method, and the complex boundary conditions of the cylindrical shell are simulated by using the non-uniform distribution of spring groups. Two new methods for solving nonlinear eigenvalue problems of the viscoelastic damped cylindrical shell are proposed. One method is named as the Vector Iteration method based on Approximate Eigenvalues (VIAE), and the solution process can be described as: the iteration equation is obtained by the characteristic equation; the approximate eigenvalue and the initial eigenvector are used for the iteration calculation; in each iteration, the eigenvectors are normalized and the nonlinear eigenvalues are obtained by using the Rayleigh quotient. Another method is named as the Secant Method of Characteristic Polynomials (SMCP). In this method, the iteration equation of the eigenvalue is derived by the secant method. Furthermore, the approximate eigenvalues of the system are used as the initial iteration eigenvalues and the nonlinear eigenvalues of the system can be approximated through several iteration calculations. Finally, the proposed modeling method and algorithm are verified by the experiment of the actual cylindrical shell attached the ZN-1 viscoelastic material at the free end. The two new methods were also compared with the classical Eigenvector Increment Method (EIM). The comparison results show that the VIAE algorithm has the highest efficiency, and both the VIAE and SMCP algorithms have wider applicability because they do not need to perform eigenvector calculations.

Detection of the Number of Signals in Uniform Arrays by Invariant-Signal-Subspace Matching

We present a novel and computationally simple solution to the problem of detecting the number of signals in the case of uniform linear arrays (ULA) and uniform rectangular arrays (URA), which is applicable to white and colored noise, and to a very small number of samples. The solution is based on novel and non-asymptotic goodness-of-fit metric, referred invariant signal subspace matching (ISSM), which is aimed at matching the signal subspaces of two subarrays which are translation invariant. We form a pair of projection matrices on the signal subspaces – one for each subarray – which are parameterized by the number of signals <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$k$</tex-math></inline-formula> and constructed from the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$k$</tex-math></inline-formula> leading eigenvectors of the sample-covariance matrices of the subarrays. The value of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$k$</tex-math></inline-formula> which minimizes this metric is selected as the number of signals. We prove the consistency of the ISSM criterion for the high signal-to-noise-ratio (SNR) limit, and also for the large-sample limit, conditioned on the noise being white. The evaluation of the ISSM criterion involves only the computation of eigenvectors of the sample-covariance matrix of the the array. Simulation results, demonstrating the improved performance of the ISSM criterion over existing criteria, especially for colored noise, are included.

Canonical Polyadic Decomposition via the Generalized Schur Decomposition

The canonical decomposition (CPD) is a fundamental tensor decomposition which expresses a tensor as a sum of rank one tensors. In stark contrast to the matrix case, with light assumptions, the CPD of a low rank tensor is (essentially) unique. The essential uniqueness of CPD makes this decomposition a powerful tool in many applications as it allows for extraction of component information from a signal of interest. One popular algorithm for algebraic computation of a CPD is the generalized eigenvalue decomposition (GEVD) which selects a matrix subpencil of a tensor, then computes the generalized eigenvectors of the pencil. In this article, we present a simplification of GEVD which improves the accuracy of the algorithm. Surprisingly, the generalized eigenvector computation in GEVD is in fact unnecessary and can be replaced by a QZ decomposition which factors a pair of matrices as a product of unitary and upper triangular matrices. Computing a QZ decomposition is a standard first step when computing generalized eigenvectors, so our algorithm can been seen as a direct simplification of GEVD.

How well-conditioned can the eigenvector problem be?

The condition number for eigenvector computations is a well-studied quantity. But how small can it possibly be? Specifically, what matrices are perfectly conditioned with respect to eigenvector computations? In this note we answer this question for n × n n \times n matrices, giving a solution that is exact to first-order as n → ∞ n \rightarrow \infty .

Lanczos Algorithm for 2DPCA

The traditional PCA algorithm (Principle Component Analysis) can obtain the feature space of face image and realize face recognition by expanding the face image matrix into vectors in face recognition. 2DPCA (Two-dimensional Principle Component Analysis) doesn’t need to spread the image matrix into one-dimensional vectors. The covariance matrix is constructed directly by using two-dimensional image matrix, so that the calculation of eigenvalue and eigenvector is simplified. We apply the Lanczos algorithm to 2 DPCA in this paper for Indian faces. The numerical experiments show that our method runs much faster and gets better recognition rates than the traditional 2DPCA algorithm.

Redundancy and Analog Slicing for Precise In-Memory Machine Learning—Part II: Applications and Benchmark

In-memory computing (IMC) is attracting interest for accelerating data-intensive computing tasks, such as artificial intelligence (AI), machine learning (ML), and scientific calculus. IMC is typically conducted in the analog domain in crosspoint arrays of resistive random access memory (RRAM) devices or memristors. However, the precision of analog operations can be hindered by various sources of noise, such as the nonlinearity of the circuit components and the programming variations due to stuck devices and stochastic switching. Here we demonstrate high-precision IMC by a custom program-verify algorithm that uses redundancy to limit the impact of stuck devices and analog slicing to encode the analog programming error in a separate memory cell. The PageRank problem, consisting of the calculation of the principal eigenvector, is shown as a reference problem, adopting a fully integrated RRAM circuit. We extend these results to also include a convolutional neural network (CNN). We demonstrate a computing accuracy of 6.7 equivalent number of bits (ENOBs). Finally, we compare our results to the solution of the same problem by a static random access memory (SRAM)-based IMC, showcasing an advantage for the RRAM implementation in terms of energy efficiency and scaling.

An efficient Nyström spectral clustering algorithm using incomplete Cholesky decomposition

Nyström method can estimate the eigenvectors of a large kernel matrix with the eigenvectors of a small sampled sub-matrix. However, we may encounter two problems when using Nyström method to speed up spectral clustering: one problem is the approximate eigenvectors generated by standard Nyström method are sub-optimal, so they may impair the performance of spectral clustering; another one is the accurate Nyström approximation needs a sufficient number of samples, which will increase the eigen-decomposition cost on the sampled sub-matrix. To solve these problems, this paper proposes an efficient Nyström spectral clustering algorithm using incomplete Cholesky decomposition, in which a new matrix factorization strategy is designed for Nyström spectral clustering to meet the orthogonal constraints, and an efficient eigensolver based on incomplete Cholesky decomposition is developed to accelerate the Nyström approximation. In this way, the obtained approximate orthogonal eigenvectors will help to improve the clustering quality, and the developed eigenvector calculation method will help to reduce the clustering complexity. The experimental results show that the proposed algorithm performs well on many challenging data sets, and it can accomplish more complex clustering tasks with limited computing resources.

Approximate spectral clustering using both reference vectors and topology of the network generated by growing neural gas.

Spectral clustering (SC) is one of the most popular clustering methods and often outperforms traditional clustering methods. SC uses the eigenvectors of a Laplacian matrix calculated from a similarity matrix of a dataset. SC has serious drawbacks: the significant increases in the time complexity derived from the computation of eigenvectors and the memory space complexity to store the similarity matrix. To address the issues, I develop a new approximate spectral clustering using the network generated by growing neural gas (GNG), called ASC with GNG in this study. ASC with GNG uses not only reference vectors for vector quantization but also the topology of the network for extraction of the topological relationship between data points in a dataset. ASC with GNG calculates the similarity matrix from both the reference vectors and the topology of the network generated by GNG. Using the network generated from a dataset by GNG, ASC with GNG achieves to reduce the computational and space complexities and improve clustering quality. In this study, I demonstrate that ASC with GNG effectively reduces the computational time. Moreover, this study shows that ASC with GNG provides equal to or better clustering performance than SC.

Random matrix eigenvalue problems in structural dynamics: An iterative approach

Uncertainties need to be taken into account in the dynamic analysis of complex structures. This is because in some cases uncertainties can have a significant impact on the dynamic response and ignoring it can lead to unsafe design. For complex systems with uncertainties, the dynamic response is characterised by the eigenvalues and eigenvectors of the underlying generalised matrix eigenvalue problem. This paper aims at developing computationally efficient methods for random eigenvalue problems arising in the dynamics of multi-degree-of-freedom systems. There are efficient methods available in the literature for obtaining eigenvalues of random dynamical systems. However, the computation of eigenvectors remains challenging due to the presence of a large number of random variables within a single eigenvector. To address this problem, we project the random eigenvectors on the basis spanned by the underlying deterministic eigenvectors and apply a Galerkin formulation to obtain the unknown coefficients. The overall approach is simplified using an iterative technique. Two numerical examples are provided to illustrate the proposed method. Full-scale Monte Carlo simulations are used to validate the new results.

STR: Secure Computation on Additive Shares Using the Share-Transform-Reveal Strategy

The rapid development of cloud computing has probably benefited each of us. However, the privacy risks brought by untrustworthy cloud servers arise the attention of more and more people and legislatures. In the last two decades, plenty of works seek to outsource various specific tasks while ensuring the security of private data. The tasks to be outsourced are countless; however, the computations involved are similar. In this paper, we construct a series of novel protocols that support the secure computation of various functions on numbers (e.g., the basic elementary functions) and matrices (e.g., the calculation of eigenvectors and eigenvalues) in arbitrary $n\geq 2$ servers. All protocols only require constant rounds of interactions and achieve the low computation complexity. Moreover, the proposed $n$-party protocols ensure the security of private data even though $n-1$ servers collude. The convolutional neural network models are utilized as the case studies to verify the protocols. The theoretical analysis and experimental results demonstrate the correctness, efficiency, and security of the proposed protocols.

Finding the resistance distance and eigenvector centrality from the network’s eigenvalues

There are different measures to classify a network’s data set that, depending on the problem, have different success rates. For example, the resistance distance and eigenvector centrality measures have been successful in revealing ecological pathways and differentiating between biomedical images of patients with Alzheimer’s disease, respectively. The resistance distance measures an effective distance between two nodes of a network taking into account all possible shortest paths between them and the eigenvector centrality measures the relative importance of each node in a network. However, both measures require knowing the network’s eigenvalues and eigenvectors. Here, we show that we can closely approximate [find exactly] the resistance distance [eigenvector centrality] of a network only using its eigenvalue spectra, where we illustrate this by experimenting on resistor circuits, real neural networks (weighted and unweighted), and paradigmatic network models — scale-free, random, and small-world networks. Our results are supported by analytical derivations, which are based on the eigenvector–eigenvalue identity. Since the identity is unrestricted to the resistance distance or eigenvector centrality measures, it can be applied to most problems requiring the calculation of eigenvectors.

Efficient Asynchronous Semi-Stochastic Block Coordinate Descent Methods for Large-Scale SVD

Eigenvector computation such as Singular Value Decomposition (SVD) is one of the most fundamental problems in machine learning, optimization and numerical linear algebra. In recent years, many stochastic variance reduction algorithms and randomized coordinate descent algorithms have been developed to efficiently solve the leading eigenvalue problem. By taking full advantage of both variance reduction and randomized coordinate descent techniques, this paper proposes a novel Semi-stochastic Block Coordinate Descent algorithm (SBCD-SVD), which is more suitable than existing algorithms for large-scale leading eigenvalue problems of SVD, and can obtain linear convergence. Unlike existing stochastic variance reduction and randomized coordinate descent methods, our algorithm inherits their advantages. Moreover, we propose a new Asynchronous parallel Semi-stochastic Block Coordinate Descent algorithm (ASBCD-SVD) and one new Asynchronous parallel Sparse approximated Variance Reduction algorithm (ASVR-SVD) for large-scale dense and sparse datasets, respectively. Finally, we prove that both dense and sparse asynchronous parallel variants can converge linearly. Extensive experimental results show that our algorithms attain high parallel speedup and achieve almost the same performance with significantly shorter time, and thus they can be widely used in various practice applications.

USED EIGENVALUES AND EIGENVECTORS TO COMPRESS THE IMAGE

The article’s main aim is to point out the significant applications of the linear algebra in the medical engineering field. Hence, the eigenvectors and eigenvalues which represent the core of linear algebra are discussed in details in order to show how they can be used in many engineering applications. The principal components analysis is one of the most important compression and feature extraction algorithms used in the engineering field (1). It mainly depends on the calculation and extraction of eigenvalues and eigenvectors that then used to represent an input; whether it's the image or a simple matrix. In this article, the use of principal component analysis for medical image compression is an important and novel application of linear algebra (2).

Exact sampling of determinantal point processes without eigendecomposition

AbstractDeterminantal point processes (DPPs) enable the modeling of repulsion: they provide diverse sets of points. The repulsion is encoded in a kernelKthat can be seen, in a discrete setting, as a matrix storing the similarity between points. The main exact algorithm to sample DPPs uses the spectral decomposition ofK, a computation that becomes costly when dealing with a high number of points. Here we present an alternative exact algorithm to sample in discrete spaces that avoids the eigenvalues and the eigenvectors computation. The method used here is innovative, and numerical experiments show competitive results with respect to the initial algorithm.

(Invited) Regression and Optimization by Analogue In-memory Computing in Crosspoint Arrays

In-memory computing (IMC) is gaining momentum for low-power acceleration of data intensive problems, particularly in the frame of artificial intelligence (AI). Analogue matrix-vector multiplication (MVM) in crosspoint arrays allows to dramatically improve energy efficiency in neural networks for inference and training [1]. Recently, it has been shown that a feedback configuration of MVM can accelerate the solution of generic linear matrix problems, such as linear systems, matrix inversion and eigenvector calculation, with a single computational step [2,3]. This is achieved by the high parallelism of MVM in the memory array, combined with the physical iteration executed naturally in the feedback loop. Eigenvector extraction allows to address general problems, such as PageRank of internet webpages, with complexity O(1), i.e., the time to solve a ranking problem remains fixed irrespective of the size of the coefficient matrix [4]. In this work, we address two specific problems of AI acceleration, namely regression and optimization.Linear regression can be viewed as one of the most basic problems of machine learning: given a sequence of points, one must predict the value of the next point in the sequence. This problem can be solved in just one computational step by using the closed-loop crosspoint array of Fig. 1, where the independent variables x_i are physically written within the two crosspoint arrays, while the dependent variables y_i are applied externally as input currents. The crosspoint arrays are connected within a closed-loop configuration by using operational amplifiers (OAs) in the rows and the columns of the arrays. The OA output yields the coefficients of the best fitting line along the sequence, i.e., the coefficients a and b in the line equation y= ax + b providing the least mean squares (LMS) regression of the sequence. This problem can be extended to any arbitrary dimension, i.e., not only data on x-y plane, but also data on x-y-z space and any hyperspace of n dimensions. The problem can also be extended to logistic regression, which enables classification in one computational step. In particular, the crosspoint regression circuit allows the training of a 2-layer perceptron in just one step, by assuming random weights in the first layer according to the extreme learning machine (ELM) approach. In-memory computing thus appears as a promising option for accelerating neural network training with complexity O(1).The continuous-time, analogue closed-loop approach can be extended to a discrete-time, digital recurrent neural network (RNN) for addressing optimization problems [5]. The RNN allows the efficient solution of constraint satisfaction problems (CSPs), such as Sudoku, by taking advantage of analogue MVM in the crosspoint array. In the RNN, stochastic iterations can allow the minimization of the cost function, hence the number of violated constraints, thus leading to the global minimum. Noise can be injected as spike jitter by phase change memory devices as integrate-and-fire neurons. The convergence to the solution can be efficiently accelerated by a 2-layer RNN, where the first layer is programmed to prevent the violation of constraints in the CSP. In-memory solution of CSPs can dramatically improve the energy efficiency and speed of data intensive optimization in transport, industry, and society in general.

Molecular aggregation in liquid water: Laplace spectra and spectral clustering of H-bonded network

Molecular dynamics simulations of pure ambient liquid water were performed. Hydrogen bond network properties were determined by calculation of eigenvectors and eigenvalues of the Laplace matrix. We investigated how these quantities depend on the system size, the coordination number and periodic boundary conditions taking into account different hydrogen bond definitions. It was found that the first peak of the Laplace spectra contains six eigenvalues. These results suggest that six communities are always formed in our simulated systems independently of the number of molecules in the cubic box. By the help of the spectral clustering method, which is an acceptable indicator of the global properties of H-bonded network, two different H-bonded environments were identified. The tetrahedrality of the water molecules is significantly larger at the surface than inside the clusters. This difference can be related to the coexistence of HDL and LDL domains in liquid water. Furthermore, our work also emphasizes that the periodic boundary conditions always cause clustering in the system.

PCA BAND SELECTION METHOD FOR A HYPERSPECTRAL SENSORS ONBOARD AN UAV

Abstract. The development of light and small sensors, like Lidar and hyperspectral sensors, has gained popularity over the last few years. In this paper we present the experience of UFPR (Brazil), in collaboration with KIT (Germany), on the use of a UAV system carrying a hyperspectral sensor for land cover studies. The sensors were integrated with the traditional IMU-GNSS systems to record data from a quadricopter. The study focuses on band selection, aiming at reducing computational effort and statistical limitations. For this purpose, the principal components of the multispectral image are computed. The best principal components are then selected according to the explained original variance, as described by the relative size of the eigenvalues. Then, each principal component is analyzed searching for contrasting spectral regions, described by consecutive positive and negative coefficients. The most representative band of each spectral region is the selected according to its information contents and contribution to the computation of the respective eigenvectors. The method is tested using images collected with the FireflEYE 185 Cubert camera with 125 channels in the wavelength between 450 nm and 950 nm, flying over the experimental Canguiri farm in Curitiba, Brazil. Finally, we discuss the advantages of the method and its limitations.

A unified linear convergence analysis of k-SVD

Eigenvector computation, e.g., k-SVD for finding top-k singular subspaces, is often of central importance to many scientific and engineering tasks. There has been resurgent interest recently in analyzing relevant methods in terms of singular value gap dependence. Particularly, when the gap vanishes, the convergence of k-SVD is considered to be capped by a gap-free sub-linear rate. We argue in this work both theoretically and empirically that this is not necessarily the case, refreshing our understanding on this significant problem. Specifically, we leverage the recently proposed structured gap in a careful analysis to establish a unified linear convergence of k-SVD to one of the ground-truth solutions, regardless of what target matrix and how large target rank k are given. Theoretical results are evaluated and verified by experiments on synthetic or real data.

The generalized trust region subproblem: solution complexity and convex hull results

We consider the Generalized Trust Region Subproblem (GTRS) of minimizing a nonconvex quadratic objective over a nonconvex quadratic constraint. A lifting of this problem recasts the GTRS as minimizing a linear objective subject to two nonconvex quadratic constraints. Our first main contribution is structural: we give an explicit description of the convex hull of this nonconvex set in terms of the generalized eigenvalues of an associated matrix pencil. This result may be of interest in building relaxations for nonconvex quadratic programs. Moreover, this result allows us to reformulate the GTRS as the minimization of two convex quadratic functions in the original space. Our next set of contributions is algorithmic: we present an algorithm for solving the GTRS up to an epsilon additive error based on this reformulation. We carefully handle numerical issues that arise from inexact generalized eigenvalue and eigenvector computations and establish explicit running time guarantees for these algorithms. Notably, our algorithms run in linear (in the size of the input) time. Furthermore, our algorithm for computing an epsilon-optimal solution has a slightly-improved running time dependence on epsilon over the state-of-the-art algorithm. Our analysis shows that the dominant cost in solving the GTRS lies in solving a generalized eigenvalue problem -- establishing a natural connection between these problems. Finally, generalizations of our convex hull results allow us to apply our algorithms and their theoretical guarantees directly to equality-, interval-, and hollow- constrained variants of the GTRS. This gives the first linear-time algorithm in the literature for these variants of the GTRS.

Augmented Block Householder Arnoldi Method Applied in Small-Signal Stability Analysis of Power Systems

Iterative methods built on Krylov subspaces have been little explored to date for the computation of eigenvalues and eigenvectors in small-signal stability analysis. Such computation is challenging and computationally expensive for matrices with a certain number of multiple and clustered eigenvalues, conditions that can be found in many dynamic state Jacobian matrices. The present paper aims to contribute with a block algorithm to perform small-signal stability analysis with this particular type of matrix, built on the Augmented Block Householder Arnoldi (ABHA) method. The advantages of using a block method lie on the fact that the searching subspace for approximate solutions is the sum of every Krylov subspace, and therefore, the solution is expected to converge in less iterations than an unblock method. The efficiency and robustness of the proposal are examined through numerical simulations using three power systems and two other methods: the conventional Arnoldi (unblock) and QR decomposition. The results indicate that the proposed numerical algorithm is more robust than the other two for handling dynamic state Jacobian matrices having a certain number of multiple and clustered eigenvalues.