Small Number Of Eigenvalues Research Articles

Forecasting High-Dimensional Covariance Matrices Using High-Dimensional Principal Component Analysis

We modify the recently proposed forecasting model of high-dimensional covariance matrices (HDCM) of asset returns using high-dimensional principal component analysis (PCA). It is well-known that when the sample size is smaller than the dimension, eigenvalues estimated by classical PCA have a bias. In particular, a very small number of eigenvalues are extremely large and they are called spiked eigenvalues. High-dimensional PCA gives eigenvalues which correct the biases of the spiked eigenvalues. This situation also happens in the financial field, especially in situations where high-frequency and high-dimensional data are handled. The research aims to estimate the HDCM of asset returns using high-dimensional PCA for the realized covariance matrix using the Nikkei 225 data, it estimates 5- and 10-min intraday asset-returns intervals. We construct time-series models for eigenvalues which are estimated by each PCA, and forecast HDCM. Our simulation analysis shows that the high-dimensional PCA has better estimation performance than classical PCA for the estimating integrated covariance matrix. In our empirical analysis, we show that we will be able to improve the forecasting performance using the high-dimensional PCA and make a portfolio with smaller variance.

Deep learning, stochastic gradient descent and diffusion maps

Stochastic gradient descent (SGD) is widely used in deep learning due to its computational efficiency, but a complete understanding of why SGD performs so well remains a major challenge. It has been observed empirically that most eigenvalues of the Hessian of the loss functions on the loss landscape of over-parametrized deep neural networks are close to zero, while only a small number of eigenvalues are large. Zero eigenvalues indicate zero diffusion along the corresponding directions. This indicates that the process of minima selection mainly happens in the relatively low-dimensional subspace corresponding to the top eigenvalues of the Hessian. Although the parameter space is very high-dimensional, these findings seems to indicate that the SGD dynamics may mainly live on a low-dimensional manifold. In this paper, we pursue a truly data driven approach to the problem of getting a potentially deeper understanding of the high-dimensional parameter surface, and in particular, of the landscape traced out by SGD by analyzing the data generated through SGD, or any other optimizer for that matter, in order to possibly discover (local) low-dimensional representations of the optimization landscape. As our vehicle for the exploration, we use diffusion maps introduced by R. Coifman and coauthors.

Benjamini–Schramm convergence and spectra of random hyperbolic surfaces of high genus

We study geometric and spectral properties of typical hyperbolic surfaces of high genus, excluding a set of small measure for the Weil-Petersson probability measure. We first prove Benjamini-Schramm convergence to the hyperbolic plane H as the genus g goes to infinity. An estimate for the number of eigenvalues in an interval [a,b] in terms of a, b and g is then proven using the Selberg trace formula. It implies the convergence of spectral measures to the spectral measure of H as g $\rightarrow$+$\infty$, and a uniform Weyl law as b $\rightarrow$+$\infty$. We deduce a bound on the number of small eigenvalues, and the multiplicity of any eigenvalue.

O(N ) ab initio calculation scheme for large-scale moiré structures

We present a two-step method specifically tailored for band structure calculation of the small-angle moir\'{e}-pattern materials which contain tens of thousands of atoms in a unit cell. In the first step, the self-consistent field calculation for ground state is performed with $O(N)$ Krylov subspace method implemented in OpenMX. Secondly, the crystal momentum dependent Bloch Hamiltonian and overlap matrix are constructed from the results obtained in the first step and only a small number of eigenvalues near the Fermi energy are solved with shift-invert and Lanczos techniques. By systematically tuning two key parameters, the cutoff radius for electron hopping interaction and the dimension of Krylov subspace, we obtained the band structures for both rigid and corrugated twisted bilayer graphene structures at the first magic angle ($\theta=1.08^\circ$) and other three larger ones with satisfied accuracy on affordable costs. The band structures are in good agreement with those from tight binding models, continuum models, plane-wave pseudo-potential based $ab~initio$ calculations, and the experimental observations. This efficient two-step method is to play a crucial role in other twisted two-dimensional materials, where the band structures are much more complex than graphene and the effective model is hard to be constructed.

Pathological Spectra of the Fisher Information Metric and Its Variants in Deep Neural Networks.

The Fisher information matrix (FIM) plays an essential role in statistics and machine learning as a Riemannian metric tensor or a component of the Hessian matrix of loss functions. Focusing on the FIM and its variants in deep neural networks (DNNs), we reveal their characteristic scale dependence on the network width, depth, and sample size when the network has random weights and is sufficiently wide. This study covers two widely used FIMs for regression with linear output and for classification with softmax output. Both FIMs asymptotically show pathological eigenvalue spectra in the sense that a small number of eigenvalues become large outliers depending on the width or sample size, while the others are much smaller. It implies that the local shape of the parameter space or loss landscape is very sharp in a few specific directions while almost flat in the other directions. In particular, the softmax output disperses the outliers and makes a tail of the eigenvalue density spread from the bulk. We also show that pathological spectra appear in other variants of FIMs: one is the neural tangent kernel; another is a metric for the input signal and feature space that arises from feedforward signal propagation. Thus, we provide a unified perspective on the FIM and its variants that will lead to more quantitative understanding of learning in large-scale DNNs.

On cut vertices and eigenvalues of character graphs of solvable groups

Given a finite group G, the character graph, denoted by Δ(G), for its irreducible character degrees is a graph with vertex set ρ(G) which is the set of prime numbers that divide the irreducible character degrees of G, and with {p,q} being an edge if there exists a non-linear χ∈Irr(G) whose degree is divisible by pq. In this paper, on one hand, we proceed by discussing the graphical shape of Δ(G) when it has cut vertices or small number of eigenvalues, and on the other hand we give some results on the group structure of G with such Δ(G). Recently, Lewis and Meng proved the character graph of each solvable group has at most one cut vertex. Now, we determine the structure of character graphs of solvable groups with a cut vertex and diameter 3. Furthermore, we study solvable groups whose character graphs have at most two distinct eigenvalues. Moreover, we investigate the solvable groups whose character graphs are regular with three distinct eigenvalues. In addition, we give some lower bounds for the number of edges of Δ(G).

Maximality of the signless Laplacian energy

We study the problem of determining the graph with n vertices having largest signless Laplacian energy. We conjecture it is the complete split graph whose independent set has (roughly) 2n∕3 vertices. We show that the conjecture is true for several classes of graphs. In particular, the conjecture holds for the set of all complete split graphs of order n, for trees, for unicyclic and bicyclic graphs. We also give conditions on the number of edges, number of cycles and number of small eigenvalues so the graph satisfies the conjecture.

Partial eigenvalue assignment and its stability in a time delayed system

Active vibration control strategy is an effective way to control dangerous vibrations in a structure, caused by resonance and to manipulate the dynamics of vibrational response. Implementation of this strategy requires real-time computations of two feedback control matrices such that a small amount of eigenvalues of the associated quadratic matrix pencil are replaced by suitably chosen ones while the remaining large number of eigenvalues and eigenvectors remain unchanged ensuring the no spill-over. This mathematical problem is referred to as the Quadratic Partial Eigenvalue Assignment problem. The greatest challenge there is to solve the problems using the knowledge of only a small number of eigenvalues and eigenvectors that are computable using state-of-the-art techniques. This paper generalizes the earlier work on partial assignment to constant time-delay systems. Furthermore, a posterior stability analysis is carried out to identify the ranges of the time-delay that maintains the closed-loop assignment while keeping the stability of the infinite number of eigenvalues for the time-delayed systems. The practical features of the proposed methods are that it is implemented in the second-order setting itself using only those small number of eigenvalues and the eigenvectors that are to be assigned and the no spill-over is established by means of mathematical results. The results of our numerical experiments support the validity of our proposed methods.

Partial eigenvalue assignment for high order system by multi-input control

An algorithm of partial eigenvalue assignment problem for high order systems is given that the spectrums are partially reassigned to predetermined locations and the remaining spectrums keep unchanged. The algorithm requires the knowledge of only a small number of eigenvalues and their corresponding eigenvectors. Numerical examples are done to illustrate the effect of the approach.

Quadratic model updating with no spill-over and incomplete measured data: Existence and computation of solution

Quadratic finite element model updating problem (QFEMUP), to be studied in this paper, is concerned with updating a symmetric nonsingular quadratic pencil in such a way that, a small set of measured eigenvalues and eigenvectors is reproduced by the updated model. If in addition, the updated model preserves the large number of unupdated eigenpairs of the original model, the model is said to be updated with no spill-over. QFEMUP is, in general, a difficult and computationally challenging problem due to the practical constraint that only a very small number of eigenvalues and eigenvectors of the associated quadratic eigenvalue problem are available from computation or measurement. Additionally, for practical effectiveness, engineering concerns such as nonorthogonality and incompleteness of the measured eigenvectors must be considered. Most of the existing methods, including those used in industrial settings, deal with updating a linear model only, ignoring damping. Only in the last few years a small number of papers been published on the quadratic model updating; several of the above issues have been dealt with both from theoretical and computational point of views. However, mathematical criterion for existence of solution has not been fully developed. In this paper, we first (i) prove a set of necessary and sufficient conditions for the existence of a solution of the no spill-over QFEMUP, then (ii) present a parametric representation of the solution, assuming a solution exists and finally, (iii) propose an algorithm for QFEMUP with no spill-over and incomplete measured eigenvectors. Interestingly, it is shown that the parametric representation can be constructed with the knowledge of only the few eigenvalues and eigenvectors that are to be updated and the corresponding measured eigenvalues and eigenvectors—complete knowledge of eigenvalues and eigenvectors of the original pencil is not needed, which makes the solution readily applicable to real-life structures.

Robust multigrid preconditioners for cell-centered finite volume discretization of the high-contrast diffusion equation

We study a conservative 5-point cell-centered finite volume discretization of the high-contrast diffusion equation. We aim to construct preconditioners that are robust with respect to the magnitude of the coefficient contrast and the mesh size simultaneously. For that, we prove and numerically demonstrate the robustness of the preconditioner proposed by Aksoylu et al. (Comput Vis Sci 11:319–331, 2008) by extending the devised singular perturbation analysis from linear finite element discretization to the above discretization. The singular perturbation analysis is more involved than that of finite element case because all the subblocks in the discretization matrix depend on the diffusion coefficient. However, as the diffusion coefficient approaches infinity, that dependence is eliminated. This allows the same preconditioner to be utilized due to similar limiting behaviours of the submatrices; leading to a narrowing family of preconditioners that can be used for different discretizations. Therefore, we have accomplished a desirable preconditioner design goal. We compare our numerical results to standard cell-centered multigrid implementations and observe that performance of our preconditioner is independent of the utilized smoothers and prolongation operators. As a side result, we also prove a fundamental qualitative property of solution of the high-contrast diffusion equation. Namely, the solution over the highly-diffusive island becomes constant asymptotically. Integration of this qualitative understanding of the underlying PDE to our preconditioner is the main reason behind its superior performance. Diagonal scaling is probably the most basic preconditioner for high-contrast coefficients. Extending the matrix entry based spectral analysis introduced by Graham and Hagger, we rigorously show that the number of small eigenvalues of the diagonally scaled matrix depends on the number of isolated islands comprising the highly-diffusive region. This indicates that diagonal scaling creates a significant clustering of the spectrum, a favorable property for faster convergence of Krylov subspace solvers.

Effondrement, spectre et propriétés diophantiennes des flots riemanniens

We study the behavior of the first eigenvalues of the Hodge Laplacian acting on differential forms under adiabatic collapsing of a riemannian flow ℱ on a closed manifold M. We show that the number of small eigenvalues is related to the basic cohomology of ℱ, and give spectral criteria for the vanishing of the Álvarez class and the Euler class of the flow. We also define a diophantine invariant of the flow which is related to the asymptotical behavior of the small eigenvalues. An appendix is devoted to arithmetic properties of riemannian flows.

Robust and minimum norm partial quadratic eigenvalue assignment in vibrating systems: A new optimization approach

Abstract The partial quadratic eigenvalue assignment problem (PQEVAP) concerns reassigning a few undesired eigenvalues of a quadratic matrix pencil to suitably chosen locations and keeping the other large number of eigenvalues and eigenvectors unchanged (no spill-over). The problem naturally arises in controlling dangerous vibrations in structures by means of active feedback control design. For practical viability, the design must be robust, which requires that the norms of the feedback matrices and the condition number of the closed-loop eigenvectors are as small as possible. The problem of computing feedback matrices that satisfy the above two practical requirements is known as the Robust Partial Quadratic Eigenvalue Assignment Problem (RPQEVAP). In this paper, we formulate the RPQEVAP as an unconstrained minimization problem with the cost function involving the condition number of the closed-loop eigenvector matrix and two feedback norms. Since only a small number of eigenvalues of the open-loop quadratic pencil are computable using the state-of-the-art matrix computational techniques and/or measurable in a vibration laboratory, it is imperative that the problem is solved using these small number of eigenvalues and the corresponding eigenvectors. To this end, a class of the feedback matrices are obtained in parametric form, parameterized by a single parametric matrix, and the cost function and the required gradient formulas for the optimization problem are developed in terms of the small number of eigenvalues that are reassigned and their corresponding eigenvectors. The problem is solved directly in quadratic setting without transforming it to a standard first-order control problem and most importantly, the significant “no spill-over property” of the closed-loop eigenvalues and eigenvectors is established by means of a mathematical result. These features make the proposed method practically applicable even for very large structures. Results on numerical experiments show that the proposed method considerably reduces both feedback norms and the sensitivity of the closed-loop eigenvalues. A study on robustness of the system responses of the method under small perturbations show that the responses of the perturbed closed-loop system are compatible with perturbations.

Quadratic partial eigenvalue assignment problem with time delay for active vibration control

Partial pole assignment in active vibration control refers to reassigning a small set of unwanted eigenvalues of the quadratic eigenvalue problem (QEP) associated with the second order system of a vibrating structure, by using feedback control force, to suitably chosen location without altering the remaining large number of eigenvalues and eigenvectors. There are several challenges of solving this quadratic partial eigenvalue assignment problem (QPEVAP) in a computational setting which the traditional pole-placement problems for first-order control systems do not have to deal with. In order to these challenges, there has been some work in recent years to solve QPEVAP in a computationally viable way. However, these works do not take into account of the practical phenomenon of the time-delay effect in the system. In this paper, a new "direct and partial modal" approach of the quadratic partial eigenvalue assignment problem with time-delay is proposed. The approach works directly in the quadratic system without requiring transformation to a standard state-space system and requires the knowledge of only a small number of eigenvalues and eigenvectors that can be computed or measured in practice. Two illustrative examples are presented in the context of active vibration control with constant time-delay to illustrate the success of our proposed approach. Future work includes generalization of this approach to a more practical complex time-delay system and extension of this work to the multi-input problem.

The Empirical Properties of Large Covariance Matrices

Our second paper also takes on an ambitious empirical study and challenges some commonly held notions. In this paper, Gilles Zumbach investigates the evolution of large covariance matrices. Having previously investigated volatility on the same datasets, Gilles turns to the joint problem of volatility and correlation. Certainly, one challenge in the study is to define meaningful ways to reduce the quantity of information, so that we can gain some intuition about how covariance evolves broadly. Gilles shows that the spectrum of the covariance matrix is actually quite static, with most of the interesting dynamics restricted to a small number of eigenvalues. This in itself would seem to support methods such as principal components analysis, where we concentrate on a small number of stable directions for correlation or covariance. Gilles’s deeper investigation shows, however, that the directions associated with the important eigenvalues change quite a bit, supporting an approach where correlation and volatility are both dynamic quantities.

ON SOLVING ENERGY-DEPENDENT PARTITIONED REAL SYMMETRIC MATRIX EIGENVALUE PROBLEM BY A PARALLEL GENETIC ALGORITHM

An energy-dependent partitioning scheme is explored for extracting a small number of eigenvalues of a real symmetric matrix with the help of a serial as well as parallel genetic algorithm (GA). The proposed method is tested on two matrices (up to 2000 × 2000) with an increasing number of processors in a master–slave architecture. A comparison is made with the Jacobi–Davidson method in serial mode as implemented in the JDQZ-package. Different partition sizes are used. Traditionally used Löwdin's method is also tested in both serial and parallel modes. The advantages and disadvantages of the parallel GA-based method in solving the partitioned eigenvalue problem are analyzed.

Domain decomposition preconditioners for elliptic equations with jump coefficients

AbstractThis paper provides a proof of the robustness of the overlapping domain decomposition preconditioners for the linear finite element approximation of second‐order elliptic boundary value problems with strongly discontinuous coefficients. By analyzing the eigenvalue distribution of the domain decomposition preconditioned system, we prove that only a small number of eigenvalues may deteriorate with respect to the discontinuous jump or mesh size, and all the other eigenvalues are bounded below and above nearly uniformly with respect to the jump and mesh size. As a result, we prove that the convergence rate of the preconditioned conjugate gradient methods is nearly uniform with respect to the large jump and mesh size. Copyright © 2008 John Wiley & Sons, Ltd.

UNIFORM CONVERGENT MULTIGRID METHODS FOR ELLIPTIC PROBLEMS WITH STRONGLY DISCONTINUOUS COEFFICIENTS

This paper gives a solution to an open problem concerning the performance of various multilevel preconditioners for the linear finite element approximation of second-order elliptic boundary value problems with strongly discontinuous coefficients. By analyzing the eigenvalue distribution of the BPX preconditioner and multigrid V-cycle preconditioner, we prove that only a small number of eigenvalues may deteriorate with respect to the discontinuous jump or meshsize, and we prove that all the other eigenvalues are bounded below and above nearly uniformly with respect to the jump and meshsize. As a result, we prove that the convergence rate of the preconditioned conjugate gradient methods is uniform with respect to the large jump and meshsize. We also present some numerical experiments to demonstrate the theoretical results.

Projection acceleration of Krylov solvers

AbstractIn many applications it appears that the initial convergence of preconditioned Krylov solvers is slow. The reason for this is that a number of small eigenvalues are present. After these bad eigenvector components are approximated, the fast superlinear convergence sets in. A way to have fast convergence from the start is to remove these components by a projection. In this paper we give a comparison of some of these projection operators. (© 2008 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)

CIRR: a Rayleigh-Ritz type method with contour integral for generalized eigenvalue problems

We consider a Rayleigh-Ritz type eigensolver for finding a limited set of eigenvalues and their corresponding eigenvectors in a certain region of generalized eigenvalue problems. When the matrices are very large, iterative methods are used to generate an invariant subspace that contains the desired eigenvectors. Approximations are extracted from the subspace through a Rayleigh-Ritz projection. In this paper, we present a Rayleigh-Ritz type method with a contour integral (CIRR method). In this method, numerical integration along a circle that contains relatively small number of eigenvalues is used to construct a subspace. Since the process to derive the subspace can be performed in parallel, the presented method is suitable for master-worker programming models. Numerical experiments illustrate the property of the proposed method.