Low Rank Perturbations Research Articles

On a nonlocal integral operator commuting with the Laplacian and the Sturm–Liouville problem: Low rank perturbations of the operator

On a nonlocal integral operator commuting with the Laplacian and the Sturm–Liouville problem: Low rank perturbations of the operator

Toeplitz momentary symbols: definition, results, and limitations in the spectral analysis of structured matrices

A powerful tool for analyzing and approximating the singular values and eigenvalues of structured matrices is the theory of Generalized Locally Toeplitz (GLT) sequences. By the GLT theory one can derive a function, called the symbol, which describes the singular value or the eigenvalue distribution of the sequence, the latter under precise assumptions. However, for small values of the matrix-size of the considered sequence, the approximations may not be as good as it is desirable, since in the construction of the GLT symbol one disregards small norm and low-rank perturbations. On the other hand, Local Fourier Analysis (LFA) can be used to construct polynomial symbols in a similar manner for discretizations, where the geometric information is present, but the small norm perturbations are retained.The main focus of this paper is the introduction of the concept of sequence of “Toeplitz momentary symbols”, associated with a given sequence of truncated Toeplitz-like matrices. We construct the symbol in the same way as in the GLT theory, but we keep the information of the small norm contributions. The low-rank contributions are still disregarded, and we give an idea on the reason why this is negligible in certain cases and why it is not in other cases, being aware that in presence of high nonnormality the same low-rank perturbation can produce a dramatic change in the eigenvalue distribution. Moreover, a difference with respect to the LFA symbols is that GLT symbols and Toeplitz momentary symbols are more general - just Lebesgue measurable - and are applicable to a larger class of matrices, while in the LFA setting only trigonometric polynomials are considered and more specifically those related to the approximation stencils. We show the applicability of the approach which leads to higher accuracy in some cases, when approximating the singular values and eigenvalues of Toeplitz-like matrices using Toeplitz momentary symbols, compared with the GLT symbol. Finally, since for many applications and their analysis it is often necessary to consider non-square Toeplitz matrices, we formalize and provide some useful definitions, applicable for non-square Toeplitz momentary symbols.

The optimal spectral gap for regular and disordered harmonic networks of oscillators

We consider one-dimensional chains and multi-dimensional networks of harmonic oscillators coupled to two Langevin heat reservoirs at different temperatures. Each particle interacts with its nearest neighbours by harmonic potentials and all individual particles are confined by harmonic potentials, too. In this article, we provide, for the first time, the sharp N-particle dependence of the spectral gap of the associated generator under various physically relevant assumptions and for different spatial dimensions. Our method of proof relies on a new approach to analyze a non self-adjoint eigenvalue problem involving low-rank non-hermitian perturbations of auxiliary discrete Schrödinger operators.

Low-Rank Perturbation of Regular Matrix Pencils with Symmetry Structures

The generic change of the Weierstraß canonical form of regular complex structured matrix pencils under generic structure-preserving additive low-rank perturbations is studied. Several different symmetry structures are considered, and it is shown that for most of the structures, the generic change in the eigenvalues is analogous to the case of generic perturbations that ignore the structure. However, for some odd/even and palindromic structures, there is a different behavior for the eigenvalues 0 and \(\infty \), respectively, \(+1\) and \(-1\). The differences arise in those cases where the parity of the partial multiplicities in the perturbed matrix pencil provided by the generic behavior in the general structure-ignoring case is not in accordance with the restrictions imposed by the structure. The new results extend results for the rank-1 and rank-2 cases that were obtained in Batzke (Linear Algebra Appl 458:638–670, 2014, Oper Matrices 10:83–112, 2016) for the case of special structure-preserving perturbations. As the main tool, we use decompositions of matrix pencils with symmetry structure into sums of rank-1 matrix pencils, as those allow a parametrization of the set of matrix pencils with a given symmetry structure and a given rank.

A geometric method for eigenvalue problems with low-rank perturbations

We consider the problem of finding the spectrum of an operator taking the form of a low-rank (rank one or two) non-normal perturbation of a well-understood operator, motivated by a number of problems of applied interest which take this form. We use the fact that the system is a low-rank perturbation of a solved problem, together with a simple idea of classical differential geometry (the envelope of a family of curves) to completely analyse the spectrum. We use these techniques to analyse three problems of this form: a model of the oculomotor integrator due to Anastasio & Gad (2007 J. Comput. Neurosci. 22, 239–254. (doi:10.1007/s10827-006-0010-x)), a continuum integrator model, and a non-local model of phase separation due to Rubinstein & Sternberg (1992 IMA J. Appl. Math. 48, 249–264. (doi:10.1093/imamat/48.3.249)).

Stability radii for real linear Hamiltonian systems with perturbed dissipation

We study linear dissipative Hamiltonian (DH) systems with real constant coefficients that arise in energy based modeling of dynamical systems. We analyze when such a system is on the boundary of the region of asymptotic stability, i.e., when it has purely imaginary eigenvalues, or how much the dissipation term has to be perturbed to be on this boundary. For unstructured systems the explicit construction of the real distance to instability (real stability radius) has been a challenging problem. We analyze this real distance under different structured perturbations to the dissipation term that preserve the DH structure and we derive explicit formulas for this distance in terms of low rank perturbations. We also show (via numerical examples) that under real structured perturbations to the dissipation the asymptotical stability of a DH system is much more robust than for unstructured perturbations.

Low Rank Perturbations of Quaternion Matrices

Low rank perturbations of right eigenvalues of quaternion matrices are considered. For real and complex matrices it is well known that under a generic rank-$k$ perturbation the $k$ largest Jordan blocks of a given eigenvalue will disappear while additional smaller Jordan blocks will remain. In this paper, it is shown that the same is true for real eigenvalues of quaternion matrices, but for complex nonreal eigenvalues the situation is different: not only the largest $k$, but the largest $2k$ Jordan blocks of a given eigenvalue will disappear under generic quaternion perturbations of rank $k$. Special emphasis is also given to Hermitian and skew-Hermitian quaternion matrices and generic low rank perturbations that are structure-preserving.

Computation of pseudospectral abscissa for large-scale nonlinear eigenvalue problems

Author(s): Meerbergen, K; Michiels, W; Van Beeumen, R; Mengi, E | Abstract: We present an algorithm to compute the pseudospectral abscissa for a nonlinear eigenvalue problem. The algorithm relies on global under-estimator and over-estimator functions for the eigenvalue and singular value functions involved. These global models follow from eigenvalue perturbation theory. The algorithm has three particular features. First, it converges to the globally rightmost point of the pseudospectrum, and it is immune to nonsmoothness. The global convergence assertion is under the assumption that a global lower bound is available for the second derivative of a singular value function depending on one parameter. It may not be easy to deduce such a lower bound analytically, but assigning large negative values works robustly in practice. Second, it is applicable to large-scale problems since the dominant cost per iteration stems from computing the smallest singular value and associated singular vectors, for which efficient iterative solvers can be used. Furthermore, a significant increase in computational efficiency can be obtained by subspace acceleration, that is, by restricting the domains of the linear maps associated with the matrices involved to small but suitable subspaces, and solving the resulting reduced problems. Occasional restarts of these subspaces further enhance the efficiency for large-scale problems. Finally, in contrast to existing iterative approaches based on constructing low-rank perturbations and rightmost eigenvalue computations, the algorithm relies on computing only singular values of complex matrices. Hence, the algorithm does not require solutions of nonlinear eigenvalue problems, thereby further increasing efficiency and reliability. This work is accompanied by a robust implementation of the algorithm that is publicly available.

Computing Girth and Cogirth in Perturbed Graphic Matroids

We give polynomial-time randomized algorithms for computing the girth and the cogirth of binary matroids that are low-rank perturbations of graphic matroids.

Parameter-Dependent Rank-One Perturbations of Singular Hermitian Or Symmetric Pencils

Structure-preserving generic low-rank perturbations are studied for classes of structured matrix pencils, including real symmetric, complex symmetric, and complex Hermitian pencils. For singular pencils it is analyzed which characteristic quantities stay invariant in the perturbed canonical form, and it is shown that the regular part of a structured matrix pencil is not affected by generic perturbations of rank one. When the rank-one perturbations involve a scaling parameter, the behavior of the canonical forms dependent on this parameter is analyzed as well.

Generic Change of the Partial Multiplicities of Regular Matrix Pencils under Low-Rank Perturbations

We describe the generic change of the partial multiplicities at a given eigenvalue $\lambda_0$ of a regular matrix pencil $A_0+\lambda A_1$ under perturbations with low normal rank. More precisely, if the pencil $A_0+\lambda A_1$ has exactly $g$ nonzero partial multiplicities at $\lambda_0$, then for most perturbations $B_0+\lambda B_1$ with normal rank $r<g$ the perturbed pencil $A_0+B_0+\lambda(A_1+B_1)$ has exactly $g-r$ nonzero partial multiplicities at $\lambda_0$, which coincide with those obtained after removing the largest $r$ partial multiplicities of the original pencil $A_0+\lambda A_1$ at $\lambda_0$. Though partial results on this problem had been previously obtained in the literature, its complete solution remained open.

On the distance to singularity via low rank perturbations

For regular matrix pencils the distance in norm to the nearest singular pencil under low rank perturbation is studied. Characterizations of this distance are derived via the Weyl function of the perturbation. Special attention is paid to the Hermitian pencil case. Estimates for the distance of a given pencil to the set of singular pencils are obtained.

Algorithms for Kullback--Leibler Approximation of Probability Measures in Infinite Dimensions

In this paper we study algorithms to find a Gaussian approximation to a target measure defined on a Hilbert space of functions; the target measure itself is defined via its density with respect to a reference Gaussian measure. We employ the Kullback--Leibler divergence as a distance and find the best Gaussian approximation by minimizing this distance. It then follows that the approximate Gaussian must be equivalent to the Gaussian reference measure, defining a natural function space setting for the underlying calculus of variations problem. We introduce a computational algorithm which is well-adapted to the required minimization, seeking to find the mean as a function, and parameterizing the covariance in two different ways: through low rank perturbations of the reference covariance and through Schrödinger potential perturbations of the inverse reference covariance. Two applications are shown: to a nonlinear inverse problem in elliptic PDEs and to a conditioned diffusion process. These Gaussian approximations also serve to provide a preconditioned proposal distribution for improved preconditioned Crank--Nicolson Monte Carlo--Markov chain sampling of the target distribution. This approach is not only well-adapted to the high dimensional setting, but also behaves well with respect to small observational noise (resp., small temperatures) in the inverse problem (resp., conditioned diffusion).

Low rank perturbations of large elliptic random matrices

We study the asymptotic behavior of outliers in the spectrum of bounded rank perturbations of large random matrices. In particular, we consider perturbations of elliptic random matrices which generalize both Wigner random matrices and non-Hermitian random matrices with iid entries. As a consequence, we recover the results of Capitaine, Donati-Martin, and Feral for perturbed Wigner matrices as well as the results of Tao for perturbed random matrices with iid entries. Along the way, we prove a number of interesting results concerning elliptic random matrices whose entries have finite fourth moment; these results include a bound on the least singular value and the asymptotic behavior of the spectral radius.

Iterative Coding for Network Coding

We consider communication over a noisy network under randomized linear network coding. Possible error mechanisms include node- or link-failures, Byzantine behavior of nodes, or an overestimate of the network min-cut. Building on the work of Kötter and Kschischang, we introduce a systematic oblivious random channel model. Within this model, codewords contain a header (this is the systematic part). The header effectively records the coefficients of the linear encoding functions, thus simplifying the decoding task. Under this constraint, errors are modeled as random low-rank perturbations of the transmitted codeword. We compute the capacity of this channel and we define an error-correction scheme based on random sparse graphs and a low-complexity decoding algorithm. By optimizing over the code degree profile, we show that this construction achieves the channel capacity in complexity which is jointly quadratic in the number of coded information bits and sublogarithmic in the error probability.

A Constrained Ordering for Solving the Equality Constrained State Estimation

This paper applies a simple constrained ordering for the solution of the equality constrained state estimation problem. By low-rank perturbations in the semidefinite (1,1) block of the coefficient matrix, while maintaining sparsity, a saddle point matrix is formed. The vectors used for generating the perturbations are rows of the matrix associated with the equality constraints that represent the zero injections. The proposed algorithm make use of the Bridson's ordering constraint for saddle-point systems, which is sufficient to guarantee the existence of a signed Cholesky factorization for the perturbed indefinite coefficient matrix, with separate symbolic and numerical phases. The need for numerical pivoting during factorization is avoided, with clear benefits for performance. Two alternative implementations are provided, either modifying a fill-reducing ordering algorithm to incorporate this constraint or modifying an existing fill-reducing ordering to respect the constraint. The proposed method is compared with existing methods in terms of computational time and convergence robustness. The IEEE 300-bus and the FRCC 3949-bus systems are used as test beds for this study.

The singular values and vectors of low rank perturbations of large rectangular random matrices

In this paper, we consider the singular values and singular vectors of finite, low rank perturbations of large rectangular random matrices. Specifically, we prove almost sure convergence of the extreme singular values and appropriate projections of the corresponding singular vectors of the perturbed matrix.As in the prequel, where we considered the eigenvalues of Hermitian matrices, the non-random limiting value is shown to depend explicitly on the limiting singular value distribution of the unperturbed matrix via an integral transform that linearizes rectangular additive convolution in free probability theory. The asymptotic position of the extreme singular values of the perturbed matrix differs from that of the original matrix if and only if the singular values of the perturbing matrix are above a certain critical threshold which depends on this same aforementioned integral transform.We examine the consequence of this singular value phase transition on the associated left and right singular eigenvectors and discuss the fluctuations of the singular values around these non-random limits.

Approximating the covariance matrix of GMMs with low-rank perturbations

Covariance matrices capture correlations that are invaluable in modeling real-life datasets. Using all d2 elements of the covariance (in d dimensions) is costly and could result in over-fitting; and the simple diagonal approximation can be over-restrictive. In this work, we present a new model, the low-rank Gaussian mixture model (LRGMM), for modeling data which can be extended to identifying partitions or overlapping clusters. The curse of dimensionality that arises in calculating the covariance matrices of the GMM is countered by using low-rank perturbed diagonal matrices. The efficiency is comparable to the diagonal approximation, yet one can capture correlations among the dimensions. Our experiments reveal the LRGMM to be an efficient and highly applicable tool for working with large high-dimensional datasets.

The eigenvalues and eigenvectors of finite, low rank perturbations of large random matrices

We consider the eigenvalues and eigenvectors of finite, low rank perturbations of random matrices. Specifically, we prove almost sure convergence of the extreme eigenvalues and appropriate projections of the corresponding eigenvectors of the perturbed matrix for additive and multiplicative perturbation models. The limiting non-random value is shown to depend explicitly on the limiting eigenvalue distribution of the unperturbed random matrix and the assumed perturbation model via integral transforms that correspond to very well-known objects in free probability theory that linearize non-commutative free additive and multiplicative convolution. Furthermore, we uncover a phase transition phenomenon whereby the large matrix limit of the extreme eigenvalues of the perturbed matrix differs from that of the original matrix if and only if the eigenvalues of the perturbing matrix are above a certain critical threshold. Square root decay of the eigenvalue density at the edge is sufficient to ensure that this threshold is finite. This critical threshold is intimately related to the same aforementioned integral transforms and our proof techniques bring this connection and the origin of the phase transition into focus. Consequently, our results extend the class of ‘spiked’ random matrix models about which such predictions (called the BBP phase transition) can be made well beyond the Wigner, Wishart and Jacobi random ensembles found in the literature. We examine the impact of this eigenvalue phase transition on the associated eigenvectors and observe an analogous phase transition in the eigenvectors. Various extensions of our results to the problem of non-extreme eigenvalues are discussed.

Extension of Range of MINRES-CN Algorithm

MINRES-CN is an iterative method for solving systems of linear equations with conjugate-normal coefficient matrices whose conspectra are located on algebraic curves of a low degree. This method was proposed in a previous publication of author and KH. D. Ikramov. In this paper, the range of applicability of MINRES-CN is extended in new direction. These are conjugate normal matrices that are low rank perturbations of Symmetric matrices. Examples are given that demonstrate a higher efficiency of MINRES-CN for this class of systems compared to the well-known algorithm GMRES.