Singular Value Decomposition Of Matrices Research Articles

Fast truncated SVD of sparse and dense matrices on graphics processors

We investigate the solution of low-rank matrix approximation problems using the truncated singular value decomposition (SVD). For this purpose, we develop and optimize graphics processing unit (GPU) implementations for the randomized SVD and a blocked variant of the Lanczos approach. Our work takes advantage of the fact that the two methods are composed of very similar linear algebra building blocks, which can be assembled using numerical kernels from existing high-performance linear algebra libraries. Furthermore, the experiments with several sparse matrices arising in representative real-world applications and synthetic dense test matrices reveal a performance advantage of the block Lanczos algorithm when targeting the same approximation accuracy.

Leader–follower consensus of uncertain variable-order fractional multi-agent systems

The leader–follower consensus of a class of variable-order fractional (VOF) uncertain linear multi-agent systems is studied in this paper. Firstly, a new general lemma is proposed to find Lyapunov candidate functions for the stability of VOF systems. Secondly, according to the proposed lemma and the stability theorem of VOF systems, and by using the singular value decomposition of matrices and some related lemmas, a sufficient condition to obtain leader–follower consensus of uncertain VOF linear systems is given in the form of a linear matrix inequality. The proposed control protocol is dependent on the order of the VOF multi-agent system and is applicable to fixed-order and nonlinear VOF multi-agent systems. Finally, the feasibility and effectiveness of the approach are verified by using some numerical simulations.

Abstract 14554: Novel Upstream Targets for Atrial Fibrillation

Introduction: Atrial fibrillation (AF) represents one of the most common arrhythmias seen clinically and is associated with a significant increase in morbidity and mortality, yet, current treatment paradigms have proven largely inadequate. One of the main contributors to the pathophysiology of AF is inflammation. Hence, reduction of inflammation associated atrial remodeling represents a novel therapeutic strategy for the treatment of AF. One of the most biologically important groups of oxylipins (oxygenated lipids) is the eicosanoids that are potent modulators of immune responses and are derived from arachidonic acid, linoleic acid among others, which are metabolized through three pathways including the cytochrome P450 (CYP) pathway. The CYP products are major anti-inflammatory metabolites with several cardioprotective effects. Hypothesis: We hypothesize that lipid metabolites may represent a new therapeutic target for AF Methods: In the cross-sectional study, human atrial appendage specimens and blood from informed and consented patients were obtained in accordance with the approved UC Davis IRB protocol. Metabolomic profiling for over 100 metabolites was assessed from the plasma of patients using LC-MS/MS. We used Functional Connectome, an extension of the singular value decomposition for matrices, to determine how the metabolites were functionally related to each other. The underlying mechanisms of AF were determined using an integrated approach with molecular biology, flow cytometry, and electrophysiology. Results: Metabolomic profiling shows a significant difference in several metabolites from the arachidonic and linoleic acid pathways in the cohort with AF compared to patients in NSR. There was a significant increase in oxidative stress, endoplasmic reticulum stress, inflammatory cytokines, nuclear translocation of NF-κB, and the activation of MAPK and TGF-b pathways from patients with AF compared to NSR cohort. Conclusions: The study provides important insights into the functional relatedness of the metabolites from the CYP pathway in patients with AF and reveals the prospect of arachidonic and linoleic acid metabolites being novel therapeutic targets in the treatment of AF.

A Krylov-Schur-like method for computing the best rank-(r1,r2,r3) approximation of large and sparse tensors

The paper is concerned with methods for computing the best low multilinear rank approximation of large and sparse tensors. Krylov-type methods have been used for this problem; here block versions are introduced. For the computation of partial eigenvalue and singular value decompositions of matrices the Krylov-Schur (restarted Arnoldi) method is used. A generalization of this method to tensors is described, for computing the best low multilinear rank approximation of large and sparse tensors. In analogy to the matrix case, the large tensor is only accessed in multiplications between the tensor and blocks of vectors, thus avoiding excessive memory usage. It is proved that if the starting approximation is good enough, then the tensor Krylov-Schur method is convergent. Numerical examples are given for synthetic tensors and sparse tensors from applications, which demonstrate that for most large problems the Krylov-Schur method converges faster and more robustly than higher order orthogonal iteration.

Elliposoidal estimates of trajectory sensitivity of multi-dimensional processes based on generalized singular values problems

The problem of studying the sensitivity of control processes to parameter variations is considered. To solve the problem, the trajectory sensitivity apparatus was used, the use of which, together with the state-space method, made it possible to construct sensitivity models. Based on the models, ellipsoidal estimates of the trajectory sensitivity functions in terms of the state, output, and error of linear multidimensional continuous systems in the form of majorants and minorants are determined. Calculations are performed using the generalized singular value decomposition of matrices composed of trajectory parametric sensitivity functions. The resulting ellipsoidal estimates, due to the meaningful possibilities of the generalized singular value decomposition of matrices, have the property of minimal sufficiency. The estimation method makes it possible, using the left singular basis corresponding to the extremal generalized singular values, to determine subspaces in the state, output, and error spaces that are characterized at each moment of time by the largest and smallest additional motion in terms of the norm. The right singular basis allows us to determine subspaces in the parameter space that generate the largest and smallest additional motion in the norm. The proposed approach solves the problem of “optimal nominal”, that is, the problem of choosing the nominal value of the parameter vector of the plant units that provide the multidimensional controlled process with the smallest value of the ellipsoidal estimates of the trajectory sensitivity functions, as well as to compare the flow of multidimensional controlled processes according to the ellipsoidal estimates of the trajectory parametric sensitivity.

The solution of the matrix equation [formula omitted] and the system of matrix equations [formula omitted] with [formula omitted

In this paper, the solvability conditions for the matrix equation AXB=D and a pair of matrix equations AX=C,XB=D with the constraint X*X=Ip are deduced by applying the spectral and singular value decompositions of matrices, and the expressions of the general solutions to these matrix equations are also provided. Furthermore, the associated optimal approximate problems to the given matrices are discussed and the optimal approximate solutions are derived. Finally, two numerical experiments are given to validate the accuracy of the results.

Level numbers of a bounded linear operator between normed linear spaces and singular value decomposition revisited

ABSTRACT We introduce the notion of level numbers of a bounded linear operator between normed linear spaces, as a generalization of the singular values of an operator between inner product spaces. We study the geometric and the analytic properties of the level numbers, in connection with Birkhoff–James orthogonality and norm optimization problems. We also illustrate the similarities and the differences between the level numbers and the singular values of an operator. As an application of the present study, we obtain a new and elementary approach to the singular value decomposition of matrices.

Bidiagonal SVD Computation via an Associated Tridiagonal Eigenproblem

The Singular Value Decomposition (SVD) is widely used in numerical analysis and scientific computing applications, including dimensionality reduction, data compression and clustering, and computation of pseudo-inverses. In many cases, a crucial part of the SVD of a general matrix is to find the SVD of an associated bidiagonal matrix. This article discusses an algorithm to compute the SVD of a bidiagonal matrix through the eigenpairs of an associated symmetric tridiagonal matrix. The algorithm enables the computation of only a subset of singular values and corresponding vectors, with potential performance gains. The article focuses on a sequential version of the algorithm, and discusses special cases and implementation details. The implementation, called BDSVDX, has been included in the LAPACK library. We use a large set of bidiagonal matrices to assess the accuracy of the implementation, both in single and double precision, as well as to identify potential shortcomings. The results show that BDSVDX can be up to three orders of magnitude faster than existing algorithms, which are limited to the computation of a full SVD. We also show comparisons of an implementation that uses BDSVDX as a building block for the computation of the SVD of general matrices.

First Order and Second Order Learning Algorithms on the Special Orthogonal Group to Compute the SVD of Data Matrices

The present paper deals with neural algorithms to learn the singular value decomposition (SVD) of data matrices. The neural algorithms utilized in the present research endeavor were developed by Helmke and Moore (HM) and appear under the form of two continuous-time differential equations over the special orthogonal group of matrices. The purpose of the present paper is to develop and compare different numerical schemes, under the form of two alternating learning rules, to learn the singular value decomposition of large matrices on the basis of the HM learning paradigm. The numerical schemes developed here are both first-order (Euler-like) and second-order (Runge-like). Moreover, a reduced Euler scheme is presented that consists of a single learning rule for one of the factors involved in the SVD. Numerical experiments performed to estimate the optical-flow (which is a component of modern IoT technologies) in real-world video sequences illustrate the features of the novel learning schemes.

An Out of Memory tSVD for Big-Data Factorization

Singular value decomposition (SVD) is a matrix factorization method widely used for dimension reduction, data analytics, information retrieval, and unsupervised learning. In general, only singular values of SVD are needed for most big-data applications. Methods such as tensor networks require an accurate computation of a substantial number of singular vectors, which can be accomplished through truncated SVD (tSVD). Additionally, many real-world datasets are too big to fit into the available memory, which mandates the development of out of memory algorithms that assume that most of the data resides on an external disk during the entire computation. These algorithms reduce communication to disk and hide part of the communication by overlapping it with communication on blocks of work. Here, building upon previous works on SVD for dense matrices, we present a method for computation of a predetermined number, $K$ , of SVD singular vectors, and the corresponding $K$ singular values, of a matrix that cannot fit in the memory. Our out of memory tSVD can be used for tensor networks algorithms. We describe ways for reducing the communication during the computation of the left and right reflectors, needed to compute the singular vectors, and introduce a method for estimating the block-sizes needed to hide the communication on parallel file systems.

Random noise attenuation via the randomized canonical polyadic decomposition

ABSTRACTTensor algebra provides a robust framework for multi‐dimensional seismic data processing. A low‐rank tensor can represent a noise‐free seismic data volume. Additive random noise will increase the rank of the tensor. Hence, tensor rank‐reduction techniques can be used to filter random noise. Our filtering method adopts the Candecomp/Parafac decomposition to approximates a N‐dimensional seismic data volume via the superposition of rank‐one tensors. Similar to the singular value decomposition for matrices, a low‐rank Candecomp/Parafac decomposition can capture the signal and exclude random noise in situations where a low‐rank tensor can represent the ideal noise‐free seismic volume. The alternating least squares method is adopted to compute the Candecomp/Parafac decomposition with a provided target rank. This method involves solving a series of highly over‐determined linear least‐squares subproblems. To improve the efficiency of the alternating least squares algorithm, we uniformly randomly sample equations of the linear least‐squares subproblems to reduce the size of the problem significantly. The computational overhead is further reduced by avoiding unfolding and folding large dense tensors. We investigate the applicability of the randomized Candecomp/Parafac decomposition for incoherent noise attenuation via experiments conducted on a synthetic dataset and field data seismic volumes. We also compare the proposed algorithm (randomized Candecomp/Parafac decomposition) against multi‐dimensional singular spectrum analysis and classical prediction filtering. We conclude the proposed approach can achieve slightly better denoising performance in terms of signal‐to‐noise ratio enhancement than traditional methods, but with a less computational cost.

Hypercomplex Tensor Completion via Convex Optimization

Expressing multidimensional information as a value in hypercomplex number systems (e.g., quaternion, octonion, etc.) has great potential, in data sciences, e.g., signal processing, to enjoy their nontrivial algebraic benefits which are not available in standard real or complex vector systems. Strategic utilizations of such benefits would include, e.g., hypercomplex singular value decomposition (SVD) and low-rank approximation of matrices. In this paper, as powerful mathematical tools for wider signal processing applications of hypercomplex number systems, we first propose novel definitions of SVD and best low-rank approximation of matrices based on algebraic translations of Cayley–Dickson (C–D) number systems. We then derive an algorithmic solution to hypercomplex tensor completion problem based on a convex optimization technique. Numerical experiments in a scenario of color tensor completion problem show that the proposed algorithm recovers much more faithfully the original color information, masked randomly by noise, than part-wise tensor completion algorithms.

An Elementary Proof of Bevan's Theorem on the Growth of Grid Classes of Permutations

AbstractBevan established that the growth rate of a monotone grid class of permutations is equal to the square of the spectral radius of a related bipartite graph. We give an elementary and self-contained proof of a generalization of this result using only Stirling's formula, the method of Lagrange multipliers, and the singular value decomposition of matrices. Our proof relies on showing that the maximum over the space of n × n matrices with non-negative entries summing to one of a certain function of those entries, parametrized by the entries of another matrix Γ of non-negative real numbers, is equal to the square of the largest singular value of Γ and that the maximizing point can be expressed as a Hadamard product of Γ with the tensor product of singular vectors for its greatest singular value.

Singular Value Decomposition of Complexes

Singular value decompositions of matrices are widely used in numerical linear algebra with many applications. In this paper, we extend the notion of singular value decompositions to finite complexe...

Methods for Computing Weighted Pseudoinverses and Weighted Normal Pseudosolutions with Singular Weights

The paper surveys articles that construct and investigate direct and iterative methods for computing weighted pseudoinverses and weighted normal pseudosolutions with singular weights. The methods considered in the paper are mainly constructed based on the authors’ articles devoted to the development of the theory of weighted pseudoinversion aimed at investigating the characteristics of both weighted pseudoinverses and weighted normal pseudosolutions with singular weights. The paper uses the following results obtained and investigated by the authors: expansions of weighted pseudoinverses into matrix power series and products, limit representations of such matrices, and determination of decompositions of weighted pseudoinverses based on weighted singular value decompositions of matrices with singular weights.

Block tridiagonal matrix enhanced multivariance products representation (BTMEMPR)

The basic aim of this work is to design a new tridiagonal matrix enhanced multivariance products representation (TMEMPR) which uses not Cartesian vectors but matrices as the support entities. What we obtain after the construction of the representation has been a singular value decomposition like structure where the core matrix becomes a block tridiagonal matrix in contrast to the diagonal and tridiagonal matrix structures of the singular value decomposition of matrices and TMEMPR respectively. We have used support matrices in the construction and not directly orthogonality of the constructed support matrices but block orthogonality which means the mutual ortho normality of the columns of produced support matrices. Certain confirmative implementations finalize the paper.

Modal Analysis of Fluid Flows: An Overview

Modal Analysis of Fluid Flows: An Overview

Tensor decompositions for learning latent variable models

This work considers a computationally and statistically efficient parameter estimation method for a wide class of latent variable models--including Gaussian mixture models, hidden Markov models, and latent Dirichlet allocation--which exploits a certain tensor structure in their low-order observable moments (typically, of second- and third-order). Specifically, parameter estimation is reduced to the problem of extracting a certain (orthogonal) decomposition of a symmetric tensor derived from the moments; this decomposition can be viewed as a natural generalization of the singular value decomposition for matrices. Although tensor decompositions are generally intractable to compute, the decomposition of these specially structured tensors can be efficiently obtained by a variety of approaches, including power iterations and maximization approaches (similar to the case of matrices). A detailed analysis of a robust tensor power method is provided, establishing an analogue of Wedin's perturbation theorem for the singular vectors of matrices. This implies a robust and computationally tractable estimation approach for several popular latent variable models.

Updating undamped structural models using displacement output feedback

A novel numerical method for updating stiffness matrix by displacement output feedback technique is presented. The required displacement output feedback gain matrix is determined by the QR-decompositions and the singular value decompositions of matrices, and thus the optimal approximation stiffness matrix is found and the large number of unmeasured and unknown eigeninformation of the original model is preserved. The proposed method is computationally efficient since it does not require iterations and the updated stiffness matrix is also symmetric and has the compact expression.

Singular vectors of orthogonally decomposable tensors

Orthogonal decomposition of tensors is a generalization of the singular value decomposition of matrices. In this paper, we study the spectral theory of orthogonally decomposable tensors. For such a tensor, we give a description of its singular vector tuples as a variety in a product of projective spaces.