Full-rank Matrices Research Articles

Properties of the matrix A − XY*

The main topic of this paper is the matrix V = A − XY*, where A is a nonsingular complex k × k matrix and X and Y are k × p complex matrices of full column rank. Because properties of the matrix V can be derived from those of the matrix Q = I − XY*, we will consider in particular the case where A = I. For the case that Y* X = I, so that Q is singular, we will derive the Moore–Penrose inverse of Q. The Moore–Penrose inverse of V in case Y* A −1 X = I then easily follows. Finally, we will focus on the eigenvalues and eigenvectors of the real matrix D − xy′ with D diagonal.

On Some Properties of Convex Matrix Sets Characterized by P -Matrices and Block P -Matrices

In this paper, we consider convex sets of real matrices and establish criteria characterizing these sets with respect to certain matrix properties of their elements. In particular, we deal with convex sets of P-matrices, block P-matrices and M-matrices, nonsingular and full rank matrices, as well as stable and Schur stable matrices. Our results are essentially based on the notion of a block P-matrix and extend and generalize some recently published results on this topic.

Linear preservers of minimal rank

It is proved that a linear transformation on the vector space of upper triangular matrices that maps the set of matrices of minimal rank 1 into itself, and either has the analogous property with respect to matrices of full minimal rank, or is bijective, is a triangular equivalence, or a flip about the south-west north-east diagonal followed by a triangular equivalence. The result can be regarded as an analogue of Marcus–Moyls theorem in the context of triangular matrices.

On Principal Angles between Subspaces of Euclidean Space

The cosines of the principal angles between the column spaces of full column rank matrices $X\in{\hbox{{\bf R}$^{m\times p}$}}$ and $Y\in{\hbox{{\bf R}$^{m\times q}$}}$ are efficiently computed, using the Björck--Golub algorithm, as the singular values of $Q_x^{T}Q_y,$ where Qx and Qy are orthonormal matrices computed by the QR factorizations of X and Y, respectively. This paper shows that the Björck--Golub algorithm is mixed stable in the following sense: the computed singular values approximate with small relative error the exact cosines of the principal angles between the column spaces of $X+\Delta X$ and $Y+\Delta Y,$ where $\Delta X,$ $\Delta Y$ are small backward errors. Further, theoretical analysis and numerical evidence show that the algorithm becomes more robust if the QR factorizations are computed with the complete pivoting scheme of Powell and Reid. Moreover, it is shown that Gaussian elimination with complete pivoting can be used as an efficient preconditioner in computation and as a useful tool in analysis of the sensitivity of the QR factorization.

A posteriori computation of the singular vectors in a preconditioned Jacobi SVD algorithm

This paper describes a novel way to implement the Jacobi algorithm for the singular value decomposition of full rank matrices. It is shown that the left and right singular vectors can be computed without explicit accumulation of Jacobi rotations. Instead, the accumulated product of Jacobi rotations is computed a posteriori as the solution of a certain well-conditioned matrix equation. Theoretical analysis provides tools to estimate, check and, if necessary, to improve the accuracy of the computed decomposition. Experimental results show that the new technique performs very well in practice.

Changes in the general linear model: A unified approach

Analysis of the general linear model with possibly rank-deficient design and dispersion matrices has sometimes generated some confusion and controversy, prompting some researchers to discuss it as quite distinct from the case of full-rank matrices. We show that linear zero functions, i.e., linear functions in observations which have zero expectations for all parameter values, provide an intuitive way of developing all the important results in connection with the general linear model, thus bridging this imaginary gap. We show that the effect of addition or deletion of a set of observations in this model can be clearly understood in statistical terms if viewed through such linear zero functions. The effect of adding or dropping a group of parameters is also explained well in this manner. Several sets of update equations were derived by a host of previous researchers in various special cases of the above set-up. The results derived here bring out the common underlying principles of these formulae and indeed help simplify most of them. These results also provide further insights into recursive residuals, design of experiments, deletion diagnostics and selection of subset models.

Achieving an Arbitrary Spatial Stiffness with Springs Connected in Parallel

In this paper, the synthesis of an arbitrary spatial stiffness matrix is addressed. We have previously shown that an arbitrary stiffness matrix cannot be achieved with conventional translational springs and rotational springs (simple springs) connected in parallel regardless of the number of springs used or the geometry of their connection. To achieve an arbitrary spatial stiffness matrix with springs connected in parallel, elastic devices that couple translational and rotational components are required. Devices having these characteristics are defined here as screw springs. The designs of two such devices are illustrated. We show that there exist some stiffness matrices that require 3 screw springs for their realization and that no more than 3 screw springs are required for the realization of full-rank spatial stiffness matrices. In addition, we present two procedures for the synthesis of an arbitrary spatial stiffness matrix. With one procedure, any rank-m positive semidefinite matrix is realized with m springs of which all may be screw springs. With the other procedure, any positive definite matrix is realized with 6 springs of which no more than 3 are screw springs.

A Tangent Algorithm for Computing the Generalized Singular Value Decomposition

We present two new algorithms for floating-point computation of the generalized singular values of a real pair $(A,B)$ of full column rank matrices and for floating-point solution of the generalized eigenvalue problem $Hx=\lambda Mx$ with symmetric, positive definite matrices H and M. The pair $(A,B)$ is replaced with an equivalent pair $(A',B')$, and the generalized singular values are computed as the singular values of the explicitly computed matrix $F=A' B'^{-1}$. The singular values of F are computed using the Jacobi method. The relative accuracy of the computed singular value approximations does not depend on column scalings of A and B; that is, the accuracy is nearly the same for all pairs $(AD_1,BD_2)$, with $D_1$, $D_2$ arbitrary diagonal, nonsingular matrices. Similarly, the pencil $H-\lambda M$ is replaced with an equivalent pencil $H'-\lambda M'$, and the eigenvalues of $H-\lambda M$ are computed as the squares of the singular values of $G=L_H L_M^{-1}$, where $L_H$, $L_M$ are the Cholesky factors of $H'$, $M'$, respectively, and the matrix G is explicitly computed as the solution of a linear system of equations. For the computed approximation $\lambda+\delta\lambda$ of any exact eigenvalue $\lambda$, the relative error $|\delta\lambda|/\lambda$ is of order $p(n) {\mbox{\boldmath$\varepsilon$}} \max\{\min_{\Delta\in{\cal D}}\kappa_2(\Delta H\Delta), \min_{\Delta\in{\cal D}}\kappa_2(\Delta M\Delta)\}$, where $p(n)$ is a modestly growing polynomial of the dimension of the problem, {\mbox{\boldmath$\varepsilon$}} is the round-off unit of floating-point arithmetic, ${\cal D}$ denotes the set of diagonal nonsingular matrices, and $\kappa_2(\cdot)$ is the spectral condition number. Furthermore, floating-point computation corresponds to an exact computation with $H+\delta H$, $M+\delta M$, where, for all i, j, $|\delta H_{ij}|/\sqrt{H_{ii}H_{jj}}$ and $|\delta M_{ij}|/\sqrt{M_{ii}M_{jj}}$ are of order of {\mbox{\boldmath$\varepsilon$}} times a modest function of n.

Accurate Computation of the Product-Induced Singular Value Decomposition with Applications

We present a new algorithm for floating-point computation of the singular value decomposition (SVD) of the product $B^{\tau}C$, where $B$ and $C$ are full row rank matrices. The algorithm replaces the pair $(B,C)$ with an equivalent pair $(B',C')$ and then it uses the Jacobi SVD algorithm to compute the SVD of the explicitly computed matrix $B'^{\tau}C'$. In this way, each nonzero singular value $\sigma$ is approximated with some $\sigma+\delta\sigma$, where the relative error $|\delta\sigma|/\sigma$ is, up to a factor of the dimensions, of order $\{\min_{\Delta\in{\cal D}}\kappa_2(\Delta B)+ \min_{\Delta\in{\cal D}} \kappa_2(\Delta C)\}$, where ${\cal D}$ denotes the set of diagonal nonsingular matrices, $\kappa_2(\cdot)$ denotes the spectral condition number, and \boldmath${\varepsilon}$\unboldmath is the roundoff unit of floating-point arithmetic. The new algorithm is applied to the eigenvalue problem $HM x = \lambda x$ with symmetric positive definite $H$ and $M$. It is shown that each eigenvalue $\lambda$ is computed with high relative accuracy and that the relative error $|\delta\lambda|/\lambda$ of the computed approximation $\lambda+\delta\lambda$ is, up to a factor of the dimension, of order \boldmath${\varepsilon}$\unboldmath$\{\min_{\Delta\in{\cal D}}\kappa_2(\Delta H\Delta) + \min_{\Delta\in{\cal D}}\kappa_2(\Delta M\Delta)\}$. The new algorithm can also be used for accurate SVD computation of a single matrix $G$ that admits an accurate factorization $G=B^{\tau}C$.

Complex recurrent neural network for computing the inverse and pseudo-inverse of the complex matrix

A complex recurrent neural network (CRNN) is formulated and applied to compute the complex matrix inverse in real time. Both full rank and rand deficient matrices are considered. This paper extends recent works which apply real recurrent networks for real-valued matrix inversion.

Efficient Solution Of The Rank-Deficient Linear Least Squares Problem

In this paper we present several new fast and reliable algorithms for solving rank-deficient linear least squares problems by means of the complete orthogonal decomposition. Experimental comparison of our algorithms with the existing implementations in LAPACK on a wide range of platforms shows considerable performance improvements. Moreover, for full-rank matrices the performance of the new algorithm is very close to that ofthe fast method based on QR factorization, thus providing a valuable general tool for full-rank matrices, rank-deficient matrices, and those matrices with unknown rank.

A new algorithm for the factorization and inversion of recursively generated matrices

In this paper we present a recursive algorithm for factorization and inversion of matrices generated by adding dyads (elementary or rank-one matrices) as it happens in recursive array signal processing. The algorithm is valid for any recursively generated rectangular matrix and it has two parts: one valid for rank-deficient and the other for full rank matrices. The second part is used to obtain a generalization of the Sherman-Morrison algorithm for the recursive inversion of the covariance matrix. From the proposed algorithm we derive two others: one to compute the inverse (pseudo-inverse) of any matrix and the other to invert simultaneously two matrices.

Maximum likelihood parameter and rank estimation in reduced-rank multivariate linear regressions

This paper considers the problem of maximum likelihood (ML) estimation for reduced-rank linear regression equations with noise of arbitrary covariance. The rank-reduced matrix of regression coefficients is parameterized as the product of two full-rank factor matrices. This parameterization is essentially constraint free, but it is not unique, which renders the associated ML estimation problem rather nonstandard. Nevertheless, the problem turns out to be tractable, and the following results are obtained. An explicit expression is derived for the ML estimate of the regression matrix in terms of the data covariances and their eigenelements. Furthermore, a detailed analysis of the statistical properties of the ML parameter estimate is performed. Additionally, a generalized likelihood ratio test (GLRT) is proposed for estimating the rank of the regression matrix. The paper also presents the results of some simulation exercises, which lend empirical support to the theoretical findings.

Nonlinear coordinate representations of smooth optimization problems

Nonlinear coordinate representations of smooth optimization problems are investigated from the point of view of variable metric algorithms. In other words, nonlinear coordinate systems, in the sense of differential geometry, are studied by taking into consideration the structure of smooth optimization problems and variable metric methods. Both the unconstrained and constrained cases are discussed. The present approach is based on the fact that the nonlinear coordinate transformation of an optimization problem can be replaced by a suitable Riemannian metric belonging to the Euclidean metric class. In the case of equality and inequality constraints, these questions are related closely to the right inverses of full-rank matrices; therefore, their characterization is a starting point of the present analysis. The main results concern a new subclass of nonlinear transformations in connection with the common supply of coordinates to two Riemannian manifolds, one immersed in the other one. This situation corresponds to the differentiable manifold structure of nonlinear optimization problems and improves the insight into the theoretical background of variable metric algorithms. For a wide class of variable metric methods, a convergence theorem in invariant form (not depending on coordinate representations) is proved. Finally, a problem of convexification by nonlinear coordinate transformations and image representations is studied.

Analytic Criteria Underlying the Full Rank Cumulant Slice Problem

Many recent methods in system identification using higher order cumulants involve solving linear equations using Hankel matrices built from cumulant slices. Uniqueness of the solution requires a full rank Hankel matrix, which is not a generic property. Non full rank matrices result in certain poles being hidden from the cumulant data. A theoetical study of this phenomenon is presented, leading to necessary and sufficient conditions for a full rank slice. A worst-case scenario, in which a given pole cannot be identified from any cumulant slice, is successfully reduced to a parametric family of transfer functions. For cumulant orders larger than three, such constructs are restricted to first order transfer functions.

Parallel solution of toeplitzlike linear systems

The known algorithms invert an n × n Toeplitz matrix in sequential arithmetic time O( n log 2 n), but their parallel time is not better than linear in n. We present a weakly numerically stable algorithm that numerically inverts an n × n well-conditioned Toeplitz matrix A in sequential time O( n log 4 n log log n) or in parallel time of O(log 3 n) using O( n log n log log n) processors. The algorithm keeps all its advantages being applied to the inversion of Toeplitzlike matrices, having smaller displacement ranks (which makes it applicable to pseudo-inversion of Toeplitzlike matrices of full rank).

Measuring Risk in Fixed Payment Securities: An Empirical Test of the Structured Full Rank Covariance Matrix

The appropriate set of parameters determining the volatility of the value of a portfolio of fixed cash flows of arbitrary maturities is the covariance matrix of unexpected interest rate changes over the term. Equilibrium models of the term structure limit the rank of the covariance matrix and implicitly impose restrictions on covariance estimation. The “full information” approach to risk measurement imposes only time stationarity assumptions on covariance matrix estimators and can result in sample matrices of full rank. Hilliard and Jordan (1989) develop a structured full rank covariance matrix that depends on only two parameters. This paper tests the Hilliard-Jordan model using likelihood ratios and criteria of forecast accuracy.

Tensorial resolution: A direct trilinear decomposition

AbstractModern instrumentation in chemistry routinely generates two‐dimensional (second‐order) arrays of data. Considering that most analyses need to compare several samples, the analyst ends up with a three‐dimensional (third‐order) array which is difficult to visualize or interpret with the conventional statistical tools.Some of these data arrays follow the so‐called trilinear model, These trilinear arrays of data are known to have unique factor analysis decompositions which correspond to the true physical factors that form the data, i.e. given the array ℝ, a unique solution can be found in many cases for each order X, Y and Z. This is in contrast to the well‐known second‐order bilinear data factor analysis, where the abstract solutions obtained are not unique and at best cannot be easily compared with the underlying physical factors owing to a rotational ambiguity.Trilinear decompositions have had the disadvantage, however, that a non‐linear optimization with many parameters is necessary to reach a least‐squares solution. This paper will introduce a method for reducing the problem to a rectangular generalized eigenvalue–eigenvector equation where the eigenvectors are the contravariant form (pseudo‐inverse) of the actual factors. It is shown that the method works well when the factors are linearly independent in at least two orders (e.g. Xir and Yjr are full rank matrices).Finally, it is shown how trilinear decompositions relate to multicomponent calibration, curve resolution and chemical analysis.

An overview of parallel algorithms for the singular value and symmetric eigenvalue problems

Solving dense symmetric eigenvalue problems and computing singular value decompositions continue to be two of the most dominating tasks in numerous scientific applications. With the advent of multiprocessor computer systems, the design of efficient parallel algorithms to determine these solutions becomes of paramount importance. In this paper, we discuss two fundamental approaches for determining the eigenvalues and eigenvectors of dense symmetric matrices on machines such as the Alliant FX/8 and CRAY X-MP. One approach capitalizes upon the inherent parallelism offered by Jacobi methods, while the other relies upon an efficient reduction to tridiagonal form via Householder's transformations followed by a multisectioning technique to obtain the eigenvalues and eigenvectors of the corresponding symmetric tridiagonal matrix. For the singular value decomposition, we discuss an efficient method for rectangular matrices in which the number of rows is substantially larger or smaller than the number of columns. This scheme performs an initial orthogonal factorization using block Householder transformation, followed by a parallel one-sided Jacobi method to obtain the singular values and singular vectors of the resulting upper-triangular matrix. Exceptional performance for this SVD scheme is demonstrated for tall matrices of full or deficient rank having clustered or multiple singular values. A hybrid method that combines one- and two-sided Jacobi schemes is also discussed. Performance results for each of the above algorithms on the Alliant FX/8 and CRAY X-MP computer systems will be presented with particular emphasis given to speedups obtained over such classical EISPACK algorithms.

Fast Toeplitz Orthogonalization Using Inner Products

A new method for the orthogonalization of complex m by n Toeplitz matrices of full rank is presented. An inverse $QR$ factorization is computed in $9mn + (27/2)n^2 $ multiplications and divisions. This method uses inner products and projections in the same spirit as lattice algorithms for linear prediction do.