Spectral Norm Research Articles

LOWER BOUNDS FOR COLUMN MATRIX APPROXIMATIONS

We show a connection between lower and upper bounds on the column matrix approximation accuracy and the bounds on the norms of the pseudoinverses of the submatrices of orthogonal matrices. This connection is exploited to derive lower bounds for column approximations accuracy in spectral and Frobenius norms.

On the Properties of r-Circulant Matrices Involving Generalized Fermat Numbers

-circulant matrices have applied in numerical computation, signal processing, coding theory, etc. In this study, our main goal is to investigate the r-circulant matrices of generalized Fermat numbers which are shown by We obtain the eigenvalues, determinants, sum identity of matrices. Also we find upper and lower bounds for the spectral norms of generalized Fermat r-circulant matrices. Beside these, we present 〖GR〗_(a,b,r)^* matrix in the form of the Hadamard product of two matrices as 〖GR〗_(a,b,r)^*=A.B. In addition, we get the right and skew-right circulant matrices for . Finally, we examine their different norms (Spectral and Euclidean) and limits for matrix norms.

Cross-modal challenging: Projection of brain response on stereoscopic image quality ranking

This work presents a novel cross-modality method for acquiring human visual perception on stereoscopic image quality ranking (SIQR), aiming to learn latent biological representations and thinking tokens from brain activity. The core idea is to directly project electroencephalogram (EEG) information onto a SIQR neural network model, rather than simulate the complex neuronal connections. To the end, we establish a multimodal brain-visual dataset, and propose a Cross-GAN model including a binocular attention-based fusion (BAF) image modality encoder, a multiscale spatiotemporal (MST) EEG modality encoder and a multimodal feature generation GAN (MF-GAN). In the BAF image encoder, the progressive interactive attention fusion is designed to highlight the significant regions of the image from monocular view to binocular view, which contributes to the global information description of single view and fused view. In the MST EEG encoder, spectral norm normalization constraint is adopted to assure the Lipschitz continuity, and the multiscale convolution blocks is presented to capture EEG representations from spatiotemporal perspective. Finally, the monomodal image and EEG features are sent to the MF-GAN for creating the optimal brain-visual manifold under the control of the generative loss function. Extensive experiments on the brain-visual multimodality SIQR database prove that the CrossGAN can improve the performance via projecting brain response onto image evaluation model in a cross-modal manner, with no need of EEG trails in application.

Decay estimates of Green's matrices for discrete-time linear periodic systems

We study periodic Lyapunov matrix equations for a general discrete-time linear periodic system B p x p − A p x p − 1 = f p , where the matrix coefficients B p and A p can be singular. The block coefficients of the inverse operator of the system are referred to as the Green matrices. We derive new decay estimates of the Green matrices in terms of the spectral norms of special solutions to the periodic Lyapunov matrix equations. The study is based on the periodic Schur decomposition of matrices.

On the optimal rank-1 approximation of matrices in the Chebyshev norm

The problem of low rank approximation is ubiquitous in science. Traditionally this problem is solved in unitary invariant norms such as Frobenius or spectral norm due to existence of efficient methods for building approximations. However, recent results reveal the potential of low rank approximations in Chebyshev norm, which naturally arises in many applications. In this paper we tackle the problem of building optimal rank-1 approximations in the Chebyshev norm. We investigate the properties of alternating minimization algorithm for building the low rank approximations and demonstrate how to use it to construct optimal rank-1 approximation. As a result we propose an algorithm that is capable of building optimal rank-1 approximations in Chebyshev norm for moderate matrices.

A Study on the Norms of Toeplitz Matrices with the Generalized Mersenne Numbers

In this article presents findings related to Toeplitz matrices with Mersenne numbers. We started by creating Toeplitz matrices whose components are Mersenne numbers. We then calculated the Euclidian, row and column norms of these matrices and established both lower and upper bounds for their spectral norms. Additionally, we determined the upper bounds for the Frobenius and spectral norms of the Kronecker and Hadamard product matrices of the Toeplitz matrices with Mersenne numbers.

Low‐rank updates of matrix square roots

AbstractModels in which the covariance matrix has the structure of a sparse matrix plus a low rank perturbation are ubiquitous in data science applications. It is often desirable for algorithms to take advantage of such structures, avoiding costly matrix computations that often require cubic time and quadratic storage. This is often accomplished by performing operations that maintain such structures, for example, matrix inversion via the Sherman–Morrison–Woodbury formula. In this article, we consider the matrix square root and inverse square root operations. Given a low rank perturbation to a matrix, we argue that a low‐rank approximate correction to the (inverse) square root exists. We do so by establishing a geometric decay bound on the true correction's eigenvalues. We then proceed to frame the correction as the solution of an algebraic Riccati equation, and discuss how a low‐rank solution to that equation can be computed. We analyze the approximation error incurred when approximately solving the algebraic Riccati equation, providing spectral and Frobenius norm forward and backward error bounds. Finally, we describe several applications of our algorithms, and demonstrate their utility in numerical experiments.

The first three largest values of the spectral norm of oriented bicyclic graphs

The first three largest values of the spectral norm of oriented bicyclic graphs

Post-processed posteriors for sparse covariances

We consider Bayesian inference of sparse covariance matrices and propose a post-processed posterior. This method consists of two steps. In the first step, posterior samples are obtained from the conjugate inverse-Wishart posterior without considering the sparse structural assumption. The posterior samples are transformed in the second step to satisfy the sparse structural assumption through a generalized thresholding function. This non-traditional Bayesian procedure is justified by showing that the post-processed posterior attains the optimal minimax rates under the spectral norm loss in high-dimensional settings. We also propose the post-processed posterior for contaminated data and apply it to the estimation of the sparse idiosyncratic covariance of the approximate factor model. The advantages of our method are demonstrated via a simulation study and a real data analysis with S&P 500 data.

Optimal linear array orientation design for 3D direct position determination via semi-Definite relaxation

This paper presents an optimization strategy for array orientations in a three-dimensional (3D) direct position determination (DPD) system. Specifically, we consider a scenario in which uniform linear arrays (ULAs) are used to locate emitters, and we seek to optimize the array orientations in terms of localization accuracy. The E-optimality criterion, which minimizes the spectral norm of the Cramér-Rao lower bound (CRLB), is exploited to formulate this problem. As the objective function is non-convex with bilinear structures, we leverage semi-definite relaxation (SDR) to transform it into a convex semi-definite programming (SDP) problem by substituting the bilinear terms with a matrix variable. In addition, we tackle the array orientation configuration problem in the context of multi-emitter scenarios and develop an SDR solution for large-scale antenna array systems. Simulation results validate that the ULAs with array orientations designed by the proposed SDR-based method have near-optimal localization performance.

Extreme Ratio Between Spectral and Frobenius Norms of Nonnegative Tensors

One of the fundamental problems in multilinear algebra, the minimum ratio between the spectral and Frobenius norms of tensors, has received considerable attention in recent years. While most values are unknown for real and complex tensors, the asymptotic order of magnitude and tight lower bounds have been established. However, little is known about nonnegative tensors. In this paper, we present an almost complete picture of the ratio for nonnegative tensors. In particular, we provide a tight lower bound that can be achieved by a wide class of nonnegative tensors under a simple necessary and sufficient condition, which helps to characterize the extreme tensors and obtain results such as the asymptotic order of magnitude. We show that the ratio for symmetric tensors is no more than that for general tensors multiplied by a constant depending only on the order of tensors, hence determining the asymptotic order of magnitude for real, complex, and nonnegative symmetric tensors. We also find that the ratio is in general different from the minimum ratio between the Frobenius and nuclear norms for nonnegative tensors, a sharp contrast to the case for real tensors and complex tensors.

Spectral complexity-scaled generalisation bound of complex-valued neural networks

Complex-valued neural networks (CVNNs) have been widely applied in various fields, primarily in signal processing and image recognition. Few studies have focused on the generalisation of CVNNs, although it is vital to ensure the performance of CVNNs on unseen data. This study is the first to prove a generalisation bound for complex-valued neural networks. The bounds increase as the spectral complexity increases, with the dominant factor being the product of the spectral norms of the weight matrices. Furthermore, this work provides a generalisation bound for CVNNs trained on sequential data, which is also affected by the spectral complexity. Theoretically, these bounds are derived using the Maurey Sparsification Lemma and Dudley entropy integral. We conducted empirical experiments on various datasets including MNIST, ashionMNIST, CIFAR-10, CIFAR-100, Tiny ImageNet, and IMDB by training complex-valued convolutional neural networks. The Spearman rank-order correlation coefficient and the corresponding p-values on these datasets provide strong proof of the statistically significant correlation between the spectral complexity of a network and its generalisation ability, as measured by the spectral norm product of the weight matrices. The code is available at https://github.com/LeavesLei/cvnn_generalization.

Hypothesis testing on compound symmetric structure of high-dimensional covariance matrix

Testing the population covariance matrix is an important topic in multivariate statistical analysis. Owing to the difficulty of establishing the central limit theorem for the test statistic based on sample covariance matrix, in most of the existing literature, it is assumed that the population covariance matrix is bounded in spectral norm as the dimension tends to infinity. Four test statistics are proposed for testing the compound symmetric structure of the population covariance matrix, which has an unbounded spectral norm as the dimension tends to infinity. Three of these tests maintain high power against different kinds of dense alternatives. The asymptotic properties of these statistics are constructed under the null hypothesis and a specific alternative hypothesis. Moreover, extensive simulation studies and an analysis of real data are conducted to evaluate the performance of our proposed tests. The simulation results show that the proposed tests outperform existing methods.

Conditioning Theory for Generalized Inverse CA‡ and Their Estimations

The conditioning theory of the generalized inverse CA‡ is considered in this article. First, we introduce three kinds of condition numbers for the generalized inverse CA‡, i.e., normwise, mixed and componentwise ones, and present their explicit expressions. Then, using the intermediate result, which is the derivative of CA‡, we can recover the explicit condition number expressions for the solution of the equality constrained indefinite least squares problem. Furthermore, using the augment system, we investigate the componentwise perturbation analysis of the solution and residual of the equality constrained indefinite least squares problem. To estimate these condition numbers with high reliability, we choose the probabilistic spectral norm estimator to devise the first algorithm and the small-sample statistical condition estimation method for the other two algorithms. In the end, the numerical examples illuminate the obtained results.

New multiplicative perturbation bounds on orthogonal projection

New multiplicative perturbation bounds for orthogonal projection in a general unitarily invariant norm, the Q-norm and the spectral norm are derived. These bounds always improve the existing bounds. An example is given to show that these new bounds are optimal.

On geometric circulant matrices with geometric sequence

In this paper, we consider a geometric circulant matrix with geometric sequence i.e. a geometric circulant matrix whose first row is ( g , gq , g q 2 , … , g q n − 1 ) , where g is a nonzero complex number and q is a nonzero real number. The determinant, the Euclidean norm and bounds for the spectral norm of such matrix are obtained. Also, in the case when such matrix is singular, we obtain the Moore-Penrose inverse of such matrix and show that the group inverse of such matrix does not exist (in most cases). The obtained results are illustrated by examples.

Horofunction Compactifications and Duality

We study the global topology and geometry of the horofunction compactification of classes of symmetric spaces under Finsler distances in three settings: bounded symmetric domains of the form B=B_1times cdots times B_r, where B_i is an open Euclidean ball in {mathbb {C}}^{n_i}, with the Kobayashi distance, symmetric cones with the Hilbert distance, and Euclidean Jordan algebras with the spectral norm. For these spaces we show, that the horofunction compactification is naturally homeomorphic to the closed unit ball of the dual norm of the Finsler metric in the tangent space at the basepoint. In each case we give an explicit homeomorphism. For finite dimensional normed spaces the link between the geometry of the horofunction compactification and the dual unit ball was suggested by Kapovich and Leeb, which we confirm for Euclidean Jordan algebras with the spectral norm. Our results also show that this duality phenomenon not only occurs in normed spaces, but also in a variety of noncompact type symmetric spaces with invariant Finsler metrics.

Inapproximability of Matrix \(\boldsymbol{p \rightarrow q}\) Norms

We study the problem of computing the norm of a matrix , defined as . This problem generalizes the spectral norm of a matrix and the Grothendieck problem ( , ) and has been widely studied in various regimes. When , the problem exhibits a dichotomy: constant factor approximation algorithms are known if , and the problem is hard to approximate within almost polynomial factors when . The regime when , known as hypercontractive norms, is particularly significant for various applications but much less well understood. The case with and was studied by Barak et al. [Proceedings of the 44th Annual ACM Symposium on Theory of Computing, 2012, pp. 307–326], who gave subexponential algorithms for a promise version of the problem (which captures small-set expansion) and also proved hardness of approximation results based on the exponential time hypothesis. However, no NP-hardness of approximation is known for these problems for any . We prove the first NP-hardness result (under randomized reductions) for approximating hypercontractive norms. We show that for any with , is hard to approximate within assuming . En route to the above result, we also prove almost tight results for the case when with .

Inequalities for products of singular values of matrices

Let A and B be positive semidefinite matrices, be an increasing convex function with , and let be a positive unital linear map. It is shown that for , where and are the singular value and the spectral norm of T, respectively. Applications of our results to sectorial matrices are given.

Bootstrapping the operator norm in high dimensions: Error estimation for covariance matrices and sketching

Although the operator (spectral) norm is one of the most widely used metrics for covariance estimation, comparatively little is known about the fluctuations of error in this norm. To be specific, let Σˆ denote the sample covariance matrix of n i.i.d. observations in Rp that arise from a population matrix Σ, and let Tn=n‖Σˆ−Σ‖ op. In the setting where the eigenvalues of Σ have a decay profile of the form λj(Σ)≍j−2β, we analyze how well the bootstrap can approximate the distribution of Tn. Our main result shows that up to factors of log(n), the bootstrap can approximate the distribution of Tn with respect to the Kolmogorov metric at the rate of n−β−1∕2 6β+4, which does not depend on the ambient dimension p. In addition, we offer a supporting result of independent interest that establishes a high-probability upper bound for Tn based on flexible moment assumptions. More generally, we discuss the consequences of our work beyond covariance matrices, and show how the bootstrap can be used to estimate the errors of sketching algorithms in randomized numerical linear algebra (RandNLA). An illustration of these ideas is also provided with a climate data example.