Sum Of Singular Values Research Articles

Measuring Dependencies between Variables of a Dynamical System Using Fuzzy Affiliations

A statistical, data-driven method is presented that quantifies influences between variables of a dynamical system. The method is based on finding a suitable representation of points by fuzzy affiliations with respect to landmark points using the Scalable Probabilistic Approximation algorithm. This is followed by the construction of a linear mapping between these affiliations for different variables and forward in time. This linear mapping, or matrix, can be directly interpreted in light of unidirectional dependencies, and relevant properties of it are quantified. These quantifications, given by the sum of singular values and the average row variance of the matrix, then serve as measures for the influences between variables of the dynamics. The validity of the method is demonstrated with theoretical results and on several numerical examples, covering deterministic, stochastic, and delayed types of dynamics. Moreover, the method is applied to a non-classical example given by real-world basketball player movement, which exhibits highly random movement and comes without a physical intuition, contrary to many examples from, e.g., life sciences.

A holey cavity for single-transducer 3D ultrasound imaging with physical optimization

Within the compressive sensing (CS) framework, one effective way to increase the likelihood of successful signal reconstruction is to employ random processes in the construction of the sensing matrix. This study presents a 3D holey cavity, with diverse frequency modes, to spectrally code, that is, randomize, the ultrasound wave fields. The simulated results show that the use of such a cavity enables imaging simple or complex targets, such as spheres or the letter E, by only a single transceiver—something that is not possible without the use of a coding structure like the cavity. The effect of noise on imaging results and the size of the targets on the first-order Born approximation (BA) are also investigated. Moreover, this study attempts to optimize the cavity, based on a single numerical metric, such as the sum of singular values (SSV) or mutual coherence (MC). Yet, it will be shown that neither of these metrics can consistently compare the norm-one imaging performance between two cavities of different materials or hole sizes. This leaves finding a quantitative metric for these kinds of optimizations an open problem.

NOWNUNM: Nonlocal Weighted Nuclear Norm Minimization for Sparse-Sampling CT Reconstruction

Computed tomography (CT) image reconstruction using classical total variation (TV)-based methods or its variations inevitably suffers from a blocky effect when the sampling number is low, because of the piecewise assumption. A low-rank based method is an effective way to circumvent this side effect. Normally, a nuclear norm is used to impose the low rank constraint, and its numerical computation depends on the sum of singular values, which are calculated by singular value decomposition. However, as larger singular values mainly deliver the structural information, treating all the singular values equally may lead to imperfect preservation of edges and textures. To deal with this problem, we here propose to reconstruct the CT image by explicitly exploring the nonlocal similarity in the target image with nonlocal weighted nuclear norm minimization. First, a matrix is constructed by grouping nonlocal patches similar to the current patch. Then, the original nuclear norm minimization is replaced by a weighted version, which treats the singular values differently according to their magnitudes. By doing this, we can eliminate noise and streak artifacts without introducing any side effects. The corresponding numerical algorithm is given by an alternating optimization strategy. Experimental results demonstrate that our method outperforms several existing reconstruction methods in both qualitative and quantitative aspects, including filtered back projection, TV, and total generalized variation methods.

Hankel Low-Rank Matrix Completion: Performance of the Nuclear Norm Relaxation

The completion of matrices with missing values under the rank constraint is a nonconvex optimization problem. A popular convex relaxation is based on minimization of the nuclear norm (sum of singular values) of the matrix. For this relaxation, an important question is whether the two optimization problems lead to the same solution. This question was addressed in the literature mostly in the case of random positions of missing elements and random known elements. In this contribution, we analyze the case of structured matrices with a fixed pattern of missing values, namely, the case of Hankel matrix completion. We extend existing results on completion of rank-one real Hankel matrices to completion of rank-r complex Hankel matrices.

Minimax risk of matrix denoising by singular value thresholding

An unknown $m$ by $n$ matrix $X_0$ is to be estimated from noisy measurements $Y=X_0+Z$, where the noise matrix $Z$ has i.i.d. Gaussian entries. A popular matrix denoising scheme solves the nuclear norm penalization problem $\operatorname {min}_X\|Y-X\|_F^2/2+\lambda\|X\|_*$, where $\|X\|_*$ denotes the nuclear norm (sum of singular values). This is the analog, for matrices, of $\ell_1$ penalization in the vector case. It has been empirically observed that if $X_0$ has low rank, it may be recovered quite accurately from the noisy measurement $Y$. In a proportional growth framework where the rank $r_n$, number of rows $m_n$ and number of columns $n$ all tend to $\infty$ proportionally to each other ($r_n/m_n\rightarrow \rho$, $m_n/n\rightarrow \beta$), we evaluate the asymptotic minimax MSE $\mathcal {M}(\rho,\beta)=\lim_{m_n,n\rightarrow \infty}\inf_{\lambda}\sup_{\operatorname {rank}(X)\leq r_n}\operatorname {MSE}(X_0,\hat{X}_{\lambda})$. Our formulas involve incomplete moments of the quarter- and semi-circle laws ($\beta=1$, square case) and the Mar\v{c}enko-Pastur law ($\beta<1$, nonsquare case). For finite $m$ and $n$, we show that MSE increases as the nonzero singular values of $X_0$ grow larger. As a result, the finite-$n$ worst-case MSE, a quantity which can be evaluated numerically, is achieved when the signal $X_0$ is "infinitely strong." The nuclear norm penalization problem is solved by applying soft thresholding to the singular values of $Y$. We also derive the minimax threshold, namely the value $\lambda^*(\rho)$, which is the optimal place to threshold the singular values. All these results are obtained for general (nonsquare, nonsymmetric) real matrices. Comparable results are obtained for square symmetric nonnegative-definite matrices.

The phase transition of matrix recovery from Gaussian measurements matches the minimax MSE of matrix denoising

Let X(0) be an unknown M by N matrix. In matrix recovery, one takes n < MN linear measurements y(1),…,y(n) of X(0), where y(i) = Tr(A(T)iX(0)) and each A(i) is an M by N matrix. A popular approach for matrix recovery is nuclear norm minimization (NNM): solving the convex optimization problem min ||X||*subject to y(i) =Tr(A(T)(i)X) for all 1 ≤ i ≤ n, where || · ||* denotes the nuclear norm, namely, the sum of singular values. Empirical work reveals a phase transition curve, stated in terms of the undersampling fraction δ(n,M,N) = n/(MN), rank fraction ρ=rank(X0)/min {M,N}, and aspect ratio β=M/N. Specifically when the measurement matrices Ai have independent standard Gaussian random entries, a curve δ*(ρ) = δ*(ρ;β) exists such that, if δ > δ*(ρ), NNM typically succeeds for large M,N, whereas if δ < δ*(ρ), it typically fails. An apparently quite different problem is matrix denoising in Gaussian noise, in which an unknown M by N matrix X(0) is to be estimated based on direct noisy measurements Y =X(0) + Z, where the matrix Z has independent and identically distributed Gaussian entries. A popular matrix denoising scheme solves the unconstrained optimization problem min|| Y-X||(2)(F)/2+λ||X||*. When optimally tuned, this scheme achieves the asymptotic minimax mean-squared error M(ρ;β) = lim(M,N → ∞)inf(λ)sup(rank(X) ≤ ρ · M)MSE(X,X(λ)), where M/N → . We report extensive experiments showing that the phase transition δ*(ρ) in the first problem, matrix recovery from Gaussian measurements, coincides with the minimax risk curve M(ρ)=M(ρ;β) in the second problem, matrix denoising in Gaussian noise: δ*(ρ)=M(ρ), for any rank fraction 0 < ρ < 1 (at each common aspect ratio β). Our experiments considered matrices belonging to two constraint classes: real M by N matrices, of various ranks and aspect ratios, and real symmetric positive-semidefinite N by N matrices, of various ranks.

Oriented incidence energy and threshold graphs

Let G be a simple graph with n vertices and m edges. Let edges of G be given an arbitrary orientation, and let Q be the vertex-edge incidence matrix of such oriented graph. The oriented incidence energy of G is then the sum of singular values of Q. We show that for any n?9, there exists at least ([n/9]/2)+1 distinct pairs of graphs on n vertices having equal oriented incidence energy.

Interior-Point Method for Nuclear Norm Approximation with Application to System Identification

The nuclear norm (sum of singular values) of a matrix is often used in convex heuristics for rank minimization problems in control, signal processing, and statistics. Such heuristics can be viewed as extensions of $\ell_1$-norm minimization techniques for cardinality minimization and sparse signal estimation. In this paper we consider the problem of minimizing the nuclear norm of an affine matrix-valued function. This problem can be formulated as a semidefinite program, but the reformulation requires large auxiliary matrix variables, and is expensive to solve by general-purpose interior-point solvers. We show that problem structure in the semidefinite programming formulation can be exploited to develop more efficient implementations of interior-point methods. In the fast implementation, the cost per iteration is reduced to a quartic function of the problem dimensions and is comparable to the cost of solving the approximation problem in the Frobenius norm. In the second part of the paper, the nuclear norm approximation algorithm is applied to system identification. A variant of a simple subspace algorithm is presented in which low-rank matrix approximations are computed via nuclear norm minimization instead of the singular value decomposition. This has the important advantage of preserving linear matrix structure in the low-rank approximation. The method is shown to perform well on publicly available benchmark data.

Consistency of Trace Norm Minimization

Regularization by the sum of singular values, also referred to as the trace norm, is a popular technique for estimating low rank rectangular matrices. In this paper, we extend some of the consisten...

Applications of Wigner’s theorem to positive maps preserving norm of operator products

We apply Wigner’s theorem to positive maps on standard operator algebras that preserve norm of operator products or sum of singular values of operator products. It follows that such preservers are of the form ϕ(A) = U AU* with U either a unitary or antiunitary operator.

A property of internally balanced and low noise structures

Given an <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</tex> th-order minimal asymptotically stable system <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">H(z)</tex> , we consider the class <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e</tex> of minimal state-space realizations which minimize the trace of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">\gamma^{2}K + W</tex> , where <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</tex> is the controllability and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">W</tex> the observability Grammian. We show that the optimal structures are defined by <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">W = \gamma^{2}K</tex> with optimal cost <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2\gammaS</tex> where <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">S</tex> is the sum of singular values. For <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">\gamma = 1</tex> , the internally balanced structure belongs to <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e</tex> while if <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">\gamma = S/ N</tex> , the optimal noise structures of Mullis and Roberts [1] belong to <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e</tex> . In particular, both structures are optimal when <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">\gamma = 1</tex> and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">S = N</tex> .