Largest Singular Value Research Articles

Robust generalized bilinear model with weighted low-rank representation for hyperspectral image unmixing

The generalized bilinear model (GBM) has been one of the most representative models for nonlinear unmixing of hyperspectral images (HSI), which can consider the second-order scattering of photons. Recently, robust GBM-low-rank representation (RGBM-LRR) for nonlinear unmixing of HSI has been introduced to capture the spatial correlation of HSI using LRR with nuclear norm minimization (NNM). However, NNM is used to approximate the matrix rank by shrinkage all singular values equally. The singular values have distinct physical significance in many real applications, and NNM may not be able to estimate the matrix rank accurately. To overcome the above issue, a robust GBM with a weighted low-rank representation (RGBM-WLRR) approach is proposed using weighted nuclear norm minimization, which mitigates the penalty on larger singular values by assigning a small weight, so the corresponding shrinkage is small and also takes serious shrinkage on small singular values by assigning larger weights to them. The proposed model is solved using an iterative alternating direction method of the multipliers. A series of experiments with real datasets and a simulated HSI with varying rank and signal-to-noise ratio reveals that RGBM-WLRR performs significantly better than the state-of-the-art algorithms in terms of signal-to-reconstruction error, root-mean-square error, and spectral angle distance.

Nonlinear subspace clustering using non-convex Schatten-p norm regularization

Subspace clustering aims to seek a multi-subspace representation that is best suitable for data points taken from a high-dimensional space. Sparse representation and low-rank approximation-based methods have become one of the main melodies for subspace clustering. In the existing methods, nuclear norm is used to approximate rank minimization. However, the common deficiency still exists for nuclear norm, which always over-penalizes large singular values and results in a biased solution. In this paper, we propose a nonlinear subspace clustering model that exploits sparsity and low-rank of data in high dimensional feature space by using Schatten-p norm surrogate ([Formula: see text]) with learned low-rank kernel. By this manner, the model guarantees that the data mapped in the high-dimensional feature spaces is lower rank and self-expressive. And we show the alternating direction method of multipliers (abbreviated as ADMM) for the corresponding problem in a reproducing kernel Hilbert space. Various experiments on motion segmentation and image clustering display that the proposed model has potentiality in outperforming most of state-of-the-art models in current literature.

Weighted Schatten [formula omitted]-norm minimization with logarithmic constraint for subspace clustering

Rank minimization-based subspace clustering methods have been widely developed in the past decades. Although some smooth surrogates, such as the nuclear norm and Schatten-p norm mitigate the NP-hard issue to some extend, these existing methods may yield unsatisfactory results, due to the singular values of the coefficient matrix not being further suppressed. To tackle this, in this paper, we propose a novel non-convex low-rank approximation based on weighted Schatten-p norm jointed logarithmic constraint, which can suppress the small and large singular values flexibly with a tighter relaxation. Specifically, we firstly proposed a low-rank approximation termed SLog by utilizing the logarithmic to tighten the Schatten-p norm, which can shrink the large singular values in a similar trend of the real rank minimization. Furthermore, to suppress the small singular values simultaneously, usually considered noise, we propose a weighted Schatten p-norm minimization named WSLog based on SLog by introducing the weight w, which behaves more robust to the sparse noise, especially w<1. Compared with recent-proposed methods, extensive experiments in subspace clustering on real datasets demonstrate the effective performance of our methods.

Tensor decomposition based on the potential low-rank and p-shrinkage generalized threshold algorithm for analyzing cancer multiomics data.

Tensor Robust Principal Component Analysis (TRPCA) has achieved promising results in the analysis of genomics data. However, the TRPCA model under the existing tensor singular value decomposition ([Formula: see text]-SVD) framework insufficiently extracts the potential low-rank structure of the data, resulting in suboptimal restored components. Simultaneously, the tensor nuclear norm (TNN) defined based on [Formula: see text]-SVD uses the same standard to handle various singular values. TNN ignores the difference of singular values, leading to the failure of the main information that needs to be well preserved. To preserve the heterogeneous structure in the low-rank information, we propose a novel TNN and extend it to the TRPCA model. Potential low-rank space may contain important information. We learn the low-rank structural information from the core tensor. The singular value space contains the association information between genes and cancers. The [Formula: see text]-shrinkage generalized threshold function is utilized to preserve the low-rank properties of larger singular values. The optimization problem is solved by the alternating direction method of the multiplier (ADMM) algorithm. Clustering and feature selection experiments are performed on the TCGA data set. The experimental results show that the proposed model is more promising than other state-of-the-art tensor decomposition methods.

Patch-based weighted SCAD prior for compressive sensing

The nuclear norm-based convex surrogate of the rank function has been widely used in compressive sensing (CS) to exploit the sparsity of nonlocal similar patches in an image. However, this method treats different singular values equally and thus may produce a result far from the optimum one. In order to alleviate the limitations of the nuclear norm, different singular values should be treated differently. The reason is that large singular values can be used to retrieve substantial contents of an image, while small ones may contain noisy information. In this paper, we propose a model via non-convex weighted Smoothly Clipped Absolute Deviation (SCAD) prior. Our motivation is that SCAD shrinkage behaves like a soft shrinkage operator for small enough inputs, whereas for large enough ones, it leaves the input intact and behaves like hard shrinkage. For moderate input values, SCAD makes a good balance between soft shrinkage and hard shrinkage. Numerically, the alternating direction method of multiplier (ADMM) is adopted to split the original problem into several sub-problems with closed-form solutions. We further analyze the convergence of the proposed method under mild conditions. Various experimental results demonstrate that the proposed model outperforms many existing state-of-the-art CS methods.

Tikhonov regularized penalty matrix construction method based on the magnitude of singular values and its application in near-field acoustic holography

Tikhonov regularization improves the ill-conditioning of the original transfer matrix by using regularization parameters and a penalty matrix. When the penalty matrix is a unit matrix, Tikhonov regularization is known as ridge estimation. Using the mean value and variance analysis of ridge estimation, it can be observed that the ridge estimation uses a regularization parameter to filter small singular values, and introduces system bias independent of the measurement error. However, due to the large order of magnitude difference between the singular values, the fixed regularization parameter cannot effectively suppress the variance component corresponding to large singular values. As a result, the variance of the estimation increases and the stability of the solution reduces. In view of the above defects, a method is proposed in this paper to construct a regularized penalty matrix based on the magnitude of singular values. Compared with the unit matrix used in ridge estimation, the constructed penalty matrix can introduce an appropriate amount of bias according to the singular values. This reduces the variance of parameter estimation under the condition that the system bias remains approximately unchanged. Finally, the constructed penalty matrix is applied to the source strength solution and sound field reconstruction of equivalent source near-field acoustic holography. The results show that the regularization method using the new penalty matrix enhances the stability of the source strength solution and improves the reconstruction accuracy of the sound field.

Tensor completion via nonconvex tensor ring rank minimization with guaranteed convergence

In recent studies, the tensor ring (TR) rank has shown high effectiveness in tensor completion due to its ability of capturing the intrinsic structure within high-order tensors. A recently proposed TR rank minimization method is based on the convex relaxation by penalizing the weighted sum of nuclear norm of TR unfolding matrices. However, this method treats each singular value equally and neglects their physical meanings, which usually leads to suboptimal solutions in practice. To alleviate this weakness, this paper proposes an enhanced low-rank sparsity measure, which can more accurately approximate the TR rank and better promote the low-rankness of the solution. In specific, we apply the logdet function onto TR unfolding matrices to shrink less the larger singular values while shrink more the smaller ones. To solve the proposed nonconvex model efficiently, we develop an alternating direction method of multipliers algorithm and theoretically prove that, under some mild assumptions, our algorithm converges to a stationary point. Extensive experiments on color images, multispectral images, and color videos demonstrate that the proposed method outperforms several state-of-the-art competitors in both visual and quantitative comparison.

Conical Statistical Optimal Near-Field Acoustic Holography with Combined Regularization.

For the sound field reconstruction of large conical surfaces, current statistical optimal near-field acoustic holography (SONAH) methods have relatively poor applicability and low accuracy. To overcome this problem, conical SONAH based on cylindrical SONAH is proposed in this paper. Firstly, elementary cylindrical waves are transformed into those suitable for the radiated sound field of the conical surface through cylinder-cone coordinates transformation, which forms the matrix of characteristic elementary waves in the conical spatial domain. Secondly, the sound pressure is expressed as the superposition of those characteristic elementary waves, and the superposition coefficients are solved according to the principle of superposition of wave field. Finally, the reconstructed conical pressure is expressed as a linear superposition of the holographic conical pressure. Furthermore, to overcome ill-posed problems, a regularization method combining truncated singular value decomposition (TSVD) and Tikhonov regularization is proposed. Large singular values before the truncation point of TSVD are not processed and remaining small singular values representing high-frequency noise are modified by Tikhonov regularization. Numerical and experimental case studies are carried out to validate the effectiveness of the proposed conical SONAH and the combined regularization method, which can provide reliable evidence for noise monitoring and control of mechanical systems.

Efficient video hashing based on low‐rank frames

Video hashing is a useful technology for diverse video applications, such as digital watermarking, copy detection and content authentication. This paper proposes a novel efficient video hashing based on low-rank frames. A key contribution is the low-rank frame calculation using the low-rank approximation of singular value decomposition (SVD). As the large singular values of SVD are stable to digital operations, video hash extraction using low-rank frames can provide good robustness. Since most energy is contained within the large singular values, low-rank frames also contribute to discrimination. Moreover, two-dimensional discrete wavelet transform (DWT) is applied to every low-rank frame and the mean of low-frequency DWT coefficients is selected as a hash element. Since these coefficients can represent input data approximately, hash discrimination is thus ensured. Experiments with 16,850 videos are carried out to test performances of the proposed algorithm. The results show that the proposed algorithm outperforms some well-known video hashing algorithms in computational time and classification about discrimination and robustness.

Perturbation expansions and error bounds for the truncated singular value decomposition

Truncated singular value decomposition is a reduced version of the singular value decomposition in which only a few largest singular values are retained. This paper presents a novel perturbation analysis for the truncated singular value decomposition for real matrices. First, we describe perturbation expansions for the singular value truncation of order r. We extend perturbation results for the singular subspace decomposition to derive the first-order perturbation expansion of the truncated operator about a matrix with rank greater than or equal to r. Observing that the first-order expansion can be greatly simplified when the matrix has exact rank r, we further show that the singular value truncation admits a simple second-order perturbation expansion about a rank-r matrix. Second, we introduce the first-known error bound on the linear approximation of the truncated singular value decomposition of a perturbed rank-r matrix. Our bound only depends on the least singular value of the unperturbed matrix and the norm of the perturbation matrix. Intriguingly, while the singular subspaces are known to be extremely sensitive to additive noises, the newly established error bound holds universally for perturbations with arbitrary magnitude. Finally, we demonstrate an application of our results to the analysis of the mean squared error associated with the TSVD-based matrix denoising solution.

Enhanced Tensor RPCA and its Application

Despite the promising results, tensor robust principal component analysis (TRPCA), which aims to recover underlying low-rank structure of clean tensor data corrupted with noise/outliers by shrinking all singular values equally, cannot well preserve the salient content of image. The major reason is that, in real applications, there is a salient difference information between all singular values of a tensor image, and the larger singular values are generally associated with some salient parts in the image. Thus, the singular values should be treated differently. Inspired by this observation, we investigate whether there is a better alternative solution when using tensor rank minimization. In this paper, we develop an enhanced TRPCA (ETRPCA) which explicitly considers the salient difference information between singular values of tensor data by the weighted tensor Schatten p-norm minimization, and then propose an efficient algorithm, which has a good convergence, to solve ETRPCA. Extensive experimental results reveal that the proposed method ETRPCA is superior to several state-of-the-art variant RPCA methods in terms of performance.

Robust singular value decomposition filtering for low signal-to-noise ratio seismic data

Reduced-rank filtering is a common method for attenuating noise in seismic data. Because conventional reduced-rank filtering distinguishes signals from noises only according to singular values, it performs poorly when the signal-to-noise ratio (S/N) is very low or when data contain high levels of isolate or coherent noise. Therefore, we have developed a novel and robust reduced-rank filtering method based on singular value decomposition in the time-space domain. In this method, noise is recognized and attenuated according to the characteristics of the singular values and the singular vectors. The left and right singular vectors corresponding to large singular values are selected first. Then, the right singular vectors are classified into different categories according to their curve characteristics, such as jump, pulse, and smooth. Each kind of right singular vector is related to a type of noise or seismic event, and it is corrected by using a different filtering technology, such as mean filtering, edge-preserving smoothing, or edge-preserving median filtering. The left singular vectors are also corrected by using the filtering methods based on frequency attributes such as main frequency and frequency bandwidth. To process seismic data containing a variety of events, local data are extracted along the local dip of the event. The optimal local dip is identified according to the singular values and singular vectors of the data matrices that are extracted along different trial directions. This new filtering method has been applied to synthetic and field seismic data, and its performance is compared with that of several conventional filtering methods. The results indicate that the new method is more robust for data with a low S/N, strong isolated noise, or coherent noise. The new method also overcomes the difficulties associated with selecting an optimal rank.

Robust Multiview Subspace Clustering of Images via Tighter Rank Approximation

In this paper, we focus on the multi-view subspace clustering problem under the framework of third order tensor. Recently, tensor nuclear norm (TNN) has been widely used in the multi-view subspace clustering problem. It is known that TNN is a convex surrogate of the tensor rank. However, TNN is linearly proportional to the sum of the singular values. Thus solving the TNN minimization problem will over-penalize the large singular values and lead to a sub-optimal solution. To address this issue, in this paper, a novel tighter tensor log-determinant (TLD) regularizer is proposed to better approximate the tensor rank. Although the proposed method is non-convex, a closed-form solution has been deduced via solving the Euler-Lagrange equation. A corresponding algorithm associated with augmented Lagrangian multipliers is established and the constructed convergent sequence to the Karush-Kuhn-Tucker (KKT) critical point solution is mathematically validated in detail. Extensive experiments indicate that the proposed model achieves significant improvements compared to the state-of-the-art convex subspace clustering models.

On the regularization effect of stochastic gradient descent applied to least-squares

We study the behavior of stochastic gradient descent applied to $\|Ax -b \|_2^2 \rightarrow \min$ for invertible $A \in \mathbb{R}^{n \times n}$. We show that there is an explicit constant $c_{A}$ depending (mildly) on $A$ such that $$ \mathbb{E} ~\left\| Ax_{k+1}-b\right\|^2_{2} \leq \left(1 + \frac{c_{A}}{\|A\|_F^2}\right) \left\|A x_k -b \right\|^2_{2} - \frac{2}{\|A\|_F^2} \left\|A^T A (x_k - x)\right\|^2_{2}.$$ This is a curious inequality: the last term has one more matrix applied to the residual $u_k - u$ than the remaining terms: if $x_k - x$ is mainly comprised of large singular vectors, stochastic gradient descent leads to a quick regularization. For symmetric matrices, this inequality has an extension to higher-order Sobolev spaces. This explains a (known) regularization phenomenon: an energy cascade from large singular values to small singular values smoothes.

Upper Bounds for the Largest Singular Value of Certain Digraph Matrices

In this paper, we consider digraphs with possible loops and the particular case of oriented graphs, i.e. loopless digraphs with at most one oriented edge between every pair of vertices. We provide an upper bound for the largest singular value of the skew Laplacian matrix of an oriented graph, the largest singular value of the skew adjacency matrix of an oriented graph and the largest singular value of the adjacency matrix of a digraph. These bounds are expressed in terms of certain parameters related to vertex degrees. We also consider some bounds for the sums of squares of singular values. As an application, for the skew (Laplacian) adjacency matrix of an oriented graph and the adjacency matrix of a digraph, we derive some upper bounds for the spectral radius and the sums of squares of moduli of eigenvalues.

An improved ridge estimation (IRE) method for troposphere water vapor tomography

Water vapor tomography technique is a research point in global navigation satellite system (GNSS) meteorology. However, GNSS tomography technique has an inherent issue of design matrix sparsity, which leads to the ill-posed problem in the inversion of normal equation. Singular value decomposition (SVD) method is commonly used to resolve this problem but performs poorly due to the large fluctuation in tomographic results caused by the small singular values. To overcome the above issue, this study proposes an improved ridge estimation (IRE) method, which only revises the relatively small singular values and retains the large singular values unchanged on the basis of the minimum deviation principle in the processing of regularization matrix. An adjustment coefficient is introduced to determine the node of small singular value with large influence on the tomographic result. Twelve GNSS stations from Hong Kong Satellite Positioning Reference Station Network (SatRef) were used to validate the performance of the proposed method over the period of day of year (Doy) 84–101, 2014. The established tomography model in SatRef was resolved using SVD and IRE methods, respectively. Statistical results show that the accuracy of average root mean square error (RMSE) of IRE method is improved by 16.7% compared with that of SVD method when the corresponding radiosonde is considered as the reference. Additionally, the comparisons of water vapor profile, relative error (RE) and the reconstructed SWV derived from IRE and SVD methods were also carried out and the numerical result indicates a good performance of the proposed IRE method. Such results verify that the applicability of proposed IRE method in this paper for GNSS water vapor tomography.

Non-Iterative Methods Based on Singular Value Decomposition for Inverse Scattering Problems

Three non-iterative methods, including major current coefficient (MCC) method, modified Born approximation (MBA) method, and connection method, are proposed to solve inverse scattering problems in this article. The three methods are motivated by the observation that the right singular vectors corresponding to large singular values have a large effect on the scattered field within the domains of interest (DOI), which has been verified both analytically and numerically for different measurement setups. The performance of the proposed methods is verified using both synthetic and experimental data. One important conclusion is that the proposed methods greatly outperform the widely used Born approximation (BA) and Rytov approximation (RA) inversion methods, with low overhead computational cost.

A note on Banach’s results concerning homogeneous polynomials associated with nonnegative tensors

A classic result of Banach states that the supreme of a multivariate homogenous polynomial is equivalent to that of its associated symmetric multilinear form over unit balls. Using the language of higher-order tensors in finite-dimensional spaces, this means that for a symmetric tensor, its largest singular value is in fact equivalent to the largest magnitude of its eigenvalues. This note strengthens Banach’s results on nonnegative higher-order tensors. It is shown that for a symmetric nonnegative irreducible tensor: (1) any singular value admitting a positive singular vector tuple must be an eigenvalue, where the singular vectors in the tuple must be equal to each other; i.e., they amount to an eigenvector; (2) when the order is odd, any nonnegative singular vector tuple corresponding to the largest singular value must also boil down to a positive eigenvector. These results give the flexibility of directly solving tensor eigenvalue problems via solving tensor singular value problems.

More results related to geometric mean and singular values for matrices

In 2016, Lin conjectured that if is a positive semi-definite matrix then where and means the jth largest singular value. In this note, we confirm this conjecture when AB = BA and prove the more general result where and f is a Liebian function. Some related inequalities are also investigated.

Smallest singular value and limit eigenvalue distribution of a class of non-Hermitian random matrices with statistical application

Suppose X is an N×n complex matrix whose entries are centered, independent, and identically distributed random variables with variance 1∕n and whose fourth moment is of order O(n−2). Suppose A is a deterministic matrix whose smallest and largest singular values are bounded below and above respectively, and z≠0 is a complex number. First we consider the matrix XAX∗−z, and obtain asymptotic probability bounds for its smallest singular value when N and n diverge to infinity and N∕n→γ,0<γ<∞. Then we consider the special case where A=J=[1i−j=1modn] is a circulant matrix. Using the above result, we show that the limit spectral distribution of XJX∗ exists when N∕n→γ,0<γ<∞ and describe the limit explicitly. Assuming that X represents a ℂN-valued time series which is observed over a time window of length n, the matrix XJX∗ represents the one-step sample autocovariance matrix of this time series. A whiteness test against an MA correlation model for this time series is introduced based on the above limit result. Numerical simulations show the excellent performance of this test.