Partial Singular Value Decomposition Research Articles

A hybrid algorithm for computing a partial singular value decomposition satisfying a given threshold

A hybrid algorithm for computing a partial singular value decomposition satisfying a given threshold

Infrared Small-Target Detection Based on Background-Suppression Proximal Gradient and GPU Acceleration

Patch-based methods improve the performance of infrared small target detection, transforming the detection problem into a Low-Rank Sparse Decomposition (LRSD) problem. However, two challenges hinder the success of these methods: (1) The interference from strong edges of the background, and (2) the time-consuming nature of solving the model. To tackle these two challenges, we propose a novel infrared small-target detection method using a Background-Suppression Proximal Gradient (BSPG) and GPU parallelism. We first propose a new continuation strategy to suppress the strong edges. This strategy enables the model to simultaneously consider heterogeneous components while dealing with low-rank backgrounds. Then, the Approximate Partial Singular Value Decomposition (APSVD) is presented to accelerate solution of the LRSD problem and further improve the solution accuracy. Finally, we implement our method on GPU using multi-threaded parallelism, in order to further enhance the computational efficiency of the model. The experimental results demonstrate that our method out-performs existing advanced methods, in terms of detection accuracy and execution time.

Explicit deflation in Golub-Kahan-Lanczos bidiagonalization methods

We discuss a simple, easily overlooked, explicit deflation procedure applied to Golub-Kahan-Lanczos Bidiagonalization (GKLB)-based methods to compute the next set of the largest singular triplets of a matrix from an already computed partial singular value decomposition. Our results here complement the vast literature on this topic, provide additional insight, and highlight the simplicity and the effectiveness of this procedure. We demonstrate how existing GKLB-based routines for the computation of the largest singular triplets can be easily adapted to take advantage of explicit deflation, thus making it more appealing to a wider range of users. Numerical examples are presented including an application of singular value thresholding.

Accurate and fast matrix factorization for low-rank learning.

In this paper, we tackle two important problems in low-rank learning, which are partial singular value decomposition and numerical rank estimation of huge matrices. By using the concepts of Krylov subspaces such as Golub-Kahan bidiagonalization (GK-bidiagonalization) as well as Ritz vectors, we propose two methods for solving these problems in a fast and accurate way. Our experiments show the advantages of the proposed methods compared to the traditional and randomized singular value decomposition methods. The proposed methods are appropriate for applications involving huge matrices where the accuracy of the desired singular values and also all of their corresponding singular vectors are essential. As a real application, we evaluate the performance of our methods on the problem of Riemannian similarity learning between two different image datasets of MNIST and USPS.

Golub\u2013Kahan vs. Monte Carlo: a comparison of bidiagonlization and a randomized SVD method for the solution of linear discrete ill-posed problems

Randomized methods can be competitive for the solution of problems with a large matrix of low rank. They also have been applied successfully to the solution of large-scale linear discrete ill-posed problems by Tikhonov regularization (Xiang and Zou in Inverse Probl 29:085008, 2013). This entails the computation of an approximation of a partial singular value decomposition of a large matrix A that is of numerical low rank. The present paper compares a randomized method to a Krylov subspace method based on Golub–Kahan bidiagonalization with respect to accuracy and computing time and discusses characteristics of linear discrete ill-posed problems that make them well suited for solution by a randomized method.

Identification of partially coherent sound sources for centrifugal fan

The signal model with multiple inputs and single output (MISO) is often used for investigating sound sources of vehicles and mechanical equipment, in which transfer path analysis (TPA) and operational transfer path analysis (OTPA) procedure are well known in industry. Sources are completely coherent, as well as completely incoherent in TPA and OTPA models. However, some mechanical devices like centrifugal fans contain not only structure-borne and air-borne sound but also vibrational source and acoustical source. The sources may be partially coherent and cannot be calculated using traditional method. In order to understand the origination of sound from the partially coherent sources of the fan, a MISO model with partially coherent inputs was developed, in which the partially coherent sources were identified by partial singular value decomposition (PSVD) method. Vibration, pressure pulsation and sound signals were measured, and partially coherent sources contribution analysis were used to calculate the contribution of sources quantitatively. A test model was built, and an experiment was conducted to verify that this method is feasible. This method is also suitable for the identification and contribution analysis of sources with partially coherent characteristics, such as automobile exhaust system. This method can calculate the contribution of partially coherent sources, which is not available in other noise source identification methods.

A novel transmissibility matrix method for identifying partially coherent noise sources on vehicles

For vehicle interior sound field, it is difficult to identify locations of main sources due to the coupling of transmission paths. The traditional operational transfer path analysis (OTPA) is unable to be used to get the contribution of partially coherent sources. A transmissibility matrix identification (TMI) method is proposed to deal with partially coherent source identification problem. Partially coherent radiation sources are located on the body panel structure and microphones are set to record sound signals. The transmissibility of sound from microphone near the partially coherent radiation source to the microphone away from the source should be smaller than the converse transmissibility. Locations of main sound sources are determined by finding the transmissibility matrix with the smallest Frobenius norm. Comparing with partial singular value decomposition (PSVD) method which is based on independent source hypothesis, main sources locations obtained by TMI method are more stable. The method was verified by experiments in the semi-anechoic chamber and successfully applied to the identification of internal noise sources in vehicles. The relationship between various sources was clear by the calculation of the partial contribution of each partially coherent source and the aliasing rate of the system.

Error Analysis of the Cholesky QR-Based Block Orthogonalization Process for the One-Sided Block Jacobi SVD Algorithm

The one-sided block Jacobi method (OSBJ) has attracted attention as a fast and accurate algorithm for the singular value decomposition (SVD). The computational kernel of OSBJ is orthogonalization of a column block pair, which amounts to computing the SVD of this block pair. Hari proposes three methods for this partial SVD, and we found through numerical experiments that the variant named V2, which is based on the Cholesky QR method, is the fastest variant and achieves satisfactory accuracy. While it is a good news from a practical viewpoint, it seems strange considering the well-known instability of the Cholesky QR method. In this paper, we perform a detailed error analysis of the V2 variant and explain why and when it can be used to compute the partial SVD accurately. Thus, our results provide a theoretical support for using the V2 variant safely in the OSBJ method.

Cost reduction of the bond-swapping part in an anisotropic tensor renormalization group

Abstract The bottleneck part of an anisotropic tensor renormalization group (ATRG) is a bond-swapping part that consists of a contraction of two tensors and a partial singular value decomposition of a matrix, and their computational costs are $O(\chi^{2d+1})$, where $\chi$ is the maximum bond dimension and $d$ is the dimensionality of the system. We propose an alternative method for the bond-swapping part and it scales with $O(\chi^{\max(d+3,7)})$, though the total cost of ATRG with the method remains $O(\chi^{2d+1})$. Moreover, the memory cost of the whole algorithm can be reduced from $O(\chi^{2d})$ to $O(\chi^{\max(d+1,6)})$. We examine ATRG with or without the proposed method in the 4D Ising model and find that the free energy density of the proposed algorithm is consistent with that of the original ATRG while the elapsed time is significantly reduced. We also compare the proposed algorithm with a higher-order tensor renormalization group (HOTRG) and find that the value of the free energy density of the proposed algorithm is lower than that of HOTRG in the fixed elapsed time.

A preconditioned Multiple Shooting Shadowing algorithm for the sensitivity analysis of chaotic systems

We propose a preconditioner that can accelerate the rate of convergence of the Multiple Shooting Shadowing (MSS) method [1]. This recently proposed method can be used to compute derivatives of time-averaged objectives (also known as sensitivities) to system parameter(s) for chaotic systems. We propose a block diagonal preconditioner, which is based on a partial singular value decomposition of the MSS constraint matrix. The preconditioner can be computed using matrix-vector products only (i.e. it is matrix-free) and is fully parallelised in the time domain. Two chaotic systems are considered, the Lorenz system and the 1D Kuramoto Sivashinsky equation. Combination of the preconditioner with a regularisation method leads to tight bracketing of the eigenvalues to a narrow range. This combination results in a significant reduction in the number of iterations, and renders the convergence rate almost independent of the number of degrees of freedom of the system, and the length of the trajectory that is used to compute the time-averaged objective. This can potentially allow the method to be used for large chaotic systems (such as turbulent flows) and optimal control applications. The singular value decomposition of the constraint matrix can also be used to quantify the effect of the system condition on the accuracy of the sensitivities. In fact, neglecting the singular modes affected by noise, we recover the correct values of sensitivity that match closely with those obtained with finite differences for the Kuramoto Sivashinsky equation in the light turbulent regime. We notice a similar improvement for the Lorenz system as well.

Sampling and multilevel coarsening algorithms for fast matrix approximations

SummaryThis paper addresses matrix approximation problems for matrices that are large, sparse, and/or representations of large graphs. To tackle these problems, we consider algorithms that are based primarily on coarsening techniques, possibly combined with random sampling. A multilevel coarsening technique is proposed, which utilizes a hypergraph associated with the data matrix and a graph coarsening strategy based on column matching. We consider a number of standard applications of this technique as well as a few new ones. Among standard applications, we first consider the problem of computing partial singular value decomposition, for which a combination of sampling and coarsening yields significantly improved singular value decomposition results relative to sampling alone. We also consider the column subset selection problem, a popular low‐rank approximation method used in data‐related applications, and show how multilevel coarsening can be adapted for this problem. Similarly, we consider the problem of graph sparsification and show how coarsening techniques can be employed to solve it. We also establish theoretical results that characterize the approximation error obtained and the quality of the dimension reduction achieved by a coarsening step, when a proper column matching strategy is employed. Numerical experiments illustrate the performances of the methods in a few applications.

Energy-Efficient Hybrid Precoding With Low Complexity for mmWave Massive MIMO Systems

Millimeter-wave (mmWave) massive multiple-input multiple-output (MIMO) utilizes large antenna arrays and is considered a promising technology for fifth-generation (5G) and beyond wireless communication systems. However, the high-power consumption of the radio-frequency (RF) chains makes it infeasible. To solve this problem, hybrid precoding is proposed, which is a combination of analog and digital precoding. The fully connected architecture hybrid precoding still requires a large number of phase shifters (PSs). The sub-connected architecture can greatly reduce the required power consumption, and however, it cannot obtain a satisfactory achievable rate. To avoid the high energy consumption and obtain a high resolution, we propose a novel partly connected architecture in this paper. In addition, we propose an energy-efficient successive interference cancelation (SIC) hybrid precoding based on the partly connected architecture, which transforms the problem of maximizing the total achievable rate with non-convex constraints into a series of sub-rate optimization problems. Furthermore, a low-complexity energy-efficient SIC hybrid precoding based on the partly connected architecture is developed, which uses the partial singular value decomposition (SVD) to realize the sub-rate optimization and significantly reduce the complexity. Theoretical analysis demonstrates the superiority of the proposed hybrid precoding in terms of complexity. The simulation results indicate that the proposed hybrid precoding algorithms enjoy better energy efficiency and achievable rate performance than some recently proposed hybrid precoding algorithms.

A new method based on the manifold-alternative approximating for low-rank matrix completion

In this paper, a new method is proposed for low-rank matrix completion which is based on the least squares approximating to the known elements in the manifold formed by the singular vectors of the partial singular value decomposition alternatively. The method can achieve a reduction of the rank of the manifold by gradually reducing the number of the singular value of the thresholding and get the optimal low-rank matrix. It is proven that the manifold-alternative approximating method is convergent under some conditions. Furthermore, compared with the augmented Lagrange multiplier and the orthogonal rank-one matrix pursuit algorithms by random experiments, it is more effective as regards the CPU time and the low-rank property.

Lanczos method for large-scale quaternion singular value decomposition

In many color image processing and recognition applications, one of the most important targets is to compute the optimal low-rank approximations to color images, which can be reconstructed with a small number of dominant singular value decomposition (SVD) triplets of quaternion matrices. All existing methods are designed to compute all SVD triplets of quaternion matrices at first and then to select the necessary dominant ones for reconstruction. This way costs quite a lot of operational flops and CPU times to compute many superfluous SVD triplets. In this paper, we propose a Lanczos-based method of computing partial (several dominant) SVD triplets of the large-scale quaternion matrices. The partial bidiagonalization of large-scale quaternion matrices is derived by using the Lanczos iteration, and the reorthogonalization and thick-restart techniques are also utilized in the implementation. An algorithm is presented to compute the partial quaternion singular value decomposition. Numerical examples, including principal component analysis, color face recognition, video compression and color image completion, illustrate that the performance of the developed Lanczos-based method for low-rank quaternion approximation is better than that of the state-of-the-art methods.

An alternating minimization method for robust principal component analysis

ABSTRACTThis paper focuses on the solution of robust principal component analysis (RPCA) problems that arise in fields such as information theory, statistics, and engineering. We adopt a model that minimizes the sum of the observation error and sparsity measurement subject to the rank constraint. To solve this problem, we propose a two-step alternating minimization method. In the first step, a symmetric low-rank product minimization, which is essentially a partial singular value decomposition, is efficiently solved with moderate accuracy. The second step then derives a closed-form solution. The proposed approach is almost parameter-free, and global convergence to a strict local minimizer is guaranteed under very loose conditions. We compare the proposed approach with some existing solvers, and numerical experiments demonstrate the outstanding performance of our approach in solving synthetic and real RPCA test problems. In particular, we illustrate the significant potential of the proposed approach to solve large-size problems with moderate accuracy.

Spatiotemporal clutter filtering methods for in vivo microvessel imaging

Ultrafast plane wave imaging enables acquisition and accumulation of a large number of Doppler ensembles in a short period of time, which substantially boosts ultrasound Doppler sensitivity by at least an order of magnitude. As a key component, spatiotemporal clutter filtering based on singular value decomposition (SVD) has recently demonstrated superior performance in tissue clutter suppression and microvessel signal extraction over conventional high-pass-filter-based clutter filtering techniques. This presentation introduces several novel SVD clutter filter techniques from our recent work, focusing on addressing the technical challenges (complex tissue motion, high computational cost, and noise) of SVD clutter filter during clinical translations of the ultrafast microvessel imaging technology for human imaging. We present a block-wise adaptive SVD filter method that can robustly threshold singular values on a local level, a real-time feasible SVD clutter filtering method based on partial SVD and randomized spatial downsampling, and a noise equalization and suppression method that can significantly improve microvessel imaging quality at a low cost. In vivo performance of these methods will be demonstrated in different organs.

Robust PCA Using Matrix Factorization for Background/Foreground Separation

Background/foreground separation has become an inevitable step in numerous image/video processing applications, such as image/video inpainting, anomaly detection, motion segmentation, augmented reality, and so on. Recent low-rank based approaches, such as robust principal component analysis separating a data matrix into a low-rank matrix with a sparse matrix, have achieved encouraging performance. However, these approaches usually need relatively high computation cost, mainly due to calculation of full or partial singular value decomposition of large matrices. On the other hand, the nuclear norm is widely exploited as a convex surrogate of the original rank function, while it is not a tighter envelope of the original rank function. To address these above-mentioned issues, this paper proposes a fast background/foreground separation algorithm in which the low-rank constraint is solved by a matrix factorization scheme, thus heavily reducing the computation cost. We further adopt two non-convex low-rank approximations to improve the robustness and flexibility of the traditional nuclear norm. In comparison with the state-of-the-art low-rank reconstruction methods, experimental results on challenging data sets, which contain different real data sets, show our superior performance in both image clarity and computation efficiency.

Denoising of Hyperspectral Image Using Low-Rank Matrix Factorization

Restoration of hyperspectral images (HSIs) is a challenging task, owing to the reason that images are inevitably contaminated by a mixture of noise, including Gaussian noise, impulse noise, dead lines, and stripes, during their acquisition process. Recently, HSI denoising approaches based on low-rank matrix approximation have become an active research field in remote sensing and have achieved state-of-the-art performance. These approaches, however, unavoidably require to calculate full or partial singular value decomposition of large matrices, leading to the relatively high computational cost and limiting their flexibility. To address this issue, this letter proposes a method exploiting a low-rank matrix factorization scheme, in which the associated robust principal component analysis is solved by the matrix factorization of the low-rank component. Our method needs only an upper bound of the rank of the underlying low-rank matrix rather than the precise value. The experimental results on the simulated and real data sets demonstrate the performance of our method by removing the mixed noise and recovering the severely contaminated images.

GCV for Tikhonov regularization by partial SVD

Tikhonov regularization is commonly used for the solution of linear discrete ill-posed problems with error-contaminated data. A regularization parameter that determines the quality of the computed solution has to be chosen. One of the most popular approaches to choosing this parameter is to minimize the Generalized Cross Validation (GCV) function. The minimum can be determined quite inexpensively when the matrix A that defines the linear discrete ill-posed problem is small enough to rapidly compute its singular value decomposition (SVD). We are interested in the solution of linear discrete ill-posed problems with a matrix A that is too large to make the computation of its complete SVD feasible, and show how upper and lower bounds for the numerator and denominator of the GCV function can be determined fairly inexpensively for large matrices A by computing only a few of the largest singular values and associated singular vectors of A. These bounds are used to determine a suitable value of the regularization parameter. Computed examples illustrate the performance of the proposed method.

Accelerating large partial EVD/SVD calculations by filtered block Davidson methods

Partial eigenvalue decomposition (PEVD) and partial singular value decomposition (PSVD) of large sparse matrices are of fundamental importance in a wide range of applications, including latent semantic indexing, spectral clustering, and kernel methods for machine learning. The more challenging problems are when a large number of eigenpairs or singular triplets need to be computed. We develop practical and efficient algorithms for these challenging problems. Our algorithms are based on a filter-accelerated block Davidson method. Two types of filters are utilized, one is Chebyshev polynomial filtering, the other is rational-function filtering by solving linear equations. The former utilizes the fastest growth of the Chebyshev polynomial among same degree polynomials; the latter employs the traditional idea of shift-invert, for which we address the important issue of automatic choice of shifts and propose a practical method for solving the shifted linear equations inside the block Davidson method. Our two filters can efficiently generate high-quality basis vectors to augment the projection subspace at each Davidson iteration step, which allows a restart scheme using an active projection subspace of small dimension. This makes our algorithms memory-economical, thus practical for large PEVD/PSVD calculations. We compare our algorithms with representative methods, including ARPACK, PROPACK, the randomized SVD method, and the limited memory SVD method. Extensive numerical tests on representative datasets demonstrate that, in general, our methods have similar or faster convergence speed in terms of CPU time, while requiring much lower memory comparing with other methods. The much lower memory requirement makes our methods more practical for large-scale PEVD/PSVD computations.