Row Norms Research Articles

An Improved Low-Rank Matrix Fitting Method Based on Weighted L1,p Norm Minimization for Matrix Completion

Low-rank matrix completion, which aims to recover a matrix with many missing values, has attracted much attention in many fields of computer science. A low-rank matrix fitting (LMaFit) method has been proposed for fast matrix completion recently. However, this method cannot converge accurately on matrices of real-world images. For improving the accuracy of LMaFit method, an improved low-rank matrix fitting (ILMF) method based on the weighted [Formula: see text] norm minimization is proposed in this paper, where the [Formula: see text] norm is the summation of the [Formula: see text]-power [Formula: see text] of [Formula: see text] norms of rows in a matrix. In the proposed method, i.e. the ILMF method, the incomplete matrix that may be corrupted by noises is decomposed into the summation of a low-rank matrix and a noise matrix at first. Then, a weighted [Formula: see text] norm minimization problem is solved by using an alternating direction method for improving the accuracy of matrix completion. Experimental results on real-world images show that the ILMF method has much better performances in terms of both the convergence accuracy and convergence speed than the compared methods.

Pylspack : Parallel Algorithms and Data Structures for Sketching, Column Subset Selection, Regression, and Leverage Scores

We present parallel algorithms and data structures for three fundamental operations in Numerical Linear Algebra: (i) Gaussian and CountSketch random projections and their combination, (ii) computation of the Gram matrix, and (iii) computation of the squared row norms of the product of two matrices, with a special focus on “tall-and-skinny” matrices, which arise in many applications. We provide a detailed analysis of the ubiquitous CountSketch transform and its combination with Gaussian random projections, accounting for memory requirements, computational complexity and workload balancing. We also demonstrate how these results can be applied to column subset selection, least squares regression and leverage scores computation. These tools have been implemented in pylspack , a publicly available Python package 1 whose core is written in C++ and parallelized with OpenMP and that is compatible with standard matrix data structures of SciPy and NumPy. Extensive numerical experiments indicate that the proposed algorithms scale well and significantly outperform existing libraries for tall-and-skinny matrices.

Squeezing a Matrix into Half Precision, with an Application to Solving Linear Systems

Motivated by the demand in machine learning, modern computer hardware is increas- ingly supporting reduced precision floating-point arithmetic, which provides advantages in speed, energy, and memory usage over single and double precision. Given the availability of such hardware, mixed precision algorithms that work in single or double precision but carry out part of a compu- tation in half precision are now of great interest for general scientific computing tasks. Because of the limited range of half precision arithmetic, in which positive numbers lie between 6 × 10−8 and 7 × 104, a straightforward rounding of single or double precision data into half precision can lead to overflow, underflow, or subnormal numbers being generated, all of which are undesirable. We develop an algorithm for converting a matrix from single or double precision to half precision. It first applies two-sided diagonal scaling in order to equilibrate the matrix (that is, to ensure that every row and column has ∞-norm 1), then multiplies by a scalar to bring the largest element within a factor θ ≤ 1 of the overflow level, and finally rounds to half precision. The second step ensures that full use is made of the limited range of half precision arithmetic, and θ must be chosen to allow sufficient headroom for subsequent computations. We apply the new algorithm to GMRES-based iterative re- finement (GMRES-IR), which solves a linear system Ax = b with single or double precision data by LU factorizing A in half precision and carrying out iterative refinement with the correction equations solved by GMRES preconditioned with the low precision LU factors. Previous implementations of this algorithm have used a crude conversion to half precision that our experiments show can cause slow convergence of GMRES-IR for badly scaled matrices or failure to converge at all. The new conversion algorithm computes ∞-norms of rows and columns of the matrix and its cost is negligible in the context of LU factorization. We show that it leads to faster convergence of GMRES-IR for badly scaled matrices and thereby allows a much wider class of problems to be solved.

Fast quantum algorithms for least squares regression and statistic leverage scores

Least squares regression is the simplest and most widely used technique for solving overdetermined systems of linear equations Ax=b, where A∈Rn×p has full column rank and b∈Rn. Though there is a well known unique solution x⁎∈Rp to minimize the squared error ‖Ax−b‖22, the best known classical algorithm to find x⁎ takes time Ω(n), even for sparse and well-conditioned matrices A, a fairly large class of input instances commonly seen in practice. In this paper, we design an efficient quantum algorithm to generate a quantum state proportional to |x⁎〉. The algorithm takes only O(log⁡n) time for sparse and well-conditioned A. When the condition number of A is large, a canonical solution is to use regularization. We give efficient quantum algorithms for two regularized regression problems, including ridge regression and δ-truncated SVD, with similar costs and solution approximation.Given a matrix A∈Rn×p of rank r with SVD A=UΣVT where U∈Rn×r, Σ∈Rr×r and V∈Rp×r, the statistical leverage scores of A are the squared row norms of U, defined as si=‖Ui‖22, for i=1,…,n. The matrix coherence is the largest statistic leverage score. These quantities play an important role in many machine learning algorithms. The best known classical algorithm to approximate these values runs in time Ω(np). In this work, we introduce an efficient quantum algorithm to approximate si in time O(log⁡n) when A is sparse and the ratio between A's largest singular value and smallest non-zero singular value is constant. This gives an exponential speedup over the best known classical algorithms. Different than previous examples which are mainly modern algebraic or number theoretic ones, this problem is linear algebraic. It is also different than previous quantum algorithms for solving linear equations and least squares regression, whose outputs compress the p-dimensional solution to a log⁡(p)-qubit quantum state.

Randomized block Kaczmarz method with projection for solving least squares

The Kaczmarz method is an iterative method for solving overcomplete linear systems of equations Ax=b. The randomized version of the Kaczmarz method put forth by Strohmer and Vershynin iteratively projects onto a randomly chosen solution space given by a single row of the matrix A and converges linearly1 in expectation to the solution of a consistent system. In this paper we analyze two block versions of the method each with a randomized projection, designed to converge in expectation to the least squares solution, often faster than the standard variants. Our approach utilizes both a row and column-paving of the matrix A to guarantee linear convergence when the matrix has consistent row norms (called nearly standardized), and a single column-paving when the row norms are unbounded. The proposed methods are an extension of the block Kaczmarz method analyzed by Needell and Tropp and the Randomized Extended Kaczmarz method of Zouzias and Freris. The contribution is thus two-fold; unlike the standard Kaczmarz method, our results demonstrate convergence to the least squares solution of inconsistent systems (both methods in the nearly standardized case and the second method in other cases). By using appropriate blocks of the matrix this convergence can be significantly accelerated, as is demonstrated by numerical experiments.

Conditioning of Leverage Scores and Computation by QR Decomposition

The leverage scores of a full-column rank matrix $A$ are the squared row norms of any orthonormal basis for $\mathrm{range}\,(A)$. We show that corresponding leverage scores of two matrices $A$ and $A+\Delta A$ are close in the relative sense if they have large magnitude and if all principal angles between the column spaces of $A$ and $A+\Delta A$ are small. We also show three classes of bounds that are based on perturbation results of QR decompositions. They demonstrate that relative differences between individual leverage scores strongly depend on the particular type of perturbation $\Delta A$. The bounds imply that the relative accuracy of an individual leverage score depends on its magnitude and the two-norm condition of $A$ if $\Delta A$ is a general perturbation; the two-norm condition number of $A$ if $\Delta A$ is a perturbation with the same normwise row-scaling as $A$; (to first order) neither condition number nor leverage score magnitude if $\Delta A$ is a componentwise row-scaled perturbation. Numerical experiments confirm the qualitative and quantitative accuracy of our bounds.

Iterative Precoder Design and User Scheduling for Block-Diagonalized Systems

The block diagonalization (BD) scheme is a low-complexity suboptimal precoding technique for multiuser multiple input-multiple output (MIMO) downlink channels, which completely precancels the multiuser interference. Accordingly, the precoder of each user lies in the null space of other users' channel matrices. In this paper, we propose an iterative algorithm using QR decompositions (QRDs) to compute the precoders. Specifically, to avoid dealing with a large concatenated matrix, we apply the QRD to a sequence of matrices of lower dimensions. One problem of BD schemes is that the number of users that can be simultaneously supported is limited due to zero interference constraints. When the number of users is large, a set of users must be selected, and selection algorithms should be designed to exploit the multiuser diversity gain. Finding the optimal set of users requires an exhaustive search, which has too high computational complexity to be practically useful. Based on the iterative precoder design, this paper proposes a low-complexity user selection algorithm using a greedy method, in which the precoders of selected users are recursively updated after each selection step. The selection metric of the proposed scheduling algorithm relies on the product of the squared row norms of the effective channel matrices, which is related to the eigenvalues by the Hadamard and Schur inequalities. An asymptotic analysis is provided to show that the proposed algorithm can achieve the optimal sum rate scaling of the MIMO broadcast channel. The numerical results show that the proposed algorithm achieves a good trade-off between sum rate performance and computational complexity. When users suffer different channel conditions, providing fairness among users is of critical importance. To address this problem, we also propose two fair scheduling (FS) algorithms, one imposing fairness in the approximation of the data rate, and another directly imposing fairness in the product of the squared row norms of the effective channel matrices.

Representation of certain homogeneous Hilbertian operator spaces and applications

Following Grothendieck's characterization of Hilbert spaces we consider operator spaces $F$ such that both $F$ and $F^*$ completely embed into the dual of a C*-algebra. Due to Haagerup/Musat's improved version of Pisier/Shlyakhtenko's Grothendieck inequality for operator spaces, these spaces are quotients of subspaces of the direct sum $C\oplus R$ of the column and row spaces (the corresponding class being denoted by $QS(C\oplus R)$). We first prove a representation theorem for homogeneous $F\in QS(C\oplus R)$ starting from the fundamental sequences defined by column and row norms of unit vectors. Under a mild regularity assumption on these sequences we show that they completely determine the operator space structure of $F$ and find a canonical representation of this important class of homogeneous Hilbertian operator spaces in terms of weighted row and column spaces. This canonical representation allows us to get an explicit formula for the exactness constant of an $n$-dimensional subspace $F_n$ of $F$ involving the fundamental sequences. Similarly, we have formulas for the the projection (=injectivity) constant of $F_n$. They also permit us to determine the completely 1-summing maps in Effros and Ruan's sense between two homogeneous spaces $E$ and $F$ in $QS(C\oplus R)$.

Analysis of MIMO Systems With Receive Antenna Selection in Spatially Correlated Rayleigh Fading Channels

We consider a receive antenna selection multiple-input-multiple-output (MIMO) system, where only one out of <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">r</sub> receive antennas is selected. Spatial channel correlation will be considered at the receiver side only. Our goal is to investigate the capacity performance of this system. In particular, we will derive a closed-form expression for the outage probability and an upper bound for the ergodic (average) capacity. These quantities will be expressed using an infinite series representation. To do so, we will first derive the joint cumulative distribution function and the joint probability density function of the squared row norms of the channel matrix. This is enabled by taking advantage of the statistical properties of multivariate chi-squared random variables. Using the outage expression derived, we demonstrate that the considered antenna selection system achieves a full diversity order that is similar to a MIMO system without antenna selection. Next, we derive the probability density function of the maximum of the squared row norms of the channel matrix and its moments, which is straightly related to the system ergodic capacity. We also analyze the error rate performance of the aforementioned receive antenna selection MIMO system while using orthogonal space-time block codes (OSTBCs) at the transmitter. Our simulation results are shown to validate our analytical findings.