Rank Of Matrices Research Articles

Theoretical Investigation of Generalization Bounds for Adversarial Learning of Deep Neural Networks

Recent studies have shown that many machine learning models are vulnerable to adversarial attacks. Much remains unknown concerning the generalization error of deep neural networks (DNNs) for adversarial learning. In this paper, we study the generalization of DNNs for robust adversarial learning with $$\ell _\infty$$ attacks, particularly focusing on attacks produced under the fast gradient sign method. We establish a tight bound for the adversarial Rademacher complexity of DNNs based on both spectral norms and ranks of weight matrices. The spectral norm and rank constraints imply that this class of networks can be realized as a subset of the class of a shallow network composed with a low-dimensional Lipschitz continuous function. This crucial observation leads to a bound that improves the dependence on the network width compared to previous works and achieves depth independence. For general machine learning tasks, we show that adversarial Rademacher complexity is always larger than natural counterpart, but the effect of adversarial perturbations can be limited under our weight normalization framework. Our theoretical findings are also confirmed by experiments.

Regularization Parameter Selection for the Low Rank Matrix Recovery

A popular approach to recover low rank matrices is the nuclear norm regularized minimization (NRM) for which the selection of the regularization parameter is inevitable. In this paper, we build up a novel rule to choose the regularization parameter for NRM, with the help of the duality theory. Our result provides a safe set for the regularization parameter when the rank of the solution has an upper bound. Furthermore, we apply this idea to NRM with quadratic and Huber functions, and establish simple formulae for the regularization parameters. Finally, we report numerical results on some signal shapes by embedding our rule into the cross validation, which state that our rule can reduce the computational time for the selection of the regularization parameter. To the best of our knowledge, this is the first attempt to select the regularization parameter for the low rank matrix recovery.

A Quaternion Matrix Equation with Two Different Restrictions

In this paper, we use the equivalence canonical form of four quaternion matrices to consider two systems of quaternion matrix equations. We derive some practical necessary and sufficient conditions for the existence of a solution to these two systems in terms of ranks of the quaternion matrices involved. We also give the general solutions to the systems when the solvability conditions are satisfied. Moreover, we present some numerical examples to illustrate the results of this paper.

Quadric surfaces in the Pfaffian hypersurface in ℙ14

We study smooth quadric surfaces in the Pfaffian hypersurface in P 14 parameterizing 6 × 6 skew-symmetric matrices of rank at most 4, not intersecting its singular locus. Such surfaces correspond to quadratic systems of skew-symmetric matrices of size 6 and constant rank 4, and give rise to a globally generated vector bundle E on the quadric. We analyse these bundles and their geometry, relating them to linear congruences in P 5 .

Stability Analysis upon High-Pressure Common Rail Fuel Injection System under Multiple Injection Modes

Fuel injection stability of high-pressure common rail fuel injection system (HPCRFIS) under multiple injection modes is a recognized technical problem. To explain it essentially, a numerical model based on bond graph method, which provides a standardized series of symbol system to describe various components in different physical fields as well as interactive effect analysis tool for different energy categories, was used to investigate the problem. As HPCRFIS is a typical nonlinear system with multi-physical coupling, it is very difficult to analyze system stability directly. Therefore, the system was linearized at typical moments. The system matrix for each typical moment was obtained thereby. To investigate the influential mechanism of system stability deeply, the variations of rank and eigenvalues distributions of state matrices were analyzed. The results show that the rank of state matrices changes abruptly when the needle or control valve moves. In terms of system stability, the oscillation characteristics are subject to the movement of needle or control valve. The ending of the former injection causes the system to show damping oscillation characteristics before the latter injection starts. The motion of control valve makes more contribution to system stability than that of needle, which could be concluded by Lyapunov stability criterion.

An approximation method of CP rank for third-order tensor completion

We study the problem of third-order tensor completion based on low CP rank recovery. Due to the NP-hardness of the calculation of CP rank, we propose an approximation method by using the sum of ranks of a few matrices as an upper bound of CP rank. We show that such upper bound is between CP rank and the square of CP rank of a tensor. This approximation would be useful when the CP rank is very small. Numerical algorithms are developed and examples are presented to demonstrate that the tensor completion performance by the proposed method is better than that of existing methods.

Dynamics of consumption inequality among occupational groups in India: Evidence from National Sample Survey data

This study analyses the dynamics of consumption inequality among various occupational groups using household survey data from National Sample Survey (NSS). Three rounds of NSS data namely, the 50th (1993–1994), 61st (2004–2005), and 66th (2009–2010) were taken for the purpose of analysis. Various measures such as Gini coefficient, overlapping index, and raking matrices are estimated for the whole as well as the subgroups using ANOGI methodology. From the analysis, it is found that the level of overall inequality as well as the between‐group inequality is on the rise during the period of analysis. Further, the measures such as relative overlap index and ranking matrices give conclusive evidence that the difference between various groups in terms of consumption expenditure pattern is increasing over time. From the ranking matrices, it is seen that the last six groups, constituting households with occupation in the unorganized sector and low level organized workers falls below the 30th percentile of consumption expenditure distribution of the first group groups that consists of individuals holding to technical, administrative, managerial, and executive occupations. Worsening inequality across various occupational groups is a matter of concern, especially from the policy perspective. As economic growth without equitable distribution of income is not a desirable outcome.

BLAS IV: A BLAS for Rk Matrix Algebra

Basic Linear Algebra Subroutines (BLAS) are well-known low-level workhorse subroutines for linear algebra vector-vector, matrixvector and matrix-matrix operations for full rank matrices. The advent of block low rank (Rk) full wave direct solvers, where most blocks of the system matrix are Rk, an extension to the BLAS III matrix-matrix work horse routine is needed due to the agony of Rk addition. This note outlines the problem of BLAS III for Rk LU and solve operations and then outlines an alternative approach, which we will call BLAS IV. This approach utilizes the thrill of Rk matrix-matrix multiply and uses the Adaptive Cross Approximation (ACA) as a methodology to evaluate sums of Rk terms to circumvent the agony of low rank addition.

Estimating Leverage Scores via Rank Revealing Methods and Randomization

We study algorithms for estimating the statistical leverage scores of rectangular dense or sparse matrices of arbitrary rank. Our approach is based on combining rank revealing methods with compositions of dense and sparse randomized dimensionality reduction transforms. We first develop a set of fast novel algorithms for rank estimation, column subset selection and least squares preconditioning. We then describe the design and implementation of leverage score estimators based on these primitives. These estimators are also effective for rank deficient input, which is frequently the case in data analytics applications. We provide detailed complexity analyses for all algorithms as well as meaningful approximation bounds and comparisons with the state-of-the-art. We conduct extensive numerical experiments to evaluate our algorithms and to illustrate their properties and performance using synthetic and real world data sets.

Testing Rank of Incomplete Unimodal Matrices

Several statistics-based detectors, based on unimodal matrix models, for determining the number of sources in a field are designed. A new variance ratio statistic is proposed, and its asymptotic distribution is analyzed. The variance ratio detector is shown to outperform the alternatives. It is shown that further improvements are achievable via optimally selected rotations. Numerical experiments demonstrate the performance gains of our detection methods over the baseline approach.

Nonisotropic symplectic graphs over finite commutative rings

<abstract><p>In this paper, we study two types of nonisotropic symplectic graphs over finite commutative rings defined by nonisotropic free submodules of rank $ 2 $ and McCoy rank of matrices. We prove that the graphs are quasi-strongly regular or Deza graphs and we find their parameters. The diameter and vertex transitivity are also analyzed. Moreover, we study subconstituents of these nonisotropic symplectic graphs.</p></abstract>

Rank $2r$ Iterative Least Squares: Efficient Recovery of Ill-Conditioned Low Rank Matrices from Few Entries

We present a new, simple, and computationally efficient iterative method for low rank matrix completion. Our method is inspired by the class of factorization-type iterative algorithms, but substantially differs from them in the way the problem is cast. Precisely, given a target rank $r$, instead of optimizing on the manifold of rank $r$ matrices, we allow our interim estimated matrix to have a specific overparametrized rank $2r$ structure. Our algorithm, denoted R2RILS, for rank $2r$ iterative least squares, thus has low memory requirements, and at each iteration it solves a computationally cheap sparse least squares problem. We motivate our algorithm by its theoretical analysis for the simplified case of a rank 1 matrix. Empirically, R2RILS is able to recover ill-conditioned low rank matrices from very few observations---near the information limit---and it is stable to additive noise.

Multiscale Low-Rank Spatial Features for Hyperspectral Image Classification

This letter presents a multiscale low-rank decomposition (MSLRD) method to extract multiscale spatial structures from hyperspectral images. The MSLRD assumes that ground objects have divergent characteristics in changing spatial scales. It decomposes each band image into a series of block-wise matrices, where these low-rank blocks take detailed spatial structures at multiple scales. It formulates the low-rank matrix decomposition problem into minimizing the ranks of all block matrices and adopts the alternative direction of the multiplier method to optimize it. Experiments on Indian Pines and Pavia University data sets show that the MSLRD can greatly improve the classification performance of regular classification on spectral features (i.e., all bands) and perform better than five state-of-the-art spatial feature extraction methods.

Rayleigh Fading Modeling and Channel Hardening for Reconfigurable Intelligent Surfaces

A realistic performance assessment of any wireless technology requires the use of a channel model that reflects its main characteristics. The independent and identically distributed Rayleigh fading channel model has been (and still is) the basis of most theoretical research on multiple antenna technologies in scattering environments. This letter shows that such a model is not physically appearing when using a reconfigurable intelligent surface (RIS) with rectangular geometry and provides an alternative physically feasible Rayleigh fading model that can be used as a baseline when evaluating RIS-aided communications. The model is used to revisit the basic RIS properties, e.g., the rank of spatial correlation matrices and channel hardening.

Tensor theta norms and low rank recovery

We study extensions of compressive sensing and low rank matrix recovery to the recovery of tensors of low rank from incomplete linear information. While the reconstruction of low rank matrices via nuclear norm minimization is rather well-understand by now, almost no theory is available so far for the extension to higher order tensors due to various theoretical and computational difficulties arising for tensor decompositions. In fact, nuclear norm minimization for matrix recovery is a tractable convex relaxation approach, but the extension of the nuclear norm to tensors is in general NP-hard to compute. In this article, we introduce convex relaxations of the tensor nuclear norm which are computable in polynomial time via semidefinite programming. Our approach is based on theta bodies, a concept from real computational algebraic geometry which is similar to the one of the better known Lasserre relaxations. We introduce polynomial ideals which are generated by the second-order minors corresponding to different matricizations of the tensor (where the tensor entries are treated as variables) such that the nuclear norm ball is the convex hull of the algebraic variety of the ideal. The theta body of order k for such an ideal generates a new norm which we call the θk-norm. We show that in the matrix case, these norms reduce to the standard nuclear norm. For tensors of order three or higher however, we indeed obtain new norms. The sequence of the corresponding unit-θk-norm balls converges asymptotically to the unit tensor nuclear norm ball. By providing the Gröbner basis for the ideals, we explicitly give semidefinite programs for the computation of the θk-norm and for the minimization of the θk-norm under an affine constraint. Finally, numerical experiments for order-three tensor recovery via θ1-norm minimization suggest that our approach successfully reconstructs tensors of low rank from incomplete linear (random) measurements.

Some new results in random matrices over finite fields

In this note we give various characterizations of random walks with possibly different steps that have relatively large discrepancy from the uniform distribution modulo a prime p, and use these results to study the distribution of the rank of random matrices over F_p and the equi-distribution behavior of normal vectors of random hyperplanes. We also study the probability that a random square matrix is eigenvalue-free, or when its characteristic polynomial is divisible by a given irreducible polynomial in the limit n to infinity in F_p. We show that these statistics are universal, extending results of Stong and Neumann-Praeger beyond the uniform model.

Linear Matroid Intersection is in Quasi-NC

Given two matroids on the same ground set, the matroid intersection problem asks to find a common independent set of maximum size. In case of linear matroids, the problem had a randomized parallel algorithm but no deterministic one. We give an almost complete derandomization of this algorithm, which implies that the linear matroid intersection problem is in quasi-NC. That is, it has uniform circuits of quasi-polynomial size $$n^{O(\log n)}$$ and O(polylog(n)) depth. Moreover, the depth of the circuit can be reduced to O(log2 n) in case of zero characteristic fields. This generalizes a similar result for the bipartite perfect matching problem. Our main technical contribution is to derandomize the Isolation lemma for the family of common bases of two matroids. We use our isolation result to give a quasi-polynomial time blackbox algorithm for a special case of Edmonds' problem, i.e., singularity testing of a symbolic matrix, when the given matrix is of the form $$A_{0} + A_{1 }x_{1} + \cdots + A_{m} x_{m}$$ , for an arbitrary matrix A0 and rank-1 matrices $$A_{1}, A_{2}, \dots, A_{m}$$ . This can also be viewed as a blackbox polynomial identity testing algorithm for the corresponding determinant polynomial. Another consequence of this result is a deterministic solution to the maximum rank matrix completion problem. Finally, we use our result to find a deterministic representation for the union of linear matroids in quasi-NC.

Fast median computation for symmetric, orthogonal matrices under the rank distance

Biological genomes can be represented as square, symmetric, orthogonal, 0-1 matrices. It turns out that the rank distance applied to two genome matrices has a biological significance: it is related to the smallest number of basic rearrangement mutations, such as reversals, translocations, transpositions (taken with weight 2), etc. that explain the differences between the two genomes. Therefore, closer genomes will produce smaller rank distances. An important tool in this context is the median problem: given three genomes A , B , and C , find a fourth genome M that minimizes d ( A , M ) + d ( B , M ) + d ( C , M ) . For genome matrices, the computational complexity of this problem is currently unknown. However, for orthogonal matrices, there are fast algorithms that solve it exactly. One such algorithm uses a “walk towards the median” paradigm. Starting from any of the input matrices, say, B , the algorithm produces rank-1 “steps”, which are rank-1 matrices that, added to B , decrease its rank distance to both A and C simultaneously. It can be shown that such steps always exist for orthogonal matrices, and can be found in polynomial time. The algorithm stops when no more improvement can be made, which is equivalent to saying that B lies between A and C in terms of the rank distance (the triangle inequality becomes an equality). There is an O ( n ω + 1 ) algorithm implementing this idea, where ω is the matrix multiplication exponent. Here we propose a novel scheme that works for symmetric orthogonal matrices, and produces a median, also guaranteed to be symmetric, in O ( n ω ) time. There is another O ( n ω ) time algorithm that produces the so-called M I median, which agrees with the majority in the subspaces where A = B , B = C , or C = A , and is equal to the identity in the orthogonal complement of the sum of these subspaces. However, this algorithm produces a different median, and has only been proved to be correct for genomic matrices. The algorithm we present here is more general.

The Channel Matrices for Determining the Entanglement Pattern of Multi-Partite State

We propose the general reduced channel matrices of multi-partite state for determining the entanglement pattern of the state. The matrix elements of reduced channel matrices are nothing but the coefficients of the state so that the rank of channel matrices can be easier determined. The entanglement pattern can be obtained from the evaluation of the rank of reduced matrices in various level of the reduced state. We apply the method on three multi-qubit states as example.

Roos bound for skew cyclic codes in Hamming and rank metric

In this paper, a Roos like bound on the minimum distance for skew cyclic codes over a general field is provided. The result holds in the Hamming metric and in the rank metric. The proofs involve arithmetic properties of skew polynomials and an analysis of the rank of parity-check matrices. For the rank metric case, a way to arithmetically construct codes with a prescribed minimum rank distance, using the skew Roos bound, is also given. Moreover, some examples of MDS codes and MRD codes over finite fields are built, using the skew Roos bound.