Fast Rectangular Matrix Multiplication Research Articles

Nonuniform ACC Circuit Lower Bounds

The class ACC consists of circuit families with constant depth over unbounded fan-in AND, OR, NOT, and MOD m gates, where m > 1 is an arbitrary constant. We prove the following. ---NEXP, the class of languages accepted in nondeterministic exponential time, does not have nonuniform ACC circuits of polynomial size. The size lower bound can be slightly strengthened to quasipolynomials and other less natural functions. ---E NP , the class of languages recognized in 2 O(n) time with an NP oracle, doesn’t have nonuniform ACC circuits of 2 n o(1) size. The lower bound gives an exponential size-depth tradeoff: for every d, m there is a δ > 0 such that E NP doesn’t have depth- d ACC circuits of size 2 n δ with MOD m gates. Previously, it was not known whether EXP NP had depth-3 polynomial-size circuits made out of only MOD 6 gates. The high-level strategy is to design faster algorithms for the circuit satisfiability problem over ACC circuits, then prove that such algorithms entail these lower bounds. The algorithms combine known properties of ACC with fast rectangular matrix multiplication and dynamic programming, while the second step requires a strengthening of the author’s prior work.

Recognition of $C_4$-Free and 1/2-Hyperbolic Graphs

The shortest-path metric ${\textup{d}}$ of a connected graph $G$ is ${1}/{2}$-hyperbolic if and only if it satisfies ${\textup{d}}(u,v) + {\textup{d}}(x,y) \leq \max \{ {\textup{d}}(u,x) + {\textup{d}}(v,y), {\textup{d}}(u,y) + {\textup{d}}(v,x) \} + 1$, for every $4$-tuple $u$, $x$, $v$, $y$ of $G$. We show that the problem of deciding whether an unweighted graph is ${1}/{2}$-hyperbolic is subcubic equivalent to the problem of determining whether there is a chordless cycle of length $4$ in a graph. An improved algorithm is also given for both problems, taking advantage of fast rectangular matrix multiplication. In the worst case it runs in $O(n^{3.26})$-time.

On Minimum Witnesses for Boolean Matrix Multiplication

Minimum witnesses for Boolean matrix multiplication play an important role in several graph algorithms. For two Boolean matrices A and B of order n, with one of the matrices having at most m nonzero entries, the fastest known algorithms for computing the minimum witnesses of their product run in either O(n2.575) time or in O(n2+mnlog(n2/m)/log2n) time. We present a new algorithm for this problem. Our algorithm runs either in time $$\tilde{O}\bigl(n^{\frac{3}{4-\omega}}m^{1-\frac{1}{4-\omega }}\bigr) $$ where ω<2.376 is the matrix multiplication exponent, or, if fast rectangular matrix multiplication is used, in time $$O\bigl(n^{1.939}m^{0.318}\bigr). $$ In particular, if ω−1<α<2 where m=nα, the new algorithm is faster than both of the aforementioned algorithms.

Faster Algorithms for All-Pairs Small Stretch Distances in Weighted Graphs

Let G=(V,E) be a weighted undirected graph, with non-negative edge weights. We consider the problem of efficiently computing approximate distances between all pairs of vertices in G. While many efficient algorithms are known for this problem in unweighted graphs, not many results are known for this problem in weighted graphs. Zwick (J. Assoc. Comput. Mach. 49:289–317, 2002) showed that for any fixed e>0, stretch 1+e distances (a path in G between u,v∈V is said to be of stretch t if its length is at most t times the distance between u and v in G) between all pairs of vertices in a weighted directed graph on n vertices can be computed in $\tilde{O}(n^{\omega})$ time, where ω 0 in G can be computed in expected time O(n 9/4). This algorithm uses a fast rectangular matrix multiplication subroutine. We also present a combinatorial algorithm (that is, it does not use fast matrix multiplication) with expected running time O(n 9/4) for computing all-pairs stretch 5/2 distances in G. This combinatorial algorithm will serve as a key step in our all-pairs stretch 2+e distances algorithm.

Fast Polynomial Factorization and Modular Composition

We obtain randomized algorithms for factoring degree $n$ univariate polynomials over $\mathbb{F}_q$ requiring $O(n^{1.5 + o(1)}\,{\rm log}^{1+o(1)} q+ n^{1 + o(1)}\,{\rm log}^{2+o(1)} q)$ bit operations. When ${\rm log}\, q < n$, this is asymptotically faster than the best previous algorithms [J. von zur Gathen and V. Shoup, Comput. Complexity, 2 (1992), pp. 187-224; E. Kaltofen and V. Shoup, Math. Comp., 67 (1998), pp. 1179-1197]; for ${\rm log}\, q \ge n$, it matches the asymptotic running time of the best known algorithms. The improvements come from new algorithms for modular composition of degree $n$ univariate polynomials, which is the asymptotic bottleneck in fast algorithms for factoring polynomials over finite fields. The best previous algorithms for modular composition use $O(n^{(\omega + 1)/2})$ field operations, where $\omega$ is the exponent of matrix multiplication [R. P. Brent and H. T. Kung, J. Assoc. Comput. Mach., 25 (1978), pp. 581-595], with a slight improvement in the exponent achieved by employing fast rectangular matrix multiplication [X. Huang and V. Y. Pan, J. Complexity, 14 (1998), pp. 257-299]. We show that modular composition and multipoint evaluation of multivariate polynomials are essentially equivalent, in the sense that an algorithm for one achieving exponent $\alpha$ implies an algorithm for the other with exponent $\alpha + o(1)$, and vice versa. We then give two new algorithms that solve the problem near-optimally: an algebraic algorithm for fields of characteristic at most $n^{o(1)}$, and a nonalgebraic algorithm that works in arbitrary characteristic. The latter algorithm works by lifting to characteristic 0, applying a small number of rounds of multimodular reduction, and finishing with a small number of multidimensional FFTs. The final evaluations are reconstructed using the Chinese remainder theorem. As a bonus, this algorithm produces a very efficient data structure supporting polynomial evaluation queries, which is of independent interest. Our algorithms use techniques that are commonly employed in practice, in contrast to all previous subquadratic algorithms for these problems, which relied on fast matrix multiplication.

A Fast Output-Sensitive Algorithm for Boolean Matrix Multiplication

We use randomness to exploit the potential sparsity of the Boolean matrix product in order to speed up the computation of the product. Our new fast output-sensitive algorithm for Boolean matrix product and its witnesses is randomized and provides the Boolean product and its witnesses almost certainly. Its worst-case time performance is expressed in terms of the input size and the number of non-zero entries of the product matrix. It runs in time $\widetilde{O}(n^{2}s^{\omega/2-1})$ , where the input matrices have size n×n, the number of non-zero entries in the product matrix is at most s, ? is the exponent of the fast matrix multiplication and $\widetilde{O}(f(n))$ denotes O(f(n)log? d n) for some constant d. By the currently best bound on ?, its running time can be also expressed as $\widetilde{O}(n^{2}s^{0.188})$ . Our algorithm is substantially faster than the output-sensitive column-row method for Boolean matrix product for s larger than n 1.232 and it is never slower than the fast $\widetilde{O}(n^{\omega})$ -time algorithm for this problem. By applying the fast rectangular matrix multiplication, we can refine our upper bound further to the form $\widetilde{O}(n^{\omega(\frac{1}{2}\log_{n}s,1,1)})$ , where ?(p,q,t) is the exponent of the fast multiplication of an n p ×n q matrix by an n q ×n t matrix. We also present a partial derandomization of our algorithm as well as its generalization to include the Boolean product of rectangular Boolean matrices. Finally, we show several applications of our output-sensitive algorithms.

Dynamic matrix rank

We consider maintaining information about the rank of a matrix under changes of the entries. For n × n matrices, we show an upper bound of O ( n 1.575 ) arithmetic operations and a lower bound of Ω ( n ) arithmetic operations per element change. The upper bound is valid when changing up to O ( n 0.575 ) entries in a single column of the matrix. We also give an algorithm that maintains the rank using O ( n 2 ) arithmetic operations per rank one update. These bounds appear to be the first nontrivial bounds for the problem. The upper bounds are valid for arbitrary fields, whereas the lower bound is valid for algebraically closed fields. The upper bound for element updates uses fast rectangular matrix multiplication, and the lower bound involves further development of an earlier technique for proving lower bounds for dynamic computation of rational functions.

Fast rectangular matrix multiplication and some applications

We study asymptotically fast multiplication algorithms for matrix pairs of arbitrary dimensions, and optimize the exponents of their arithmetic complexity bounds. For a large class of input matrix pairs, we improve the known exponents. We also show some applications of our results: (i) we decrease from O(n 2 + n 1+o(1)logq) to O(n 1.9998 + n 1+o(1)logq) the known arithmetic complexity bound for the univariate polynomial factorization of degree n over a finite field with q elements; (ii) we decrease from 2.837 to 2.7945 the known exponent of the work and arithmetic processor bounds for fast deterministic (NC) parallel evaluation of the determinant, the characteristic polynomial, and the inverse of an n × n matrix, as well as for the solution to a nonsingular linear system of n equations; (iii) we decrease from O(m 1.575 n) to O(m 1.5356 n) the known bound for computing basic solutions to a linear programming problem with m constraints and n variables.

Fast sparse matrix multiplication

Let A and B two n × n matrices over a ring R (e.g., the reals or the integers) each containing at most m nonzero elements. We present a new algorithm that multiplies A and B using O ( m 0.7 n 1.2 + n 2+ o (1) ) algebraic operations (i.e., multiplications, additions and subtractions) over R . The naïve matrix multiplication algorithm, on the other hand, may need to perform Ω( mn ) operations to accomplish the same task. For m ≤ n 1.14 , the new algorithm performs an almost optimal number of only n 2+ o (1) operations. For m ≤ n 1.68 , the new algorithm is also faster than the best known matrix multiplication algorithm for dense matrices which uses O ( n 2.38 ) algebraic operations. The new algorithm is obtained using a surprisingly straightforward combination of a simple combinatorial idea and existing fast rectangular matrix multiplication algorithms. We also obtain improved algorithms for the multiplication of more than two sparse matrices. As the known fast rectangular matrix multiplication algorithms are far from being practical, our result, at least for now, is only of theoretical value.

Fast Rectangular Matrix Multiplication and Applications

First we study asymptotically fast algorithms for rectangular matrix multiplication. We begin with new algorithms for multiplication of ann×nmatrix by ann×n2matrix in arithmetic timeO(nω),ω=3.333953…, which is less by 0.041 than the previous record 3.375477…. Then we present fast multiplication algorithms for matrix pairs of arbitrary dimensions, estimate the asymptotic running time as a function of the dimensions, and optimize the exponents of the complexity estimates. For a large class of input matrix pairs, we improve the known exponents. Finally we show three applications of our results: (a)we decrease from 2.851 to 2.837 the known exponent of the work bounds for fast deterministic (NC) parallel evaluation of the determinant, the characteristic polynomial, and the inverse of ann×nmatrix, as well as for the solution to a nonsingular linear system ofnequations, (b)we asymptotically accelerate the known sequential algorithms for the univariate polynomial composition modxn, yielding the complexity boundO(n1.667) versus the old record ofO(n1.688), and for the univariate polynomial factorization over a finite field, and (c)we improve slightly the known complexity estimates for computing basic solutions to the linear programming problem withmconstraints andnvariables.

Fast rectangular matrix multiplication and QR decomposition

In the last twenty-five years there has been much research into “fast” matrix multiplication methods: ones that have an asymptotically smaller operation count than conventional multiplication. Most fast methods are derived for square matrices, but they can be applied to rectangular matrices by a blocking technique. We obtain an expression for the order of the operation count for this blocked multiplication of rectangular matrices. We derive an exact operation count for Strassen's method with rectangular matrices and determine the recursion threshold that minimizes the operation count. We also show that when Strassen's method is used to multiply rectangular matrices it is more efficient to use the method on the whole product than to apply the method to square submatrices. Fast multiplication methods can be exploited in calculating a QR decomposition of an m × n matrix. We show that the operation count can be reduced from O( mn 2) to O(mn 1+( 1 (4-α) ) ) by using a fast multiplication method with exponent α in conjunction with Bischof and Van Loan's WY representation of a product of Householder transformations.

Rapid Multiplication of Rectangular Matrices

The number of essential multiplications required to multiply matrices of size $N \times N$ and $N \times N^{0.172} $ is bounded by $CN^2 \log ^2 N$.