Fast Matrix Multiplication Algorithms Research Articles

Semi-analytical solution of Brent equations

A parametrization of Brent equations is proposed which admit for a several times reduction of the number of unknowns and equations. The arising equations are solved numerically, and for the resulting fast matrix multiplication algorithms many known values of rank are reproduced and even improved, in particular, the designs (4,4,4;48) and (2,4,5;32) are found.

Correction of ‘J. Laderman, V. Pan, X.–H. Sha, On practical Algorithms for Accelerated Matrix Multiplication, Linear Algebra and its Applications. Vol. 162-164 (1992) pp. 557-588’

In this article, we corrected the trilinear formula for triple disjoint matrix multiplication given in the article ‘J. Laderman, V. Pan, X. H. Sha, On practical Algorithms for Accelerated Matrix Multiplication, Linear Algebra and its Applications. Vol. 162-164 (1992) pp. 557-588’, which is incorrect for matrix dimensions equal to two or greater. That formula is a base of two algorithms, for disjoint and single matrix multiplication. The necessary correction made the amount of non–scalar products rise, slightly increasing the algorithm time complexity. We also corrected explicit formulas, in the bilinear form, for triple disjoint matrix multiplication. They are explicit, and thus convenient for the practical use of the fast matrix multiplication algorithms in question.

Research on fast matrix multiplication algorithm

This research mainly focuses on fast matrix multiplication algorithms. Fast matrix multiplication is one of the most fundamental problems in computer science. The fast matrix multiplication algorithm differs from conventional matrix multiplication in that it offers a faster computational approach that can perform the operation in less than O(n3) time complexity. This algorithm provides a more efficient method for multiplying matrices, significantly reducing the computational requirements. The Laser method, developed by Coppersmith and Winograd, is an algorithm for matrix multiplication that does not involve direct computation. It establishes a relationship between matrix multiplication and tensors and simplifies the operation by finding an intermediate tensor that is computationally manageable. This method applies a series of simplification operations to determine an upper bound on the computational complexity of matrix multiplication. However, as matrices become larger, the computational and memory requirements increase, posing challenges for practical implementation. This research will present the main ideas and performance of the Laser method and discuss the improvements made to the Laser method, including refined analysis and asymmetric hashing techniques. Additionally, it highlights the need for further exploration, such as parallel computing and optimization strategies, to enhance the efficiency of matrix multiplication algorithms. Furthermore, this research will also provide a prospectus for the future of matrix multiplication algorithms, such as the practical implementation of the Laser method.

Computing Generalized Convolutions Faster Than Brute Force

In this paper, we consider a general notion of convolution. Let D be a finite domain and let D^n be the set of n-length vectors (tuples) of D. Let f :Dtimes Drightarrow D be a function and let oplus _f be a coordinate-wise application of f. The f-Convolution of two functions g,h :D^n rightarrow {-M,ldots ,M} is (g⊛fh)(v):=∑vg,vh∈Dns.t.v=vg⊕fvhg(vg)·h(vh)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\begin{aligned} (g \\mathbin {\\circledast _{f}}h)(\ extbf{v}) {:}{=}\\sum _{\\begin{array}{c} \ extbf{v}_g,\ extbf{v}_h \\in D^n\\\\ \ ext {s.t. } \ extbf{v}= \ extbf{v}_g \\oplus _f \ extbf{v}_h \\end{array}} g(\ extbf{v}_g) \\cdot h(\ extbf{v}_h) \\end{aligned}$$\\end{document}for every textbf{v}in D^n. This problem generalizes many fundamental convolutions such as Subset Convolution, XOR Product, Covering Product or Packing Product, etc. For arbitrary function f and domain D we can compute f-Convolution via brute-force enumeration in widetilde{{mathcal {O}}}(|D|^{2n} cdot textrm{polylog}(M)) time. Our main result is an improvement over this naive algorithm. We show that f-Convolution can be computed exactly in widetilde{{mathcal {O}}}( (c cdot |D|^2)^{n} cdot textrm{polylog}(M)) for constant c {:}{=}3/4 when D has even cardinality. Our main observation is that a cyclic partition of a function f :Dtimes Drightarrow D can be used to speed up the computation of f-Convolution, and we show that an appropriate cyclic partition exists for every f. Furthermore, we demonstrate that a single entry of the f-Convolution can be computed more efficiently. In this variant, we are given two functions g,h :D^n rightarrow {-M,ldots ,M} alongside with a vector textbf{v}in D^n and the task of the f-Query problem is to compute integer (g mathbin {circledast _{f}}h)(textbf{v}). This is a generalization of the well-known Orthogonal Vectors problem. We show that f-Query can be computed in widetilde{{mathcal {O}}}(|D|^{frac{omega }{2} n} cdot textrm{polylog}(M)) time, where omega in [2,2.372) is the exponent of currently fastest matrix multiplication algorithm.

Privacy-preserving cancer type prediction with homomorphic encryption

Cancer genomics tailors diagnosis and treatment based on an individual’s genetic information and is the crux of precision medicine. However, analysis and maintenance of high volume of genetic mutation data to build a machine learning (ML) model to predict the cancer type is a computationally expensive task and is often outsourced to powerful cloud servers, raising critical privacy concerns for patients’ data. Homomorphic encryption (HE) enables computation on encrypted data, thus, providing cryptographic guarantees to protect privacy. But restrictive overheads of encrypted computation deter its usage. In this work, we explore the challenges of privacy preserving cancer type prediction using a dataset consisting of more than 2 million genetic mutations from 2713 patients for several cancer types by building a highly accurate ML model and then implementing its privacy preserving version in HE. Our solution for cancer type inference encodes somatic mutations based on their impact on the cancer genomes into the feature space and then uses statistical tests for feature selection. We propose a fast matrix multiplication algorithm for HE-based model. Our final model achieves 0.98 micro-average area under curve improving accuracy from 70.08 to 83.61% , being 550 times faster than the standard matrix multiplication-based privacy-preserving models. Our tool can be found at https://github.com/momalab/octal-candet.

Equivalent polyadic decompositions of matrix multiplication tensors

Invariance transformations of polyadic decompositions of matrix multiplication tensors define an equivalence relation on the set of such decompositions. In this paper, we present an algorithm to efficiently decide whether two polyadic decompositions of a given matrix multiplication tensor are equivalent. With this algorithm, we analyze the equivalence classes of decompositions of several matrix multiplication tensors. This analysis is relevant for the study of fast matrix multiplication as it relates to the question of how many essentially different fast matrix multiplication algorithms there exist. This question has been first studied by de Groote, who showed that for the multiplication of 2 ×2 matrices with 7 active multiplications, all algorithms are essentially equivalent to Strassen’s algorithm. In contrast, the results of our analysis show that for the multiplication of larger matrices (e.g., 2 ×3 by 3 ×2 or 3 ×3 by 3 ×3 matrices), two decompositions are very likely to be essentially different. We further provide a necessary criterion for a polyadic decomposition to be equivalent to a polyadic decomposition with integer entries. Decompositions with specific integer entries, e.g., powers of two, provide fast matrix multiplication algorithms with better efficiency and stability properties. This condition can be tested algorithmically and we present the conclusions obtained for the decompositions of small/medium matrix multiplication tensors.

Universal points in the asymptotic spectrum of tensors

Motivated by the problem of constructing fast matrix multiplication algorithms, Strassen (FOCS 1986, Crelle 1987–1991) introduced and developed the theory of asymptotic spectra of tensors. For any sub-semiringX\mathcal {X}of tensors (under direct sum and tensor product), the duality theorem that is at the core of this theory characterizes basic asymptotic properties of the elements ofX\mathcal {X}in terms ofthe asymptotic spectrum ofX\mathcal {X}, which is defined as the collection of semiring homomorphisms fromX\mathcal {X}to the non-negative reals with a natural monotonicity property. The asymptotic properties characterized by this duality encompass fundamental problems in complexity theory, combinatorics and quantum information.Universal spectral pointsare elements in the asymptotic spectrum of the semiring ofalltensors. Finding all universal spectral points suffices to find the asymptotic spectrum ofanysub-semiring. The construction of non-trivial universal spectral points has been an open problem for more than thirty years. We construct, for the first time, a family of non-trivial universal spectral points over the complex numbers, calledquantum functionals. We moreover prove that the quantum functionals precisely characterise the asymptotic slice rank of complex tensors. Our construction, which relies on techniques from quantum information theory and representation theory, connects the asymptotic spectrum of tensors to the quantum marginal problem and entanglement polytopes.

GCSA Codes With Noise Alignment for Secure Coded Multi-Party Batch Matrix Multiplication

A secure multi-party batch matrix multiplication problem (SMBMM) is considered, where the goal is to allow a master to efficiently compute the pairwise products of two batches of massive matrices, by distributing the computation across S servers. Any X colluding servers gain no information about the input, and the master gains no additional information about the input beyond the product. A solution called Generalized Cross Subspace Alignment codes with Noise Alignment (GCSA- NA) is proposed in this work, based on cross-subspace alignment codes. The state of art solution to SMBMM is a coding scheme called polynomial sharing (PS) that was proposed by Nodehi and Maddah-Ali. GCSA-NA outperforms PS codes in several key aspects—more efficient and secure inter-server communication, lower latency, flexible inter-server network topology, efficient batch processing, and tolerance to stragglers. The idea of noise alignment can also be combined with N-source Cross Subspace Alignment (N-CSA) codes and fast matrix multiplication algorithms like Strassen’s construction. Moreover, noise alignment can be applied to symmetric secure private information retrieval to achieve the asymptotic capacity.

Matrix Multiplication, a Little Faster

Strassen’s algorithm (1969) was the first sub-cubic matrix multiplication algorithm. Winograd (1971) improved the leading coefficient of its complexity from 6 to 7. There have been many subsequent asymptotic improvements. Unfortunately, most of these have the disadvantage of very large, often gigantic, hidden constants. Consequently, Strassen-Winograd’s O ( n log 2 7 ) algorithm often outperforms other fast matrix multiplication algorithms for all feasible matrix dimensions. The leading coefficient of Strassen-Winograd’s algorithm has been generally believed to be optimal for matrix multiplication algorithms with a 2 × 2 base case, due to the lower bounds by Probert (1976) and Bshouty (1995). Surprisingly, we obtain a faster matrix multiplication algorithm, with the same base case size and asymptotic complexity as Strassen-Winograd’s algorithm, but with the leading coefficient reduced from 6 to 5. To this end, we extend Bodrato’s (2010) method for matrix squaring, and transform matrices to an alternative basis. We also prove a generalization of Probert’s and Bshouty’s lower bounds that holds under change of basis, showing that for matrix multiplication algorithms with a 2 × 2 base case, the leading coefficient of our algorithm cannot be further reduced, and is therefore optimal. We apply our method to other fast matrix multiplication algorithms, improving their arithmetic and communication costs by significant constant factors.

Distillation of Greenberger–Horne–Zeilinger States by Combinatorial Methods

We prove a lower bound on the rate of Greenberger-Horne-Zeilinger states distillable from pure multipartite states by local operations and classical communication (LOCC). Our proof is based on a modification of a combinatorial argument used in the fast matrix multiplication algorithm of Coppersmith and Winograd. Previous use of methods from algebraic complexity in quantum information theory concerned transformations with stochastic local operations and classical operation (SLOCC), resulting in an asymptotically vanishing success probability. In contrast, our new protocol works with asymptotically vanishing error.

If the Current Clique Algorithms Are Optimal, so Is Valiant's Parser

The CFG recognition problem is as follows: given a context-free grammar $\mathcal{G}$ and a string $w$ of length $n$, decide whether $w$ can be obtained from $\mathcal{G}$. This is the most basic parsing question and is a core computer science problem. Valiant's parser from 1975 solves the problem in $O(n^{\omega})$ time, where $\omega<2.373$ is the matrix multiplication exponent. Dozens of parsing algorithms have been proposed over the years, yet Valiant's upper bound remains unbeaten. The best combinatorial algorithms have mildly subcubic $O(n^3/\log^3{n})$ time complexity. Evidence for the nonexistence of more efficient algorithms was given in a result of Lee (JACM'01), who showed that any algorithm with running time $O(|\mathcal{G}|\cdot n^{3-\varepsilon})$, which can solve a more general parsing problem in the unusual case that $|\mathcal{G}|=\Omega(n^6)$, can be converted into a surprising subcubic time algorithm for Boolean Matrix Multiplication. However, nothing was known for the more relevant case of constant size grammars. In this work, we make the first progress on the constant size grammar case in 40 years, and prove that any improvement on Valiant's algorithm, either in terms of runtime or by avoiding the inefficiencies of the known fast matrix multiplication algorithms, would imply a breakthrough algorithm for the $k$-Clique problem: given a graph of $n$ nodes, decide whether there are $k$ nodes that form a clique. Using similar techniques, we derive similar lower bounds for more modern and well-studied cubic time problems for which faster algorithms are highly desirable in practice: RNA Folding, a central problem in computational biology, and Dyck Language Edit Distance, answering an open question of Saha (FOCS'14).

Improving the Numerical Stability of Fast Matrix Multiplication

Fast algorithms for matrix multiplication, namely those that perform asymptotically fewer scalar operations than the classical algorithm, have been considered primarily of theoretical interest. Apart from Strassen's original algorithm, few fast algorithms have been efficiently implemented or used in practical applications. However, there exist many practical alternatives to Strassen's algorithm with varying performance and numerical properties. Fast algorithms are known to be numerically stable, but because their error bounds are slightly weaker than the classical algorithm, they are not used even in cases where they provide a performance benefit. We argue in this paper that the numerical sacrifice of fast algorithms, particularly for the typical use cases of practical algorithms, is not prohibitive, and we explore ways to improve the accuracy both theoretically and empirically. The numerical accuracy of fast matrix multiplication depends on properties of the algorithm and of the input matrices, and we consider both contributions independently. We generalize and tighten previous error analyses of fast algorithms and compare their properties. We discuss algorithmic techniques for improving the error guarantees from two perspectives: manipulating the algorithms, and reducing input anomalies by various forms of diagonal scaling. Finally, we benchmark performance and demonstrate our improved numerical accuracy.

A framework for practical parallel fast matrix multiplication

Matrix multiplication is a fundamental computation in many scientific disciplines. In this paper, we show that novel fast matrix multiplication algorithms can significantly outperform vendor implementations of the classical algorithm and Strassen's fast algorithm on modest problem sizes and shapes. Furthermore, we show that the best choice of fast algorithm depends not only on the size of the matrices but also the shape. We develop a code generation tool to automatically implement multiple sequential and shared-memory parallel variants of each fast algorithm, including our novel parallelization scheme. This allows us to rapidly benchmark over 20 fast algorithms on several problem sizes. Furthermore, we discuss a number of practical implementation issues for these algorithms on shared-memory machines that can direct further research on making fast algorithms practical.

Communication costs of Strassen's matrix multiplication

Algorithms have historically been evaluated in terms of the number of arithmetic operations they performed. This analysis is no longer sufficient for predicting running times on today's machines. Moving data through memory hierarchies and among processors requires much more time (and energy) than performing computations. Hardware trends suggest that the relative costs of this communication will only increase. Proving lower bounds on the communication of algorithms and finding algorithms that attain these bounds are therefore fundamental goals. We show that the communication cost of an algorithm is closely related to the graph expansion properties of its corresponding computation graph. Matrix multiplication is one of the most fundamental problems in scientific computing and in parallel computing. Applying expansion analysis to Strassen's and other fast matrix multiplication algorithms, we obtain the first lower bounds on their communication costs. These bounds show that the current sequential algorithms are optimal but that previous parallel algorithms communicate more than necessary. Our new parallelization of Strassen's algorithm is communication-optimal and outperforms all previous matrix multiplication algorithms.&lt;!-- END_PAGE_1 --&gt;

On sunflowers and matrix multiplication

We present several variants of the sunflower conjecture of Erdős & Rado (J Lond Math Soc 35:85–90, 1960) and discuss the relations among them. We then show that two of these conjectures (if true) imply negative answers to the questions of Coppersmith & Winograd (J Symb Comput 9:251–280, 1990) and Cohn et al. (2005) regarding possible approaches for obtaining fast matrix-multiplication algorithms. Specifically, we show that the Erdős–Rado sunflower conjecture (if true) implies a negative answer to the “no three disjoint equivoluminous subsets” question of Coppersmith & Winograd (J Symb Comput 9:251–280, 1990); we also formulate a “multicolored” sunflower conjecture in $${\mathbb{Z}_3^n}$$ and show that (if true) it implies a negative answer to the “strong USP” conjecture of Cohn et al. (2005) (although it does not seem to impact a second conjecture in Cohn et al. (2005) or the viability of the general group-theoretic approach). A surprising consequence of our results is that the Coppersmith–Winograd conjecture actually implies the Cohn et al. conjecture. The multicolored sunflower conjecture in $${\mathbb{Z}_3^n}$$ is a strengthening of the well-known (ordinary) sunflower conjecture in $${\mathbb{Z}_3^n}$$ , and we show via our connection that a construction from Cohn et al. (2005) yields a lower bound of (2.51 . . .) n on the size of the largest multicolored 3-sunflower-free set, which beats the current best-known lower bound of (2.21 . . . ) n Edel (2004) on the size of the largest 3-sunflower-free set in $${\mathbb{Z}_3^n}$$ .

Fast Matrix Multiplication

We give an overview of the history of fast algorithms for matrix multiplication. Along the way, we look at some other fundamental problems in algebraic complexity like polynomial evaluation. This exposition is self-contained. To make it accessible to a broad audience, we only assume a minimal mathematical background: basic linear algebra, familiarity with polynomials in several variables over rings, and rudimentary knowledge in combinatorics should be sufficient to read (and understand) this article. This means that we have to treat tensors in a very concrete way (which might annoy people coming from mathematics), occasionally prove basic results from combinatorics, and solve recursive inequalities explicitly (because we want to annoy people with a background in theoretical computer science, too).

Graph expansion and communication costs of fast matrix multiplication

The communication cost of algorithms (also known as I/O-complexity) is shown to be closely related to the expansion properties of the corresponding computation graphs. We demonstrate this on Strassen's and other fast matrix multiplication algorithms, and obtain the first lower bounds on their communication costs. In the sequential case, where the processor has a fast memory of size M , too small to store three n -by- n matrices, the lower bound on the number of words moved between fast and slow memory is, for a large class of matrix multiplication algorithms, Ω( ( n /√ M ) ω 0 · M ), where ω 0 is the exponent in the arithmetic count (e.g., ω 0 = lg 7 for Strassen, and ω 0 = 3 for conventional matrix multiplication). With p parallel processors, each with fast memory of size M , the lower bound is asymptotically lower by a factor of p . These bounds are attainable both for sequential and for parallel algorithms and hence optimal.

Universal and efficient compressed sensing by spread spectrum and application to realistic Fourier imaging techniques

We advocate a compressed sensing strategy that consists of multiplying the signal of interest by a wide bandwidth modulation before projection onto randomly selected vectors of an orthonormal basis. Firstly, in a digital setting with random modulation, considering a whole class of sensing bases including the Fourier basis, we prove that the technique is universal in the sense that the required number of measurements for accurate recovery is optimal and independent of the sparsity basis. This universality stems from a drastic decrease of coherence between the sparsity and the sensing bases, which for a Fourier sensing basis relates to a spread of the original signal spectrum by the modulation (hence the name "spread spectrum"). The approach is also efficient as sensing matrices with fast matrix multiplication algorithms can be used, in particular in the case of Fourier measurements. Secondly, these results are confirmed by a numerical analysis of the phase transition of the l1- minimization problem. Finally, we show that the spread spectrum technique remains effective in an analog setting with chirp modulation for application to realistic Fourier imaging. We illustrate these findings in the context of radio interferometry and magnetic resonance imaging.

Fast matrix multiplication techniques based on the Adleman-Lipton model

On distributed memory electronic computers, the implementation and association of fast parallel matrix multiplication algorithms has yielded astounding results and insights. In this discourse, we use the tools of molecular biology to demonstrate the theoretical encoding of Strassen's fast matrix multiplication algorithm with DNA based on an $n$-moduli set in the residue number system, thereby demonstrating the viability of computational mathematics with DNA. As a result, a general scalable implementation of this model in the DNA computing paradigm is presented and can be generalized to the application of \emph{all} fast matrix multiplication algorithms on a DNA computer. We also discuss the practical capabilities and issues of this scalable implementation. Fast methods of matrix computations with DNA are important because they also allow for the efficient implementation of other algorithms (i.e. inversion, computing determinants, and graph theory) with DNA.

A Deterministic Algorithm for the Frieze–Kannan Regularity Lemma

The Frieze–Kannan regularity lemma is a powerful tool in combinatorics. It has also found applications in the design of approximation algorithms and recently in the design of fast combinatorial algorithms for boolean matrix multiplication. The algorithmic applications of this lemma require one to efficiently construct a partition satisfying the conditions of the lemma. R. Williams recently asked if one can construct a partition satisfying the conditions of the Frieze–Kannan regularity lemma in deterministic subcubic time. We resolve this problem by designing an $\tilde O(n^{\omega})$ time algorithm for constructing such a partition, where $\omega < 2.376$ is the exponent of fast matrix multiplication. The algorithm relies on a spectral characterization of vertex partitions satisfying the properties of the Frieze–Kannan regularity lemma.