Approximate Matrix-vector Multiplication Research Articles

An Efficient Limited Memory Multi-Step Quasi-Newton Method

This paper is dedicated to the development of a novel class of quasi-Newton techniques tailored to address computational challenges posed by memory constraints. Such methodologies are commonly referred to as “limited” memory methods. The method proposed herein showcases adaptability by introducing a customizable memory parameter governing the retention of historical data in constructing the Hessian estimate matrix at each iterative stage. The search directions generated through this novel approach are derived from a modified version closely resembling the full memory multi-step BFGS update, incorporating limited memory computation for a singular term to approximate matrix–vector multiplication. Results from numerical experiments, exploring various parameter configurations, substantiate the enhanced efficiency of the proposed algorithm within the realm of limited memory quasi-Newton methodologies category.

Scaling the discrete-time Wright-Fisher model to biobank-scale datasets.

The discrete-time Wright-Fisher (DTWF) model and its diffusion limit are central to population genetics. These models can describe the forward-in-time evolution of allele frequencies in a population resulting from genetic drift, mutation, and selection. Computing likelihoods under the diffusion process is feasible, but the diffusion approximation breaks down for large samples or in the presence of strong selection. Existing methods for computing likelihoods under the DTWF model do not scale to current exome sequencing sample sizes in the hundreds of thousands. Here, we present a scalable algorithm that approximates the DTWF model with provably bounded error. Our approach relies on two key observations about the DTWF model. The first is that transition probabilities under the model are approximately sparse. The second is that transition distributions for similar starting allele frequencies are extremely close as distributions. Together, these observations enable approximate matrix-vector multiplication in linear (as opposed to the usual quadratic) time. We prove similar properties for Hypergeometric distributions, enabling fast computation of likelihoods for subsamples of the population. We show theoretically and in practice that this approximation is highly accurate and can scale to population sizes in the tens of millions, paving the way for rigorous biobank-scale inference. Finally, we use our results to estimate the impact of larger samples on estimating selection coefficients for loss-of-function variants. We find that increasing sample sizes beyond existing large exome sequencing cohorts will provide essentially no additional information except for genes with the most extreme fitness effects.

XMAP: Programming Memristor Crossbars for Analog Matrix–Vector Multiplication: Toward High Precision Using Representable Matrices

Linear transformations are the dominating computation within many important applications. The natural multiply and accumulate feature of memristor crossbar arrays promise unprecedented processing capabilities to resistive dot-product engines (DPEs), which can accelerate approximate matrix-vector multiplication (MVM). Unfortunately, the precision of the analog computation may be degraded by parasitics, non-linear device characteristics, and variations. In this paper, we propose a framework called XMAP for mapping an arbitrary matrix into appropriate memristor conductance values (or state variables for non-linear devices). The specified conductance values are next programmed to the memristor hardware using accurate closed-loop tuning. XMAP is based on formulating the mapping problem as a mathematical optimization problem, which can be elegantly minimized using the concept of representable matrices, i.e., the matrices that can be represented on a crossbar. Compared with the state-of-the-art conversion algorithm, the computational accuracy is improved with up to 3.29X at the expense of overhead in run-time. The precision improvements translate into noteworthy application level benefits within signal compression and neural network inference.

Efficient complementary resistive switch-based crossbar array Booth multiplier

Recent advances of memristive devices allow high endurance, non-volatile storage and low leakage power. Thus, these devices are suitable candidates for in-memory computing. Several recent studies explored the usage of memristive crossbar array for approximate and neuromorphic computing, including approximate matrix-vector multiplication. However, accurate digital circuit realization using device-level simulation, accounting for more realistic ReRAM device behavior, is only studied for adder circuits so far. In this paper, we report the first study of a multiplier scheme with complementary resistive switch-based crossbar arrays. An efficient mapping of Booth multiplication algorithm with different area-timing trade-offs, is discussed. Simulation studies are performed using 4-bit numbers to validate our approach.

Approximate multiplication in adaptive wavelet methods

Abstract Cohen, Dahmen and DeVore designed in [Adaptive wavelet methods for elliptic operator equations: convergence rates, Math. Comp., 2001, 70(233), 27–75] and [Adaptive wavelet methods II¶beyond the elliptic case, Found. Comput. Math., 2002, 2(3), 203–245] a general concept for solving operator equations. Its essential steps are: transformation of the variational formulation into the well-conditioned infinite-dimensional l 2-problem, finding the convergent iteration process for the l 2-problem and finally using its finite dimensional approximation which works with an inexact right-hand side and approximate matrix-vector multiplication. In our contribution, we pay attention to approximate matrix-vector multiplication which is enabled by an off-diagonal decay of entries of the wavelet stiffness matrices. We propose a more efficient technique which better utilizes actual decay of matrix and vector entries and we also prove that this multiplication algorithm is asymptotically optimal in the sense that storage and number of floating point operations, needed to resolve the problem with desired accuracy, remain proportional to the problem size when the resolution of the discretization is refined.

Rapid solution of first kind boundary integral equations in [formula omitted

Weakly singular boundary integral equations (BIEs) of the first kind on polyhedral surfaces Γ in R 3 are discretized by Galerkin BEM on shape-regular, but otherwise unstructured meshes of meshwidth h. Strong ellipticity of the integral operator is shown to give nonsingular stiffness matrices and, for piecewise constant approximations, up to O( h 3) convergence of the farfield. The condition number of the stiffness matrix behaves like O( h −1) in the standard basis. An O( N) agglomeration algorithm for the construction of a multilevel wavelet basis on Γ is introduced resulting in a preconditioner which reduces the condition number to O(| log h|). A class of kernel-independent clustering algorithms (containing the fast multipole method as special case) is introduced for approximate matrix–vector multiplication in O(N( log N) 3) memory and operations. Iterative approximate solution of the linear system by CG or GMRES with wavelet preconditioning and clustering-acceleration of matrix–vector multiplication is shown to yield an approximate solution in log-linear complexity which preserves the O( h 3) convergence of the potentials. Numerical experiments are given which confirm the theory.

On the fast matrix multiplication in the boundary element method by panel clustering

The boundary element method (BEM) leads to a system of linear equations with a full matrix, while FEM yields sparse matrices. This fact seems to require much computational work for the definition of the matrix, for the solution of the system, and, in particular, for the matrix-vector multiplication, which always occurs as an elementary. In this paper a method for the approximate matrix-vector multiplication is described which requires much less arithmetical work. In addition, the storage requirements are strongly reduced.