Vector Multiplication Research Articles

A New Operational Matrices Based-Technique for Fractional Integro Reaction-Diffusion Equation Involving Spatiotemporal Variable-Order Derivative

In this article, a novel generalized model of the nonlinear fractional integro-reaction-diffusion equation (NFIRDE) with spatiotemporal variable-order (SVO) is introduced, where the variable order derivatives are equipped with the Atangana–Baleanu–Caputo (ABC) sense. This model represents a great generalization of a significant type of NFIRDE and their applications. Moreover, A novel, efficient and fully spectral shifted Legendre tau technique is developed to solve the proposed model. Despite the difficulty of applying this mechanism to solve this type of equations, due to the presence of nonlinear terms and the SVO functions that appear in the traditional differential and integral operational matrices. We deduce some new operational matrices that play the fundamental role in facilitating the implementation of the tau method. These operational matrices represent the SVO ABC-derivative, the integro term within the model, as well as the vector multiplications with the space-time Kronecker product. As a result, the proposed model is restructured into a system of nonlinear algebraic equations, which simplifying the solving process. We illustrate our method’s effectiveness and validity with numerical examples with both smooth and non-smooth solutions. Our findings show that the proposed tau method delivers accurate results and exhibits non-local properties.

SMT efficiency in supervised ML methods: a throughput and interference analysis

The microarchitecture of general-purpose processors is continuously evolving to adapt to the new computation and memory demands of incoming workloads. In this regard, new circuitry is added to execute specific instructions like vector multiplication or string operations. These enhancements and the support of multiple threads per core make simultaneous multithreading (SMT) processors dominate the market for data center processors. Regarding emerging workloads, machine learning is taking an important role in many research domains like biomedicine, economics, and social sciences. This paper analyzes the efficiency of machine learning workloads running in SMT mode (two threads per core) versus running them in ST mode (single-threaded) with twice the number of cores. Experimental results in an Intel Xeon Skylake-X processor show an SMT efficiency falling between 80% and 100% across the studied workloads. These results prove two main findings: i) last-generation SMT processors are excellent candidates to execute ML workloads as they achieve a high SMT efficiency, and ii) if the performance of two major resources (i.e., FP double operator and core’s caches) was boosted, all the workloads would achieve an almost perfect SMT efficiency. Moreover, results show that there is still room to support more threads without adding extra hardware. The discussed findings are aimed at providing insights to design future processors for ML workloads.

Variational quantum algorithm for the Poisson equation based on the banded Toeplitz systems

Abstract For solving the Poisson equation it is usually possible to discretize it into solving the corresponding linear system $Ax=b$. Variational quantum algorithms (VQAs) for \textcolor{red}{the discreted} Poisson equation have been studied before. We give a VQA based on the banded Toeplitz systems for solving the Poisson equation with respect to the structural features of matrix $A$. In detail, we decompose the matrix $A$ and $A^2$ into a linear combination of the corresponding banded Toeplitz matrix and sparse matrices with only a few non-zero elements. For the one-dimensional Poisson equation with different boundary conditions and the $d$-dimensional Poisson equation with Dirichlet boundary conditions, \textcolor{red}{the number of decomposition terms is less than the work in [Phys. Rev. A108, 032418 (2023)]}. Based on the decomposition of the matrix, we design quantum circuits that \textcolor{red}{evaluate efficiently} the cost function. \textcolor{red}{Additionally, numerical simulation verifies the feasibility of the proposed algorithm. In the end,} the VQAs for linear systems of equations and matrix–vector multiplications with $K$-banded Teoplitz matrix $T_n^K$ \textcolor{red}{are} given, where $T_n^K\in R^{n\times n}$ and $K\in O(ploy\log n)$.

Real-time Blood Pressure Prediction on Wearable Devices using Edge Based Deep Neural Networks: A Hardware-software Co-design Approach

This paper presents the hardware realization of a real-time blood pressure (BP) prediction model for wearable devices, utilizing long short-term memory (LSTM) deep neural networks (DNNs). The proposed system uses both electrocardiogram (ECG) and photoplethysmogram (PPG) signal values for BP prediction. It aims to address the limitations of traditional BP measurement methods, providing a low error, minimal computational overhead, more accurate and convenient alternative system for individuals with hypertension or at risk for cardiovascular diseases. The utilization of split matrix approach leads to a reduction in hardware complexity across the entire system. This technique involves breaking down the larger weight matrices used in the computations of DNNs into smaller matrices. This fragmentation results in a decrease in the complexity of the hardware responsible for performing matrix vector multiplications (MVMs) within LSTMs. The resultant architecture of the predictive model gains several advantages, including a lowered level of complexity in terms of the space occupied by individual cells, decreased processing delay, and reduced power consumption. Furthermore, this approach enables the achievement of a notably improved minimum achievable clock period of 2.972 ns. This prediction model can operate locally on wearable devices, reducing the reliance on cloud computing and improving privacy and security. The performance evaluations are carried out using both analytical and implementation results. The results indicate that the proposed model can be practically applied to real-world problems and can potentially enhance the accuracy of various machine-learning tasks.

PF‐GEMV: Utilization maximizing architecture in fast matrix–vector multiplication for GPT‐2 inference

AbstractOwing to the widespread advancement of transformer‐based artificial neural networks, artificial intelligence (AI) processors are now required to perform matrix–vector multiplication in addition to the conventional matrix–matrix multiplication. However, current AI processor architectures are optimized for general matrix–matrix multiplications (GEMMs), which causes significant throughput degradation when processing general matrix–vector multiplications (GEMVs). In this study, we proposed a port‐folding GEMV (PF‐GEMV) scheme employing multiformat and low‐precision techniques while reusing an outer product‐based processor optimized for conventional GEMM operations. This approach achieves 93.7% utilization in GEMV operations with an 8‐bit format on an 8 8 processor, thus resulting in a 7.5 increase in throughput compared with that of the original scheme. Furthermore, when applied to the matrix operation of the GPT‐2 large model, an increase in speed by 7 is achieved in single‐batch inferences.

Statistically optimal firstorder algorithms: a proof via orthogonalization

Abstract We consider a class of statistical estimation problems in which we are given a random data matrix ${\boldsymbol{X}}\in{\mathbb{R}}^{n\times d}$ (and possibly some labels ${\boldsymbol{y}}\in{\mathbb{R}}^{n}$) and would like to estimate a coefficient vector ${\boldsymbol{\theta }}\in{\mathbb{R}}^{d}$ (or possibly a constant number of such vectors). Special cases include low-rank matrix estimation and regularized estimation in generalized linear models (e.g. sparse regression). Firstorder methods proceed by iteratively multiplying current estimates by ${\boldsymbol{X}}$ or its transpose. Examples include gradient descent or its accelerated variants. Under the assumption that the data matrix ${\boldsymbol{X}}$ is standard Gaussian, Celentano, Montanari, Wu (2020, Conference on Learning Theory, pp. 1078–1141. PMLR) proved that for any constant number of iterations (matrix vector multiplications), the optimal firstorder algorithm is a specific approximate message passing algorithm (known as ‘Bayes AMP’). The error of this estimator can be characterized in the high-dimensional asymptotics $n,d\to \infty $, $n/d\to \delta $, and provides a lower bound to the estimation error of any firstorder algorithm. Here we present a simpler proof of the same result, and generalize it to broader classes of data distributions and of firstorder algorithms, including algorithms with non-separable nonlinearities. Most importantly, the new proof is simpler, does not require to construct an equivalent tree-structured estimation problem, and is therefore susceptible of a broader range of applications.

Quantum Algorithms for the Multiplication of Circulant Matrices and Vectors

Quantum Algorithms for the Multiplication of Circulant Matrices and Vectors

Implementation and optimization of SpMV algorithm based on SW26010P many-core processor and stored in BCSR format

The irregular distribution of non-zero elements of large-scale sparse matrix leads to low data access efficiency caused by the unique architecture of the Sunway many-core processor, which brings great challenges to the efficient implementation of sparse matrix–vector multiplication (SpMV) computing by SW26010P many-core processor. To address this problem, a study of SpMV optimization strategies is carried out based on the SW26010P many-core processor. Firstly, we design a memorized data storage transformation strategy to transform the matrix in CSR storage format into BCSR (Block Compressed Sparse Row) storage. Secondly, the dynamic task scheduling method is introduced to the algorithm to realize the load balance between slave cores. Thirdly, the LDM memory is refined and designed, and the slave core dual cache strategy is optimized to further improve the performance. Finally, we selected a large number of representative sparse matrices from the Matrix Market for testing. The results show that the scheme has obviously speedup the processing procedure of sparse matrices with various sizes and sizes, and the master–slave speedup ratio can reach up to 38 times. The optimization method used in this paper has implications for other complex applications of the SW26010P many-core processor.

Fixed‐time prescribed performance control method for nonlinear systems with unknown time‐varying parameters

AbstractIn this study, it is considered that the question of adaptive tracking control of nonlinear systems containing unknown time‐varying parameters. To optimize the control effect, a fixed‐time control strategy is adopted, in which the upper bound of the settling time is connected with the designed parameters, but not to the initial conditions. In the design, the unknown time‐varying parameters are treated as the multiplication of known and unknown vectors, and the adaptive laws are constructed to estimate the mean value of the components in the unknown ones, which availably solve the difficulty of unknown control gains. Besides, the application of command filtered technique, which combined with the compensation signals containing the adaptive parameters, effectively avoids the “complexity explosion” problem. Further, Lyapunov stability theory verifies that the tracking error can always be kept within the boundaries of the performance functions, and converges to a designated neighbourhood of the origin within a fixed time, beyond that all signals of the closed‐loop system are bounded. Finally, the simulation results prove the effectiveness of the proposed control scheme.

HyperGen: Compact and Efficient Genome Sketching using Hyperdimensional Vectors.

Genomic distance estimation is a critical workload since exact computation for whole-genome similarity metrics such as Average Nucleotide Identity (ANI) incurs prohibitive runtime overhead. Genome sketching is a fast and memory-efficient solution to estimate ANI similarity by distilling representative k-mers from the original sequences. In this work, we present HyperGen that improves accuracy, runtime performance, and memory efficiency for large-scale ANI estimation. Unlike existing genome sketching algorithms that convert large genome files into discrete k-mer hashes, HyperGen leverages the emerging hyperdimensional computing (HDC) to encode genomes into quasi-orthogonal vectors (Hypervector, HV) in high-dimensional space. HV is compact and can preserve more information, allowing for accurate ANI estimation while reducing required sketch sizes. In particular, the HV sketch representation in HyperGen allows efficient ANI estimation using vector multiplication, which naturally benefits from highly optimized general matrix multiply (GEMM) routines. As a result, HyperGen enables the efficient sketching and ANI estimation for massive genome collections. We evaluate HyperGen 's sketching and database search performance using several genome datasets at various scales. HyperGen is able to achieve comparable or superior ANI estimation error and linearity compared to other sketch-based counterparts. The measurement results show that HyperGen is one of the fastest tools for both genome sketching and database search. Meanwhile, HyperGen produces memory-efficient sketch files while ensuring high ANI estimation accuracy. A Rust implementation of HyperGen is freely available under the MIT license as an open-source software project at https://github.com/wh-xu/Hyper-Gen. The scripts to reproduce the experimental results can be accessed at https://github.com/wh-xu/experiment-hyper-gen.

Efficiency analysis of discontinuous Galerkin approaches for the application onto quantum Liouville-type equations

The simulation of nanodevices is computationally inefficient with current algorithms. The discontinuous Galerkin approach has been demonstrated in the field of computational fluid dynamics to deliver high order accuracy and efficiency due to its reliance on matrix–vector multiplications. Previously, the discontinuous Galerkin approach was successfully used in conjunction with the finite volume technique to solve the Liouville–von Neumann equation in center-mass coordinates and thus simulate nanodevices. To exploit its full potential regarding high-performance computing, this work aims to substitute the aforementioned finite volume technique with the discontinuous Galerkin method. To arrive at the said formalism, a finite element method is implemented as an intermediate step.

CRPIM: An efficient compute-reuse scheme for ReRAM-based Processing-in-Memory DNN accelerators

Resistive random access memory (ReRAM) is a promising technology for AI Processing-in-Memory (PIM) hardware because of its compatibility with CMOS, small footprint, and ability to complete matrix–vector multiplication workloads inside the memory device itself. However, redundant computations are brought on by duplicate weights and inputs when an MVM has to be split into smaller-granularity sequential sub-works in the real world. Recent studies have proposed repetition-pruning to address this issue, but the buffer allocation strategy for enhancing buffer device utilization remains understudied. In preliminary experiments observing input patterns of neural layers with different datasets, the similarity of repetition allows us to transfer the buffer allocation strategy obtained from a small dataset to the computation with a large dataset. Hence, this paper proposes a practical compute-reuse mechanism for ReRAM-based PIM, called CRPIM, which replaces repetitive computations with buffering and reading. Moreover, the subsequent buffer allocation problem is resolved at both inter-layer and intra-layer levels. Our experimental results demonstrate that CRPIM significantly reduces ReRAM cells and execution time while maintaining adequate buffer and energy overhead.

Variable transformations in combination with wavelets and ANOVA for high-dimensional approximation

We use hyperbolic wavelet regression for the fast reconstruction of high-dimensional functions having only low-dimensional variable interactions. Compactly supported periodic Chui-Wang wavelets are used for the tensorized hyperbolic wavelet basis on the torus. With a variable transformation, we are able to transform the approximation rates and fast algorithms from the torus to other domains. We perform and analyze scattered data approximation for smooth but arbitrary density functions by using a least squares method. The corresponding system matrix is sparse due to the compact support of the wavelets, which leads to a significant acceleration of the matrix vector multiplication. For non-periodic functions, we propose a new extension method. A proper choice of the extension parameter together with the piecewise polynomial Chui-Wang wavelets extends the functions appropriately. In every case, we are able to bound the approximation error with high probability. Additionally, if the function has a low effective dimension (i.e., only interactions of a few variables), we qualitatively determine the variable interactions and omit ANOVA terms with low variance in a second step in order to decrease the approximation error. This allows us to suggest an adapted model for the approximation. Numerical results show the efficiency of the proposed method.

Cerberus: Triple Mode Acceleration of Sparse Matrix and Vector Multiplication

The multiplication of sparse matrix and vector (SpMV) is one of the most widely used kernels in high-performance computing as well as machine learning acceleration for sparse neural networks. The design space of SpMV accelerators has two axes: algorithm and matrix representation. There have been two widely used algorithms and data representations. Two algorithms, scalar multiplication and dot product, can be combined with two sparse data representations, compressed sparse and bitmap formats for the matrix and vector. Although the prior accelerators adopted one of the possible designs, it is yet to be investigated which design is the best one across different hardware resources and workload characteristics. This paper first investigates the impact of design choices with respect to the algorithm and data representation. Our evaluation shows that no single design always outperforms the others across different workloads, but the two best designs (i.e., compressed sparse format and bitmap format with dot product) have complementary performance with trade-offs incurred by the matrix characteristics. Based on the analysis, this study proposes Cerberus, a triple-mode accelerator supporting two sparse operation modes in addition to the base dense mode. To allow such multi-mode operation, it proposes a prediction model based on matrix characteristics under a given hardware configuration, which statically selects the best mode for a given sparse matrix with its dimension and density information. Our experimental results show that Cerberus provides 12.1× performance improvements from a dense-only accelerator, and 1.5× improvements from a fixed best SpMV design.

Accuracy of linear transformations performed on a nonideal Mach–Zehnder interferometer

We consider a mathematical model of a planar Mach–Zehnder interferometer with nonideal beam splitters. For this model, we obtain two fidelity estimates of the matrix–vector multiplication in the form of dependences of the multiplication error on the beam splitting error in directional couplers. The first estimate is obtained as a measure of the difference between the transfer matrices implemented by interferometers and is presented in the form of the norm of the difference between two unitary matrices corresponding to ideal and non-ideal interferometers. The second estimate is obtained as a measure of the difference in output intensities. It is shown that in the latter case the fidelity depends both on the beam splitter error and on the parameters of the input signals. We verified the second estimate using computer simulations of MZI in COMSOL Multiphysics.

Characterisation of optical waveguides for photonic integrated circuits

Fast signal processing at the speed of light is the main advantage of photonic integrated circuits. Therefore, these circuits have good prospects for the implementation of mathematical calculations, including matrix to vector multiplication. The purpose of the research was to create and investigate a technique for automatic measurement of brightness distribution along optical waveguides of analogue photonic integrated circuits. Empirical methods (observation, measurement, comparison, experiment) and a complex method (analysis and synthesis) have been used during the research. The proposed technique uses a digital camera that captures images of optical waveguide illuminated by light emitting diodes and image processing software to calculate brightness distribution. This technique determines the best approximation of this distribution, calculates parameters of brightness non-uniformity and losses of optical radiation. Measurements of a set of optical waveguides help to identify the best candidates for photonic integrated circuits. It has been found that optical waveguides with grinded surfaces acting as diffusive scattering have good combination of smooth brightness distribution and small losses of optical radiation. Due to multiple diffuse reflection and scattering within waveguide material, these waveguides are promising candidates for analogue photonic integrated circuits. All other waveguides with non-processed surface, with grooves or grinded with a large grain have sufficient losses of optical radiation. These losses are usually caused by the exit of optical radiation from waveguide surface. The obtained results are necessary for accurate design of circuits that takes into account scattering and losses in optical waveguides. The proposed technique can be applied in automatic technological process of manufacturing a fast and economical photonic matrix to vector multiplication, which does not require expensive electron-beam, optical or laser lithographic equipment

Stochastic dynamics of mechanical systems with impacts via the Step Matrix multiplication based Path Integration method

AbstractIn this work we propose the Step Matrix Multiplication based Path Integration method (SMM-PI) for nonlinear vibro-impact oscillator systems. This method allows the efficient and accurate deterministic computation of the time-dependent response probability density function by transforming the corresponding Chapman–Kolmogorov equation to a matrix–vector multiplication using high-order numerical time-stepping and interpolation methods. Additionally, the SMM-PI approach yields the computation of the joint probability distribution for response and impact velocity, as well as the time between impacts and other important characteristics. The method is applied to a nonlinear oscillator with a pair of impact barriers, and to a linear oscillator with a single barrier, providing relevant densities and analysing energy accumulation and absorption properties. We validate the results with the help of stochastic Monte-Carlo simulations and show the superior ability of the introduced formulation to compute accurate response statistics.

Feed Forward Neural Network With Column-Wise Matrix–Vector Multiplication on FPGAs

This article presents a reconfigurable accelerator for recurrent neural networks with fine grained column wise matrix vector multiplication (RENOWN).We propose a novel latency hiding architecture for recurrent neural network accelerator using column wise matrix vector multiplication instead of the state of the art row wise operation. This hardware (HW) architecture can eliminate data dependencies to improve the throughput of RNN inference systems. Besides, we introduce a configurable checkerboard tiling strategy which allows large weight matrices, while incorporating various configurations of element-based parallelism (EP) and vector-based parallelism (VP). These optimizations improve the exploitation of parallelism to increase HW utilization and enhance system throughput. Evaluation results show that our design can achieve over 29.6 tera operations per second (TOPS) which would be among the highest for field-programmable gate array (FPGA)-based RNN designs. Compared to state-of-the-art accelerators on FPGAs, our design achieves 3.7–14.8 times better performance and has the highest HW.

An Integrated All‐Optical Ising Machine with Unlimited Spin Array Size and Coupling

Artificial spin systems are used to solve combinatorial optimization problems by mapping them to the ground state search of the Ising model. Ising machines of various designs, in fields as diverse as photonics and electronics, have been proposed and demonstrated in recent years. One important mathematical operation required in photonic Ising machines for nearly all Hamiltonian minimization algorithms is repeated matrix–vector multiplication with the vector size limited by the physical size of the optical system. Herein, an integrated photonic Ising machine based on matrix partitioning and phase encoding, which effectively allows the use of physically small optical circuits to handle arbitrarily large spin vectors, is proposed. All the complicated calculations are carried out optically, rather than electronically, in this approach. How the required surface area of a hypothetical chip‐based photonic Ising machine can be resized according to convenience, with the trade‐off being the solution time, and that this trade‐off can be done in a way that does not impact the algorithm's efficacy, is shown. Through simulation, the system is predicted to have a high success rate, high noise tolerance, and high error tolerance.

Machine Learning-Based Kernel Selector for SpMV Optimization in Graph Analysis

Sparse Matrix and Vector multiplication (SpMV) is one of the core algorithms in various large-scale scientific computing and real-world applications. With the rapid development of AI and big data, the input vector in SpMV becomes sparse in many application fields. Especially in some graph analysis calculations, the sparsity of the input vector will change with the running of the program, and the non-zero element distribution of the adjacency matrix of some graph data has the power law property, leading to serious load imbalance, which requires additional optimization means. Therefore, the optimal SpMV kernel may be different, and a single SpMV kernel can no longer meet the acceleration requirements. In this paper, we propose a decision tree-based adaptive SpMV framework, named DTSpMV, that can automatically select appropriate SpMV kernels according to different input data in iterations of graph computation. Based on the analysis of computing patterns, bit-array compression algorithms, and serial and parallel algorithms, we encapsulate nine SpMV kernels within the framework. We explore machine learning-based kernel selectors in terms of both accuracy and runtime overhead. Experimental results on NVIDIA Tesla T4 GPU show that our adaptive framework achieves the arithmetic average performance improvement of \(152\% \) compared to the SpMV kernel in cuSPARSE.