Matrix Multiplication Algorithm Research Articles

Multi-Disease Classification Model Using Strassen’s Half of Threshold (SHoT) Training Algorithm in Healthcare Sector

In healthcare industry, Neural Network has attained a milestone in solving many real-life classification problems varies from very simple to complex and from linear to non-linear. To improve the training process by reducing the training time, Adaptive Skipping Training algorithm named as Half of Threshold (HOT) has been proposed. To perform the fast classification and also to improve the computational efficiency such as accuracy, error rate, etc., the highlighted characteristics of proposed HOT algorithm has been integrated with Strassen’s matrix multiplication algorithm and derived a novel, hybrid and computationally efficient algorithm for training and validating the neural network named as Strassen’s Half of Threshold (SHoT) Training Algorithm. The experimental outcome based on the simulation demonstrated that the proposed SHOT algorithm outperforms both BPN and HOT algorithm in terms of training time which is reduced with the range of 7% to 54% and its efficiency which is improved with the range of 3% to 15% on various dataset such as Hepatitis, SPeCT, Heart, Liver Disorders, Breast Cancer Wisconsin (Diagnostic), Drug Consumption, Cardiotocography, Splice-junction Gene Sequences and Thyroid Disease dataset that are extracted from Machine Learning Dataset Repository of UCI. It can be integrated with any type of supervised training algorithm.

Randomization of approximate bilinear computation for matrix multiplication

ABSTRACT We present a method for randomizing formulas for bilinear computation of matrix products which does not increase the leading order complexity of the computation. We consider the implications of such randomization when there are two sources of error. The first source is due to the computation formula itself only being approximately correct. Such formulas come up when numerically searching for faster matrix multiplication algorithms. The second source is due to using floating point arithmetic. This kind of error is especially important when computing on low precision hardware like GPUs. Our theoretical results and numerical experiments indicate that our method can improve performance when the two kinds of error are present individually, as well as when they are present at the same time.

High-Performance 3D Mesh-Based NOC Architecture Using Node-Layer Clustering

Network-on-chip (NOC) architectures are being extensively employed following the growing number of cores in such networks as well as a rising trend in power/energy consumption and latency development. Proper NOC architectures can thus significantly contribute to the performance of these networks. Accordingly, a new architecture is proposed in this paper to decrease network diameter. A new particular topological structure is also presented based on node-layer clustering algorithm (NLCA) together with several rules for the main node as cluster-head (CH). A deadlock-free routing is subsequently suggested using this topology. To examine the effect of the given architecture on algorithm speed, the Scalable Universal Matrix Multiplication Algorithm (SUMMA) is further implemented and evaluated. Upon a decrease in the network diameter, the simulation results indicate a 10% improvement in energy consumption, 5.3% growth in network latency, and 20% enhancement in throughput as the given architecture is utilized. Moreover, SUMMA is employed in which a better cost in the proposed architecture can be established compared with its counterparts.

HPMaX: heterogeneous parallel matrix multiplication using CPUs and GPUs

We present a novel heterogeneous parallel matrix multiplication algorithm that utilizes both central processing units (CPUs) and graphics processing units (GPUs) for large-scale matrices. Based on Strassen’s method, we represent matrix multiplication work as a set of matrix addition and multiplication tasks among their sub-matrices. Then, we distribute the tasks to CPUs and GPUs while considering the characteristics of the tasks and computing resources to minimize the data communication overhead and fully utilize the available computing power. To handle a large matrix efficiently with limited GPU memory, we also propose a block-based work decomposition method. We then further improve the performance of our method by exploiting the concurrent execution abilities of a heterogeneous parallel computing system. We implemented our method on five different heterogeneous systems and applied it to matrices of various sizes. Our method generally shows higher performance than the prior GPU-based matrix multiplication methods. Moreover, compared with the state-of-the-art GPU matrix multiplication library (i.e., CUBLAS), our method achieved up to 1.97 times higher performance using the same GPUs and CPU cores. In some cases, our method using a low-performance GPU (e.g., GTX 1060, 3 GB) achieved performance comparable to that of CUBLAS using a high-performance GPU (e.g., RTX 2080, 8 GB). Also, our method continually improves performance as we use more computing resources like additional CPU cores and GPUs. We could achieve such high performance because our approach fully utilized the capacities of the given heterogeneous parallel computing systems while employing the Strassen’s method, which has a lower asymptotic complexity. These results demonstrate the efficiency and robustness of our algorithm.

Efficient parallel linear scaling method to get the response density matrix in all-electron real-space density-functional perturbation theory

The real-space density-functional perturbation theory (DFPT) for the computations of the response properties with respect to the atomic displacement and homogeneous electric field perturbation has been recently developed and implemented into the all-electron, numeric atom-centered orbitals electronic structure package FHI-aims. It is found that the bottleneck for large scale applications is the computation of the response density matrix, which scales as O(N3). Here for the response properties with respect to the homogeneous electric field, we present an efficient parallel linear scaling algorithm for the response density matrix calculation. Our scheme is based on the second-order trace-correcting purification and the parallel sparse matrix–matrix multiplication algorithms. The new scheme reduces the formal scaling from O(N3) to O(N), and shows good parallel scalability over tens of thousands of cores. As demonstrated by extensive validation, we achieve a rapid computation of accurate polarizabilities using DFPT. Finally, the computational efficiency of this scheme has been illustrated by making the scaling tests and scalability tests on massively parallel computer systems.

Parallel Theatre: An actor framework in Java for high performance computing

Theatre is an actor-based system currently implemented in Java, which enables modelling, analysis and implementation of predictable time-dependent distributed systems like cyber-physical systems. Theatre, though, can also be used for untimed complex systems. The core feature of Theatre is its control-based character. A reflective control layer, which can be tailored to the needs of specific application domains, is responsible for scheduling and dispatching messages asynchronously exchanged among actors. A Theatre system is a federation of theatres (computing nodes), upon which application actors are partitioned. Actors can move from a theatre to another. In its distributed implementation, Theatre rests on a socket network initially established among the interacting theatres. Messages and actors exploit Java serialization/deserialization when moving from a theatre to another. This paper proposes a novel extension of Theatre, Parallel Theatre, which is developed for an exploitation of the computing potential of nowadays multi-core machines with shared memory. Parallel Theatre avoids sockets and Java serialization. A key factor of Parallel Theatre consists in being a completely lock-free computing framework. Locks are avoided not only at the application level, as it was already demonstrated by classical actor systems, but also in the runtime system, thus enabling high-performance computing of scalable systems. This paper describes Parallel Theatre and the particular control forms which were developed for untimed and timed parallel systems. Then two scalable models are presented. The first one is concerned with a parallel algorithm for matrix multiplication. The second example is related to a parallel simulation of a multi-agent model for the Iterated Prisoner's Dilemma (IPD), which permits to observe the emergence of cooperation in large populations of players. The experimental results confirm the achievement of good execution performance.

Minimizing Time Complexity by Using Modified Strassen’s Matrix Multiplication Algorithm on Convolutional Layers

Minimizing Time Complexity by Using Modified Strassen’s Matrix Multiplication Algorithm on Convolutional Layers

A tutorial and open source software for the efficient evaluation of gravity and magnetic kernels

Fast computation of three-dimensional gravity and magnetic forward models is considered. When the measurement data is assumed to be obtained on a uniform grid which is staggered with respect to the discretization of the parameter volume, the resulting kernel sensitivity matrices exhibit block-Toeplitz–Toeplitz-block (BTTB) structure. These matrices are symmetric for the gravity problem but unsymmetric for the magnetic problem. In each case, the structure facilitates fast forward computation using two-dimensional fast Fourier transforms. The construction of the kernel matrices and the application of the transform for fast forward multiplication, for each problem, is carefully described. But, for purposes of comparison with the non-transform approach, the generation of the unique entries that define a given kernel matrix is also explained. It is also demonstrated how the matrices, and hence transforms, are adjusted when padding around the volume domain is introduced. The transform algorithms for fast forward matrix multiplication with the sensitivity matrix and its transpose, without the direct construction of the relevant matrices, are presented. Numerical experiments demonstrate the significant reduction in computation time and memory requirements that are achieved using the transform implementation. Thus, it becomes feasible, both in terms of reduced memory requirements and computational time, to implement the transform algorithms for large three-dimensional volumes. All presented algorithms, including with variable padding, are coded for optimal memory, storage and computation as an open source Matlab code which can be adapted for any convolution kernel which generates a BTTB matrix, whether or not it is symmetric. This work, therefore, provides a general tool for the efficient simulation of gravity and magnetic field data, as well as any formulation which admits a sensitivity matrix with the required structure.

A configurable multiplex data transfer model for asynchronous and heterogeneous FPGA accelerators on single DMA device

To reduce DMA utilization for multiple algorithm IPs on FPGA, a channel configurable and multiplex DMA device (CMDMA) is proposed for asynchronous and heterogeneous algorithm IPs. Firstly, we abstract the entities and data-flow in CMDMA system with a formal description for function definition and work-flow analysis. Then based on the functions and work-flow, we design and implement a prototype of CMDMA, which includes CMDMA software driver (SW) and hardware circuits (HW) of one DMA IP, a configurable input switch (CISwitch), algorithm IPs, and an asynchronous output switch (AOSwitch). The configurable function of CMDMA is implemented by CISwitch through a configuration port in HW-level, and a configurable Round-Robin (CRR) algorithm is proposed to implement channel and input data schedule in SW-level. For output, a channel distinguishable output buffer (ChnDistBuf) is proposed, which is able to deliver channel ID and data size to SW earlier than the end time of an algorithm IP. With a double interrupt coordination method of both ChnDistBuf and algorithm IPs, CMDMA is able to successively store complete output data from different algorithm IPs. With a double interrupt coordination method of both ChnDistBuf and algorithm IPs, CMDMA is able to successively store complete output data from different algorithm IPs. The experiments based on 4 heterogeneous matrix multiplication algorithm IPs on Xilinx Zynq platform show that CMDMA is able to improve about 8%-29% average algorithm acceleration rates on single algorithm IP compared to the exclusive method that one DMA works for one algorithm IP only, and it is able to increase about 10–40 MB/s and 5–15 MB/s of DMA input and output data throughput with multiple algorithm IPs running in parallel. Moreover, the extended LUT and FF resources in CMDMA are 756 and 1219, both of which are about 1% of Zynq platform. Besides, in a double CNN algorithm IPs test on Mnist application, an enhanced function of data broadcasting in CMDMA is able to improve 4 s than the system with 4 exclusive DMA running in parallel, meanwhile reduce 3 DMA utilization and 0.03 W power consumption.

A novel publicly delegable secure outsourcing algorithm for large-scale matrix multiplication

A novel publicly delegable secure outsourcing algorithm for large-scale matrix multiplication

Fast approximate shortest paths in the congested clique

We design fast deterministic algorithms for distance computation in the Congested Clique model. Our key contributions include:A (2+epsilon )-approximation for all-pairs shortest paths in O(log ^2{n} / epsilon ) rounds on unweighted undirected graphs. With a small additional additive factor, this also applies for weighted graphs. This is the first sub-polynomial constant-factor approximation for APSP in this model.A (1+epsilon )-approximation for multi-source shortest paths from O(sqrt{n}) sources in O(log ^2{n} / epsilon ) rounds on weighted undirected graphs. This is the first sub-polynomial algorithm obtaining this approximation for a set of sources of polynomial size. Our main techniques are new distance tools that are obtained via improved algorithms for sparse matrix multiplication, which we leverage to construct efficient hopsets and shortest paths. Furthermore, our techniques extend to additional distance problems for which we improve upon the state-of-the-art, including diameter approximation, and an exact single-source shortest paths algorithm for weighted undirected graphs in tilde{O}(n^{1/6}) rounds.

Comparison Study and Operation Collapsing Issues for Serial Implementation of Square Matrix Multiplication Approach Suitable in High Performance Computing Environment

Multiplication between two square matrices is one of the fundamental computational approach in the domain of mathematics and computer science where it is fully recognized as a fore most technique for several interdisciplinary domain and sub domains like linear algebra, graph theory, multidimensional graphics, cryptographic computation, convolution neural network, deep learning, digital signal processing, medical image processing, steganography, relativity theory, quantum computing and many others. In a high-performance computing environment computational complexity analysis of matrix multiplication algorithm ensures a powerful paradox that takes a massive data processing approach to the world where problem solution feasibility comes down in respect of operation and time. In our paper we have analyze different methods to multiply two square matrices (like 2×2 matrices) and their arithmetic complexity analysis in a asymptotic flavor along with operation collapsing issues through serial processing technique. We have analytically and experimentally explained and shown using MATLAB 9.3 simulator that fast matrix multiplication approach like Strassen and Winograd perform much better than conventional matrix multiplication algorithm especially for large amount of data.

An Error-Free Transformation for Matrix Multiplication with Reproducible Algorithms and Divide and Conquer Methods

This paper discusses accurate numerical algorithms for matrix multiplication. Matrix multiplication is a basic and important problem in numerical linear algebra. Numerical computations using floating-point arithmetic can be quickly performed on existing computers. However, the accumulation of rounding errors due to finite precision arithmetic is a critical problem. An error-free transformation for matrix multiplication is reviewed in this paper. Such a transformation is extremely useful for developing accurate numerical algorithms for matrix multiplication. One advantage of the transformation is that it exploits Basic Linear Algebra Subprograms (BLAS). We provide a rounding error analysis of reproducible algorithms for matrix multiplication based on the error-free transformation. In addition, we propose an error-free transformation for matrix multiplication that can be utilized with the divide and conquer methods.

Matrix Multiplication, a Little Faster

Strassen’s algorithm (1969) was the first sub-cubic matrix multiplication algorithm. Winograd (1971) improved the leading coefficient of its complexity from 6 to 7. There have been many subsequent asymptotic improvements. Unfortunately, most of these have the disadvantage of very large, often gigantic, hidden constants. Consequently, Strassen-Winograd’s O ( n log 2 7 ) algorithm often outperforms other fast matrix multiplication algorithms for all feasible matrix dimensions. The leading coefficient of Strassen-Winograd’s algorithm has been generally believed to be optimal for matrix multiplication algorithms with a 2 × 2 base case, due to the lower bounds by Probert (1976) and Bshouty (1995). Surprisingly, we obtain a faster matrix multiplication algorithm, with the same base case size and asymptotic complexity as Strassen-Winograd’s algorithm, but with the leading coefficient reduced from 6 to 5. To this end, we extend Bodrato’s (2010) method for matrix squaring, and transform matrices to an alternative basis. We also prove a generalization of Probert’s and Bshouty’s lower bounds that holds under change of basis, showing that for matrix multiplication algorithms with a 2 × 2 base case, the leading coefficient of our algorithm cannot be further reduced, and is therefore optimal. We apply our method to other fast matrix multiplication algorithms, improving their arithmetic and communication costs by significant constant factors.

Cannon-type triangular matrix multiplication for the reduction of generalized HPD eigenproblems to standard form

We first develop a new variant of Cannon’s algorithm for parallel matrix multiplication on rectangular process grids. Then we tailor it to selected situations where at least one triangular matrix is involved, namely “upper triangle of (full × upper triangular),” “lower triangle of (lower triangular × upper triangular),” and “all of (upper triangular × rectangular).” These operations arise in the transformation of generalized hermitian positive definite eigenproblems AX=BXΛ to standard form A˜X˜=X˜Λ, and making use of the triangular structure enables savings in arithmetic operations and communication. Numerical results show that the new implementations outperform previously available routines, and they are particularly effective if a whole sequence of generalized eigenproblems with the same matrix B must be solved, but they can also be competitive for the solution of a single generalized eigenproblem.

Non-polynomial Worst-Case Analysis of Recursive Programs

We study the problem of developing efficient approaches for proving worst-case bounds of non-deterministic recursive programs. Ranking functions are sound and complete for proving termination and worst-case bounds of non-recursive programs. First, we apply ranking functions to recursion, resulting in measure functions. We show that measure functions provide a sound and complete approach to prove worst-case bounds of non-deterministic recursive programs. Our second contribution is the synthesis of measure functions in non-polynomial forms. We show that non-polynomial measure functions with logarithm and exponentiation can be synthesized through abstraction of logarithmic or exponentiation terms, Farkas Lemma, and Handelman’s Theorem using linear programming. While previous methods obtain polynomial worst-case bounds, our approach can synthesize bounds of various forms including O( n log n ) and O( n r ), where r is not an integer. We present experimental results to demonstrate that our approach can efficiently obtain worst-case bounds of classical recursive algorithms such as (i) Merge sort, Heap sort, and the divide-and-conquer algorithm for the Closest Pair problem, where we obtain O( n log n ) worst-case bound, and (ii) Karatsuba’s algorithm for polynomial multiplication and Strassen’s algorithm for matrix multiplication, for which we obtain O( n r ) bounds such that r is not an integer and is close to the best-known bound for the respective algorithm. Besides the ability to synthesize non-polynomial bounds, we also show that our approach is equally capable of obtaining polynomial worst-case bounds for classical programs such as Quick sort and the dynamic programming algorithm for computing Fibonacci numbers.

Distillation of Greenberger–Horne–Zeilinger States by Combinatorial Methods

We prove a lower bound on the rate of Greenberger-Horne-Zeilinger states distillable from pure multipartite states by local operations and classical communication (LOCC). Our proof is based on a modification of a combinatorial argument used in the fast matrix multiplication algorithm of Coppersmith and Winograd. Previous use of methods from algebraic complexity in quantum information theory concerned transformations with stochastic local operations and classical operation (SLOCC), resulting in an asymptotically vanishing success probability. In contrast, our new protocol works with asymptotically vanishing error.

Hyper-power zero forcing detector for massive MIMO systems

In massive multiple-input multiple-output (MIMO) systems when the number of base station antennas is much higher than the number of users, linear detectors, such as zero forcing (ZF) and minimum mean-square error (MMSE), are able to achieve the near-optimal performance due to the favorable massive MIMO channel propagation. But, these detectors employ, in general, exact matrix inversion which is computationally complex for such systems. In this paper, we affirm that computing the exact matrix inversion by direct methods is not necessary to find ZF or MMSE solution. An iterative matrix inversion procedure would yield similar performance. Thus, an efficient iterative matrix inversion based on the hyper-power (HP) method is proposed for massive MIMO detection. The computing efficiency of the iterative matrix inversion is further improved by optimizing the number of terms from the infinite series used in the HP method. Analytical results show that the optimum choice for the number of terms is three from the HP method. Simulation results show that the HP method with the optimum number of terms achieves the near-optimal ZF performance in a small number of iterations. Finally, the Coppersmith–Winograd algorithm for matrix multiplication is employed in order to reduce the computational complexity from \(O(K^{3})\) to \(O(K^{2.373})\), where K represents the number of users.

Coded Computation Over Heterogeneous Clusters

In large-scale distributed computing clusters, such as Amazon EC2, there are several types of “system noise” that can result in major degradation of performance: system failures, bottlenecks due to limited communication bandwidth, latency due to straggler nodes, and so on. There have been recent results that demonstrate the impact of coding for efficient utilization of computation and storage redundancy to alleviate the effect of stragglers and communication bottlenecks in homogeneous clusters. In this paper, we focus on general heterogeneous distributed computing clusters consist of a variety of computing machines with different capabilities. We propose a coding framework for speeding up distributed computing in heterogeneous clusters by trading redundancy for reducing the latency of computation. In particular, we propose heterogeneous coded matrix multiplication (HCMM) algorithm for performing distributed matrix multiplication over heterogeneous clusters that are provably asymptotically optimal for a broad class of processing time distributions. Moreover, we show that HCMM is unboundedly faster than any uncoded scheme that partitions the total workload among the workers. To demonstrate how the proposed HCMM scheme can be applied in practice, we provide results from numerical studies and Amazon EC2 experiments comparing HCMM with three benchmark load allocation schemes—uniform uncoded, load-balanced uncoded, and uniform coded. In particular, in our numerical studies, HCMM achieves speedups of up to 73%, 56%, and 42%, respectively, over the three benchmark schemes mentioned earlier. Furthermore, we carry out experiments over Amazon EC2 clusters and demonstrate how HCMM can be combined with rateless codes with nearly linear decoding complexity. In particular, we show that HCMM combined with the Luby transform codes can significantly reduce the overall execution time. HCMM is found to be up to 61%, 46%, and 36% faster than the aforementioned three benchmark schemes, respectively. Additionally, we provide a generalization to the problem of optimal load allocation in heterogeneous settings, where we take into account the monetary costs associated with distributed computing clusters. We argue that HCMM is asymptotically optimal for budget-constrained scenarios as well. In particular, we characterize the minimum possible expected cost associated with a computation task over a given cluster of machines. Furthermore, we develop a heuristic algorithm for (HCMM) load allocation for the distributed implementation of budget-limited computation tasks.

효율적인 행렬 곱 알고리즘 및 이를 활용한 고성능 임베디드 시스템 개발

최근 항공우주 및 방위산업 분야에서 다양한 임베디드 시스템들의 소형화 및 저가화 개발이 많이 요구되고 있다. 본 논문에서는 고속 DSP를 이용하여 임베디드 시스템을 개발한다. 또한 비행 조종과 같은 고정밀 알고리즘의 연산시간을 줄이기 위해 행렬 곱을 위한 두 가지 알고리즘을 제안하고 구현한다. 개발한 임베디드 시스템을 이용하여 성능을 검증한 결과, 2×2 단위 계산방법을 이용한 기존 방법과 성능을 비교했을 때 첫 번째 제안방법은 행렬의 사이즈가 작을때 성능이 개선된다. 두 번째 제안방법은 2×2 단위 계산방법보다 전체적으로 성능이 우세하다.