Matrix Multiplication Operations Research Articles

Deep Learning-Driven Compiler Enhancements for Efficient Matrix Multiplication

Matrix multiplication is a fundamental operation in many computational fields, requiring optimization to handle increasing data sizes efficiently. In this paper, the implementation of Deep Learning in Matrix multiplication is reviewed, which is considered important nowadays due to the growing complexity of matrix multiplication for gaming and complex programs. The current standard matrix multiplication and the time taken by it on different matrix sizes are described. The Tiled Matrix multiplication, which trims the matrix into various pieces and calculates the product for each piece, and thereafter combines the result, is also described. The times taken by both methods for different matrix sizes were compared. The main idea was to use Deep Neural Networks (DNN) to compare and rank code variants that are obtained in pieces and determine their relative performance. A tournament-based ranking system is used for assigning ranks to the code versions. The effectiveness of these techniques was evaluated on various matrix multiplication operations commonly found in deep learning workloads. Up to 8.844x speedup over the naive implementation for a matrix size of 1024 is achieved by this approach. The results demonstrate the effectiveness of combining compiler optimization techniques and deep learning models in optimizing matrix multiplication.

LL-GNN: Low Latency Graph Neural Networks on FPGAs for High Energy Physics

This work presents a novel reconfigurable architecture for Low Latency Graph Neural Network (LL-GNN) designs for particle detectors, delivering unprecedented low latency performance. Incorporating FPGA-based GNNs into particle detectors presents a unique challenge since it requires sub-microsecond latency to deploy the networks for online event selection with a data rate of hundreds of terabytes per second in the Level-1 triggers at the CERN Large Hadron Collider experiments. This article proposes a novel outer-product based matrix multiplication approach, which is enhanced by exploiting the structured adjacency matrix and a column-major data layout. In addition, we propose a custom code transformation for the matrix multiplication operations, which leverages the structured sparsity patterns and binary features of adjacency matrices to reduce latency and improve hardware efficiency. Moreover, a fusion step is introduced to further reduce the end-to-end design latency by eliminating unnecessary boundaries. Furthermore, a GNN-specific algorithm-hardware co-design approach is presented which not only finds a design with a much better latency but also finds a high accuracy design under given latency constraints. To facilitate this, a customizable template for this low latency GNN hardware architecture has been designed and open-sourced, which enables the generation of low-latency FPGA designs with efficient resource utilization using a high-level synthesis tool. Evaluation results show that our FPGA implementation is up to 9.0 times faster and achieves up to 13.1 times higher power efficiency than a GPU implementation. Compared to the previous FPGA implementations, this work achieves 6.51 to 16.7 times lower latency. Moreover, the latency of our FPGA design is sufficiently low to enable deployment of GNNs in a sub-microsecond, real-time collider trigger system, enabling it to benefit from improved accuracy. The proposed LL-GNN design advances the next generation of trigger systems by enabling sophisticated algorithms to process experimental data efficiently.

Processing‐in‐memory based multilateration localization in wireless sensor networks using memristor crossbar arrays

SummaryMost of the multilateration algorithms for object localization rely heavily on matrix multiplication for calculating the precise location of the target from all the reference points in a 3D domain. A significant disadvantage is that performing matrix multiplication can be computationally expensive, particularly for larger matrices, leading to slower performance and longer processing times. In this article, we propose a Processing‐in‐Memory (PIM) based localization approach for multilateration using a memristor crossbar array for performing faster matrix multiplication. The experiments were conducted on real‐time data collected through ultra‐wideband (UWB) chip. The simulation results demonstrate that the proposed model is around 50% faster than the fastest known localization algorithm considered for evaluation and exhibits an average of 55% improvement in localization accuracy.

Efficient GPU implementation of randomized SVD and its applications

Matrix decompositions are ubiquitous in machine learning, including applications in dimensionality reduction, data compression and deep learning algorithms. Typical solutions for matrix decompositions have polynomial complexity which significantly increases their computational cost and time. In this work, we leverage efficient processing operations that can be run in parallel on modern Graphical Processing Units (GPUs), predominant computing architecture used e.g. in deep learning, to reduce the computational burden of computing matrix decompositions. More specifically, we reformulate the randomized decomposition problem to incorporate fast matrix multiplication operations (BLAS-3) as building blocks. We show that this formulation, combined with fast random number generators, allows to fully exploit the potential of parallel processing implemented in GPUs. Our extensive evaluation confirms the superiority of this approach over the competing methods and we release the results of this research as a part of the official CUDA implementation.11https://docs.nvidia.com/cuda/cusolver/index.html.

Outcome for the group SL(2,57)

The set of all (n×n) non-singular matrices over the field F this set forms a group under the operation of matrix multiplication. This group is called the general linear group of dimension over the field F, denoted by . The determinant of these matrices is a homomorphism from into F* and the kernel of this homomorphism was the special linear group and denoted by Thus is the subgroup of which contains all matrices of determinant one. The rational valued characters of the rational representations written as a linear combination of the induced characters for the groups discuss in this paper and find the Artin indicator for this group after study the rational valued characters of the rational representations and the induced characters.

Research on Mushroom Image Classification Algorithm Based on Deep Sparse Dictionary Learning

The traditional mushroom feature extraction method has low classification efficiency and unsatisfactory effect. Dictionary learning is widely used in image classification. However, the previous work is to learn dictionaries in the original space, which limits the performance of sparse representation classification. In order to solve the problem of spatial redundancy in traditional convolutional neural networks and the weak performance of deep learning in small samples, an improved dictionary learning algorithm, Deep Sparse Dictionary learning (DSDL), is proposed. The input to DSDL is not a matrix gathered from the original grayscale image or a hand-created feature, but rather a relatively deeper feature extraction via a stack autoencoder. Then, a structured dictionary is designed to reconstruct the deep features according to different categories of distinguishing features. In addition, it is necessary to learn the associated structured projection sparse dictionary to ensure that the decoder updates in the direction of the deconvolution operator error is minimal. By utilizing sparse dictionary learning loss functions and autoencoder loss functions, DSDL can simultaneously learn deep latent features and corresponding dictionary pairs. In the testing phase of DSDL, the minimum errors of deep feature and structured projection components for different classes can be directly represented by basic matrix multiplication operations. Experimental results show that the proposed method achieves a good classification effect on mushroom images, which shows the effectiveness of the method.

Algorithm/Hardware Co-Optimization for Sparsity-Aware SpMM Acceleration of GNNs

In recent years, graph neural networks (GNNs) have achieved impressive performance in various application fields by extracting information from graph-structured data. It contains extensive feature aggregation operations and has become a performance bottleneck, which can be abstracted as a specialized Sparse-Dense Matrix Multiplication (SpMM) operation. Previous works have leveraged the inner product or outer product to accelerate the feature aggregation process. However, inefficient execution leads to extremely unbalanced workloads and extensive intermediate data, hampering the performance of previous processors. So in this paper, we demonstrate an algorithm/hardware co-optimization chance to enhance SpMM acceleration for GNNs. First, the algorithm part develops a dataflow-efficient SpMM algorithm that integrates three optimization methods to mitigate computation and memory access inefficiencies. Specifically, 1) the proposed equal-value partition method achieves fine-grained data partition and enables load balancing during data movement. 2) After observing the vertex aggregation phenomenon, a vertex-clustering optimization method is presented to enable significant data locality. 3) The adaptive dataflow based on Gustavson’s algorithm is further implemented to enable the efficient distribution of sparse elements and improves computing resource utilization. Then, the hardware part features the proposed SpMM algorithm and customizes SDMA, a flexible and efficient accelerator to boost SpMM acceleration, which follows the adaptive dataflow to eliminate sparsity and explore the regular parallelism dimension. Finally, we prototype SDMA on the Xilinx Alveo U280 FPGA accelerator card. The results demonstrate that SDMA achieves 5.68× 14.68× energy efficiency over the previous GPU implementations deployed on the Nvidia GTX 1080Ti and 1.32× higher throughput over the state-of-the-art FPGA prototype.

Explicit solutions of conjugate, periodic, time-varying Sylvester equations

Solutions of a group of conjugate time-varying matrix equations are discussed in this paper. Through mathematical derivation, the solutions to this group of equations are equivalent to the solutions to a class of conjugate time-invariant matrix equations. Further, the related conditions of solvability are obtained and the general explicit solutions are represented by using quasicontrollability and quasiobservability matrices. A detailed algorithm is presented to make the calculation process clear, and the effectiveness of the algorithm is verified by a concrete example. The proposed algorithm can provide complete solutions to the considered equation in explicit parametric form and its main computation includes solving an ordinary linear algebraic equation and some matrix multiplication operations.

Developing a Static Kinematic Model for Continuum Robots Using Dual Quaternions for Efficient Attitude and Trajectory Planning

Kinematic modeling is essential for planning and controlling continuum robot motion. The traditional Denavit Hartenberg (DH) model involves complex matrix multiplication operations, resulting in computationally intensive inverse solutions and trajectory planning. Solving position and orientation changes in continuum robots using the double quaternion rule can reduce computational complexity. However, existing dual quaternion methods are direct equational transformations of DH rules and do not give a complete modeling process. They usually require more interpretability when applying continuum robot kinematic modeling. This paper uses the dual quaternion method to establish a kinematic model of a continuum robot. It uses a two-section continuum robot model to compare the advantages of dual quaternion and traditional modeling methods. In addition, this paper proposes a five-polynomial interpolation algorithm based on the dual quaternion method for trajectory planning of continuum robots. This method accurately models spatial bending and torsional motions of singularity-free continuum robots.

Silicon Programmable Photonic Circuits Based on Periodic Bimodal Waveguides

AbstractProgrammable photonic circuits are dense assemblies of waveguide meshes in which the flow of light can be reconfigured by software to implement a wide variety of functions, ranging from radiofrequency filtering to optical computing. However, most programmable architectures to date rely on rather bulky Mach‐Zehnder interferometers (MZIs), which are not suitable for large‐scale and high‐density integration. Here, an alternative approach to MZI‐based programmable photonic circuits by using slow‐light‐enhanced periodic bimodal waveguides (PBWs) as programmable units is presented. This study experimentally demonstrates low‐loss short tuning elements of 30× 1.7 µm2 in area, achieving a two‐orders of magnitude integration density improvement compared to conventional MZIs. A rectangular arrangement of these tunable units is proposed for 3× 3 and 4× 4 matrix multiplication operations to design a feedforward circuit with a footprint of only 100× 250 µm2. Finally, the performance trade‐off and benchmark with alternative programmable unit cells are analyzed in order to address the ever‐growing demand for large computing requirements in next‐generation applications such as artificial intelligence and quantum information processing.

An Efficient Transformer Model for Synthesizing Dual Energy CT from Single Energy Scanner

Dual-energy CT can be used to optimize radiation treatment. Recently, deep learning has been demonstrated to synthesize high-energy CT images from low-energy ones for dose reduction and lower CT system burden. As the state-of-the-art deep learning architecture, the computation burden of Transformer increases quadratically with the feature size, making the model training resource-demanding or even infeasible. Here, we introduce an efficient transformer for the balance between CT image synthesis quality and computational burden. The model is a U-shape deep neural network with encoders and decoders built by Transformer blocks. The model input is low-energy 100kVp CT image and the output is high-energy 140kVp one. Each block has a Self Channel Correlation Unit (SCCU) and a Self Spatial Attention Unit (SSAU). Local shortcuts are applied for both units. Under-sampling operation achieved by pixel shuffling is used to obtain multi-scale feature maps, and the transformer block is applied on each feature scale. Symmetric skip connection sending features from shallow layers to deep layers, thus an additional 1 × 1 convolution is used for feature fusion in each decoder. In a SCCU, the feature is first mapped to one Query, one Key, and one Value. Then the Query and the Key tensors perform matrix multiplication to compute cross covariance of feature channels. The channel correlation score can be obtained by normalization of the covariance, and it is used to weight the Value tensor. As a result, the model complexity only increases linearly with the feature size. Besides the channel weighting, we enhance spatial information using SSAU, where the feature is mapped to two tensors. One tensor after activation is used to point-wisely calibrate another tensor. Additional Transformer blocks are cascaded to the last decoder for feature refinement. Because of the structure similarity of low- and high-energy CT images, a global shortcut is used to ease model training. Clinical iodine contrast-enhanced dual energy CT image datasets of 19 patients are used in this study. The dual-energy scanning is performed by a SOMATOM Definition Flash DECT scanner. We split the datasets into training dataset of 15 patients, validation dataset of 1 patient, and testing dataset of 3 patients. The image size is 512 × 512 with pixel size 0.5 × 0.5 mm2. The U-Net model with 1.95M parameters and 44.87G FLOPS achieved the averaged PSNR value of 44.55 dB (s.t.d. 1.34) and averaged RMSE value of 0.0060 (s.t.d. 0.001). In comparison, our efficient Transformer with 1.408M parameters and 31.375G FLOPS achieved the averaged PSNR value of 44.78 dB (s.t.d. 1.37) and RMSE value of 0.0059 (s.t.d. 0.001), demonstrating our model has better performance with small model size and less computation. The efficient Transformer model allows high-resolution CT image synthesis with small model scale and computation burden from low-energy CT image.

Multi-view clustering via optimal transport algorithm

The surge in data with multiple views has propelled significant interest in the domain of multi-view clustering. Unlike conventional single-view data, multi-view data offers a more accurate representation of objects. However, the pivotal challenge remains in the effective categorization of data through feature extraction from multiple views within clustering tasks. Notably, prevailing multi-view clustering algorithms often emphasize the derivation of appropriate view weights, inadvertently sidestepping optimization intricacies. This approach frequently leads to protracted computational time due to the resource-intensive matrix multiplication operations involved in optimization, coupled with the necessity of weight allocation for diverse views. Addressing this, we propose an optimization framework founded on the optimal transport algorithm that operates independently of view weights. A paramount advantage of the optimal transport algorithm lies in its rapid convergence to a closed-form solution. This study diverges from the conventional focus on view weights and centers on the optimization process within clustering algorithms for resolving multi-view challenges. The introduced framework employing the optimal transport algorithm significantly mitigates computational complexity while handling multiple views. Rigorous experimentation duly substantiates the efficacy of the proposed framework in the realm of multi-view clustering. Across publicly available multi-view datasets, our framework exhibits superior performance over existing state-of-the-art algorithms.

Linear Layer Architecture Based on Cyclic Shift and XOR

One of the nodes of a block symmetric encryption algorithm is represented by a linear layer, the purpose of which is to distribute the mutual influence of bits within the processed data block. Several methods exist for constructing a linear layer, the most common of which are matrix multiplication operations and the permutation of bits. Both approaches have high computational complexity and are not equally effective for both hardware and software implementations. This paper presents an approach for constructing linear functions for block symmetric encryption algorithms utilizing cyclic shift, and bitwise addition operations are formulated. We provide a preliminary assessment of certain properties of such functions, including the branch number. This linear operation can accommodate binary words of any length, allowing for the design of an optimal linear layer for software or hardware architectures with any word size. Furthermore, the developed architecture allows for balancing the laboriousness of linear operations and related branch numbers. The proposed novel linear layer architecture facilitates the creation of fast lightweight encryption algorithms as well as robust classical algorithms with a high level of cryptographic strength. For efficient implementation on software and hardware platforms, no additional optimizations are required, as the proposed linear layer allows for achieving high performance in both cases.

Score for the Group SL(2,38)

The set of all (n×n) non-singular matrices over the field F. And this set forms a group under the operation of matrix multiplication. This group is called the general linear group of dimension over the field F, denoted by . The determinant of these matrices is a homomorphism from into F* and the kernel of this homomorphism was the special linear group and denoted by Thus is the subgroup of which contains all matrices of determinant one. The rationally valued characters of the rational representations are written as a linear combination of the induced characters for the groups discussed in this paper. We find the Artin indicator for this group after studying the rationally valued characters of the rational representations and the induced characters.

Efficient GEMM Implementation for Vision-Based Object Detection in Autonomous Driving Applications

Convolutional Neural Networks (CNNs) have been incredibly effective for object detection tasks. YOLOv4 is a state-of-the-art object detection algorithm designed for embedded systems. It is based on YOLOv3 and has improved accuracy, speed, and robustness. However, deploying CNNs on embedded systems such as Field Programmable Gate Arrays (FPGAs) is difficult due to their limited resources. To address this issue, FPGA-based CNN architectures have been developed to improve the resource utilization of CNNs, resulting in improved accuracy and speed. This paper examines the use of General Matrix Multiplication Operations (GEMM) to accelerate the execution of YOLOv4 on embedded systems. It reviews the most recent GEMM implementations and evaluates their accuracy and robustness. It also discusses the challenges of deploying YOLOv4 on autonomous vehicle datasets. Finally, the paper presents a case study demonstrating the successful implementation of YOLOv4 on an Intel Arria 10 embedded system using GEMM.

ClipSim: A GPU-friendly Parallel Framework for Single-Source SimRank with Accuracy Guarantee

SimRank is an important metric to measure the topological similarity between two nodes in a graph. In particular, single-source and top-k SimRank has numerous applications in recommendation systems, network analysis, and web mining, etc. Mathematically, given a vertex, the computation of single-machine and single-source SimRank mainly lies in matrix-matrix operations. However, it is almost impossible to directly compute on large graphs. Thus, existing works yield to two main operations: a series of random walks, and sparse matrix and dense vector multiplication operations. This brings about high computation cost for SimRank on large graphs. In real-world applications, there is always the query time and accuracy trade-off, which hinders the computation of high-precision SimRank on large-scale graphs. To handle this problem, this paper proposesClipSim, the first GPU-friendly parallel framework that accelerates the single-source SimRank on GPU with accuracy guarantee. We design a novel data structure and GPU-friendly parallel algorithms for efficient computation of all the operations of SimRank on GPU. Moreover, our theoretical derivation enables ClipSim to largely reduce the number of random walks required for each node, while maintaining the same theoretical accuracy as the state-of-the-art algorithm, ExactSim. We conduct extensive experiments on real-world and synthetic datasets to demonstrate the accuracy and efficiency of ClipSim. The results show that compared with ExactSim, ClipSim obtains single-source SimRank vectors with the same accuracy and up to 160× faster computation time.

Minimizing Multiplication of Kernel Computation in Convolutional Neural Networks Using Strassen Algorithm

Convolution neural networks (CNN) have been widely applied for the computer vision task. However, the success of CNN is limited by the computational complexity of the network, so it is difficult for the model to run the inference process in real time. In this paper, we apply Strassen matrix multiplication to reduce multiplications in convolution operations in CNN, in order to get faster execution for CNN. First, we transform the convolution operation into a matrix multiplication operation using the Toeplitz mapping method, then after that, we apply the Strassen method to these matrices. In the end, we compare the number of arithmetic operations (multiplication and addition) in the convolutional layer using Strassen and the standard algorithm. We apply this algorithm implementation in convolution layers 1 and 3 in LeNet-5 Architecture.

Power-Intent Systolic Array Using Modified Parallel Multiplier for Machine Learning Acceleration.

Systolic arrays are an integral part of many modern machine learning (ML) accelerators due to their efficiency in performing matrix multiplication that is a key primitive in modern ML models. Current state-of-the-art in systolic array-based accelerators mainly target area and delay optimizations with power optimization being considered as a secondary target. Very few accelerator designs directly target power optimizations and that too using very complex algorithmic modifications that in turn result in a compromise in the area or delay performance. We present a novel Power-Intent Systolic Array (PI-SA) that is based on the fine-grained power gating of the multiplication and accumulation (MAC) block multiplier inside the processing element of the systolic array, which reduces the design power consumption quite significantly, but with an additional delay cost. To offset the delay cost, we introduce a modified decomposition multiplier to obtain smaller reduction tree and to further improve area and delay, we also replace the carry propagation adder with a carry save adder inside each sub-multiplier. Comparison of the proposed design with the baseline Gemmini naive systolic array design and its variant, i.e., a conventional systolic array design, exhibits a delay reduction of up to 6%, an area improvement of up to 32% and a power reduction of up to 57% for varying accumulator bit-widths.

An Artificial Intelligence-Assisted Expectation Propagation Detection for MIMO Systems

Multiple-input multiple-output (MIMO) technology is one of the key technical approaches to improve the spectrum efficiency of wireless communication. Modern communication systems employ MIMO and high-order quadrature amplitude modulation (QAM) to maximize spectral efficiency. However, with the increase in the number of antennas and modulation orders, it is very challenging to design a low-complexity and high-efficiency MIMO receiver. In recent years, with the rapid development of new technologies such as artificial intelligence, more and more researchers have tried to apply machine learning techniques in the field of communication to break through the performance of traditional communication algorithms. In this paper, we propose a new low-complexity MIMO detection algorithm: an artificial intelligence-assisted expectation propagation (EP) detection algorithm. Neural network models are used to learn and map some of the time-consuming steps in the EP detection algorithm, converting the complex operation process into a few matrix multiplication operations in order to reduce the complexity of the detection algorithm. It is verified that the method proposed in this paper can approximate the performance of the original EP detection algorithm with reduced complexity and is applicable in different scenarios.

An Introduction to the Theory of Matrix Near-Rings

Matrix rings are prominent in abstract algebra. In this paper we give an overview of the theory of matrix near-rings. A near-ring differs from a ring in that it does not need to be abelian and one of the distributive laws does not hold in general. We introduce two ways in which matrix near-rings can be defined and discuss the structure of each. One is as given by Beildeman and the other is as defined by Meldrum. Beildeman defined his matrix near-rings as normal arrays under the operation of matrix multiplication and addition. He showed that we have a matrix near-ring over a near-ring if, and only if, it is a ring. In this case it is not possible to obtain a matrix near-ring from a proper near-ring. Later, in 1986, Meldrum and van der Walt defined matrix near-rings over a near-ring as mappings from the direct sum of n copies of the additive group of the near-ring to itself. In this case it can be shown that a proper near-ring is obtained. We prove several properties, introduce some special matrices and show that a matrix notation can be introduced to make calculations easier, provided that n is small.