Matrix Multiplication Research Articles

Graph-OPU: A highly flexible FPGA-Based Overlay Processor for Graph Neural Networks

Field-programmable gate arrays (FPGAs) are an ideal candidate for accelerating graph neural networks (GNNs). However, the FPGA redeployment process is time-consuming when updating or switching between diverse GNN models across different applications. Existing GNN processors eliminate the need for FPGA redeployment when switching between different GNN models. However, adapting matrix multiplication types by switching processing units decreases hardware utilization. In addition, the bandwidth of DDR limits further improvements in hardware performance. This paper proposes a highly flexible FPGA-based overlay processor for GNN accelerations. Graph-OPU provides excellent flexibility and programmability for users, as the executable code of GNN models is automatically compiled and reloaded without requiring FPGA redeployment. First, we customize the compiler and instruction sets for the inference process of different GNN models. Second, we customize the datapath and optimize the data format in the microarchitecture to fully leverage the advantages of high bandwidth memory (HBM). Third, we design a unified matrix multiplication to handle both sparse-dense matrix multiplication (SpMM) and general matrix multiplication (GEMM), enhancing Graph-OPU performance. During Graph-OPU execution, the computational units are shared between SpMM and GEMM instead of being switched, which improves the hardware utilization. Finally, we implement a hardware prototype on the Xilinx Alveo U50 and test the mainstream GNN models using various datasets. Experimental results show that Graph-OPU achieves up to 1654 \(\times\) and 63 \(\times\) speedup, as well as up to 5305 \(\times\) and 422 \(\times\) energy efficiency boosts, compared to implementations on CPU and GPU, respectively. Graph-OPU outperforms state-of-the-art (SOTA) end-to-end overlay accelerators for GNN, reducing latency by an average of 1.36 \(\times\) and improving energy efficiency by 1.41 \(\times\) on average. Moreover, Graph-OPU exhibits an average 1.45 \(\times\) speed improvement in end-to-end latency over the SOTA GNN processor. Graph-OPU represents an in-depth study of an FPGA-based overlay processor for GNNs, offering high flexibility, speedup, and energy efficiency.

Local Second Order Mo̷ller-Plesset Theory with a Single Threshold Using Orthogonal Virtual Orbitals: A Distributed Memory Implementation.

In order to alleviate the computational burden associated with superlinear compute scalings with molecular size in electron correlation methods, researchers have developed local correlation methods that wisely treat relatively small contributions as zeros but still yield accurate energy approximation. Such local correlation techniques can also be combined with parallel computing resources to obtain further efficiency and scalability. This work focuses on the distributed memory parallel implementation of a local correlation method for second order Mo̷ller-Plesset (MP2) theory. This method also only has a single threshold to control the dropping of terms and accuracy of different computing kernels in the algorithm. The process partitioning strategy and distributed parallel implementation with the message passing interface (MPI) are discussed. In particular, the algorithm relies on a fixed sparsity pattern matrix multiplication and a corresponding distributed conjugate gradient solver, which exhibits almost linear scaling in both strong and weak scaling analyses. Numerical experiments on a range of molecules, including linear chains and molecules with 2 and 3-dimensional characters, are reported. For example, with only 32 MPI ranks, this MP2 implementation can calculate the correlation energy of vancomycin in def2-TZVP basis within 0.003% accuracy (10-6.5 threshold) in half an hour, where the same problem is unfeasible to solve with sequential or pure shared memory implementations.

The vibration characteristics of LFGMs plates with openings based on spectral shift Legendre method

This paper introduces a novel spectral shift Legendre method for analyzing the vibrational characteristics of laminated functionally graded materials (LFGMs) plates with openings. By employing shifted Legendre series as permissible displacement functions, the computational process is significantly simplified by transforming the integral and differential calculations in the vibration characteristic equation into matrix multiplication. The stability and accuracy of the method are ensured by expanding the displacement at shifted Gauss-Lobatto points (SGLP). The proposed method is then extended to study the dynamic performance of LFGMs plates with openings. To achieve this, a dynamic equation for LFGMs plates without openings is constructed based on the first-order shear deformation theory (FSDT) and LFGMs theory. Subsequently, by setting the weights of the SGLP corresponding to the opening locations to zero, a rapid and accurate solution to the vibration problems of LFGMs plates with openings is realized. To verify the reliability and accuracy of the method, vibrational analyses of several typical LFGMs plates with openings are conducted, and the results are compared with existing literature and results obtained using the finite element method (FEM). The comparisons indicate that the method is reliable and effective, offering a more direct and convenient modeling process. This work significantly expands the application scope of spectral methods and provides a new perspective for the vibrational analysis of LFGMs with openings.

Dual in-memory computing of matrix-vector multiplication for accelerating neural networks

Dual in-memory computing of matrix-vector multiplication for accelerating neural networks

Tuning high-level synthesis SpMV kernels in Alveo FPGAs

Sparse Matrix-Vector Multiplication (SpMV) is an essential operation in scientific and engineering fields, with applications in areas like finite element analysis, image processing, and machine learning. To address the need for faster and more energy-efficient computing, this paper investigates the acceleration of SpMV through Field-Programmable Gate Arrays (FPGAs), leveraging High-Level Synthesis (HLS) for design simplicity. Our study focuses on the AMD-Xilinx Alveo U280 FPGA, assessing the performance of the SpMV kernel from Vitis Libraries, which is the state of the art on SpMV acceleration on FPGAs. We explore kernel modifications, transition to single precision, and varying partition sizes, demonstrating the impact of these changes on execution time. Furthermore, we investigate matrix preprocessing techniques, including Reverse Cuthill-McKee (RCM) reordering and a hybrid sparse storage format, to enhance efficiency. Our findings reveal that the performance of FPGA-accelerated SpMV is influenced by matrix characteristics, by smaller partition sizes, and by specific preprocessing techniques delivering notable performance improvements. By selecting the best results from these experiments, we achieved execution time enhancements of up to 3.2×. This study advances the understanding of FPGA-accelerated SpMV, providing insights into key factors that impact performance and potential avenues for further improvement.

Investigation of Different Chemical Realizations for Molecular Matrix Multiplications

Investigation of Different Chemical Realizations for Molecular Matrix Multiplications

Boosting co-creation of Nature‐based Solutions within Living Labs: Interrelating enablers using Interpretive Structural Modelling

Co-creation is at the intersection of Nature-based Solutions and Living Lab concepts, which adopt collaboration and innovation amongst different stakeholders to address complex issues such as climate change. Nature-based Solutions have been increasingly recognized for their potential to address diverse societal challenges in a multifunctional, action-oriented, and contextualized manner. Living Labs have proliferated as a participatory approach for the co-development and co-testing of Nature-based Solutions, particularly in Europe. Nevertheless, there is a persistent challenge regarding enhancing stakeholder engagement, which is recognized as crucial for the implementation of Nature-based Solutions. In this article, we examined ten empirical cases of Living Labs applying Nature-based Solutions to increase climate resilience in European coastal cities. We conducted an Interpretive Structural Modelling and Cross-impact Matrix Multiplication Applied to Classification analysis methodology to systematically analyse and interrelate co-creation enablers. We have identified fifteen enablers to co-create Nature-based Solutions in Living Labs across five levels of influence among each other. A key finding is that ‘sharing and learning from real-life examples’ and ‘opening opportunities for informal inputs’ have the most driving power among the studied enablers. Having a robust engagement toolbox also appears to be a crucial point in enhancing co-creation. Exploring the potential of such enablers seems vital to fostering the co-creation of Nature-based Solutions within Living Labs. By understanding how enablers are interlinked and can be prioritized based on their driving and dependence powers, environmental managers can better use their resources and engage with their key stakeholders more successfully.

A Communication-Efficient Distributed Matrix Multiplication Scheme with Privacy, Security, and Resiliency.

Secure distributed matrix multiplication (SDMM) schemes are crucial for distributed learning algorithms where extensive data computation is distributed across multiple servers. Inspired by the application of repairing Reed-Solomon (RS) codes in distributed storage and secret sharing, we propose SDMM schemes with reduced communication overhead through the use of trace polynomials. Specifically, these schemes are designed to address three critical concerns: (i) ensuring information-theoretic privacy against collusion among servers; (ii) providing security against Byzantine servers; and (iii) offering resiliency against stragglers to mitigate computing delays. To the best of our knowledge, security and resiliency are being considered for the first time within trace polynomial-based approaches. Furthermore, our schemes offer the advantage of reduced sub-packetization and a lower server-count requirement, which diminish the computational complexity and download cost for the user.

FPGA-Based Sparse Matrix Multiplication Accelerators: From State-of-the-art to Future Opportunities

Sparse matrix multiplication (SpMM) plays a critical role in high-performance computing applications, such as deep learning, image processing, and physical simulation. Field-Programmable Gate Arrays (FPGAs), with their configurable hardware resources, can be tailored to accelerate SpMMs. There has been considerable research on deploying sparse matrix multipliers across various FPGA platforms. However, the FPGA-based design of sparse matrix multipliers still presents numerous challenges. Therefore, it is necessary to summarize and organize the current work to provide a reference for further research. This paper first introduces the computational method of SpMM, and categorizes the different challenges of FPGA deployment. Following this, we introduce and analyze a variety of state-of-the-art FPGA-based accelerators tailored for SpMMs. In addition, a comparative analysis of these accelerators is performed, examining metrics including compression rate, throughput, and resource utilization. Finally, we propose potential research directions and challenges for further study of FPGA-based SpMM acclerators.

VLR-BPP: An intelligent virtual location replacement based bilateral privacy-preserving architecture for edge cloud systems

Mobile Crowdsourcing (MCS) has emerged as a significant edge-cloud computing paradigm in which workers perceive data at the network edge and report it to cloud-based computing services for processing, enabling the construction of various applications. Consequently, it is imperative to achieve Bilateral Location Privacy-Preserving (BLPP) to protect the privacy of both Data Requester (DR) and workers, as disclosing location information entails many sensitive details that can result in losses for DR and workers alike. The Local Differential Privacy (LDP) approach is widely employed in Privacy-Preserving (PP) techniques due to its inherent advantages, wherein owners release data with added noise, allowing for proactive customization of privacy strength without relying on any third party. However, the current state of LDP methods presents a dilemma: when privacy protection is strong, introducing excessive location noise can lead to a decrease in the accuracy of task-worker matching, while a high rate of task-worker matching necessitates the compromise of privacy strength. In this paper, an intelligent Virtual Location Replacement based enhanced Bilateral Privacy-Preserving (VLR-BPP) architecture is proposed to improve privacy protection strength and matching accuracy in MCS simultaneously. Within the VLR-BPP architecture, a Bipartite-Graph-based Matrix Completion (BGMC) model is employed to establish the spatiotemporal correlations among data. Then, a Virtual Location Replacement (VLR) strategy is proposed to obfuscate the locations of tasks or workers to their highly correlated virtual location before publishing. Based on VLR, three preemptive location virtualization approaches are introduced: Only Task Location Virtual (OTLV), Only Workers Location Virtual (OWLV), and Both Task and Workers Location Virtual (BTWLV). For workers and DR, Randomized Response (RR) techniques and Random Matrix Multiplication Mechanism (RMM) are used to implement LDP independently. A greedy algorithm is adopted to recruit workers for tasks. In response to the data submitted by workers, BGMC imputation mechanism is utilized to enhance data quality. Finally, simulations based on real-world datasets demonstrate that the performance of our architecture surpasses existing state-of-the-art methods in privacy protection and data collection quality by 18.92∼38.17% and 15.49∼50.77%, respectively.

Nijenhuis geometry IV: conservation laws, symmetries and integration of certain non-diagonalisable systems of hydrodynamic type in quadratures

Abstract The paper contains two lines of results: the first one is a study of symmetries and conservation laws of gl -regular Nijenhuis operators. We prove the splitting theorem for symmetries and conservation laws of Nijenhuis operators, show that the space of symmetries of a gl -regular Nijenhuis operator forms a commutative algebra with respect to (pointwise) matrix multiplication. Moreover, all the elements of this algebra are strong symmetries of each other. We establish a natural relationship between symmetries and conservation laws of a gl -regular Nijenhuis operator and systems of the first and second companion coordinates. Moreover, we show that the space of conservation laws is naturally related to the space of symmetries in the sense that any conservation laws can be obtained from a single conservation law by multiplication with an appropriate symmetry. In particular, we provide an explicit description of all symmetries and conservation laws for gl -regular operators at algebraically generic points. The second line of results contains an application of the theoretical part to a certain system of partial differential equations of hydrodynamic type, which was previously studied by different authors, but mainly in the diagonalisable case. We show that this system is integrable in quadratures, i.e. its solutions can be found for almost all initial curves by integrating closed 1-forms and solving some systems of functional equations. The system is not diagonalisable in general, and construction and integration of such systems is an actively studied and explicitly stated problem in the literature.

Heparan Sulfate-Collagen Surface Multilayers Support Serum-Free Microcarrier Culture of Mesenchymal Stem Cells.

The increasing cost of high-volume cultures and dependence on serum and growth factor supplementation limit the affordability of mesenchymal stromal cell (MSC) therapies. This has spurred interest in developing strategies that support adherent cell expansion while reducing raw material costs. Culture surfaces coated with sulfated glycosaminoglycans (GAGs), specifically heparan sulfate (HS), are an alternative to prolong growth factor retention in cell cultures. Unlike heparin, recombinant HS (rHS) offers strong binding affinity for multiple growth factors and extracellular matrix components, such as collagen I, without undesirable anticoagulant effects or xenobiotic health risks. The potential of rHS as a factor reservoir in MSC cultures remains underexplored. This study investigated the impact of rHS on the growth and anti-inflammatory properties of undifferentiated bone marrow MSCs in both planar and microcarrier-based cultures. It was hypothesized that rHS would enable MSC growth with minimal growth factor supplementation in a sulfation level-dependent manner. Cell culture surfaces were assembled via the layer-by-layer (LbL) method, combining alternating collagen I (COL) and rHS. These bilayers support cell adhesion and enable the incorporation of distinct sulfation levels on the culture surface. Examination of pro-mitogenic FGF and immunostimulatory IFN-γ release dynamics confirmed prolonged availability and sulfate level dependencies. Sulfated surfaces supported cell growth in low serum (2% FBS) and serum-free (SF) media at levels equivalent to standard culture conditions. Cell growth on rHS-coated surfaces in SF was comparable to that on heparin-coated surfaces and commercial surface-coated microcarriers in low serum. These growth benefits were observed in both planar and microcarrier (μCs) cultures. Additionally, rHS surfaces reduced β-galactosidase expression relative to uncoated surfaces, delaying cell senescence. Multivariate analysis of cytokines in conditioned media indicated that rHS-containing surfaces enhanced cytokine levels relative to uncoated surfaces during IFN-γ stimulation and correlated with decreased pro-inflammatory macrophage activity. Overall, utilizing highly sulfated rHS with COL reduces the need for exogenous growth factors and effectively supports MSC growth and anti-inflammatory potency on planar and microcarrier surfaces under minimal factor supplementation.

Mentor: A Memory-Efficient Sparse-dense Matrix Multiplication Accelerator Based on Column-Wise Product

Sparse-dense matrix multiplication (SpMM) is the performance bottleneck of many high-performance and deep-learning applications, making it attractive to design specialized SpMM hardware accelerators. Unfortunately, existing hardware solutions do not take full advantage of data reuse opportunities of the input and output matrices or suffer from irregular memory access patterns. Their strategies increase the off-chip memory traffic and bandwidth pressure, leaving much room for improvement. We present Mentor , a new approach to designing SpMM accelerators. Our key insight is that column-wise dataflow, while rarely exploited in prior works, can address these issues in SpMM computations. Mentor is a software-hardware co-design approach for leveraging column-wise dataflow to improve data reuse and regular memory accesses of SpMM. On the software level, Mentor incorporates a novel streaming construction scheme to preprocess the input matrix for enabling a streaming access pattern. On the hardware level, it employs a fully pipelined design to unlock the potential of column-wise dataflow further. The design of Mentor is underpinned by a carefully designed analytical model to find the trade-off between performance and hardware resources. We have implemented an FPGA prototype of Mentor . Experimental results show that Mentor achieves speedup by geomean 2.05 × (up to 3.98 ×), reduces the memory traffic by geomean 2.92 × (up to 4.93 ×), and improves bandwidth utilization by geomean 1.38 × (up to 2.89 ×), compared to the state-of-art hardware solutions.

Secure and efficient general matrix multiplication on cloud using homomorphic encryption

Secure and efficient general matrix multiplication on cloud using homomorphic encryption

MAFT-SO: A novel multi-atlas fusion template based on spatial overlap for ASD diagnosis

Autism spectrum disorder (ASD) is a common neurological condition. Early diagnosis and treatment are essential for enhancing the life quality of individuals with ASD. However, most existing studies either focus solely on the brain networks of subjects within a single atlas or merely employ simple matrix concatenation to represent the fusion of multi-atlas. These approaches neglected the natural spatial overlap that exists between brain regions across multi-atlas and did not fully capture the comprehensive information of brain regions under different atlases. To tackle this weakness, in this paper, we propose a novel multi-atlas fusion template based on spatial overlap degree of brain regions, which aims to obtain a comprehensive representation of brain networks. Specifically, we formally define a measurement of the spatial overlap among brain regions across different atlases, named spatial overlap degree. Then, we fuse the multi-atlas to obtain brain networks of each subject based on spatial overlap. Finally, the GCN is used to perform the final classification. The experimental results on Autism Brain Imaging Data Exchange (ABIDE) demonstrate that our proposed method achieved an accuracy of 0.757. Overall, our method outperforms SOTA methods in ASD/TC classification.

The Enablers and Interdependencies of Blockchain Technology in Supply Chain Management: Evidence From Luxury Products

ABSTRACTBlockchain is a technology that combines a set of properties to guarantee network security, transparency, and visibility, including a decentralized structure, distributed notes and storage mechanism, consensus algorithm, intelligent contracts, and asymmetric encryption. Supply chain management activities, including supply chain management provenance, business process reengineering, and security enhancement, have enormous potential to be transformed by blockchain. This research uses the integrated interpretive structural modeling–cross‐impact matrix multiplication applied to classification (ISM‐MICMAC) technique to ascertain the hierarchical relationships and to comprehend the severity of interrelationships among various components in tackling our research questions. In MICMAC analysis, the enablers were classified into four categories based on their dependence and driving powers. The combined ISM‐MICMAC methodology employed for this study relies on experts’ individual evaluations and subjective assessments. Therefore, even with extreme caution, it is impossible to guarantee that the results are entirely devoid of personal biases. To further validate the linkages discovered in this study, we suggest employing more multiple‐criteria decision‐making (MCDM) methodologies and comparing the results with those of our research. Another method for confirming the results of the current study is to use an empirical research design based on survey methods.

Barriers to blockchain-enabled drug recycling: A TISM-MICMAC approach

Pharmaceutical and personal care products have been recognized as contaminants of emerging concern, and many compounds are entering the environment through non-source points like households. Integrating circular economy principles into the supply chain has been proposed as a sustainable waste minimization strategy. Drug recycling programs facilitate the return of end-of-use medicine from consumers, quality checks, screening for reusability, and redistribution to people in need. In this study, we conceptualize a blockchain-enabled drug recycling (BCT-DR) process and identify the barriers to implementing BCT-DR. The contextual relationships between the barriers are examined using Total interpretive structural modeling(TISM) and classified using the Cross-Impact Matrix Multiplication Applied to Classification (MICMAC) approach. Among the factors considered for the study, inadequate regulatory policies, technological complexity, and data risks are found to be the key barriers to the adoption of BCT-DR. The estimation of the contextual relationships of the barriers to BCT-DR has implications for managers, policymakers, researchers, and health system actors in designing effective interventional strategies to address the barriers to BCT-DR.

Correction of ‘J. Laderman, V. Pan, X.–H. Sha, On practical Algorithms for Accelerated Matrix Multiplication, Linear Algebra and its Applications. Vol. 162-164 (1992) pp. 557-588’

In this article, we corrected the trilinear formula for triple disjoint matrix multiplication given in the article ‘J. Laderman, V. Pan, X. H. Sha, On practical Algorithms for Accelerated Matrix Multiplication, Linear Algebra and its Applications. Vol. 162-164 (1992) pp. 557-588’, which is incorrect for matrix dimensions equal to two or greater. That formula is a base of two algorithms, for disjoint and single matrix multiplication. The necessary correction made the amount of non–scalar products rise, slightly increasing the algorithm time complexity. We also corrected explicit formulas, in the bilinear form, for triple disjoint matrix multiplication. They are explicit, and thus convenient for the practical use of the fast matrix multiplication algorithms in question.

Computationally-efficient locking-free isogeometric discretizations of geometrically nonlinear Kirchhoff–Love shells

Discretizations based on the Bubnov-Galerkin method and the isoparametric concept suffer from membrane locking when applied to Kirchhoff–Love shell formulations. Membrane locking causes not only smaller displacements than expected, but also large-amplitude spurious oscillations of the membrane forces. Continuous-assumed-strain (CAS) elements were originally introduced to remove membrane locking in quadratic NURBS-based discretizations of linear plane curved Kirchhoff rods (Casquero et al., CMAME, 2022). In this work, we propose CASs and CASns elements to overcome membrane locking in quadratic NURBS-based discretizations of geometrically nonlinear Kirchhoff–Love shells. CASs and CASns elements are interpolation-based assumed-strain locking treatments. The assumed strains have C0 continuity across element boundaries and different components of the membrane strains are interpolated at different interpolation points. CASs elements use the assumed strains to obtain both the physical strains and the virtual strains, which results in a global tangent matrix which is a symmetric matrix. CASns elements use the assumed strains to obtain only the physical strains, which results in a global tangent matrix which is a non-symmetric matrix. To the best of the authors’ knowledge, CASs and CASns elements are the first assumed-strain treatments to effectively overcome membrane locking in quadratic NURBS-based discretizations of geometrically nonlinear Kirchhoff–Love shells while satisfying the following important characteristics for computational efficiency: (1) No additional unknowns are added, (2) No additional systems of algebraic equations need to be solved, (3) The same elements are used to approximate the displacements and the assumed strains, (4) No additional matrix operations such as matrix inversions or matrix multiplications are needed to obtain the stiffness matrix, and (5) The nonzero pattern of the stiffness matrix is preserved. Analogously to the interpolation-based assumed-strain locking treatments for Lagrange polynomials that are widely used in commercial FEA software, the implementation of CASs and CASns elements only requires to modify the subroutine that computes the element residual vector and the element tangent matrix. The benchmark problems show that CASs and CASns elements, using either 2 × 2 or 3 × 3 Gauss–Legendre quadrature points per element, are effective locking treatments since these element types result in more accurate displacements for coarse meshes and excise the spurious oscillations of the membrane forces.

GPU accelerated sparse curvelet-constrained wavefield reconstruction inversion with source estimation: Application to Chevron Benchmark 2014 blind test data set

Full-waveform inversion (FWI) is a pivotal tool in seismic exploration and seismology but encounters a persistent challenge known as “cycle-skipping,” causing it to frequently converge to local minima. Wavefield reconstruction inversion (WRI) has emerged as a potential solution to mitigate the risk of cycle skipping. Researchers have provided numerical examples indicating that WRI is less prone to being ensnared in local minima caused by the absence of low-frequency component data and the absence of a well-defined initial model compared with conventional FWI. Despite its potential, the computational demands of WRI have hindered its widespread application, especially in scenarios involving ocean towed streamer seismic acquisition and unknown sources, where the augmented systems differ from source to source. In our study, we introduce a novel approach — sparse curvelet-constrained WRI with source estimation (WRI-SE-CC) — accelerated by graphics processing unit (GPU). Real-time source function estimation is achieved through the variable projection method, and noise-related artifacts are suppressed using sparse curvelet constraints. By optimizing the utilization of hundreds of computation processors within a GPU for parallel computing of matrix-vector multiplications, we present a GPU-based grouped conjugate gradient method to accelerate the computation of WRI-SE-CC. Numerical experiments demonstrate a significant 240-fold acceleration compared with the preconditioned conjugate gradient using one CPU core for computations involving multiple sources. Inversion experiments with the overthrust model demonstrate the capability of our method in mitigating local minima and suppressing noise-related artifacts. Furthermore, we validate the framework on the Chevron 2014 blind test data set, showcasing its effectiveness in addressing practical challenges in the field.