Matrix Multiplication Research Articles

VeriCNN: Integrity verification of large-scale CNN training process based on zk-SNARK

Cloud environments are frequently employed to train Convolutional Neural Networks (CNNs). On the other hand, the training procedure is exposed to potential integrity threats due to the cloud’s lack of trustworthiness. Although there are now available methods for zero-knowledge proof-based training integrity verification for small models, the low-proof performance of large-scale CNNs like LeNet-5 and VGG16 makes this task challenging. To solve the training integrity verification of large-scale CNNs, this research suggests a technique called VeriCNN. Particularly, VeriCNN builds proof of integrity for CNN training using zk-SNARK. It adopts the Winograd algorithm to optimize convolution operations and introduces a high-probability matrix multiplication to optimize zero-knowledge proofs. Experimental results demonstrate the effectiveness and practicality of VeriCNN in verifying the training integrity of large-scale models. For instance, VeriCNN can generate a training integrity proof for the LeNet-5 model in just 25 s and for the VGG16 model in 390 s.

Two-dimensional data partitioning for non-negative matrix tri-factorization

As a two-sided clustering and dimensionality reduction paradigm, Non-negative Matrix Tri-Factorization (NMTF) has attracted much attention in machine learning and data mining researchers due to its excellent performance and reliable theoretical support. Unlike Non-negative Matrix Factorization (NMF) methods applicable to one-sided clustering only, NMTF introduces an additional factor matrix and uses the inherent duality of data to realize the mutual promotion of sample clustering and feature clustering, thus showing great advantages in many scenarios (e.g., text co-clustering). However, the existing methods for solving NMTF usually involve intensive matrix multiplication, which is characterized by high time and space complexities, that is, there are limitations of slow convergence of the multiplicative update rules and high memory overhead. In order to solve the above problems, this paper develops a distributed parallel algorithm with a 2-dimensional data partition scheme for NMTF (i.e., PNMTF-2D). Experiments on multiple text datasets show that the proposed PNMTF-2D can substantially improve the computational efficiency of NMTF (e.g., the average iteration time is reduced by up to 99.7% on Amazon) while ensuring the effectiveness of convergence and co-clustering.

Evaluation of competency dimensions for employee performance assessment: evidence from micro, small, and medium enterprises

PurposeIn recent years, there has been a growing emphasis on competency-based systems as a means of assessing employee performance. These systems assess the degree to which the competencies of employees align with the requirements of their employment positions. This study aims to identify, prioritize, and make contextual interrelationships of the competency dimensions that are relevant for evaluating employees in the context of Indian manufacturing MSMEs.Design/methodology/approachThese dimensions were identified through an extensive literature review and interviews with industry experts. Further, a mixed-methods approach, including the “Bayesian Best-Worst Method” (BBWM), is applied for prioritizing important dimensions, whereas for making mutual relationships, the “Interpretive Structural Modeling” (ISM) method is utilized. “Matrice d'impacts croisés multiplication appliquée á un classment” (MICMAC) is also known as “cross-impact matrix multiplication applied to classification” is used for clustering competency dimensions based on their “driving power” and “dependence power”.FindingsThe findings reveal that among the primary dimensions, “creative performance,” and among the sub-dimensions, “innovative behaviors,” are the most critical competency dimensions for an employee assessment. The study also found that “smart working”, “factual and theoretical knowledge”, “empathy at work”, “understanding of specific knowledge”, and “engagement ideas and activities” are the main dimensions driving employees' competency.Originality/valueThis paper provides contribution to the competence literature by identifying and evaluating competency dimensions for assessing employees' performance within manufacturing MSMEs in an emerging economy such as India. The study also assesses the rank and contextual relationship between the identified dimensions as no past research focused on the same by using BBWM and ISM in the Indian manufacturing MSMEs context.

Algorithms for Matrix Multiplication via Sampling and Opportunistic Matrix Multiplication

Algorithms for Matrix Multiplication via Sampling and Opportunistic Matrix Multiplication

SpChar: Characterizing the sparse puzzle via decision trees

Sparse matrix computation is crucial in various modern applications, including large-scale graph analytics, deep learning, and recommender systems. The performance of sparse kernels varies greatly depending on the structure of the input matrix, making it difficult to gain a comprehensive understanding of sparse computation and its relationship to inputs, algorithms, and target machine architecture. Despite extensive research on certain sparse kernels, such as Sparse Matrix-Vector Multiplication (SpMV), the overall family of sparse algorithms has yet to be investigated as a whole. This paper introduces SpChar, a workload characterization methodology for general sparse computation. SpChar employs tree-based models to identify the most relevant hardware and input characteristics, starting from hardware and input-related metrics gathered from Performance Monitoring Counters (PMCs) and matrices. Our analysis enables the creation of a characterization loop that facilitates the optimization of sparse computation by mapping the impact of architectural features to inputs and algorithmic choices. We apply SpChar to more than 600 matrices from the SuiteSparse Matrix collection and three state-of-the-art Arm Central Processing Units (CPUs) to determine the critical hardware and software characteristics that affect sparse computation. In our analysis, we determine that the biggest limiting factors for high-performance sparse computation are (1) the latency of the memory system, (2) the pipeline flush overhead resulting from branch misprediction, and (3) the poor reuse of cached elements. Additionally, we propose software and hardware optimizations that designers can implement to create a platform suitable for sparse computation. We then investigate these optimizations using the gem5 simulator to achieve a significant speedup of up to 2.63× compared to a CPU where the optimizations are not applied.

A single-sided all-at-once preconditioning for linear system from a non-local evolutionary equation with weakly singular kernels

In Lin et al. (2021) [21] and Zhao et al. (2023) [37], two-sided preconditioning techniques are proposed for non-local evolutionary equations, which possesses (i) mesh-size independent theoretical bound of condition number of the two-sided preconditioned matrix; (ii) small and stable iteration numbers in numerical tests. In this paper, we modify the two-sided preconditioning by multiplying the left-sided and the right-sided preconditioners together as a single-sided preconditioner. Such a single-sided preconditioner essentially derives from approximating the spatial matrix with a fast diagonalizable matrix and keeping the temporal matrix unchanged. Clearly, the matrix-vector multiplication of the single-sided preconditioning is faster to compute than that of the two-sided one, since the single-sided preconditioned matrix has a simpler structure. More importantly, we show theoretically that the single-sided preconditioned generalized minimal residual (GMRES) method has a convergence rate no worse than the two-sided preconditioned one. As a result, the one-sided preconditioned GMRES solver requires less computational time than the two-sided preconditioned GMRES solver in total. Numerical results are reported to show the efficiency of the proposed single-sided preconditioning technique.

Random Projection‐Based Locality‐Sensitive Hashing in a Memristor Crossbar Array with Stochasticity for Sparse Self‐Attention‐Based Transformer

AbstractSelf‐attention mechanism is critically central to the state‐of‐the‐art transformer models. Because the standard full self‐attention has quadratic complexity with respect to the input's length L, resulting in prohibitively large memory for very long sequences, sparse self‐attention enabled by random projection (RP)‐based locality‐sensitive hashing (LSH) has recently been proposed to reduce the complexity to O(L log L). However, in current digital computing hardware with a von Neumann architecture, RP, which is essentially a matrix multiplication operation, incurs unavoidable time and energy‐consuming data shuttling between off‐chip memory and processing units. In addition, it is known that digital computers simply cannot generate provably random numbers. With the emerging analog memristive technology, it is shown that it is feasible to harness the intrinsic device‐to‐device variability in the memristor crossbar array for implementing the RP matrix and perform RP‐LSH computation in memory. On this basis, sequence prediction tasks are performed with a sparse self‐attention‐based Transformer in a hybrid software‐hardware approach, achieving a testing accuracy over 70% with much less computational complexity. By further harnessing the cycle‐to‐cycle variability for multi‐round hashing, 12% increase in the testing accuracy is demonstrated. This work extends the range of applications of memristor crossbar arrays to the state‐of‐the‐art large language models (LLMs).

Strassen's rank additivity for small tensors, including tensors of rank less or equal 7

The article is concerned with the problem of additivity of the tensor rank. That is, for two independent tensors, we study when the rank of their direct sum is equal to the sum of their individual ranks. The statement saying that additivity always holds was previously known as Strassen's conjecture (1969) until Shitov proposed counterexamples (2019). They are not explicit and only known to exist asymptotically for very large tensor spaces. In this article, we present families of pairs of small three-way tensors for which the additivity holds. For instance, over the base field C, it is the case if both tensors are of rank less or equal 7. This proves that a pair of 2×2 matrix multiplication tensors has the rank additivity property. We show also that the Alexeev-Forbes-Tsimerman substitution method preserves the structure of a direct sum of tensors.

A novel and simple spectral method for nonlocal PDEs with the fractional Laplacian

We propose a novel and simple spectral method based on the semi-discrete Fourier transforms to discretize the fractional Laplacian (−Δ)α2. Numerical analysis and experiments are provided to study its performance. Our method has the same symbol |ξ|α as the fractional Laplacian (−Δ)α2 at the discrete level, and thus it can be viewed as the exact discrete analogue of the fractional Laplacian. This unique feature distinguishes our method from other existing methods for the fractional Laplacian. Note that our method is different from the Fourier pseudospectral methods in the literature which are usually limited to periodic boundary conditions (see Remark 1.1). Numerical analysis shows that our method can achieve a spectral accuracy. The stability and convergence of our method in solving the fractional Poisson equations were analyzed. Our scheme yields a multilevel Toeplitz stiffness matrix, and thus fast algorithms can be developed for efficient matrix-vector multiplications. The computational complexity is O(2Nlog⁡(2N)), and the memory storage is O(N) with N the total number of points. Extensive numerical experiments verify our analytical results and demonstrate the effectiveness of our method in solving various problems.

Environment influences the genetic structure and genetic differentiation of Sassafras tzumu (Lauraceae)

BackgroundSassafras tzumu, an elegant deciduous arboreal species, belongs to the esteemed genus Sassafras within the distinguished family Lauraceae. With its immense commercial value, escalating market demands and unforeseen human activities within its natural habitat have emerged as new threats to S. tzumu in recent decades, so it is necessary to study its genetic diversity and influencing factors, to propose correlative conservation strategies.ResultsBy utilizing genotyping-by-sequence (GBS) technology, we acquired a comprehensive database of single nucleotide polymorphisms (SNPs) from a cohort of 106 individuals sourced from 13 diverse Sassafras tzumu natural populations, scattered across various Chinese mountainous regions. Through our meticulous analysis, we aimed to unravel the intricate genetic diversity and structure within these S. tzumu populations, while simultaneously investigating the various factors that potentially shape genetic distance. Our preliminary findings unveiled a moderate level of genetic differentiation (FST = 0.103, p < 0.01), accompanied by a reasonably high genetic diversity among the S. tzumu populations. Encouragingly, our principal component analysis painted a vivid picture of two distinct genetic and geographical regions across China, where gene flow appeared to be somewhat restricted. Furthermore, employing the sophisticated multiple matrix regression with randomization (MMRR) analysis method, we successfully ascertained that environmental distance exerted a more pronounced impact on genetic distance when compared to geographical distance (βE = 0.46, p < 0.01; βD = 0.16, p < 0.01). This intriguing discovery underscores the potential significance of environmental factors in shaping the genetic landscape of S. tzumu populations.ConclusionsThe genetic variance among populations of S. tzumu in our investigation exhibited a moderate degree of differentiation, alongside a heightened level of genetic diversity. The environmental distance of S. tzumu had a greater impact on its genetic diversity than geographical distance. It is of utmost significance to formulate and implement meticulous management and conservation strategies to safeguard the invaluable genetic resources of S. tzumu.

Improving compressed matrix multiplication using control variate method

The seminal work by Pagh [1] proposed a matrix multiplication algorithm for real-valued squared matrices called Compressed Matrix Multiplication (CMM) having a sparse matrix output product. The algorithm is based on a popular sketching technique called Count-Sketch [2] and Fast Fourier Transform (FFT). For input square matrices A and B of order n and the product matrix AB with Frobenius norm ||AB||F, the algorithm offers an unbiased estimate for each entry, i.e., (AB)i,j of the product matrix AB with a variance bounded by ||AB||F2/b, where b is the compressed bucket size. Thus, the variance will eventually become high for a small bucket size. In this work, we address the high variance problem of CMM with the help of a simple and practical technique based on classical variance reduction methods in statistics. Our techniques rely on the Control Variate (CV) method. We suggest rigorous theoretical analysis for variance reduction and complement it via supporting empirical evidence.

Inverse-free zeroing neural network for time-variant nonlinear optimization with manipulator applications

In this paper, the problem of time-variant optimization subject to nonlinear equation constraint is studied. To solve the challenging problem, methods based on the neural networks, such as zeroing neural network and gradient neural network, are commonly adopted due to their performance on handling nonlinear problems. However, the traditional zeroing neural network algorithm requires computing the matrix inverse during the solving process, which is a complicated and time-consuming operation. Although the gradient neural network algorithm does not require computing the matrix inverse, its accuracy is not high enough. Therefore, a novel inverse-free zeroing neural network algorithm without matrix inverse is proposed in this paper. The proposed algorithm not only avoids the matrix inverse, but also avoids matrix multiplication, greatly reducing the computational complexity. In addition, detailed theoretical analyses of the convergence performance of the proposed algorithm is provided to guarantee its excellent capability in solving time-variant optimization problems. Numerical simulations and comparative experiments with traditional zeroing neural network and gradient neural network algorithms substantiate the accuracy and superiority of the novel inverse-free zeroing neural network algorithm. To further validate the performance of the novel inverse-free zeroing neural network algorithm in practical applications, path tracking tasks of three manipulators (i.e., Universal Robot 5, Franka Emika Panda, and Kinova JACO2 manipulators) are conducted, and the results verify the applicability of the proposed algorithm.

Neuro-Immunomodulatory Potential of Nanoenabled 4D Bioprinted Microtissue for Cartilage Tissue Engineering.

Cartilage defects trigger post-traumatic inflammation, leading to a catabolic metabolism in chondrocytes and exacerbating cartilage degradation. Current treatments aim to relieve pain but fail to target the inflammatory process underlying osteoarthritis (OA)progression. Here, a human cartilage microtissue (HCM) nanoenabled with ibuprofen-loaded poly(lactic-co-glycolic acid) nanoparticles (ibu-PLGA NPs) is 4D-bioprinted to locally mitigate inflammation and impair nerve sprouting. Under an in vitro inflamed environment, the nanoenabled HCM exhibits chondroprotective potential by decreasing the interleukin (IL)1β and IL6 release, while sustaining extracellular matrix (ECM) production. In vivo, assessments utilizing the air pouch mouse model affirm the nanoenabled HCM non-immunogenicity. Nanoenabled HCM-derived secretomes do not elicit a systemic immune response and decrease locally the recruitment of mature dendritic cells and the secretion of multiple inflammatory mediators and matrix metalloproteinases when compared to inflamed HCM condition. Notably, the nanoenabled HCM secretome has no impact on the innervation profile of the skin above the pouch cavity, suggesting a potential to impede nerve growth. Overall, HCM nanoenabled with ibu-PLGA NPs emerges as a potent strategy to mitigate inflammation and protect ECM without triggering nerve growth, introducing an innovative and promising approach in the cartilage tissue engineering field.

Efficiency analysis of discontinuous Galerkin approaches for the application onto quantum Liouville-type equations

The simulation of nanodevices is computationally inefficient with current algorithms. The discontinuous Galerkin approach has been demonstrated in the field of computational fluid dynamics to deliver high order accuracy and efficiency due to its reliance on matrix–vector multiplications. Previously, the discontinuous Galerkin approach was successfully used in conjunction with the finite volume technique to solve the Liouville–von Neumann equation in center-mass coordinates and thus simulate nanodevices. To exploit its full potential regarding high-performance computing, this work aims to substitute the aforementioned finite volume technique with the discontinuous Galerkin method. To arrive at the said formalism, a finite element method is implemented as an intermediate step.

A Common Source 3D FeFET with Disturb Inhibition Program and Erase Method

A common source p-type single-crystal channel three-dimensional ferroelectric field-effect transistor (3D FeFET) in a 2 × 2 × 3 array is proposed. Two programming and erasing conditions are introduced. A large memory window (>1.2 V), good retention (>10 years), and high speed (<100 ns) was presented under high voltage (±6 V) conditions. The endurance, >103, was observed under relatively low voltage (±3 V) conditions. Based on these two conditions, a novel asymmetric bias program and erase method is proposed to obtain good disturb inhibition. A more than 0.5 V threshold voltage shift in target cell was achieved while threshold voltage shift in unselected cell was limited, and analysis of long term disturb in novel method is proposed, showing good disturb inhibition. Additional investigation in word line disturbance shows causation and efficiency of disturb. Building upon the proposed structure of the 3D FeFET array, a vector matrix multiplication able to calculate 2-bit weights was designed and demonstrated. This work provides a potential solution for increasing integration density with 3D FeFET array.

Applications of Matrix Multiplication

In this paper we present some interesting applications of the matrix’s multiplication, that include the Leslie matrix and population change, which we calculate this kind of changes from year to other year by matrix multiplication. Another important part of the paper is Analysis of Traffic Flow, we represent the flow of traffic through a network of one-way streets. Another much important part is the production costs, this is fantastic usage of matrix multiplication, in which, A company manufactures three products. Its production expenses are divided into three categories, here in this paper we well describe this beautiful issue. By matrix multiplication, we can encode and decode messages. To encode a message, we choose an invertible matrix and multiply the uncoded row matrices (on the right) by to obtain coded row matrices, this idea will be clarify by some useful examples. Also we used to study certain relationships between objects by matrix multiplication. We will clarify all of these applications by useful examples. In this paper, we present six different applications of matrix multiplication.

Quantization and Hardware Architecture Co-Design for Matrix-Vector Multiplications of Large Language Models

Quantization and Hardware Architecture Co-Design for Matrix-Vector Multiplications of Large Language Models

Privacy-preserving outsourcing scheme of face recognition based on locally linear embedding

Face recognition is a biometric-based technology that identifies or verifies a person’s identity by analyzing and comparing patterns in their facial features. Among kinds of face recognition algorithms, Locally Linear Embedding (LLE) algorithm has the advantage of preserving the local structure of face data. However, the LLE algorithm requires some computationally intensive matrix calculations, which might introduce latency to the device. In this paper, we propose a privacy-preserving outsourcing scheme of face recognition based on LLE algorithm for the first time. By outsourcing matrix multiplication, linear equations, and eigenvalue decomposition of the LLE algorithm to a cloud server, we reduce the computational load on the client device. To ensure data security, we generate a novel type of sparse orthogonal matrix through square root application and random permutation of diagonal elements, used as secret keys to blind the client’s data. Meanwhile, due to the sparsity of key matrices, the complexity of client-side computation complexity is reduced from O(n3) to O(n2). The experimental results show that our proposed scheme reduces computational costs while maintaining almost identical face recognition accuracy as the original LLE-based algorithm.

Evaluation of Air Pollution Control Management In Jakarta by Crossed-Impact Matrix Multiplications Applied to Classification (MICMAC)

Abstract The rapid activity in Jakarta due to high economic growth increases air pollution, which impacts health problems and environmental, social, and ecological disturbances. The research objectives are to determine factors for controlling air pollution, score and evaluate the matrix of direct influence (MDI), draw a direct and indirect influence graph, and make rating and classification comparisons. The information collection was carried out by applying the Center Gather Dialog strategy to decide the factors that impact contamination control. Information evaluation was carried out utilizing the interpretive structural modelling (ISM) with matrice d’ impacts cruises multiplication appliqué a classement (cross-impact matrix multiplication applied to classification, abbreviated as MICMAC). The results show that the factor that influences air pollution control is high concentrations resulting from activities in the transportation sector and industrial sector. Economic growth has an indirect effect because economic growth is correlated in the same direction as air pollution. The transportation sector consists of motorized vehicle activities such as cars, two-wheeled motorbikes, aeroplanes, trains, and ships that use oil fuel, which contributes to the high concentration of air pollution in Jakarta. Controlling air pollution must involve all parties, including government responsibility, public awareness, transportation, and industrial actors. All parties should implement and comply with regulations effectively and efficiently. Air pollution monitoring activities must be carried out appropriately and regularly.

A fast amplitude preserving three-parameter 3D parabolic Radon transform and its application on multiple attenuation

Seismic wavefields propagate through three-dimensional (3D) space, and their precise characterization is crucial for understanding subsurface structures. Traditional 2D algorithms, due to their limitations, are insufficient to fully represent three-dimensional wavefields. The classic 3D Radon transform algorithm assumes that the wavefield's propagation characteristics are consistent in all directions, which often does not hold true in complex underground media. To address this issue, we present an improved 3D three-parameter Radon algorithm that considers the wavefield variation with azimuth and provides a more accurate wavefield description. However, introducing new parameters to describe the azimuthal variation also poses computational challenges. The new Radon transform operator involves five variables and cannot be simply decomposed into small matrices for efficient computation; instead, it requires large matrix multiplication and inversion operations, significantly increasing the computational load. To overcome this challenge, we have integrated the curvature and frequency parameters, simplifying all frequency operators to the same, thereby significantly improving computation efficiency. Furthermore, existing transform algorithms neglect the lateral variation of seismic amplitudes, leading to discrepancies between the estimated multiples and those in the data. To enhance the amplitude preservation of the algorithm, we employ orthogonal polynomial fitting to capture the amplitude spatial variation in 3D seismic data. Combining these improvements, we propose a fast, amplitude-preserving, 3D three-parameter Radon transform algorithm. This algorithm not only enhances computational efficiency while maintaining the original wavefield characteristics, but also improves the representation of seismic data by increasing amplitude fidelity. We validated the algorithm in multiple attenuation using both synthetic and real seismic data. The results demonstrate that the new algorithm significantly improves both accuracy and computational efficiency, providing an effective tool for analyzing seismic wavefields in complex subsurface structures.