Matrix Multiplication Research Articles

Industry 4.0 Implementation Drivers in Manufacturing Companies in Algeria: A Structural Analysis via ISM Method

The new industrial era is characterised by an incredible speed of development, in addition to fierce competition, which prompts companies to quickly adopt Industry 4.0 to maintain their position in the future industrial market. This paper aims to identify the main drivers that facilitate the implementation of Industry 4.0 in manufacturing companies in Algeria. The Interpretative Structural Modelling (ISM) method and the Cross-Impact Matrix Multiplication Applied to a Ranking (MICMAC) method were used. The ISM-MICMAC method aims to determine interrelationships and levels of influence between different factors. This allows the identification of the key drivers for the success of Industry 4.0 implementation in manufacturing companies. The results showed that awareness of the concept of Industry 4.0, research and development, support and vision of senior management, and society acceptance of the technology, in addition to cooperation and openness, are the most important drivers that contribute to the success of the implementation of Industry 4.0. Moreover, improving production, maintenance efficiency, and improving mass customisation also play a crucial role in the adoption of Industry 4.0 in manufacturing companies.

Configurable sparse matrix - matrix multiplication accelerator on FPGA: A systematic design space exploration approach with quantization effects

High-performance sparse matrix multipliers are essential for deep learning applications, and as big data analytics continues to evolve, specialized accelerators are also needed to efficiently handle sparse matrix operations. This paper proposes a modified, configurable, outer product based architecture for sparse matrix multiplication, and explores design space of the proposed architecture. The performance of various architecture configurations has been examined for input samples with similar characteristics. The proposed architecture has been implemented on Xilinx Kintex-7 FPGA using 32-bit single precision floating-point arithmetic and also in 8-bit, 16-bit and 32-bit fixed-point arithmetic formats. The effect of quantization in the proposed architecture has been analyzed extensively and the results have been reported. The performance of the proposed architecture has been compared with state-of-the-art implementations, and an improvement of 9.21% has been observed in the performance.

Low-Dimensional Invariant Embeddings for Universal Geometric Learning

AbstractThis paper studies separating invariants: mappings on D-dimensional domains which are invariant to an appropriate group action and which separate orbits. The motivation for this study comes from the usefulness of separating invariants in proving universality of equivariant neural network architectures. We observe that in several cases the cardinality of separating invariants proposed in the machine learning literature is much larger than the dimension D. As a result, the theoretical universal constructions based on these separating invariants are unrealistically large. Our goal in this paper is to resolve this issue. We show that when a continuous family of semi-algebraic separating invariants is available, separation can be obtained by randomly selecting $$2D+1 $$ 2 D + 1 of these invariants. We apply this methodology to obtain an efficient scheme for computing separating invariants for several classical group actions which have been studied in the invariant learning literature. Examples include matrix multiplication actions on point clouds by permutations, rotations, and various other linear groups. Often the requirement of invariant separation is relaxed and only generic separation is required. In this case, we show that only $$D+1$$ D + 1 invariants are required. More importantly, generic invariants are often significantly easier to compute, as we illustrate by discussing generic and full separation for weighted graphs. Finally we outline an approach for proving that separating invariants can be constructed also when the random parameters have finite precision.

Design methodology of compact edge vision transformer CiM considering non-volatile memory bit precision and memory error tolerance

This paper proposes a design methodology for a compact edge vision transformer (ViT) Computation-in-Memory (CiM). ViT has attracted much attention for its high inference accuracy. However, to achieve high inference accuracy, the conventional ViT requires fine-tuning many parameters with pre-trained models on large datasets and a large number of matrix multiplications in inference. Thus, to map ViT to non-volatile memory (NVM)-based CiM compactly for edge applications (IoT/Mobile devices) in inference, this paper analyses fine-tuning in training, clipping, and quantization in inference. The proposed compact edge ViT CiM can be optimized by three design methods according to use cases considering the required fine-tuning time, ease of setting memory bit precision, and memory error tolerance of ViT CiM. As a result, in CIFAR-10, the most compact type successfully reduces the total memory size of ViT by 85.8% compared with the conventional ViT. Furthermore, the high accuracy type and high error-tolerant type improve inference accuracy by 4.4% and memory-error tolerance by more than four times compared with convolutional neural networks, respectively.

MILP modeling of matrix multiplication: cryptanalysis of KLEIN and PRINCE

MILP modeling of matrix multiplication: cryptanalysis of KLEIN and PRINCE

An analysis of the inhibitors of resilience in the global value chains of multinational enterprises

PurposeThe COVID-19 pandemic has showcased the lack of resilience found in the global value chains (GVCs) of multinational enterprises (MNEs). Existing evidence shows that MNEs have only recently and slowly started recovering and attempting to rebuild the resilience of their GVCs. This paper analyzes the challenges/inhibitors faced by MNEs in building their resilience through their GVCs.Design/methodology/approachA four-stage hybrid model was used to identify the interrelationship among the identified inhibitors and to distinguish the most critical ones by ranking them. In the first stage, we employed a modified total interpretive structural modeling (m-TISM) approach to determine the inter-relationship among the inhibitors. Additionally, we identified the inhibitors' driving power and dependency by performing a matrix multiplication applied to classification (MICMAC) analysis. In the second stage, we employed the Pythagorean fuzzy analytic hierarchy process (PF-AHP) method to determine the weight of the criteria. The next stage followed, in which we used the Pythagorean fuzzy combined compromise solution (PF-CoCoSo) method to rank the inhibitors. Finally, we performed a sensitivity analysis to determine the robustness of the framework we had built based on the criteria and inhibitors.FindingsWe find business sustainability to have the highest importance and managerial governance as the most critical inhibitor hindering the path to resilience. Based on these insights, we derive four research propositions aimed at strengthening the resilience of such GVCs, followed by their implications for theory and practice.Originality/valueOur findings contribute to the extant literature by uncovering key inhibitors that act as barriers to MNEs. We link out our findings with a number of propositions that we derive, which may be considered for implementation by MNEs and could help them endow their GVCs with resilience.

Construction land reduction projects as a pathway to sustainability: an empirical analysis of risks factors in China.

Construction land reduction (CLR) is an effective instrument to improve intensive land use, restrict the expansion of construction land, safeguard the requisition-compensation balance of construction land in China, and realize sustainable development. But multiple risks arise from the process of construction land reduction. In that case, identifying and analyzing the key risks of CLR is the prerequisite for formulating practical policy guidelines. This study is conducted to identify the risk factors of CLR and analyze these risks based on expert opinion. Initially, the original risk factors are sourced from existing literature. In order to tailor them to China's specific context, the Delphi method is employed to systematically refine risk definitions, consolidate similar risk elements, and identify any previously unrecognized risks in the literature. Following an in-depth review of the literature, we create a contextual relationship-based model employing an integrated technique of interpretive structural modeling (ISM) and Cross-Impact Matrix Multiplication Applied to Classification (MICMAC) analysis. Based on the ISM and MICMAC analysis, five key risks were identified, and the prevention strategies and policy recommendations for CLR project risks are put forward.

Adaptive neural acceleration unit based on heterogeneous multicore hardware architecture FPGA and software-defined hardware

ABSTRACT This study is anchored in a heterogeneous multicore hardware architecture, specifically Field Programmable Gate Arrays (FPGAs), and software-defined hardware. It employs dynamic planning for the reconfiguration of a software-defined System-on-Chip (SOC) to expedite neural network processes. The reconfigured software-defined SOC serves as an efficient data processor, facilitating the hybrid development and verification of domain-specific System-on-Chip architectures, adhering to the Advanced Microcontroller Bus Architecture (AMBA) standard. The proposed block data trimming methodology significantly enhances overall hybrid computing efficiency by mitigating throughput limitations inherent in Systolic array hardware. The designed Systolic array Matrix Multiply Unit (MMU) is evaluated for a maximum MMU size of 32 × 32 and 1,024 Multiply Accumulator (MAC) units. Hybrid dynamic circuits are devised for the synthesis of int8, int16, int32, and int64 data types and associated logic circuits to validate the performance of parallel computing. It is anticipated that this proposed system can find utility in the development and deployment of unmanned devices, as well as in the integration of deep learning and multi-sensory fusion involving acoustic, tactile, and force sensory components.

Randomized Block Kaczmarz Methods for Inner Inverses of a Matrix

In this paper, two randomized block Kaczmarz methods to compute inner inverses of any rectangular matrix A are presented. These are iterative methods without matrix multiplications and their convergence is proved. The numerical results show that the proposed methods are more efficient than iterative methods involving matrix multiplications for the high-dimensional matrix.

Distributed computing with the cloud

We investigate the effect of omnipresent cloud storage on distributed computing. To this end, we specify a network model with links of prescribed bandwidth that connect standard processing nodes, and, in addition, passive storage nodes. Each passive node represents a cloud storage system, such as Dropbox, Google Drive etc. We study a few tasks in this model, assuming a single cloud node connected to all other nodes, which are connected to each other arbitrarily. We give implementations for basic tasks of collaboratively writing to and reading from the cloud, and for more advanced applications such as matrix multiplication and federated learning. Our results show that utilizing node-cloud links as well as node-node links can considerably speed up computations, compared to the case where processors communicate either only through the cloud or only through the network links. We first show how to optimally read and write large files to and from the cloud in general graphs using flow techniques. We use these primitives to derive algorithms for combining, where every processor node has an input value and the task is to compute a combined value under some given associative operator. In the special but common case of “fat links,” where we assume that links between processors are bidirectional and have high bandwidth, we provide near-optimal algorithms for any commutative combining operator (such as vector addition). For the task of matrix multiplication (or other non-commutative combining operators), where the inputs are ordered, we present tight results in the simple “wheel” network, where procesing nodes are arranged in a ring, and are all connected to a single cloud node.

Abstract B023: Ectopic Expression of the Obscurin PH-domain in Aggressive Breast Cancer Cells Modulates PI3K/Akt Activity and Inhibits Cellular Migration and Invasion

Abstract New therapies targeting metastatic spread are greatly needed to improve breast cancer patient survival. Recently, obscurin, an ~800kDa protein localized to the cell membrane, has risen to the spotlight as an overlooked metastasis suppressor commonly lost in breast cancer. Specifically, breast epithelial cells lacking obscurin undergo epithelial-mesenchymal transition (EMT) and demonstrate increased migration and invasion. Biochemical studies in our lab have linked breast epithelial obscurin loss to increased activation of the pro-metastatic PI3K/Akt signaling pathway. Mechanistically, the obscurin pleckstrin homology (PH)-domain directly binds the SH3-domain of the p85 regulatory subunit of PI3K. Thus far, due to obscurin’s massive size, gain of function/rescue experiments of obscurin have not been feasible. To combat this challenge, we have generated a mini-obscurin protein construct composed of an obscurin core functional unit: a myristilated obscurin PH-domain (Myr-PH). We hypothesized that expression of Myr-PH in obscurin-low breast cancer cells will reduce PI3K/Akt pathway activity and suppress metastasis. Remarkably, we report that ectopic expression of Myr-PH in the obscurin-low MDA-MB-231 (triple negative) and SKBr3 (HER2 positive) breast cancer cell lines diminishes PI3K/Akt pathway activity. Furthermore, Myr-PH sequesters p85 to the cell membrane, anchoring actin at the cell periphery. Functionally, we demonstrate that expression of Myr-PH in both cell line receptor subtypes: 1) reduces cellular migration collectively through decreased scratch assay wound closure and decreased transwell assay 8um pore migration or at the single cell level, through reduced 3um diameter microchannel migration, 2) diminishes cell seeding on multiple extracellular matrix (ECM) substrates known to populate metastatic microenvironments, and 3) decreases cell dissemination and tumorsphere invasion into a 3D collagen-1 matrix. Underlying these functional changes, we show that Myr-PH inhibits actin-dependent filopodia and invadopodia formation and decreases the expression of downstream PI3K/Akt target matrix metalloproteinases, all of which are potent drivers of metastatic dissemination. To this end, we are currently delivering the obscurin-PH domain in animals via lipid nanoparticles, aiming to examine its anti-metastatic efficacy in vivo. If successful, this work would pioneer the first gene therapy-based, targeted use of a single metastasis suppressor protein domain to block PI3K signaling and inhibit breast cancer metastasis. Citation Format: Matthew K Eason, Se-Jong Lee, Poornima Dubey, Anthony Kim, Konstantinos Konstantopoulos, Aikaterini Kontrogianni-Konstantopoulos. Ectopic Expression of the Obscurin PH-domain in Aggressive Breast Cancer Cells Modulates PI3K/Akt Activity and Inhibits Cellular Migration and Invasion [abstract]. In: Proceedings of the AACR Special Conference in Cancer Research: Advances in Breast Cancer Research; 2023 Oct 19-22; San Diego, California. Philadelphia (PA): AACR; Cancer Res 2024;84(3 Suppl_1):Abstract nr B023.

A Conv‐GEMM reconfigurable accelerator with WS‐RS dataflow for high throughput processing

AbstractConvolution and matrix operations are both important computations in Deep Neural Networks (DNNs). However, the significant differences between convolution and matrix computation patterns have posed a challenge in efficiently supporting both convolution (Conv) and general matrix multiplication (GEMM) on hardware design. This paper proposes a Conv‐GEMM reconfigurable accelerator architecture for high throughput edge processing. A weight stationary‐row streaming (WS‐RS) dataflow scheme is proposed, which maximizes data reuse through hierarchical memory structures and flexible PE connections, and supports high throughput edge‐based deep learning algorithms. Based on the proposed dataflow, multi‐scale memory access network (MMAN), reconfigurable accumulator array (RAA), and configurable instruction set architecture (ISA) are designed to optimize computation throughput and energy efficiency. The accelerator is designed under 65 nm technology, achieves peak performance of 1.15 TOPS at 250 MHz, with an energy efficiency of 1.14 TOPS/W. The GEMM computation achieves 85.7% latency improvement and the Mobilenet‐V1 processing achieves a throughput of 529 fps under a 256 × 224 image size and an 87.15% (top‐5) accuracy on the ImageNet dataset.

Parallel Accelerating Number Theoretic Transform for Bootstrapping on a Graphics Processing Unit

The bootstrapping procedure has become the main bottleneck affecting the efficiency of all known fully homomorphic encryption (FHE) schemes. The state-of-the-art scheme for efficient bootstrapping, which is called fully homomorphic encryption over the torus (TFHE), accelerates polynomial multiplication by leveraging number theoretic transform (NTT) and implementing NTT in parallel on a GPU. Unfortunately, almost none of the recent advancements in NTT take full advantage of a GPU, leading to the need for more time. With this in mind, in this work, a novel faster number theoretic transform based on a GPU is proposed, in which matrix multiplication is used to implement a decomposed small-point NTT. When implementing matrix multiplication, we introduce a merging preprocessing method to merge multiple inputs of the small-point NTT, aiming to effectively minimize the count of modulo operations. Subsequently, when the merged result is multiplied by rotation factors, we use logical left shift rather than arithmetic multiplication to improve the computational efficiency. Our scheme can easily be used to realize a 1024-point NTT and the results of the experiments show that the speedup ratio of our method over the butterfly algorithm is about 2.49.

Spatial supply-demand balance of health and failure consequences for urban sewer pipes

Sewer pipes health and failure consequences evaluation are vital to ensure the safety of urban sewer networks. Sewer pipes with low health and severe failure consequences should be the primary focus of daily maintenance and management for drainage authorities. Conversely, sewer pipes with high health and low failure consequences are temporarily not a primary concern, in order to reduce resource wastage within the constraints of limited economic and time resources. Thus, the scientific analysis of their relationships is deemed essential. This paper applies supply-demand balance theory to analyze the relationship. On the supply side, a health Fuzzy Comprehensive Evaluation model (HFCE) is proposed with weighted hierarchies established. On the demand side, a failure consequences FCE model (FFCE) is established. Each sewer pipe's supply-demand balance score is analyzed using matrix multiplication, highlighting two types of imbalanced pipes and sub-districts. Taking the Liangshui River Basin in Beijing as an example, the results show that the health evaluation can identify the spatial distribution of the health of sewer pipes, with low health levels V and IV accounting for 10.23 % and 4.43 %, respectively. The failure consequences evaluation can evaluate the spatial distribution of the severity of sewer pipe failure consequences, with high failure consequences levels V and IV respectively accounting for 10.97 % and 16.77 %. The supply-demand balance analysis identifies the proportion of the first and second types of imbalanced pipes, respectively 14.24 % and 6.80 %. This method, by analyzing the relationship of health and failure consequences, provides a decision-making basis for drainage authorities to prioritize sewer pipe operation management, repair, and maintenance.

Compact, efficient, and scalable nanobeam core for photonic matrix-vector multiplication

Optical neural networks have emerged as a promising avenue for implementing artificial intelligence applications, with matrix computations being a crucial component. However, the existing implementations based on microring resonators (MRRs) face bottlenecks in integration, power efficiency, and scalability, hindering the practical applications of wavelength division multiplexing (WDM)-based matrix-vector multiplications at the hardware level. Here we present a photonic crystal nanobeam cavity (PCNC) matrix core. Remarkably compact with dimensions reduced to 20µm×0.5µm, the PCNC unit exhibits a thermal tuning efficiency more than three times that of MRRs. Crucially, it is immune to the free spectral range constraint, thus able to harness the wealth of independent wavelength channels provided by WDM. A 3×3 PCNC core chip is demonstrated for animal face recognition and a six-channel chip is employed for handwritten digit classification to demonstrate the scalability. The PCNC solution holds immense promise, offering a versatile platform for next-generation photonic artificial intelligence chips.

Research on fast matrix multiplication algorithm

This research mainly focuses on fast matrix multiplication algorithms. Fast matrix multiplication is one of the most fundamental problems in computer science. The fast matrix multiplication algorithm differs from conventional matrix multiplication in that it offers a faster computational approach that can perform the operation in less than O(n3) time complexity. This algorithm provides a more efficient method for multiplying matrices, significantly reducing the computational requirements. The Laser method, developed by Coppersmith and Winograd, is an algorithm for matrix multiplication that does not involve direct computation. It establishes a relationship between matrix multiplication and tensors and simplifies the operation by finding an intermediate tensor that is computationally manageable. This method applies a series of simplification operations to determine an upper bound on the computational complexity of matrix multiplication. However, as matrices become larger, the computational and memory requirements increase, posing challenges for practical implementation. This research will present the main ideas and performance of the Laser method and discuss the improvements made to the Laser method, including refined analysis and asymmetric hashing techniques. Additionally, it highlights the need for further exploration, such as parallel computing and optimization strategies, to enhance the efficiency of matrix multiplication algorithms. Furthermore, this research will also provide a prospectus for the future of matrix multiplication algorithms, such as the practical implementation of the Laser method.

Efficient FPGA Implementation of Convolutional Neural Networks and Long Short-Term Memory for Radar Emitter Signal Recognition

In recent years, radar emitter signal recognition has enjoyed a wide range of applications in electronic support measure systems and communication security. More and more deep learning algorithms have been used to improve the recognition accuracy of radar emitter signals. However, complex deep learning algorithms and data preprocessing operations have a huge demand for computing power, which cannot meet the requirements of low power consumption and high real-time processing scenarios. Therefore, many research works have remained in the experimental stage and cannot be actually implemented. To tackle this problem, this paper proposes a resource reuse computing acceleration platform based on field programmable gate arrays (FPGA), and implements a one-dimensional (1D) convolutional neural network (CNN) and long short-term memory (LSTM) neural network (NN) model for radar emitter signal recognition, directly targeting the intermediate frequency (IF) data of radar emitter signal for classification and recognition. The implementation of the 1D-CNN-LSTM neural network on FPGA is realized by multiplexing the same systolic array to accomplish the parallel acceleration of 1D convolution and matrix vector multiplication operations. We implemented our network on Xilinx XCKU040 to evaluate the effectiveness of our proposed solution. Our experiments show that the system can achieve 7.34 giga operations per second (GOPS) data throughput with only 5.022 W power consumption when the radar emitter signal recognition rate is 96.53%, which greatly improves the energy efficiency ratio and real-time performance of the radar emitter recognition system.

Simplified implicit finite-difference method of spatial derivative using explicit schemes with optimized constant coefficients based on L1 norm

The explicit finite-difference (EFD) method is widely used in numerical simulation of seismic wave propagation to approximate spatial derivatives. However, the traditional and optimized high-order EFD methods suffer from the saturation effect, which seriously restricts the improvement of numerical accuracy. In contrast, the implicit FD (IFD) method approximates the spatial derivatives in the form of rational functions and thus can obtain much higher numerical accuracy with relatively low orders; however, its computational cost is expensive due to the need to invert a multidiagonal matrix. We derive an explicit strategy for the IFD method to reduce the computational cost by constructing the IFD method with the discrete Fourier matrix; then, we transform the inversion of the multidiagonal matrix into an explicit matrix multiplication; next, we construct an objective function based on the [Formula: see text] norm to reduce approximation error of the IFD method. This explicit strategy of the IFD method can avoid inverting the multidiagonal matrix, thus improving the computational efficiency. This constant coefficient optimization method reduces the approximation error in the medium-wavenumber range at the cost of tolerable deviation (smaller than 0.0001) in the low-wavenumber range. For the 2D Marmousi model, the root-mean-square error of the numerical results obtained by this method is one-fifth that of the traditional IFD method with the same order (i.e., 5/3) and one-third that of the traditional EFD method with much higher orders (i.e., 72). The significant reduction of numerical error makes the developed method promising for numerical simulation in large-scale models, especially for long-time simulations.

Reprogrammable Non-Linear Circuits Using ReRAM for NN Accelerators

As the massive usage of artificial intelligence techniques spreads in the economy, researchers are exploring new techniques to reduce the energy consumption of Neural Network (NN) applications, especially as the complexity of NNs continues to increase. Using analog Resistive RAM devices to compute matrix-vector multiplication in O (1) time complexity is a promising approach, but it is true that these implementations often fail to cover the diversity of non-linearities required for modern NN applications. In this work, we propose a novel approach where Resistive RAMs themselves can be reprogrammed to compute not only the required matrix multiplications but also the activation functions, Softmax, and pooling layers, reducing energy in complex NNs. This approach offers more versatility for researching novel NN layouts compared to custom logic. Results show that our device outperforms analog and digital field-programmable approaches by up to 8.5× in experiments on real-world human activity recognition and language modeling datasets with convolutional neural network, generative pre-trained Transformer, and long short-term memory models.

A non-probabilistic reliability topology optimization method based on aggregation function and matrix multiplication considering buckling response constraints

A non-probabilistic reliability topology optimization method based on aggregation function and matrix multiplication considering buckling response constraints