Optimized Matrix Multiplication Research Articles

Improved privacy-preserving PCA using optimized homomorphic matrix multiplication

Principal Component Analysis (PCA) is a pivotal technique widely utilized in the realms of machine learning and data analysis. It aims to reduce the dimensionality of a dataset while minimizing the loss of information. In recent years, there have been endeavors to utilize homomorphic encryption in privacy-preserving PCA algorithms for secure cloud computing. These approaches commonly employ a PCA routine known as PowerMethod, which takes the covariance matrix as input and generates an approximate eigenvector corresponding to the primary component of the dataset. However, their performance is constrained by the absence of an efficient homomorphic covariance matrix computation circuit and an accurate homomorphic vector normalization strategy in the PowerMethod algorithm. In this study, we propose a novel approach to privacy-preserving PCA that addresses these limitations, resulting in superior efficiency, accuracy, and scalability compared to previous approaches.We attain such efficiency and precision through the following contributions: (i) We implement space and speed optimization techniques for a homomorphic matrix multiplication method, specifically tailored for parallel computing scenarios. (ii) Leveraging the benefits of this optimized matrix multiplication, we devise an efficient homomorphic circuit for computing the covariance matrix homomorphically. (iii) Utilizing the covariance matrix, we develop a novel and efficient homomorphic circuit for the PowerMethod that incorporates a universal homomorphic vector normalization strategy to enhance both its accuracy and practicality.Our privacy-preserving PCA scheme, implemented using our innovative homomorphic PowerMethod circuit, surpasses the state-of-the-art approach with an average speedup of 1.9 times on datasets with size 200×256. Notably, our scheme demonstrates an even more remarkable estimated speedup of 25 times when applied to larger datasets of size 60000×256, along with an R2 accuracy improvement of up to 0.285, showcasing efficiency and accuracy that has not been reported by previous approaches.

Weighted slice rank and a minimax correspondence to Strassen's spectra

Structural and computational understanding of tensors is the driving force behind faster matrix multiplication algorithms, the unraveling of quantum entanglement, and the breakthrough on the cap set problem. Strassen's asymptotic spectra program (FOCS 1986) characterizes optimal matrix multiplication algorithms through monotone functionals. Our work advances and makes novel connections among two recent developments in the study of tensors, namely•the slice rank of tensors, a notion of rank for tensors that emerged from the resolution of the cap set problem (Ann. Math. 2017),•and the quantum functionals of tensors (STOC 2018), monotone functionals defined as optimizations over moment polytopes. More precisely, we introduce an extension of slice rank that we call weighted slice rank and we develop a minimax correspondence between the asymptotic weighted slice rank and the quantum functionals. Weighted slice rank encapsulates different notions of bipartiteness of quantum entanglement.The correspondence allows us to give a rank-type characterization of the quantum functionals. Moreover, whereas the original definition of the quantum functionals only works over the complex numbers, this new characterization can be extended to all fields. Thereby, in addition to gaining deeper understanding of Strassen's theory for the complex numbers, we obtain a proposal for quantum functionals over other fields. The finite field case is crucial for combinatorial and algorithmic problems where the field can be optimized over.

More Efficient Secure Matrix Multiplication for Unbalanced Recommender Systems

With recent advances in homomorphic encryption (HE), it becomes feasible to run non-interactive machine learning (ML) algorithms on encrypted data without decryption. In this work, we propose novel encoding methods to pack matrix in a compact way and more efficient methods to perform matrix multiplication on homomorphically encrypted data, leading to a speed boost of <inline-formula><tex-math notation="LaTeX">$1.5\times - 20\times$</tex-math></inline-formula> for slim rectangular matrix multiplication compared with state-of-the-art. Moreover, we integrate our optimized secure matrix arithmetic with the MPI distributed computing framework, achieving scalable parallel secure matrix computation. Equipped with the optimized matrix multiplication, we propose <i>uSCORE</i> , a privacy-preserving cross-domain recommendation system for the unbalanced scenario, where a big data owner provides recommendation as a service to a client who has less data and computation power. Our design delegates most of the computation to the service provider, and has a low communication cost, which previous works failed to achieve. For a client who has 16 million user-item pairs to update, it only needs about 3 minutes (in the LAN setting) to prepare the encrypted data. The server can finish the update process on the encrypted data in less than half an hour, effectively reducing the client's test error from 0.72 to 0.62.

High‐performance direct algorithms for computing the sign function of triangular matrices

SummaryAlgorithms and implementations for computing the sign function of a triangular matrix are fundamental building blocks for computing the sign of arbitrary square real or complex matrices. We present novel recursive and cache‐efficient algorithms that are based on Higham's stabilized specialization of Parlett's substitution algorithm for computing the sign of a triangular matrix. We show that the new recursive algorithms are asymptotically optimal in terms of the number of cache misses that they generate. One algorithm that we present performs more arithmetic than the nonrecursive version, but this allows it to benefit from calling highly optimized matrix multiplication routines; the other performs the same number of operations as the nonrecursive version, suing custom computational kernels instead. We present implementations of both, as well as a cache‐efficient implementation of a block version of Parlett's algorithm. Our experiments demonstrate that the blocked and recursive versions are much faster than the previous algorithms and that the inertia strongly influences their relative performance, as predicted by our analysis.

High-Performance Evaluation of Finite Element Variational Forms via Commuting Diagrams and Duality

We revisit the question of optimizing the construction and application of finite element matrices. By using commuting properties of the reference mappings and duality, we reorganize stiffness matrix construction and matrix-free application so that the bulk of the work can be done by optimized matrix multiplication libraries. We provide examples, including numerical experiments, with the Laplace and curl-curl operators as well as develop a general framework. Our techniques are applicable in general geometry and are not restricted to constant coefficient operators.

An Optimized Matrix Multiplication on ARMv7 Architecture

A sufficiently optimized matrix multiplication on embedded systems can facilitate data processing in high performance mobile measuring equipment since plenty of the kernel mathematical algorithms are based on matrix multiplication. In this paper, we propose a matrix multiplication specially optimized for ARMv7 architecture. The performance-critical differences between ARMv7 and conventional desktop/server architecture are considered to block the simple implementation. The Advanced-SIMD (Single Instruction Multiple Data) engine NEON is additionally exploited to increase the arithmetic computing performance and decrease the memory access latency. Experimental results demonstrate that the proposed scheme is 7-20 times faster than the simple implementation and superior to popular algorithm and open source libraries.

A Parallel Implementation of the Gustafson-Kessel Clustering Algorithm with CUDA

Despite the benefits of the Gustafson-Kessel (GK) clustering algorithm, it becomes computationally inefficient when applied to high-dimensional data. In this letter, a parallel implementation of the GK algorithm on the GPU with CUDA is proposed. Using an optimized matrix multiplication algorithm with fast access to shared memory, the CUDA version achieved a maximum 240-fold speedup over the single-CPU version.