Linear Algebra Computations Research Articles

Enhancing Linear Algebraic Computation of Logic Programs Using Sparse Representation

Algebraic characterization of logic programs has received increasing attention in recent years. Researchers attempt to exploit connections between linear algebraic computation and symbolic computation to perform logical inference in large-scale knowledge bases. In this paper, we analyze the complexity of the linear algebraic methods for logic programs and propose further improvement by using sparse matrices to embed logic programs in vector spaces. We show its great power of computation in reaching the fixed point of the immediate consequence operator. In particular, performance for computing the least models of definite programs is dramatically improved using the sparse matrix representation. We also apply the method to the computation of stable models of normal programs, in which the guesses are associated with initial matrices, and verify its effect when there are small numbers of negation. These results show good enhancement in terms of performance for computing consequences of programs and depict the potential power of tensorized logic programs.

Prediction of missing values. Comparison of approaches

AbstractWe compare four numerical methods for the prediction of missing values in four different datasets [1]. These methods are 1) the hierarchical maximum likelihood estimation (ℋ‐MLE), and three machine learning (ML) methods, which include 2) k‐nearest neighbors (kNN), 3) random forest, and 4) Deep Neural Network (DNN). From the ML methods, the best results (for considered datasets) were obtained by the kNN method with three (or seven) neighbors. On one dataset, the MLE method showed a smaller error than the kNN method, whereas, on another, the kNN method was better. The MLE method requires a lot of linear algebra computations and works fine on almost all datasets. Its result can be improved by taking a smaller threshold and more accurate hierarchical matrix arithmetics. To our surprise, the well‐known kNN method produces similar results as ℋ‐MLE and worked much faster.

Linnea

The translation of linear algebra computations into efficient sequences of library calls is a non-trivial task that requires expertise in both linear algebra and high-performance computing. Almost all high-level languages and libraries for matrix computations (e.g., Matlab, Eigen) internally use optimized kernels such as those provided by BLAS and LAPACK; however, their translation algorithms are often too simplistic and thus lead to a suboptimal use of said kernels, resulting in significant performance losses. To combine the productivity offered by high-level languages, and the performance of low-level kernels, we are developing Linnea, a code generator for linear algebra problems. As input, Linnea takes a high-level description of a linear algebra problem; as output, it returns an efficient sequence of calls to high-performance kernels. Linnea uses a custom best-first search algorithm to find a first solution in less than a second, and increasingly better solutions when given more time. In 125 test problems, the code generated by Linnea almost always outperforms Matlab, Julia, Eigen, and Armadillo, with speedups up to and exceeding 10×.

Compiling basic linear algebra subroutines for quantum computers

Efficiently processing basic linear algebra subroutines is of great importance for a wide range of computational problems. In this paper, we consider techniques to implement matrix functions on a quantum computer. We embed given matrices into 3 times larger Hermitian matrices and assume as input a given set of unitary operators generated by the embedding matrices. With the matrix embedding formula, we give Trotter-based quantum subroutines for elementary matrix operations include addition, multiplication, Kronecker sum, tensor product, Hadamard product, and arbitrary real eigenvalue single-matrix functions. We then discuss the composed matrix functions in terms of the estimation of scalar quantities such as inner products, traces, determinants, and Schatten p-norms with bounded errors. We thus provide a framework for compiling instructions for linear algebraic computations into gate sequences on actual quantum computers. The framework for calculating the matrix functions is more efficient than the best classical counterpart for a set of matrices.

A linear-algebraic method to compute polynomial PDE conservation laws

We present a method to compute polynomial conservation laws for systems of partial differential equations (PDEs). The method only relies on linear algebraic computations and is complete, in the sense it can find a basis for all polynomial fluxes that yield conservation laws, up to a specified order of derivatives and degree. We compare our method to state-of-the-art algorithms based on the direct approach on a few PDE systems drawn from mathematical physics.

A family of parallel quasi-Newton algorithms for unconstrained minimization

This paper deals with the solution of the unconstrained optimization problems on parallel computers using quasi-Newton methods. The algorithm is based on that parallelism can be exploited in function and derivative evaluation costs and linear algebra calculations in the standard sequential algorithm. Computational problem is reported for showing that the parallel algorithm is superior to the sequential one.

Qeirreps: An open-source program for Quantum ESPRESSO to compute irreducible representations of Bloch wavefunctions

Bloch wavefunctions in solids form a representation of crystalline symmetries. Recent studies revealed that symmetry representations in band structure can be used to diagnose the topological properties of weakly interacting materials. In this work, we introduce an open-source program qeirreps that computes the representation characters in a band structure based on the output file of Quantum ESPRESSO. Our program also calculates the Z4 index, i.e., the sum of inversion parities at all time-reversal invariant momenta, for materials with inversion symmetry. When combined with the symmetry indicator method, this program can be used to explore new topological materials. Program summaryProgram title: qeirrepsCPC Library link to program files:https://doi.org/10.17632/dfgw6wgvmt.1Licensing provisions: GNU General Public Licence 3.0Programming language: Fortran 90External routines/libraries:•BLAS (http://www/netlib.org/blas)•LAPACK (http://www.netlib.org/lapack)Nature of problem: Irreducible representations of Bloch wavefunctionsSolution method: Linear algebra calculation for Bloch wavefunctions

Contextual contracts for component‐oriented resource abstraction in a cloud of high performance computing services

SummaryEfforts to support high performance computing (HPC) applications' requirements in the context of cloud computing have motivated us to design HPC Shelf, a cloud computing services platform to build and deploy large‐scale parallel computing systems. We introduce Alite, the contextual contract system of HPC Shelf, to select component implementations according to requirements of the host application, target parallel computing platform characteristics (e.g., clusters and MPPs), quality of service (QoS) properties, and cost restrictions. It is evaluated through a small‐scale case study employing two complementary component‐based frameworks. The first one aims to represent components that implement linear algebra computations based on the BLAS interface. In turn, the second one aims to represent parallel computing platforms on the IaaS cloud offered by Amazon EC2 Service.

GPU Tensor Cores for Fast Arithmetic Reductions

This article proposes a parallel algorithm for computing the arithmetic reduction of n numbers as a set of matrix-multiply accumulate (MMA) operations that are executed simultaneously by GPU tensor cores. The analysis, assuming tensors of size m x m, shows that the proposed algorithm has a parallel running time of T(n) = 5log <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">m2</sub> n and a speedup of S = 45log <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2m</sub> 2 over a canonical parallel reduction. Experimental performance results on a Tesla V100 GPU show that the tensor-core based approach is energy efficient and runs up to ~ 3:2× and 2× faster than a standard GPU-based reduction and Nvidia's CUB library, respectively, while keeping the numerical error below 1 percent with respect to a double precision CPU reduction. The chained design of the algorithm allows a flexible configuration of GPU thread-blocks and the optimal values found through experimentation agree with the theoretical ones. The results obtained in this work show that GPU tensor cores are relevant not only for Deep Learning or Linear Algebra computations, but also for applications that require the acceleration of large summations.

BootCMatchG: An adaptive Algebraic MultiGrid linear solver for GPUs

Abstract Sparse solvers are one of the building blocks of any technology for reliable and high-performance scientific and engineering computing. In this paper we present a software package which implements an efficient multigrid sparse solver running on Graphics Processing Units. The package is a branch of a wider initiative of software development for sparse Linear Algebra computations on emergent HPC architectures involving a large research group working in many application projects over the last ten years.

Rapid Discrete Optimization via Simulation with Gaussian Markov Random Fields

Inference-based optimization via simulation, which substitutes Gaussian process (GP) learning for the structural properties exploited in mathematical programming, is a powerful paradigm that has been shown to be remarkably effective in problems of modest feasible-region size and decision-variable dimension. The limitation to “modest” problems is a result of the computational overhead and numerical challenges encountered in computing the GP conditional (posterior) distribution on each iteration. In this paper, we substantially expand the size of discrete-decision-variable optimization-via-simulation problems that can be attacked in this way by exploiting a particular GP—discrete Gaussian Markov random fields—and carefully tailored computational methods. The result is the rapid Gaussian Markov Improvement Algorithm (rGMIA), an algorithm that delivers both a global convergence guarantee and finite-sample optimality-gap inference for significantly larger problems. Between infrequent evaluations of the global conditional distribution, rGMIA applies the full power of GP learning to rapidly search smaller sets of promising feasible solutions that need not be spatially close. We carefully document the computational savings via complexity analysis and an extensive empirical study. Summary of Contribution: The broad topic of the paper is optimization via simulation, which means optimizing some performance measure of a system that may only be estimated by executing a stochastic, discrete-event simulation. Stochastic simulation is a core topic and method of operations research. The focus of this paper is on significantly speeding-up the computations underlying an existing method that is based on Gaussian process learning, where the underlying Gaussian process is a discrete Gaussian Markov Random Field. This speed-up is accomplished by employing smart computational linear algebra, state-of-the-art algorithms, and a careful divide-and-conquer evaluation strategy. Problems of significantly greater size than any other existing algorithm with similar guarantees can solve are solved as illustrations.

Enhancing Linear Algebraic Computation of Logic Programs Using Sparse Representation

Algebraic characterization of logic programs has received increasing attention in recent years. Researchers attempt to exploit connections between linear algebraic computation and symbolic computation in order to perform logical inference in large scale knowledge bases. This paper proposes further improvement by using sparse matrices to embed logic programs in vector spaces. We show its great power of computation in reaching the fixpoint of the immediate consequence operator from the initial vector. In particular, performance for computing the least models of definite programs is dramatically improved in this way. We also apply the method to the computation of stable models of normal programs, in which the guesses are associated with initial matrices, and verify its effect when there are small numbers of negation. These results show good enhancement in terms of performance for computing consequences of programs and depict the potential power of tensorized logic programs.

Efficient emulators for scattering using eigenvector continuation

Eigenvector continuation (EC) has been shown to accurately and efficiently reproduce ground states for targeted sets of Hamiltonian parameters. It uses as variational basis vectors the corresponding ground-state eigensolutions from selected other sets of parameters. Here we extend the EC approach to scattering using the Kohn variational principle. We first test it using a model for S-wave nucleon-nucleon scattering and then demonstrate that it also works to give accurate predictions for non-local potentials, charged-particle scattering, complex optical potentials, and higher partial waves. These proofs-of-principle validate EC as an accurate emulator for applying Bayesian inference to parameter estimation constrained by scattering observables. The efficiency of such emulators is because the accuracy is achieved with a small number of variational basis elements and the central computations are just linear algebra calculations in the space spanned by this basis.

Scalable linear algebra on a relational database system

As data analytics has become an important application for modern data management systems, a new category of data management system has appeared recently: the scalable linear algebra system. We argue that a parallel or distributed database system is actually an excellent platform upon which to build such functionality. Most relational systems already have support for cost-based optimization---which is vital to scaling linear algebra computations---and it is well known how to make relational systems scalable. We show that by making just a few changes to a parallel/distributed relational database system, such a system can become a competitive platform for scalable linear algebra. Taken together, our results should at least raise the possibility that brand new systems designed from the ground up to support scalable linear algebra are not absolutely necessary, and that such systems could instead be built on top of existing relational technology.

Randomized numerical linear algebra: Foundations and algorithms

This survey describes probabilistic algorithms for linear algebraic computations, such as factorizing matrices and solving linear systems. It focuses on techniques that have a proven track record for real-world problems. The paper treats both the theoretical foundations of the subject and practical computational issues.Topics include norm estimation, matrix approximation by sampling, structured and unstructured random embeddings, linear regression problems, low-rank approximation, subspace iteration and Krylov methods, error estimation and adaptivity, interpolatory and CUR factorizations, Nyström approximation of positive semidefinite matrices, single-view (‘streaming’) algorithms, full rank-revealing factorizations, solvers for linear systems, and approximation of kernel matrices that arise in machine learning and in scientific computing.

Simulation of Fluid–Structure Interaction Problems with Thin Elastic Plate via the Coupling of Finite Element and Lattice Boltzmann Methods

A numerical framework is proposed to couple the finite element (FE) and lattice Boltzmann methods (LBM) for simulating fluid–structure interaction (FSI) problems. The LBM is used as an efficient method for solving the weakly-compressible fluid flows. The corotational FE method for beam elements is used to solve the thin plate deformation. The two methods are coupled via a direct-forcing immersed boundary (IB) method with a sub-iteration scheme. A virtual structure method has been developed to improve the computational accuracy. Validations of the proposed coupling method have been carried out by testing a vortex-induced vibration problem. The numerical results are in good agreement with [Li and Favier (2017), “A non-staggered coupling of finite element and lattice Boltzmann methods via an immersed boundary scheme for fluid-structure interaction,” Comput. Fluids 143, 90–102]. The proposed method does not require heavy linear algebra calculation, which is suitable for parallel computation.

Parallel radiation dose computations with GENOCOP III on GPUs

In radiotherapy planning, which involves optimization, efforts to produce better (more effective and at the same time with lesser adverse effects) patient radiation plans must trade-off with higher computing times needed to achieve this goal. Computing times is the key (but not only) factor in radiotherapy planning, which is always performed in clinical workflows regimes. ‘Win–win’ is when better plans can be produced within a non-increasing time budget. This work reports on the authors’ attempt to put radiotherapy planning in a ‘win–win’ situation. We looked into unexploited till now reserves, which lie in the performance of optimization methods and algorithms, namely the reserves in performing linear algebra computations when such methods are applied to radiotherapy. A specific feature of such applications is the necessity of numerous sparse matrix $$\times$$ vector computations, with matrices and vectors of large sizes. Our first step was to propose ways to include sparse matrix procedures from existing libraries into an optimization algorithm for testing experiments. Our second step in our quest for reserves was to resort to High Performance Computing. We chose graphical processing units because of their versatility, low cost, accessibility, and the existence of linear algebra libraries dedicated to these platforms. We report on the reserves identified in this way, i.e., on speedups of optimization computations achievable by such an optimization algorithm hybridization. We tested our hybrid algorithm numerically on a clinical case.

Hierarchical algorithms on hierarchical architectures.

A traditional goal of algorithmic optimality, squeezing out flops, has been superseded by evolution in architecture. Flops no longer serve as a reasonable proxy for all aspects of complexity. Instead, algorithms must now squeeze memory, data transfers, and synchronizations, while extra flops on locally cached data represent only small costs in time and energy. Hierarchically low-rank matrices realize a rarely achieved combination of optimal storage complexity and high-computational intensity for a wide class of formally dense linear operators that arise in applications for which exascale computers are being constructed. They may be regarded as algebraic generalizations of the fast multipole method. Methods based on these hierarchical data structures and their simpler cousins, tile low-rank matrices, are well proportioned for early exascale computer architectures, which are provisioned for high processing power relative to memory capacity and memory bandwidth. They are ushering in a renaissance of computational linear algebra. A challenge is that emerging hardware architecture possesses hierarchies of its own that do not generally align with those of the algorithm. We describe modules of a software toolkit, hierarchical computations on manycore architectures, that illustrate these features and are intended as building blocks of applications, such as matrix-free higher-order methods in optimization and large-scale spatial statistics. Some modules of this open-source project have been adopted in the software libraries of major vendors.This article is part of a discussion meeting issue ‘Numerical algorithms for high-performance computational science’.

FT-PBLAS: PBLAS-Based Fault-Tolerant Linear Algebra Computation on High-performance Computing Systems

As high-performance computing (HPC) systems have scaled up, resilience has become a great challenge. To guarantee resilience, various kinds of hardware and software techniques have been proposed. However, among popular software fault-tolerant techniques, both the checkpoint-restart approach and the replication technique face challenges of scalability in the era of peta- and exa-scale systems due to their numerous processes. In this situation, algorithm-based approaches, or algorithm-based fault tolerance (ABFT) mechanisms, have become attractive because they are efficient and lightweight. Although the ABFT technique is algorithm-dependent, it is possible to implement it at a low level (e.g., in libraries for basic numerical algorithms) and make it application-independent. However, previous ABFT approaches have mainly aimed at achieving fault tolerance in integrated circuits (ICs) or at the architecture level and are therefore not suitable for HPC systems; e.g., they use checksums of rows and columns of matrices rather than checksums of blocks to detect errors. Furthermore, they cannot deal with errors caused by node failure, which are common in current HPC systems. To solve these problems, this paper proposes FT-PBLAS, a PBLAS-based library for fault-tolerant parallel linear algebra computations that can be regarded as a fault-tolerant version of the parallel basic linear algebra subprograms (PBLAS), because it provides a series of fault-tolerant versions of interfaces in PBLAS. To support the underlying error detection and recovery mechanisms in the library, we propose a block-checksum approach for non-fatal errors and a scheme for addressing node failure, respectively. We evaluate two fault-tolerant mechanisms and FT-PBLAS on HPC systems, and the experimental results demonstrate the performance of our library.

Ara: A 1-GHz+ Scalable and Energy-Efficient RISC-V Vector Processor With Multiprecision Floating-Point Support in 22-nm FD-SOI

In this article, we present Ara, a 64-bit vector processor based on the version 0.5 draft of RISC-V's vector extension, implemented in GlobalFoundries 22FDX fully depleted silicon-on-insulator (FD-SOI) technology. Ara's microarchitecture is scalable, as it is composed of a set of identical lanes, each containing part of the processor's vector register file and functional units. It achieves up to 97% floating-point unit (FPU) utilization when running a 256 × 256 double-precision matrix multiplication on 16 lanes. Ara runs at more than 1 GHz in the typical corner (TT/0.80 V/25 °C), achieving a performance up to 33 DP-GFLOPS. In terms of energy efficiency, Ara achieves up to 41 DP-GFLOPS W <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-1</sup> under the same conditions, which is slightly superior to similar vector processors found in the literature. An analysis on several vectorizable linear algebra computation kernels for a range of different matrix and vector sizes gives insight into performance limitations and bottlenecks for vector processors and outlines directions to maintain high energy efficiency even for small matrix sizes where the vector architecture achieves suboptimal utilization of the available FPUs.