Level-3 Basic Linear Algebra Subprograms Research Articles

Parallel Optimization of BLAS on a New-Generation Sunway Supercomputer

The new-generation Sunway supercomputer has ultra-high computing capacity. But due to the unique heterogeneous architecture of the supercomputer, the open-source versions of basic linear algebra subprograms (BLAS) are insufficient for performance or compatibility. In addition, due to the update of the architecture, BLAS based on the previous Sunway could not fully exploit the performance of the successor. To address the challenges, we propose an optimized BLAS on the new-generation Sunway supercomputer in this paper. Specially, for achieving efficient computation, a parallel optimization method based on the new-generation Sunway for the Level-1 BLAS computing between vectors and the Level-2 BLAS computing between vectors and matrices is first proposed. Then, an adaptive scheduling algorithm for various data sizes is proposed, which is used to balance the tasks of core groups. Finally, to achieve highly efficient general matrix multiplication (GEMM) kernels, a parallel optimization method based on the new-generation Sunway for the Level-3 BLAS computing between matrices is proposed, which includes source-level optimization as well as assembly-level optimization. Experimental results show that the memory bandwidth utilization of the optimized Level-1/2 BLAS exceeds 95%, and the computational efficiency of the optimized GEMM kernel exceeds 94%.

Integration and exploitation of intra-routine malleability in BLIS

Malleability is a property of certain applications (or tasks) that, given an external request or autonomously, can accommodate a dynamic modification of the degree of parallelism being exploited at runtime. Malleability improves resource usage (core occupation) on modern multicore architectures for applications that exhibit irregular and divergent execution paths and heavily depend on the underlying library performance to attain high performance. The integration of malleability within high-performance instances of the Basic Linear Algebra Subprograms (BLAS) is nonexistent, and, in addition, it is difficult to attain given the rigidity of current application programming interfaces (APIs). In this paper, we overcome these issues presenting the integration of a malleability mechanism within BLIS, a high-performance and portable framework to implement BLAS-like operations. For this purpose, we leverage low-level (yet simple) APIs to integrate on-demand malleability across all Level-3 BLAS routines, and we demonstrate the performance benefits of this approach by means of a higher-level dense matrix operation: the LU factorization with partial pivoting and look-ahead.

Free vibration reanalysis of structures with added degrees of freedom

This paper deals with free vibration reanalysis for structural topological modifications with added degrees of freedom. A modified initial analysis system (MIAS) is first constructed to include the newly added degrees of freedom. The MIAS is then used to analyze the modified structure, keeping the number of degrees of freedom unchanged. The block combined approximation method with shifting is finally utilized to calculate multiple eigensolutions of the modified structure. The implementation of the approach involves only LDLT factorizations of shifted sub-stiffness matrices corresponding to the newly added degrees of freedom. The proposed method consists of matrix–matrix operations. These operations can be implemented by using Level-3 Basic Linear Algebra Subprograms, the execution efficiency is thus enhanced. Numerical examples are given to demonstrate the effectiveness of the proposed method and the accuracy of the approximate solutions.

The approximate reanalysis method for topology optimization under harmonic force excitations with multiple frequencies

In mode acceleration method for topology optimization related harmonic response with multiple frequencies, most of the computation effort is invested in the solution of the eigen-problem. This paper is focused on reduction of the computational effort in repeated solution of the eigen-problem involved in mode acceleration method. The block combined approximation with shifting method is adopted for eigen-problem reanalysis, which simultaneously calculates some eigenpairs of modified structures. The triangular factorizations of shifted stiffness matrices generated within a certain number of design iterations are utilized to calculate the modes. For improving computational efficiency, Basic Linear Algebra Subprograms (BLAS) are utilized. The reanalysis method is based on matrix-matrix operations with Level-3 BLAS and can provide very fast development of approximate solutions of high quality for frequencies and associated mode shapes of the modified structure. Numerical examples are given to demonstrate the efficiency of the proposed topology optimization procedure and the accuracy of the approximate solutions.

BLIS: A Framework for Rapidly Instantiating BLAS Functionality

The BLAS-like Library Instantiation Software (BLIS) framework is a new infrastructure for rapidly instantiating Basic Linear Algebra Subprograms (BLAS) functionality. Its fundamental innovation is that virtually all computation within level-2 (matrix-vector) and level-3 (matrix-matrix) BLAS operations can be expressed and optimized in terms of very simple kernels. While others have had similar insights, BLIS reduces the necessary kernels to what we believe is the simplest set that still supports the high performance that the computational science community demands. Higher-level framework code is generalized and implemented in ISO C99 so that it can be reused and/or reparameterized for different operations (and different architectures) with little to no modification. Inserting high-performance kernels into the framework facilitates the immediate optimization of any BLAS-like operations which are cast in terms of these kernels, and thus the framework acts as a productivity multiplier. Users of BLAS-dependent applications are given a choice of using the traditional Fortran-77 BLAS interface, a generalized C interface, or any other higher level interface that builds upon this latter API. Preliminary performance of level-2 and level-3 operations is observed to be competitive with two mature open source libraries (OpenBLAS and ATLAS) as well as an established commercial product (Intel MKL).

Code Generation and Optimization of Distributed-memory Dense Linear Algebra Kernels

Design by Transformation (DxT) is an approach to software development that encodes domain-specific programs as graphs and expert design knowledge as graph transformations. The goal of DxT is to mechanize the generation of highly-optimized code. This paper demonstrates how DxT can be used to transform sequential specifications of an important set of Dense Linear Algebra (DLA) kernels, the level-3 Basic Linear Algebra Subprograms (BLAS3), into high-performing library routines targeting distributed-memory (cluster) architectures. Getting good BLAS3 performance for such platforms requires deep domain knowledge, so their implementations are manually coded by experts. Unfortunately, there are few such experts and developing the full variety of BLAS3 implementations takes a lot of repetitive effort. A prototype tool, DxTer, automates this tedious task. We explain how we build on previous work to represent loops and multiple loop-based algorithms in DxTer. Performance results on a BlueGene/P parallel supercomputer show that the generated code meets or beats implementations that are hand-coded by a human expert and outperforms the widely used ScaLAPACK library.

Codesign Tradeoffs for High-Performance, Low-Power Linear Algebra Architectures

As technology is reaching physical limits, reducing power consumption is a key issue on our path to sustained performance. In this paper, we study fundamental tradeoffs and limits in efficiency (as measured in energy per operation) that can be achieved for an important class of kernels, namely the level-3 Basic Linear Algebra Subprograms (BLAS). It is well-accepted that specialization is the key to efficiency. This paper establishes a baseline by studying GEneral Matrix-matrix Multiplication (GEMM) on a variety of custom and general-purpose CPU and GPU architectures. Our analysis shows that orders of magnitude improvements in efficiency are possible with relatively simple customizations and fine-tuning of memory hierarchy configurations. We argue that these customizations can be generalized to perform other representative linear algebra operations. In addition to exposing the sources of inefficiencies in current CPUs and GPUs, our results show our prototype Linear Algebra Processor (LAP) implementing Double-precision GEMM (DGEMM) can achieve 600 GFLOPS while consuming less than 25 Watts in standard 45 nm technology, which is up to 50 × more energy efficient than cutting-edge CPUs.

A parallel block implementation of Level-3 BLAS for MIMD vector processors

We describe an implementation of Level-3 BLAS (Basic Linear Algebra Subprograms) based on the use of the matrix-matrix multiplication kernel (GEMM). Blocking techniques are used to express the BLAS in terms of operations involving triangular blocks and calls to GEMM. A principal advantage of this approach is that most manufacturers provide at least an efficient serial version of GEMM so that our implementation can capture a significant percentage of the computer performance. A parameter which controls the blocking allows an efficient exploitation of the memory hierarchy of the various target computers. Furthermore, this blocked version of Level-3 BLAS is naturally parallel. We present results on the ALLIANT FX/80, the CONVEX C220, the CRAY-2, and the IBM 3090/VF. For GEMM, we always use the manufacturer-supplied versions. For the operations dealing with triangular blocks, we use assembler or tuned Fortran (using loop-unrolling) codes, depending on the efficiency of the available libraries.