Linear Algebra Programming Research Articles

Design and Implementation for Nonblocking Execution in GraphBLAS: Tradeoffs and Performance

GraphBLASis a recent standard that allows the expression of graph algorithms in the language of linear algebra and enables automatic code parallelization and optimization. GraphBLAS operations are memory bound and may benefit from data locality optimizations enabled by nonblocking execution. However, nonblocking execution remains under-evaluated. In this article, we present a novel design and implementation that investigates nonblocking execution in GraphBLAS. Lazy evaluation enables runtime optimizations that improve data locality, and dynamic data dependence analysis identifies operations that may reuse data in cache. The nonblocking execution of an arbitrary number of operations results in dynamic parallelism, and the performance of the nonblocking execution depends on two parameters, which are automatically determined, at run-time, based on a proposed analytic model. The evaluation confirms the importance of nonblocking execution for various matrices of three algorithms, by showing up to 4.11× speedup over blocking execution as a result of better cache utilization. The proposed analytic model makes the nonblocking execution reach up to 5.13× speedup over the blocking execution. The fully automatic performance is very close to that obtained by using the best manual configuration for both small and large matrices. Finally, the evaluation includes a comparison with other state-of-the-art frameworks for numerical linear algebra programming that employ parallel execution and similar optimizations to those discussed in this work, and the presented nonblocking execution reaches up to 16.1× speedup over the state of the art.

Synthesis of Incremental Linear Algebra Programs

This article targets the Incremental View Maintenance (IVM) of sophisticated analytics (such as statistical models, machine learning programs, and graph algorithms) expressed as linear algebra programs. We present LAGO, a unified framework for linear algebra that automatically synthesizes efficient incremental trigger programs, thereby freeing the user from error-prone manual derivations, performance tuning, and low-level implementation details. The key technique underlying our framework is abstract interpretation, which is used to infer various properties of analytical programs. These properties give the reasoning power required for the automatic synthesis of efficient incremental triggers. We evaluate the effectiveness of our framework on a wide range of applications from regression models to graph computations.

Tuning a general purpose software cache library for TaihuLight’s SW26010 processor

The Sunway TaihuLight supercomputer has been installed for several years and many applications have been ported or built for TaihuLight. Initially most applications running on TaihuLight are with regular memory access patterns, such as dense linear algebra, structured grids and dynamic programming. At the year of 2018, developers have published a general purpose graph processing framework, a ported version of LAMMPS and a sparse triangular solver. These applications are with irregular memory access patterns which need a lot of special processings to make use of the computing processing elements (CPEs) of TaihuLight. While those strategies are efficient, doing such processing may be difficult for wider range of applications, especially for the constantly changing molecular dynamics applications or dynamic unstructured grids. In this paper, we present our work of designing a general purpose software cache library, SWCache, for simplifying the work of applying software cache in kernels, as well as a series of tools for tuning and modelling the performance of our software cache. After a series of optimizations including reordering branches for better branch prediction, hand-tuning register allocation, we evaluate our implementation in two mini-apps: miniFE and miniMD. Experiments show that our tuned software cache library can be applied in these applications, and can provide 20% speedup in miniMD compared to the strategies in a previous port of LAMMPS. Also, the workload of writing code can be reduced by using our library. Besides, the experience of efficient macro-based programming should be valuable for further application development on CPEs which are lack of C++ support.

On the Existence of Block-Diagonal Solutions to Lyapunov and ${\mathcal {H}_\infty }$ Riccati Inequalities

In this paper, we describe sufficient conditions when block-diagonal solutions to Lyapunov and $\mathcal{H}_{\infty}$ Riccati inequalities exist. In order to derive our results, we define a new type of comparison systems, which are positive and are computed using the state-space matrices of the original (possibly nonpositive) systems. Computing the comparison system involves only the calculation of $\mathcal{H}_{\infty}$ norms of its subsystems. We show that the stability of this comparison system implies the existence of block-diagonal solutions to Lyapunov and Riccati inequalities. Furthermore, our proof is constructive and the overall framework allows the computation of block-diagonal solutions to these matrix inequalities with linear algebra and linear programming. Numerical examples illustrate our theoretical results.

Theorems of the alternative revisited

Theorems of the alternative for linear algebraic equations and inequalities are considered in this paper. Classical theorems of the alternative, such as Gordan’s theorem, Farkas’ theorem, Stiemke’s theorem, Motzkin’s theorem, Slater’s theorem, Tucker’s theorem, Duffin’s theorem, Gale’s theorems, etc. are recalled. Theorems of this form are important for both linear algebra and mathematical programming, especially for mathematical programming problems with linear equality and/or inequality constraints. Some new theorems of the alternative are formulated and proved.

Model Study and Analysis Based on Student Performance Data under SPSS Statistics— Take a College from Hangzhou Normal University as an Example

With reasonable assumption, this paper established four mathematic models by SPSS Statistics software. According to these models, the paper analyzed the correlativity of three courses, advanced mathematics A2, linear algebra A3 and program development foundation (hereinafter referred to briefly as AM, LA and PDF) from two point of view in the course and the class respectively, and came up with the perspective of teaching science and engineering course in college.

PolyJIT: Polyhedral Optimization Just in Time

While polyhedral optimization appeared in mainstream compilers during the past decade, its profitability in scenarios outside its classic domain of linear-algebra programs has remained in question. Recent implementations, such as the LLVM plugin Polly, produce promising speedups, but the restriction to affine loop programs with control flow known at compile time continues to be a limiting factor. PolyJIT combines polyhedral optimization with multi-versioning at run time, at which one has access to knowledge enabling polyhedral optimization, which is not available at compile time. By means of a fully-fledged implementation of a light-weight just-in-time compiler and a series of experiments on a selection of real-world and benchmark programs, we demonstrate that the consideration of run-time knowledge helps in tackling compile-time violations of affinity and, consequently, offers new opportunities of optimization at run time.

On optimizing operator fusion plans for large-scale machine learning in systemML

Many machine learning (ML) systems allow the specification of ML algorithms by means of linear algebra programs, and automatically generate efficient execution plans. The opportunities for fused operators---in terms of fused chains of basic operators---are ubiquitous, and include fewer materialized intermediates, fewer scans of inputs, and sparsity exploitation across operators. However, existing fusion heuristics struggle to find good plans for complex operator DAGs or hybrid plans of local and distributed operations. In this paper, we introduce an exact yet practical cost-based optimization framework for fusion plans and describe its end-to-end integration into Apache SystemML. We present techniques for candidate exploration and selection of fusion plans, as well as code generation of local and distributed operations over dense, sparse, and compressed data. Our experiments in SystemML show end-to-end performance improvements of up to 22x, with negligible compilation overhead.

Varimax 회전 및 그 이후: R 및 Q 분석을 위한 선형 대수, 시각화 및 Python 프로그래밍을 사용한 PCA에 대한 사용지침서

Varimax 회전 및 그 이후: R 및 Q 분석을 위한 선형 대수, 시각화 및 Python 프로그래밍을 사용한 PCA에 대한 사용지침서

A structure-preserving pivotal method for affine variational inequalities

Affine variational inequalities (AVI) are an important problem class that subsumes systems of linear equations, linear complementarity problems and optimality conditions for quadratic programs. This paper describes PathAVI, a structure-preserving pivotal approach, that can efficiently process (solve or determine infeasible) large-scale sparse instances of the problem with theoretical guarantees and at high accuracy. PathAVI implements a strategy known to process models with good theoretical properties without reducing the problem to specialized forms, since such reductions may destroy sparsity in the models and can lead to very long computational times. We demonstrate formally that PathAVI implicitly follows the theoretically sound iteration paths, and can be implemented in a large scale setting using existing sparse linear algebra and linear programming techniques without employing a reduction. We also extend the class of problems that PathAVI can process. The paper illustrates the effectiveness of our approach by comparison to the Path solver used on a complementarity reformulation of the AVI in the context of applications in friction contact and Nash Equilibria. PathAVI is a general purpose solver, and freely available under the same conditions as Path.

Graph Programming Interface (GPI): A Linear Algebra Programming Model for Large Scale Graph Computations

Graph processing is becoming a crucial component for analyzing big data arising in many application domains such as social and biological networks, fraud detection, and sentiment analysis. As a result, a number of computational models for graph analytics have been proposed in the literature to help users write efficient large scale graph algorithms. In this paper we present an alternative model for implementing graph algorithms using a linear algebra based specification. We first specify a set of linear algebra primitives that allows users to express graph algorithms by composition of linear algebra operations. We then describe a high performance implementation of these primitives using C $$++$$ and subsequently its integration with the Spark framework to achieve the scalability we need for large systems. We provide an overview of our implementation and also compare and contrast the expressiveness and performance of various algorithms implemented with our approach with that of the current Spark GraphX implementation of those algorithms.

Matching Bills of Materials Using Tree Reconciliation

A product Bill of Materials (BOM) is a structured tree which represents its components and their hierarchal relationships. The BOMs are traditionally used for Material Requirement Planning (MRP). However, they do have other useful applications in product modeling and variety management. Recent research used graph difference operations, linear algebra and integer programming to match trees of BOM and find pairwise similarity measures for applications such as clustering product variants into families and retrieval of design and manufacturing data. Matching phylogenetic trees has been utilized in biological science for decades and is referred to as “tree reconciliation”. A new application of this approach in manufacturing to match pairs of BOM trees and retrieve the most similar design is presented. This novel method can help speeding-up other downstream planning activities such as process planning, hence, improving productivity and shortening time to market. Assembly of chemical processing centrifugal pumps is used as a case study for demonstration. This novel matching of Bills of Materials uses linear time algorithms, compared to state-of- the-art algorithms which use integer programming and matrix approximation, hence, leading to more computational efficiency.

Asset Pricing, Financial Markets, and Linear Algebra

SummaryConcepts from asset pricing and financial markets theory are used to illustrate concepts of linear algebra and linear programming.

A LINEAR APPROACH TO LIE TRIPLE AUTOMORPHISMS OF H*-ALGEBRAS

By developing a linear algebra program involving many dif- ferent structures associated to a three-graded H � -algebra, it is shown that if L is a Lie triple automorphism of an innite-dimension al topologically simple associative H � -algebra A, then L is either an automorphism, an anti-automorphism, the negative of an automorphism or the negative of an anti-automorphism. If A is nite-dimensional, then there exists an automorphism, an anti-automorphism, the negative of an automorphism or the negative of an anti-automorphism F : A ! A such that := F L is a linear map from A onto its center sending commutators to zero. We also describe L in the case of having A zero annihilator.

Problem Based Learning (PBL): Analysis of Continuous Stirred Tank Chemical Reactors with a Process Control Approach

This work is focused on a project that integrates the curriculum such as thermodynamic, chemical reactor engineering, linear algebra, differential equations and computer programming. The purpose is that students implement the most knowledge and tools to analyse the stirred tank chemical reactor as a simple dynamic system. When the students finished this practice they should have learned about analysis of dynamic system through bifurcation analysis, hysteresis phenomena, find equilibrium points, stability type, and phase portrait. Once the steps were accomplished, we concluded that the purpose was satisfactorily reached with an increment in creative ability. The student showed a bigger interesting in this practice, since they worked in group. The most important fact is that the percentage of failure among students was 10%. Finally, using alternative teaching-learning process improves the Mexican system education.

Generalized switch-setting problems

Switch-setting games like Lights Out are typically modelled as a graph, where the vertices represent switches and lamps, and the edges capture the switching rules. We generalize the concepts used for a mathematical description of Lights Out and its relatives. Our approach uses bipartite graphs and allows for the analysis of a broader class of switch-setting games, which is demonstrated at a new variant of the Lights Out puzzle. Our method exhibits full duality between switches and lamps, and we get rid of some insufficiencies inherent in the modelling with non-bipartite graphs. We present a detailed analysis of the new Lights Out variant, formulate solvability conditions, give graphically aesthetic interpretations and discuss aspects on minimum solutions. In our study of parity domination in bipartite graphs we incorporate methods from linear algebra and linear programming. We point out the close relations between graph theoretic terms and the language of algebra over Z 2 .

Computing photonic band structures by Dirichlet-to-Neumann maps: The triangular lattice

An efficient semi-analytic method is developed for computing the band structures of two-dimensional photonic crystals which are triangular lattices of circular cylinders. The problem is formulated as an eigenvalue problem for a given frequency using the Dirichlet-to-Neumann (DtN) map of a hexagon unit cell. This is a linear eigenvalue problem even if the material is dispersive, where the eigenvalue depends on the Bloch wave vector. The DtN map is constructed from a cylindrical wave expansion, without using sophisticated lattice sums techniques. The eigenvalue problem can be efficiently solved by standard linear algebra programs, since it involves only matrices of relatively small size.

A cost-effective implementation of multilevel tiling

This paper presents a new cost-effective algorithm to compute exact loop bounds when multilevel tiling is applied to a loop nest having affine functions as bounds (nonrectangular loop nest). Traditionally, exact loop bounds computation has not been performed because its complexity is doubly exponential on the number of loops in the multilevel tiled code and, therefore, for certain classes of loops (i.e., nonrectangular loop nests), can be extremely time consuming. Although computation of exact loop bounds is not very important when tiling only for cache levels, it is critical when tiling includes the register level. This paper presents an efficient implementation of multilevel tiling that computes exact loop bounds and has a much lower complexity than conventional techniques. To achieve this lower complexity, our technique deals simultaneously with all levels to be tiled, rather than applying tiling level by level as is usually done. For loop nests having very simple affine functions as bounds, results show that our method is between 15 and 28 times faster than conventional techniques. For loop nests caving not so simple bounds, we have measured speedups as high as 2,300. Additionally, our technique allows eliminating redundant bounds efficiently. Results show that eliminating redundant bounds in our method is between 22 and 11 times faster than in conventional techniques for typical linear algebra programs.

Introducing parallel manipulators through laboratory experiments

Describes how simple, inexpensive prototypes of parallel manipulators can be employed as an efficient educational tool to teach undergraduate students not only the basics of parallel manipulators but also reinforce material learned in linear algebra, kinematics, statics, and computer programming.

Learning missing values from summary constraints

Real-world data sets often contain errors and inconsistency. Even though this is a very important problem it has received relatively little attention in the research community. In this paper we examine the problem of learning missing values when some summary information is available. We use linear algebra and constraint programming techniques to learn the missing values using apriori-known summary information and that derived from the raw data. We reconstruct the missing values by different methods in three scenarios: ideal-constrained, under-constrained, and over-constrained. Furthermore, for a range query involving missing values, we also give the lower bound and upper bound for the values using constraint programming techniques. We believe that theory of linear algebra and constraint programming constitutes a sound basis for learning missing values when summary information is available.