Regular Parallelism Research Articles

Algorithm/Hardware Co-Optimization for Sparsity-Aware SpMM Acceleration of GNNs

In recent years, graph neural networks (GNNs) have achieved impressive performance in various application fields by extracting information from graph-structured data. It contains extensive feature aggregation operations and has become a performance bottleneck, which can be abstracted as a specialized Sparse-Dense Matrix Multiplication (SpMM) operation. Previous works have leveraged the inner product or outer product to accelerate the feature aggregation process. However, inefficient execution leads to extremely unbalanced workloads and extensive intermediate data, hampering the performance of previous processors. So in this paper, we demonstrate an algorithm/hardware co-optimization chance to enhance SpMM acceleration for GNNs. First, the algorithm part develops a dataflow-efficient SpMM algorithm that integrates three optimization methods to mitigate computation and memory access inefficiencies. Specifically, 1) the proposed equal-value partition method achieves fine-grained data partition and enables load balancing during data movement. 2) After observing the vertex aggregation phenomenon, a vertex-clustering optimization method is presented to enable significant data locality. 3) The adaptive dataflow based on Gustavson’s algorithm is further implemented to enable the efficient distribution of sparse elements and improves computing resource utilization. Then, the hardware part features the proposed SpMM algorithm and customizes SDMA, a flexible and efficient accelerator to boost SpMM acceleration, which follows the adaptive dataflow to eliminate sparsity and explore the regular parallelism dimension. Finally, we prototype SDMA on the Xilinx Alveo U280 FPGA accelerator card. The results demonstrate that SDMA achieves 5.68× 14.68× energy efficiency over the previous GPU implementations deployed on the Nvidia GTX 1080Ti and 1.32× higher throughput over the state-of-the-art FPGA prototype.

Multiple metrics assessment method for a reliable evaluation of corneal suturing skills

This study aimed to evaluate the efficiency of a multiple metrics assessment method to differentiate between surgeons of differing experience while performing a corneal suturing task. Volunteer ophthalmologists were assigned to three groups (senior [SG], junior [JG] and novice [NG]) according to their experience in corneal suturing. All participants performed three sessions of corneal wound closure by three stitches. Suturing and participant posture were recorded with cameras, and assessed by two blind assessors for stitch quality (using Zhang score) and ergonomics (using Rapid Upper Limb Assessment [RULA] score). Task duration was recorded. Objective analyses of stitches geometry and instrument position were carried out. We included 24 participants: 5 in the SG, 8 in the JG and 11 in the NG. Stitch quality was significantly better and time to perform the procedure significantly lower in more experienced groups (p < 0.001 and p = 0.002, respectively). SG participants better respected regular distance and parallelism between stitches compared to others (p = 0.01). Instrument position was similar between groups, although SG participants minimized their back-and-forth movements compared to NG participants. Ergonomics assessment was similar. Multiple metrics assessment efficiently determined how to differentiate between novices and experienced surgeons on corneal suturing skills, providing hints for future training studies.

Regular parallelisms on $\mathrm{PG}(3,\mathbb R)$ admitting a 2-torus action

A regular parallelism of real projective 3-space $\mathrm{PG}(3,\mathbb R)$ is an equivalence relation on the line space such that every class is equivalent to the set of 1-dimensional complex subspaces of $\mathbb C^2 = \mathbb R^4$. We shall assume that the set of classes is compact, and characterize those regular parallelisms that admit an action of a 2-dimensional torus group. We prove that there is a one-dimensional subtorus fixing every parallel class. From this property alone we deduce that the parallelism is a 2- or 3-dimensional regular parallelism in the sense of Betten and Riesinger [6]. If a 2-torus acts, then the parallelism can be described using a so-called generalized line star or \it gl star \rm which admits a 1-torus action. We also study examples of such parallelisms by constructing gl stars. In particular, we prove a claim which was presented in [6] with an incorrect proof. The present article continues a series of papers by the first author on parallelisms with large groups.

GPU-accelerated sparse matrices parallel inversion algorithm for large-scale power systems

State-of-the-art Graphics Processing Unit (GPU) has superior performances on float-pointing calculation and memory bandwidth, and therefore has great potential in many computationally intensive power system applications, one of which is the inversion of large-scale sparse matrix. It is a fundamental component for many power system analyses which requires to solve massive number of forward and backward substitution (F&B) subtasks and seems to be a good GPU-accelerated candidate application. By means of solving multiple F&B subtasks concurrently and a serial of performance tunings in compliance with GPU’s architectures, we successfully develop a batch F&B algorithm on GPUs, which not only extracts the intra-level and intra-level parallelisms inside single F&B subtask but also explores a more regular parallelism among massive F&B subtasks, called inter-task parallelism. Case study on a 9241-dimension case shows that the proposed batch F&B solver consumes 2.92 μs per forward substitution (FS) subtask when the batch size is equal to 3072, achieving 65 times speedup relative to KLU library. And on the basis the complete design process of GPU-based inversion algorithm is proposed. By offloading the tremendous computational burden to GPU, the inversion of 9241-dimension case consumes only 97 ms, which can achieve 8.1 times speedup relative to the 12-core CPU inversion solver based on KLU library. The proposed batch F&B solver is practically very promising in many other power system applications requiring solving massive F&B subtasks, such as probabilistic power flow analysis.

Regular parallelisms from generalized line stars in $${P_{3} \mathbb{R}}$$ P 3 R : a direct proof

A generalized line star with respect to an elliptic quadric contained in the Klein quadric gives rise to a regular parallelism in real projective 3-space. This was shown by Betten and Riesinger (Results Math 47:226–241, 2005) using the Thas–Walker construction. They remark that the resulting description of the parallelism is equivalent to a much simpler one, but again the proof is hard. In practice, they always work with that simpler construction. We show that the Thas–Walker approach is not needed here. In fact, one can derive all relevant properties of the parallelism from the simple construction. We also give coordinate-free, short proofs of the topological properties of regular parallelisms in general and of those obtained from generalized line stars in particular.

Heterogeneous-race-free memory models

Commodity heterogeneous systems (e.g., integrated CPUs and GPUs), now support a unified, shared memory address space for all components. Because the latency of global communication in a heterogeneous system can be prohibi-tively high, heterogeneous systems (unlike homogeneous CPU systems) provide synchronization mechanisms that only guarantee ordering among a subset of threads, which we call a scope. Unfortunately, the consequences and se-mantics of these scoped operations are not yet well under-stood. Without a formal and approachable model to reason about the behavior of these operations, we risk an array of portability and performance issues. In this paper, we embrace scoped synchronization with a new class of memory consistency models that add scoped synchronization to data-race-free models like those of C++ and Java. Called sequential consistency for heterogeneous-race-free (SC for HRF), the new models guarantee SC for programs with "sufficient" synchronization (no data races) of "sufficient" scope. We discuss two such models. The first, HRF-direct, works well for programs with highly regular parallelism. The second, HRF-indirect, builds on HRF-direct by allowing synchronization using different scopes in some cases involving transitive communication. We quanti-tatively show that HRF-indirect encourages forward-looking programs with irregular parallelism by showing up to a 10% performance increase in a task runtime for GPUs.

Heterogeneous-race-free memory models

Commodity heterogeneous systems (e.g., integrated CPUs and GPUs), now support a unified, shared memory address space for all components. Because the latency of global communication in a heterogeneous system can be prohibi-tively high, heterogeneous systems (unlike homogeneous CPU systems) provide synchronization mechanisms that only guarantee ordering among a subset of threads, which we call a scope. Unfortunately, the consequences and se-mantics of these scoped operations are not yet well under-stood. Without a formal and approachable model to reason about the behavior of these operations, we risk an array of portability and performance issues. In this paper, we embrace scoped synchronization with a new class of memory consistency models that add scoped synchronization to data-race-free models like those of C++ and Java. Called sequential consistency for heterogeneous-race-free (SC for HRF), the new models guarantee SC for programs with "sufficient" synchronization (no data races) of "sufficient" scope. We discuss two such models. The first, HRF-direct, works well for programs with highly regular parallelism. The second, HRF-indirect, builds on HRF-direct by allowing synchronization using different scopes in some cases involving transitive communication. We quanti-tatively show that HRF-indirect encourages forward-looking programs with irregular parallelism by showing up to a 10% performance increase in a task runtime for GPUs.

Clifford parallelism: old and new definitions, and their use

Parallelity in the real elliptic 3-space was defined by W. K. Clifford in 1873 and by F. Klein in 1890; we compare the two concepts. A Clifford parallelism consists of all regular spreads of the real projective 3-space $${{\rm PG}(3,\mathbb{R})}$$ whose (complex) focal lines (=directrices) form a regulus contained in an imaginary quadric (D1 = Klein’s definition). Our new access to the topic ‘Clifford parallelism’ is free of complexification and involves Klein’s correspondence λ of line geometry together with a bijective map γ from all regular spreads of $${{\rm PG}(3,\mathbb{R})}$$ onto those lines of $${{\rm PG}(5,\mathbb{R})}$$ having no common point with the Klein quadric; a regular parallelism P of $${{\rm PG}(3,\mathbb{R})}$$ is Clifford, if the spreads of P are mapped by γ onto a plane of lines (D2 = planarity definition). We prove the equivalence of (D1) and (D2). Associated with γ is a simple dimension concept for regular parallelisms which allows us to say instead of (D2): the 2-dimensional regular parallelisms of $${{\rm PG}(3,\mathbb{R})}$$ are Clifford (D3 = dimensionality definition). Submission of (D2) to λ−1 yields a complexification free definition of a Clifford parallelism which uses only elements of $${{\rm PG}(3,\mathbb{R})}$$ : A regular parallelism P is Clifford, if the union of any two distinct spreads of P is contained in a general linear complex of lines (D4 = line geometric definition). In order to see (D1) and (D2) simultaneously at work we discuss the following two examples using, at the one hand, complexification and (D1) and, at the other hand, (D2) under avoidance of complexification. Example 1. In the projectively extended real Euclidean 3-space a rotational regular spread with center o is submitted to the group of all rotations about o; we prove, that a Clifford parallelism is generated. Example 2. We determine the group $${Aut_e({\bf P}_{\bf C})}$$ of all automorphic collineations and dualities of the Clifford parallelism P C and show $${Aut_e({\bf P}_{\bf C})\hspace{1.5mm} \cong ({\rm SO}_3\mathbb{R} \times {\rm SO}_3\mathbb{R})\rtimes \mathbb{Z}_2}$$ .

A Model-Driven Design Framework for Massively Parallel Embedded Systems

Modern embedded systems integrate more and more complex functionalities. At the same time, the semiconductor technology advances enable to increase the amount of hardware resources on a chip for the execution. Massively parallel embedded systems specifically deal with the optimized usage of such hardware resources to efficiently execute their functionalities. The design of these systems mainly relies on the following challenging issues: first, how to deal with the parallelism in order to increase the performance; second, how to abstract their implementation details in order to manage their complexity; third, how to refine these abstract representations in order to produce efficient implementations. This article presents the Gaspard design framework for massively parallel embedded systems as a solution to the preceding issues. Gaspard uses the repetitive Model of Computation (MoC), which offers a powerful expression of the regular parallelism available in both system functionality and architecture. Embedded systems are designed at a high abstraction level with the MARTE (Modeling and Analysis of Real-time and Embedded systems) standard profile, in which our repetitive MoC is described by the so-called Repetitive Structure Modeling (RSM) package. Based on the Model-Driven Engineering (MDE) paradigm, MARTE models are refined towards lower abstraction levels, which make possible the design space exploration. By combining all these capabilities, Gaspard allows the designers to automatically generate code for formal verification, simulation and hardware synthesis from high-level specifications of high-performance embedded systems. Its effectiveness is demonstrated with the design of an embedded system for a multimedia application.

Accelerating Physical Simulations Using Graphics Processing Units

Abstract Graphics processors are used in many fields of applications that require high computational power. Especially in scientific computing, the programming of graphics processing units is an active field of research. Because of their hardware characteristics, graphics processors are well-suited for regular parallelism, however the implementation of irregular problems requires more advanced strategies. In this article, the hardware architecture of graphics processors and different frameworks for graphics processor programming, such as CAL, Brook+, CUDA and OpenCL with their specific properties, are presented. Additionally, an overview of different physical applications that have been implemented successfully on graphics processors is given. The parallel implementation of a specific irregular physical application on graphics processors is presented in more detail. This application simulates anomalous diffusion in porous media using random walk on Random Sierpinski Carpets.

Programming the memory hierarchy revisited

We describe two novel constructs for programming parallel machines with multi-level memory hierarchies: call-up, which allows a child task to invoke computation on its parent, and spawn, which spawns a dynamically determined number of parallel children until some termination condition in the parent is met. Together we show that these constructs allow applications with irregular parallelism to be programmed in a straightforward manner, and furthermore these constructs complement and can be combined with constructs for expressing regular parallelism. We have implemented spawn and call-up in Sequoia and we present an experimental evaluation on a number of irregular applications.

Automata and processes on multisets of communicating objects

Inspired by P systems initiated by Gheorghe Paun, we study a computation model over a multiset of communicating objects. The objects in our model are instances of finite automata. They interact with each other by firing external transitions between two objects. Our model, called a service automaton, is intended to specify, at a high level, a service provided on top of network devices abstracted as communicating objects. We formalize the concept of processes, running over a multiset of objects, of a service automaton and study the computing power of both single-process and multiprocess service automata. In particular, in the multiprocess case, regular maximal parallelism is defined for inter-process synchronization. It turns out that single-process service automata are equivalent to vector addition systems and hence can define nonregular processes. Among other results, we also show that Presburger reachability problem for single-process service automata is decidable, while it becomes undecidable in the multiprocess case. Hence, multiprocess service automata can not be effectively simulated by vector addition systems.

Regular parallelisms in kinematic spaces

Here we propose a definition of regular parallelism in a linear space not necessarily embedded onto a projective space and we investigate its properties in the particular case of kinematic spaces. We prove that the kinematic parallelisms are always regular in that sense and we deduce some results on the group of translations acting transitively on the pointset.

Hyperflock determining line sets and totally regular parallelisms of PG $${(3,\,{\mathbb{R}})}$$

By a totally regular parallelism of the real projective 3-space \({\Pi_3:={{\rm PG}}(3, \mathbb {R})}\) we mean a family T of regular spreads such that each line of Π3 is contained in exactly one spread of T. For the investigation of totally regular parallelisms the authors mainly employ Klein’s correspondence λ of line geometry and the polarity π5 associated with the Klein quadric H5 (for details see Chaps. 1 and 3). The λ-image of a totally regular parallelism T is a hyperflock of H5, i.e., a family H of elliptic subquadrics of H5 such that each point of H5 is on exactly one subquadric of H. Moreover, \({\{\pi_5({{\rm span}} \,\lambda(\mathcal {X}))\vert\mathcal {X}\in\bf{T}\}=:\mathcal {H}_{\bf{T}}}\) is a hyperflock determining line set, i.e., a set \({\mathcal {Z}}\) of 0-secants of H5 such that each tangential hyperplane of H5 contains exactly one line of \({\mathcal {Z}}\) . We say that \({{{\rm dim}}({{\rm span}}\,\mathcal {H}_{\bf{T}})=:d_{\bf{T}}}\) is the dimension of T and that T is a dT- parallelism. Clifford parallelisms and 2-parallelisms coincide. The examples of non-Clifford parallelisms exhibited in Betten and Riesinger [Result Math 47:226–241, 2004; Adv Geom 8:11–32, 2008; J Geom (to appear)] are totally regular and of dimension 3. If \({\mathcal{G}}\) is a hyperflock determining line set, then \({\{\lambda^{-1}\,{\rm (}\pi_5(X){\,\cap H_5)\,|\, X\in\mathcal{G}\}}}\) is a totally regular parallelism. In the present paper the authors construct examples of topological (see Definition 1.1) 4- and 5-parallelisms via hyperflock determining line sets.

Topological parallelisms of the real projective 3-space

A parallelism of a projective 3-space Π is a family P of spreads such that each line of Π is contained in exactly one spread of P. A parallelism is said to be totally regular, if all its members are regular spreads. By a generalized line star with respect to an elliptic quadric Q of a classical projective 3-space we understand a set \(\cal A\) of 2-secants of Q such that each non-interior point of Q is incident with exactly one line of \(\cal A\). From each generalized line star we can construct a totally regular parallelism which we do in essential by the Thas-Walker construction. A parallelisms of the real projective 3-space PG(3, ℝ) is called topological, if the operation of drawing a line parallel to a given line through a given point is continuous. Clifford parallelisms are topological. Using generalized line stars we exhibit examples of non-Clifford topological parallelisms and of non-topological parallelisms.

GigaOp DSP on FPGA

DSP algorithms such as automated target recognition and SONAR beamforming are a good match for FPGA technology due to their regular structure, available parallelism, pipeline-ability, and modest data word sizes. FPGA implementations of these applications outperformed their DSP and microprocessor counterparts by factors ranging from 10× on up with an equivalent sustained computational rate of more than 2 GOps/second per FPGA. This paper introduces a set of criteria which have a great impact on how well an application maps to FPGA technology. It then describes two applications in detail and the process of mapping each to FPGA technology. Comparisons with software implementations are made and followed by conclusions and future challenges.

The Cyclic Parallelisms of PG(3,5)

We determine, by a computer search, all the cyclic parallelisms of PG(3,5). There are 45 of them, up to projective equivalence. In particular, there are two cyclic regular parallelisms of PG(3,5). Previously, no example was known of a regular parallelism in PG(3,q), forqodd.

An Implementation of Multiple and Multivariate Fourier Transforms on Vector Processors

Several algorithms for multiple and multivariate complex Fourier transforms are suggested. They use little or no scratch space, i.e., they are essentially in-place. Furthermore, they display high levels of regular parallelism needed for vector-parallel computers. Data access is mainly by stride 1. These algorithms correspond to matrix factorizations using transposition permutations. An essential tool used in defining the permutations and the factorizations are Kronecker products. The suggested algorithms are combined with a method for one-dimensional fast Fourier transforms which is also in-place and accesses data with stride 1. Although the number of floating point operations and the total degree of parallelism is left unchanged by the methods used here, data access is much more regular. This is reflected in substantially lower values of a new indicator called loop overhead indicator and often by execution times approximately 2 to 5 times lower, even for moderate problem sizes on the Fujitsu VP 2200, compared to the implementation of the fundamental tensor product factorization.

Regular parallelisms from translation planes

Let π be a translation plane of order q 4 admitting a collineation group G isomorphic to SL(2, q), p r = q, in the translation complement. It is shown that if the p-elements of G are elations and L is an elation axis then the kernel of π is GF( q) and G acts 1 2 - transitive on l ∞− N∩l ∞ (where N denotes the elation net) if and only if there is a regular parallelism of L (considering L as PG(3, q)). With no assumption on the p-elements, it is shown that the existence SL(2, q)× z 1+ q+ q 2 as a collineation group implies that the p-elements are elations, the kernel is GF( q) and any elation axis L admits a regular parallelism as PG(3, q).

Soldadura por uma das extremidades de dois cromossômios homólogos do tityus

In this paper the author describes a very interesting case of union of two homologous chromosomes of the scorpion Tityus bahiensis just by the opposite extremities. The two normal pairs of chromosomes behave as ordinarily, the members of each pair showing at times a slight disturbance in their regular parallelism. The complex chromosome, on the contrary, behaves itself as if it were devoid of kinetochores, that is, it does not orient like normal chromosomes nor reveal any kind of active movement. The fusion of the chromosomes has resulted from terminal breakage at the opposite ends, the correspondig fragments having been found unpaired in a cell in which two pairs of chromosomes were present. Consequently, the compound chromosome, like the normal ones, is provided with a kinetochore at each one of the free ends. Being thus a centric chromosome its behavior, or more exactly, its kinetic inactivity may be compared with that of the monovalents found elsewhere in meioses. It is due o the failure of a partner. The fusion of two homologous chromosomes has transformed them into a new chromosomal unit in whose corresponding parts the ability of pairing was entirely abolished. This result is in full contradiction with the theory of a point-to point attraction between homologous chromosomes attributed to particular power of the genes, since, if genes really exist, being placed in their original loci, they would promote the union side by side of the members of the compound chromosome. If an attraction loci-to-loci should prevail the compound chromosome would be bent as in Fig. 8, C or form a ring similar to the loops observed in the inverted segment of sailvary chromosomes of Drosophila, as represented in the Fig. 8, D and this, in accordance with the order of the loci resulting from an union of corresponding or opposite ends of the fused chromosomes, as indicated in the Fig, 8 A and B. The evidence in hand points to a fusion by non homologous extremities. The expected rings, however, have never been found in metaphase plates. From this fact the author concludes that there is no point-to-point attraction between chromosomes, a conclusion in full agreement with the behavior of Hemipteran chromosomes which, in spite of geing composed of two equivalent halves do not bend in order to adjust the corresponding loci. (Cf. the papers on Hemiptera published by the author in this volume).