Distributed Memory Multiprocessors Research Articles

The t/k-diagnosability of m-ary n-cube networks

A crucial index used to evaluate the reliability of a multiprocessor system is diagnosability. The t/k-diagnosis strategy greatly improves the diagnosability of a system by allowing up to k fault-free processors to be misdiagnosed as faulty processors. Compared with the t-diagnosability and t1/t1-diagnosability, the t/k-diagnosability can better reflect the fault modes of a practical system. The m-ary n-cube Qnm (in order not to be confused with the “k” of the t/k-diagnosability, we replace k-ary n-cube Qnk with m-ary n-cube Qnm) is a significant and common network topology, which is used as the underlying network in the construction of many distributed memory multiprocessors. In this paper, we explore several useful lemmas and the t/k-diagnosability of Qnm. According to the sufficient condition for determining that a system is t/k-diagnosable, we show that for n≥2, m≥4, and 0≤k≤2n, Qnm is tk,n/k-diagnosable, where tk,n=2(k+1)n−12(k+1)(k+2)+1. Based on this, we provide a comparison among the t-diagnosability, the t1/t1-diagnosability, and the t/k-diagnosability of Qnm.

A Linguistic Study of Speech Comprehension and Production in Aphasic Speakers

Distributed memory is a term used in computer science to describe a multiprocessor computer system in which each processor has its own private memory. Computational jobs can only work with local data, so if you need remote data, you'll have to communicate with one or more remote processors. Parallel and distributed computing are frequently used together. Distributed parallel computing employs many computing devices to process tasks in parallel, whereas parallel computing on a single computer uses multiple processors to execute tasks in parallel. Distributed systems are designed separately from the core network. There are different kinds of distributed systems such as peer-to-peer (P2P) networks, groups, grids, distributed storage systems. The multicore processor can be classified into two types: homogeneous and heterogeneous. This paper reviews the impact of the distributed-memory parallel processing approach on performance-enhancing of multicomputer-multicore systems. Also, number of methods have been introduced which used in distributed-memory systems and discuss which method is the best and enhance multicore performance in distributed systems. The best methods were those which used an operating system named gun/Linux 4.8.0-36, intel Xeon 2.5, python programming language.

3D nonlinear conjugate gradient inversion for frequency-domain airborne EM based on vector finite element method

The 3D inversion is a challenging task in the research of the frequency-domain airborne electromagnetic (AEM) method due to the massive amounts of data and limited computational efficiency. In this paper, we introduce the 3D FAEM inversion based on the vector finite element method (FEM). Compare with the traditional nodal FEM, the vector FEM can improve the computational efficiency by reducing the number of variables and the scale of linear equations. To tackle the computational demands, in the calculation process we apply the PARDISO solver with shared-memory distributed-memory multi-processors to solve the multi-source forward linear equations and calculate the gradient of the objective function by adjoint equations. Then typical resistivity model with the anomalous body is carried out to verify the validity of the proposed method. Meanwhile, in allusion to the multi-solution problem caused by the inversion problem, we study the influence of typical initial models on inversion results. Finally, we applied our algorithm to invert field data. The inversion results demonstrate that the proposed method can obtain reliable AEM resistivity inverted results.

Evolving PDC curriculum and tools: A study in responding to technological change

Much has changed about parallel and distributed computing (PDC) since the author began teaching the topic in the late 1990s. This paper reviews some of the key changes to the field and describes their impacts on his work as a PDC educator. Such changes include: the availability of free implementations of the message passing interface (MPI) for distributed-memory multiprocessors; the development of the Beowulf cluster; the advent of multicore architectures; the development of free multithreading languages and libraries such as OpenMP; the availability of (relatively) inexpensive manycore accelerator devices (e.g., GPUs); the availability of free software platforms like CUDA, OpenACC, OpenCL, and OpenMP for using accelerators; the development of inexpensive single board computers (SBCs) like the Raspberry Pi, and other changes. The paper details the evolution of PDC education at the author's institution in response to these changes, including curriculum changes, seven different Beowulf cluster designs, and the development of pedagogical tools and techniques specifically for PDC education. The paper also surveys many of the hardware and software infrastructure options available to PDC educators, provides a strategy for choosing among them, and provides practical advice for PDC pedagogy. Through these discussions, the reader may see how much PDC education has changed over the past two decades, identify some areas of PDC that have remained stable during this same time period, and so gain new insight into how to efficiently invest one's time as a PDC educator.

Software implementation of the conjugate gradient method for shared and distributed memory multiprocessor systems

The article examines different approaches to parallelizing the conjugate gradient algorithms for shared and distributed memory multiprocessor systems. We use the standards of MPI and OpenMP libraries of the C programming language for parallel programming. Testing the algorithms was carried out on the HP BLc3000 supercomputer, which contains 392 Intel cores and the InfiniBand network. The article presents the test results – acceleration and efficiency of parallel implementation for each type of multiprocessor systems depending on the dimension of a system of linear equations and the number of parallel processes. We present short descriptions of parallelization methods and excerpts of the code.

Automatic parallelization of representative-based clustering algorithms for multicore cluster systems

Ease of programming and optimal parallel performance have historically been on the opposite side of a trade-off, forcing the user to choose. With the advent of the Big Data era and the rapid evolution of sequential algorithms, the data analytics community can no longer afford the trade-off. We observed that several clustering algorithms often share common traits—particularly, algorithms belonging to the same class of clustering exhibit significant overlap in processing steps. Here, we present our observation on domain patterns in representative-based clustering algorithms and how they manifest as clearly identifiable programming patterns when mapped to a Domain Specific Language (DSL). We have integrated the signatures of these patterns in the DSL compiler for parallelism identification and automatic parallel code generation. The compiler either generates MPI C++ code for distributed memory parallel processing or MPI–OpenMP C++ code for hybrid memory parallel processing, depending upon the target architecture. Our experiments on different state-of-the-art parallelization frameworks show that our system can achieve near-optimal speedup while requiring a fraction of the programming effort, making it an ideal choice for the data analytics community. Results are presented for both distributed and hybrid memory systems.

HDOT — An approach towards productive programming of hybrid applications

A wealth of important scientific and engineering applications are configured for use on high performance computing architectures using functionality found in the MPI specification. This specification provides application developers with a straightforward means for implementing their ideas for execution on distributed-memory parallel processing computers. OpenMP directives provide a means for operating on shared-memory regions of those computers. With the advent of machines composed of many-core processors, the strict synchronisation required by the bulk synchronous parallel (BSP) communication model can hinder performance increases. This is due to the complexity to handle load imbalances, to reduce serialisation imposed by blocking communication patterns, to overlap communication with computation and, finally, to deal with increasing memory overheads. The MPI specification provides advanced features such as non-blocking calls or shared memory to mitigate some of these factors. However, applying these features efficiently usually requires significant changes on the application structure.Task parallel programming models are being developed as a means of mitigating the abovementioned issues but without requiring extensive changes on the application code. In this work, we present a methodology to develop hybrid applications based on tasks called hierarchical domain over-decomposition with tasking (HDOT). This methodology overcomes most of the issues found on MPI-only and traditional hybrid MPI+OpenMP applications. However, by emphasising the reuse of data partition schemes from process-level and applying them to task-level, it enables a natural coexistence between MPI and shared-memory programming models. The proposed methodology shows promising results in terms of programmability and performance measured on a set of applications.

Analytical Estimation of the Scalability of Iterative Numerical Algorithms on Distributed Memory Multiprocessors

This article presents a new high-level parallel computational model named BSF - Bulk Synchronous Farm. The BSF model extends the BSP model to deal with the compute-intensive iterative numerical methods executed on distributed-memory multiprocessor systems. The BSF model is based on the master-worker paradigm and the SPMD programming model. The BSF model makes it possible to predict the upper scalability bound of a BSF-program with great accuracy. The BSF model also provides equations for estimating the speedup and parallel efficiency of a BSF-program.

Generalized Parallel Computational Schemes for Time-Consuming Global Optimization

This paper addresses computationally intensive global optimization problems, for solving of which the supercomputing systems with exaflops performance can be required. To overcome such computational complexity, the paper proposes the generalized parallel computational schemes, which may involve numerous efficient parallel algorithms of global optimization. The proposed schemes include various ways of multilevel decomposition of parallel computations to guarantee the computational efficiency of supercomputing systems with shared and distributed memory multiprocessors with thousands of processors to meet global optimization challenges.

Модель параллельных вычислений для многопроцессорных систем с распределенной памятью

The paper is devoted to an analytical study of the "master-worker" framework scalability on multiprocessors with distributed memory. A new model of parallel computations called BSF is proposed. The BSF model is based on BSP and SPMD models. The scope of BSF model is the compute-intensive applications. The architecture of BSF-computer is defined. The structure of BSF-program is described. The Using this metric, the upper scalability bounds of BSF programs on distributed memory multiprocessors are evaluated. The formulas for estimating the parallel efficiency of BSF programs also proposed.

Sufficient conditions for Hamiltonicity in multiswapped networks

OTIS networks are interconnection networks amenable to deployment as hybrid networks containing both electronic and optical links. Deficiencies as regards symmetry led to the subsequent formulation of biswapped networks which were later generalized to multiswapped networks so as to still enable optoelectronic implementation (as it happens, multiswapped networks also generalize previously studied hierarchical crossed cubes). Multiswapped networks of the form Msw(H;G) are known to possess good (graph-theoretic) properties as regards their use as (optoelectronic) interconnection networks (in distributed-memory multiprocessors) and in relation to those of the component networks G and H. Combinatorially, they provide a hierarchical mechanism to define new networks from existing networks (so that the properties of the new network can be controlled in terms of the constituent networks). In this paper, we prove that if G and H are Hamiltonian networks then the multiswapped network Msw(H;G) is also Hamiltonian. At the core of our proof is finding specially designed Hamiltonian cycles in 2-dimensional and heavily pruned 3-dimensional tori, irrespective of the actual networks G and H we happen to be working with. This lends credence to the role of tori as fundamental networks within the study of interconnection networks.

A Decomposition of the Tikhonov Regularization Functional Oriented to Exploit Hybrid Multilevel Parallelism

We introduce a decomposition of the Tikhonov Regularization (TR) functional which split this operator into several TR functionals, suitably modified in order to enforce the matching of their solutions. As a consequence, instead of solving one problem we can solve several problems reproducing the initial one at smaller dimensions. Such approach leads to a reduction of the time complexity of the resulting algorithm. Since the subproblems are solved in parallel, this decomposition also leads to a reduction of the overall execution time. Main outcome of the decomposition is that the parallel algorithm is oriented to exploit the highest performance of parallel architectures where concurrency is implemented both at the coarsest and finest levels of granularity. Performance analysis is discussed in terms of the algorithm and software scalability. Validation is performed on a reference parallel architecture made of a distributed memory multiprocessor and a Graphic Processing Unit. Results are presented on the Data Assimilation problem, for oceanographic models.

Embedding hypercubes and folded hypercubes onto Cartesian product of certain trees

The hypercube network is one of the most popular interconnection networks since it has simple structure and is easy to implement. The folded hypercube is an important variation of the hypercube. Interconnection networks play a major role in the performance of distributed memory multiprocessors and the one primary concern for choosing an appropriate interconnection network is the graph embedding ability. A graph embedding of a guest graph G into a host graph H is an injective map on the vertices such that each edge of G is mapped into a path of H. The wirelength of this embedding is defined to be the sum of the lengths of the paths corresponding to the edges of G. In this paper we embed hypercube and folded hypercube onto Cartesian product of trees such as 1-rooted complete binary tree and path, sibling tree and path to minimize the wirelength.

Throughput-memory footprint trade-off in synthesis of streaming software on embedded multiprocessors

We study the trade-off between throughput and memory footprint of embedded software that is synthesized from acyclic static dataflow (task graph) specifications targeting distributed memory multiprocessors. We identify iteration overlapping as a knob in the synthesis process by which one can trade application throughput for its memory requirement. Given an initial processor assignment and non-overlapped task schedule, we formally present underlying properties of the problem, such as constraints on a valid iteration overlapping, maximum possible throughput, and minimum memory footprint. Moreover, we develop an effective algorithm for generation of a rich set of design points that provide a range of trade-off options. Experimental results on a number of applications and architectures validate the effectiveness of our approach.

Range query processing on single and multi GPU environments

Metric-space similarity search has proven suitable in a number of application domains such as multimedia retrieval and computational biology to name a few. These applications usually work on very large databases that are often indexed to speed-up on-line searches. To achieve efficient throughput, it is essential to exploit the intrinsic parallelism in the respective search query processing algorithms. Many strategies have been proposed in the literature to parallelize these algorithms either on shared or distributed memory multiprocessor systems. Lately, GPUs have been used to implement brute-force parallel search strategies instead of using index data structures. Indexing poses difficulties when it comes to achieve efficient exploitation of GPU resources. In this paper we propose single and multi GPU metric space techniques that efficiently exploit GPU tailored index data structures for parallel similarity search in large databases. The experimental results show that our proposal outperforms previous index-based sequential and OpenMP search strategies.

Linearly many faults in dual-cube-like networks

The dual cubes were introduced as a better interconnection network than the hypercubes for large scale distributed memory multiprocessors. In this paper we introduce a generalization of these networks, called dual-cube-like networks, which preserve the basic structure of dual cubes and retain many of its topological properties. We investigate structural properties of these networks beyond simple measures such as connectivity. We prove that if up to kn−k(k+1)2 vertices are deleted from a dual-cube-like n-regular network, then the resulting graph will either be connected or will have a large component and small components having at most k−1 vertices in total, and this result is sharp for k≤n. As an application, we derive additional results such as the cyclic vertex-connectivity and the restricted vertex-connectivity of these networks.

FORMLESS

Variants of dataflow specification models are widely used to synthesize streaming applications for distributed-memory parallel processors. We argue that current practice of specifying streaming applications using rigid dataflow models, implicitly prohibits a number of platform oriented optimizations and hence limits portability and scalability with respect to number of processors. We motivate Functionally-cOnsistent stRucturally-MalLEabe Streaming Specification , dubbed FORMLESS, which refers to raising the abstraction level beyond fixed-structure dataflow to address its portability and scalability limitations. To demonstrate the potential of the idea, we develop a design space exploration scheme to customize the application specification to better fit the target platform. Experiments with several common streaming case studies demonstrate improved portability and scalability over conventional dataflow specification models, and confirm the effectiveness of our approach.

JOSEPHINE: A parallel SPH code for free-surface flows

JOSEPHINE is a parallel Smoothed Particle Hydrodynamics program, designed to solve unsteady free-surface flows. The adopted numerical scheme is efficient and has been validated on a first case, where a liquid drop is stretched over the time. Boundary conditions can also be modelled, as it is demonstrated in a second case: the collapse of a water column. Results show good agreement with both reference numerical solutions and experiments. The use of parallelism allows significant reduction of the computational time, even more with large number of particles. JOSEPHINE has been written so that any untrained developers can handle it easily and implement new features. Program summaryProgram title:JOSEPHINECatalogue identifier: AELV_v1_0Program summary URL:http://cpc.cs.qub.ac.uk/summaries/AELV_v1_0.htmlProgram obtainable from: CPC Program Library, Queenʼs University, Belfast, N. IrelandLicensing provisions: Standard CPC licence, http://cpc.cs.qub.ac.uk/licence/licence.htmlNo. of lines in distributed program, including test data, etc.: 5139No. of bytes in distributed program, including test data, etc.: 22 833Distribution format: tar.gzProgramming language: Fortran 90 and OpenMPIComputer: All shared or distributed memory parallel processors, tested on a Xeon W3520, 2.67 GHz.Operating system: Any system with a Fortran 90 compiler and MPI, tested on Debian Linux.Has the code been vectorised or parallelised?: The code has been parallelised but has not been explicitly vectorised.RAM: Dependent upon the number of particles.Classification: 4.12Nature of problem:JOSEPHINE is designed to solve unsteady incompressible flows with a free-surface and large deformations.Solution method:JOSEPHINE is an implementation of Smoothed Particle Hydrodynamics. SPH is a Lagrangian mesh free particle method, thus, no explicit tracking procedure is required to catch the free surface. Incompressibility is satisfied using a weakly compressible model. Boundary conditions at walls are enforced by means of the ghost particles technique. The free-surface dynamic and kinematic conditions are applied implicitly.Running time: 15 mn on 4 processors for the dam-break case with 5000 particles, dependent upon the real duration (2 s here).

Faster exact maximum parsimony search with XMP

Despite trends towards maximum likelihood and Bayesian criteria, maximum parsimony (MP) remains an important criterion for evaluating phylogenetic trees. Because exact MP search is NP-complete, the computational effort needed to find provably optimal trees skyrockets with increasing numbers of taxa, limiting analyses to around 25-30 taxa. This is, in part, because currently available programs fail to take advantage of parallelism. We present XMP, a new program for finding exact MP trees that comes in both serial and parallel versions. The serial version is faster in nearly all tests than existing software. The parallel version uses a work-stealing algorithm to scale to hundreds of CPUs on a distributed-memory multiprocessor with high efficiency. An optimized SSE2 inner loop provides additional speedup for Pentium 4 and later CPUs. C source code and several binary versions are freely available from http://www.massey.ac.nz/~wtwhite/xmp. The parallel version requires an MPI implementation, such as the freely available MPICH2.

Parallel implementation of an adaptive and parameter-free N-body integrator

Previously, Pruett et al. (2003) [3] described an N-body integrator of arbitrarily high order M with an asymptotic operation count of O ( M 2 N 2 ) . The algorithm's structure lends itself readily to data parallelization, which we document and demonstrate here in the integration of point-mass systems subject to Newtonian gravitation. High order is shown to benefit parallel efficiency. The resulting N-body integrator is robust, parameter-free, highly accurate, and adaptive in both time-step and order. Moreover, it exhibits linear speedup on distributed parallel processors, provided that each processor is assigned at least a handful of bodies. Program summary Program title: PNB.f90 Catalogue identifier: AEIK_v1_0 Program summary URL: http://cpc.cs.qub.ac.uk/summaries/AEIK_v1_0.html Program obtainable from: CPC Program Library, Queen's University, Belfast, N. Ireland Licensing provisions: Standard CPC license, http://cpc.cs.qub.ac.uk/licence/licence.html No. of lines in distributed program, including test data, etc.: 3052 No. of bytes in distributed program, including test data, etc.: 68 600 Distribution format: tar.gz Programming language: Fortran 90 and OpenMPI Computer: All shared or distributed memory parallel processors Operating system: Unix/Linux Has the code been vectorized or parallelized?: The code has been parallelized but has not been explicitly vectorized. RAM: Dependent upon N Classification: 4.3, 4.12, 6.5 Nature of problem: High accuracy numerical evaluation of trajectories of N point masses each subject to Newtonian gravitation. Solution method: Parallel and adaptive extrapolation in time via power series of arbitrary degree. Running time: 5.1 s for the demo program supplied with the package.