Distributed Memory Multicomputers Research Articles

The Path to Scalable Distributed Quantum Computing

Experimental groups are now fabricating quantum processors powerful enough to execute small instances of quantum algorithms and definitively demonstrate quantum error correction that extends the lifetime of quantum data, adding urgency to architectural investigations. Although other options continue to be explored, effort is coalescing around topological coding models as the most practical implementation option for error correction on realizable microarchitectures. Scalability concerns have also motivated architects to propose distributed memory multicomputer architectures, with experimental efforts demonstrating some of the basic building blocks to make such designs possible. We compile the latest results from a variety of different systems aiming at the construction of a scalable quantum computer.

Parallel Algorithm with Parameters Based on Alternating Direction for Solving Banded Linear Systems

An efficient parallel iterative method with parameters on distributed-memory multicomputer is investigated for solving the banded linear equations in this work. The parallel algorithm at each iterative step is executed using alternating direction by splitting the coefficient matrix and using parameters properly. Only it twice requires the communications of the algorithm between the adjacent processors, so this method has high parallel efficiency. Some convergence theorems for different coefficient matrices are given, such as a Hermite positive definite matrix or anM-matrix. Numerical experiments implemented on HP rx2600 cluster verify that our algorithm has the advantages over the multisplitting one of high efficiency and low memory space, which has a considerable advantage in CPU-times costs over the BSOR one. The efficiency for Example 1 is better than BSOR one significantly. As to Example 2, the acceleration rates and efficiency of our algorithm are better than the PEk inner iterative one.

Parallel Algorithm for Solving Block-Tridiagonal Linear Systems

Efficient parallel iterative algorithm is investigated for solving block-tridiagonal linear systems on distributed-memory multi-computers. Based on Galerkin theory, the communication only need twice between the adjacent processors per iteration step. Furthermore, the condition for convergence is given when the coefficient matrix A is a symmetric positive definite matrix. Numerical experiments implemented on the cluster verify that our algorithm parallel acceleration rates and efficiency are higher than the multisplitting one, and has the advantages over the multisplitting method of high efficiency and low memory space.

Evaluation of P-Scheme/G Algorithm for Solving Recurrence Equations

 Abstract—A parallel algorithm called P-scheme/G is proposed for solving recurrence equations on GPGPU systems. This is based on P-scheme algorithm that has been originally developed for distributed memory multicomputers. In order to achieve a high performance computation on GPGPU systems, our method alleviates branch divergences by reducing the stride data accesses. We also illustrate the effectiveness of the optimal thread configuration for the recurrence equation. Our experiments with GTX 590 show that the implementation of the rearrangement using the shared memory improves the performance by 200\% to 300\% and the validity of the policy of the thread configuration is confirmed for both the constant and the non-constant parameter cases. We achieve the speedup of around 400 as a recurrence equation solver with non-constant parameters.

A 3D Hybrid Model for Tissue Growth: The Interplay between Cell Population and Mass Transport Dynamics

To provide theoretical guidance for the design and in vitro cultivation of bioartificial tissues, we have developed a multiscale computational model that can describe the complex interplay between cell population and mass transport dynamics that governs the growth of tissues in three-dimensional scaffolds. The model has three components: a transient partial differential equation for the simultaneous diffusion and consumption of a limiting nutrient; a cellular automaton describing cell migration, proliferation, and collision; and equations that quantify how the varying nutrient concentration modulates cell division and migration. The hybrid discrete-continuous model was parallelized and solved on a distributed-memory multicomputer to study how transport limitations affect tissue regeneration rates under conditions encountered in typical bioreactors. Simulation results show that the severity of transport limitations can be estimated by the magnitude of two dimensionless groups: the Thiele modulus and the Biot number. Key parameters including the initial seeding mode, cell migration speed, and the hydrodynamic conditions in the bioreactor are shown to affect not only the overall rate, but also the pattern of tissue growth. This study lays the groundwork for more comprehensive models that can handle mixed cell cultures, multiple nutrients and growth factors, and other cellular processes, such as cell death.

Full-system simulation of distributed memory multicomputers

In this paper we discuss environments for the full-system simulation of multicomputers. These environments are composed of a large collection of modules that simulate the compute nodes and the network, plus additional linking elements that perform communication and synchronization. We present our own environment, in which we integrate Simics with INSEE. We reuse as many Simics modules as possible to reduce the effort of hardware modeling, and also to simulate standard machines running unmodified operating systems. This way we avoid the error-prone effort of developing drivers and libraries. The environment we propose in this paper enables us to show some of the difficulties we found when integrating diverse tools, and how we were able to overcome them. Furthermore we show some important details to have into account in order to do a valid full-system simulation of multicomputers, mostly related with synchronization and timing. Thus, a trade-off has to be found between simulation speed and accuracy of results.

Dynamic Behavior Of Heterogeneous Cell Populations Growing Under Mass Transport Limitations

Tissue growth in biomimetic scaffolds is strongly influenced by the dynamics and heterogeneity of cell populations. A significant source of heterogeneity is nutrient (or growth factor) depletion. Cells slow down, stop dividing or die when the concentrations of key nutrients or growth factors drop below critical levels in the scaffold interior. As a result, we still cannot grow in vitro tissue samples thicker than a few millimeters for metabolically active cells.To provide theoretical guidance for the in vitro cultivation of bioartificial tissues, we have developed a multi-scale model that can describe how the complex interplay among key intracellular processes, cell population dynamics and nutrient depletion regulates the growth of tissues in 3D scaffolds. We use a discrete, stochastic algorithm to describe the population dynamics of migrating, interacting and proliferating cells. Diffusion and consumption of a limiting nutrient is modeled by a partial differential equation subject to boundary conditions appropriate for common bioreactors. This PDE is solved numerically and the computed concentration profiles are used to modulate cell proliferation rates and migration speeds. The hybrid discrete-continuous model was parallelized and solved on a distributed-memory multicomputer to study how mass transport limitations affect tissue regeneration rates under conditions encountered in typical bioreactors.Simulation results show that the severity of mass transport limitations can be estimated by the magnitude of two dimensionless groups. Critical system parameters like cell population heterogeneity, the initial spatial distribution of seed cells, the distribution of cell migration speeds, and the hydrodynamic environment are shown to affect not only the overall rate, but also the pattern of tissue growth. More specifically, the interplay of cell population heterogeneity and cell death due to nutrient depletion can lead to dynamic self-assembly of cells and the formation of stratified structures.

A sparse nonsymmetric eigensolver for distributed memory architectures

In this work, we propose an efficient parallel implementation of the nonsymmetric block Lanczos algorithm for the computation of few extreme eigenvalues, and corresponding eigenvectors, of real nonhermitian matrices for distributed memory multicomputers. The reorganisation of the block Lanczos algorithm implemented allows to exploit a coarse-grained parallelism and to harness the computational power of the target architectures. The computational kernels of the algorithm are matrix–matrix multiplications, with dense and sparse factors, QR factorisation and singular value decomposition. To reduce the total amount of communication involved in the matrix–matrix multiplication with a sparse factor, we substitute each matrix appearing in the algorithm with its transpose. Then, we develop an efficient parallelisation of the matrix–matrix multiplication when the second factor is sparse. Some other linear algebra operations are performed using ScaLAPACK library. The parallel eigensolver has been tested on a cluster of PCs. All reported results show the proposed algorithm is efficient on the target architectures for problems of adequate dimension.

A flexible processor mapping technique toward data localization for block-cyclic data redistribution

Array redistribution is usually needed for more efficiently executing a data-parallel program on distributed memory multicomputers. To minimize the redistribution data transfer cost, processor mapping techniques were proposed to reduce the amount of redistributed data elements. Theses techniques demand that the beginning data elements on a processor not be redistributed in the redistribution. On the other hand, for satisfying practical computation needs, a programmer may require other data elements to be un-redistributed (localized) in the redistribution. In this paper, we propose a flexible processor mapping technique for the Block-Cyclic redistribution to allow the programmer to localize the required data elements in the redistribution. We also present an efficient redistribution method for the redistribution employing our proposed technique. The data transfer cost reduction and system performance improvement for the redistributions with data localization are analyzed and presented in our experimental results.

Performance analysis of explicit group parallel algorithms for distributed memory multicomputer

Since their introduction, the four-point explicit group (EG) and explicit decoupled group (EDG) methods in solving elliptic PDE’s have been implemented on various parallel computing architectures such as shared memory parallel computer and distributed computer systems. However, no detailed study on the performance analysis of these algorithms was done in any of these implementations. In this paper we developed performance models for these explicit group methods and present detailed study of their hypothetical implementation on two distributed memory multicomputers with different computation speed and communication bandwidth. Detailed performance analysis based on these models predicted different theoretical performance if the methods were implemented on the clusters. This was confirmed by the experimental results performed on the two distinct clusters. Theoretical analysis and experimental results indicated that both explicit group methods are scalable with respect to number of processors and the problem size.

Regular Paper: Parallel Implementation of a Cellular Automaton Modeling the Growth of Three-Dimensional Tissues

A promising approach for treating tissue or organ failure involves the use of bioartificial tissue substitutes grown in scaffolds with appropriate structure and shape. Currently, however, the engineering of tissue substitutes is a long and costly process based exclusively on experimentation. Predictive computer models can greatly reduce the development costs of tissue-engineered therapies by enabling scientists to rapidly evaluate the effect of system parameters on the growth rates and quality of regenerated tissues. We report here the parallel implementation of a three-dimensional model that employs cellular automata to describe the dynamic behavior of a population of mammalian cells that migrate, interact and proliferate to generate new tissues. The simulator uses MPI for interprocessor communication and is suitable for distributed memory multi-computers. Three parallel algorithms are developed to approximate the sequential algorithm describing this dynamic process of tissue growth. The parallel algorithms progressively relax the correctness requirements using different approaches to handle the cells that either move/ divide in the boundary layers of processors or cross sub-domain boundaries. Finally, a systematic study is carried out to evaluate the accuracy and performance of these algorithms.

Data distribution schemes of sparse arrays on distributed memory multicomputers

A data distribution scheme of sparse arrays on a distributed memory multicomputer, in general, is composed of three phases, data partition, data distribution, and data compression. To implement the data distribution scheme, many methods proposed in the literature first perform the data partition phase, then the data distribution phase, followed by the data compression phase. We called a data distribution scheme with this order as Send Followed Compress (SFC) scheme. In this paper, we propose two other data distribution schemes, Compress Followed Send (CFS) and Encoding-Decoding (ED), for sparse array distribution. In the CFS scheme, the data compression phase is performed before the data distribution phase. In the ED scheme, the data compression phase can be divided into two steps, encoding and decoding. The encoding step and the decoding step are performed before and after the data distribution phase, respectively. To evaluate the CFS and the ED schemes, we compare them with the SFC scheme. In the data partition phase, the row partition, the column partition, and the 2D mesh partition with/without load-balancing methods are used for these three schemes. In the compression phase, the CRS/CCS methods are used to compress sparse local arrays for the SFC and the CFS schemes while the encoding/decoding step is used for the ED scheme. Both theoretical analysis and experimental tests were conducted. In the theoretical analysis, we analyze the SFC, the CFS, and the ED schemes in terms of the data distribution time and the data compression time. In experimental tests, we implemented these three schemes on an IBM SP2 parallel machine. From the experimental results, for most of test cases, the CFS and the ED schemes outperform the SFC scheme. For the CFS and the ED schemes, the ED scheme outperforms the CFS scheme for all test cases.

TRLE—an efficient data compression scheme for image composition of volume rendering on distributed memory multicomputers

Data compression is a well-known method to improve the image composition time of parallel volume rendering on distributed memory multicomputers. In this paper, we propose an efficient data compression scheme, the template run-length encoding (TRLE) scheme, for image composition. Given an image with 2n×2n pixels, in the TRLE scheme, the image is treated as n×n blocks and each block has 2×2 pixels. Since a pixel can be a blank or non-blank pixel, there 16 templates in a block. To compress an image, the TRLE scheme encodes an image block by block similar to the run-length encoding scheme. However, the TRLE scheme can filter out or use small space to encode blocks whose four pixels are blank pixels, that is, the TRLE scheme can encode a partial image according to the shape of non-blank pixels. To evaluate the performance of the TRLE scheme, we compare the proposed scheme with the BR, the RLE, and the BRLC schemes. Since a data compression scheme needs to cooperate with some data communication schemes, in the implementation, the binary-swap, the parallel-pipelined, and the rotate-tiling data communication schemes are used. By combining the four data compression schemes with the three data communication schemes, we have twelve image composition methods. These twelve methods are implemented on an IBM SP2 parallel machine. Four volume datasets are used as test samples. The data computation time and the data communication time are measured. The experimental results show that the TRLE data compression scheme with the rotate-tiling data communication scheme outperforms other eleven image composition methods for all test samples.

Optimizing Communications of Dynamic Data Redistribution on Symmetrical Matrices in Parallelizing Compilers

Dynamic data redistribution is used to enhance data locality and algorithm performance by reducing interprocessor communication in many parallel scientific applications on distributed memory multicomputers. Since the redistribution is performed at runtime, there is a performance tradeoff between the efficiency of the new data decomposition for a subsequent phase of an algorithm and the cost of redistributing data among processors. In this paper, we present a processor replacement scheme to minimize the cost of interprocessor data exchange during runtime. The main idea of the proposed technique is to develop a replacement function for reordering logical processors in the destination phase. Based on the replacement function, a realigned sequence of destination processors can be derived and is then used to perform data decomposition in the receiving phase. Together with local matrix and compressed CRS vectors transposition schemes, the interprocessor communication can be eliminated during runtime. A significant improvement of this approach is that the realignment of data can be performed without interprocessor communication for special cases. The second contribution of the present technique is that the complicated communication sets generation could be simplified by applying local matrix transposition. Consequently, the indexing cost could be reduced significantly. The proposed techniques can be applied in both dense and sparse applications. A generalized symmetric redistribution algorithm is also presented in this work. To analyze the efficiency of the proposed technique, the theoretical analysis proves that up to (p-1)/p data transmission cost can be saved. For general cases, the symmetric redistribution algorithm saves 1/p communication overheads compared with the traditional method. Experimental results also show that the proposed techniques provide superior performance in most data redistribution instances

A parallel algorithm based on Galerkin theory for block-tridiagonal linear systems

The work presented in this paper focuses on parallel iterative method for solving block-tridiagonal linear systems. Being based on Galerkin theory predetermining W m = V m , and forming a suitable matrix V m , a parallel iterative method on distributed-memory multi-computer is established. Then Eq. (3.1)–(3.3) are provided for carrying out computation required by our parallel algorithm. Furthermore, convergence is proved when the coefficient matrix A is a symmetric positive definite matrix, and the sufficient condition is given. In the end, two illustrative examples implemented on HP rx2600 cluster show that our algorithm’s parallel acceleration rates and efficiency are higher.

An Efficient Communication Scheduling Method for the Processor Mapping Technique Applied Data Redistribution

Array redistribution is usually required for more efficiently executing a data-parallel program on distributed memory multi-computers. In performing array redistribution using synchronous communication mode, data communications among the processors should be properly arranged to avoid incurring higher data transfer cost. Some efficient communication scheduling methods for the Block-Cyclic redistribution have been proposed. On the other hand, the processor mapping technique can help reduce the data transfer cost of redistribution. To avoid degrading the benefit of data transfer cost reduction, it is needed to construct optimal communication schedules for the redistribution in which the processor mapping technique is applied. In this paper, we present a unified approach to constructing optimal communication schedules for the processor mapping technique applied Block-Cyclic redistribution. The proposed method is founded on the processor mapping technique and can more efficiently construct the required communication schedules than other optimal scheduling methods.

A parallel iterative method for solving periodical block-tridiagonal linear equations

In this paper, based upon splitting the coefficient matrix, we propose a parallel iterative algorithm for periodical block-tridiagonal linear equations on distributed-memory multi-computers, which is more general applied than presented in [Lihua Chi, Jie Liu, Xiaomei Li. An effective parallel algorithm for tridiagonal linear equations, Journal of Computer 22 (2) (1999) 218–221]. Our algorithm can be easily generalized to solve the tridiagonal systems, the block-tridiagonal, and the periodical block-tridiagonal systems. The communication only needs twice between the adjacent processors per iteration. Furthermore, analysis of convergence and error about this algorithm is given. Finally, some numerical results on HP r×2600 cluster show that practice computing is consistent with theory. The algorithm has great parallel efficiency.

A parallel algorithm for band linear systems

The work presented in this paper focuses on parallel iterative algorithm for solving band linear systems. With suitable decomposition of coefficient matrix and using the form of the iterative formula of the BSOR serial method, a parallel algorithm on distributed-memory multi-computer is established. Then Eq. (7) and (8) are provided for carrying out computation required by our parallel algorithm. Furthermore, convergence is proved when the coefficient matrix A is Hermite positive definite matrix or M-matrix, and the sufficient condition is given. According to theoretical analysis, this algorithm has the same convergent velocity as BSOR method, and has the same parallelism as BJ method. In the end, two illustrative examples implemented on HP rx2600 cluster show that our algorithm’s parallel acceleration rates and efficiency are higher.

Essential Cycle Calculation Method for Irregular Array Redistribution

In many parallel programs, run-time array redistribution is usually required to enhance data locality and reduce remote memory access on the distributed memory multicomputers. In general, array distribution can be classified into regular distribution and irregular distribution according to the distribution fashion. Many methods for performing regular array redistribution have been presented in the literature. However, for the heterogeneous computation environment, irregular array redistributions can be used to adjust data assignment at run-time. In this paper, an Essential Cycle Calculation method for unequal block sizes array redistribution is presented. In the ECC method, a processor first computes the source/destination processor/data sets of array elements in the first essential cycle of the local array it owns. From the source/destination processor/data sets of array elements in the first essential cycle, we can construct packing/unpacking pattern tables. Since each essential cycle has the same communication pattern, based on the packing/unpacking pattern tables, a processor can pack/unpack array elements efficiently. To evaluate the performance of the ECC method, we have implemented this method on an IBM SP2 parallel machine and compare it with the Sequence method. The cost models for these methods are also presented. The experimental results show that the ECC method greatly outperforms the Sequence method for all test samples.

A parallel algorithm for block-tridiagonal linear systems

In this paper, a parallel algorithm for solving block-tridiagonal linear systems on distributed-memory multi-computer is presented. According to theoretical analysis, convergent velocity and complexity of this algorithm are the same as BSOR method’s, and parallelism is the same as BJ method’s. Moreover, two examples have been implemented on HP rx2600 cluster, and the numerical experiments indicate that our algorithm is feasible and effective.