Asynchronous Iterations Research Articles

Block-asynchronous Multigrid Smoothers for GPU-accelerated Systems

This paper explores the need for asynchronous iteration algorithms as smoothers in multigrid methods. The hardware target for the new algorithms is top-of-the-line, highly parallel hybrid architectures – multicore-based systems enhanced with GPGPUs. These architectures are the most likely candidates for future high-end supercomputers. To pave the road for their effcient use, we must resolve challenges related to the fact that data movement, not floatingpoint operations, is the bottleneck to performance. Our work is in this direction — we designed block-asynchronous multigrid smoothers that perform more flops in order to reduce synchronization, and hence data movement. We show that the extra flops are done for “free,” while synchronization is reduced and the convergence properties of multigrid with classical smoothers like Gauss-Seidel can be preserved.

Adaptation and Evaluation of the Multisplitting-Newton and Waveform Relaxation Methods Over Distributed Volatile Environments

This paper presents new adaptations of two methods that solve large differential equations systems, to the grid context. The first method is based on the Multisplitting concept and the second on the Waveform Relaxation concept. Their adaptations are implemented according to the asynchronous iteration model which is well suited to volatile architectures that suffer from high latency networks. Many experiments were conducted to evaluate and compare the accuracy and performance of both methods while solving the advection-diffusion problem over heterogeneous, distributed and volatile architectures. The JACEP2P-V2 middleware provided the fault tolerant asynchronous environment, required for these experiments.

Extracting state-based performance metrics using asynchronous iterative techniques

Solutions of large sparse linear fixed-point problems lie at the heart of many important performance analysis calculations. These calculations include steady-state, transient and passage-time computations in discrete-time Markov chains, continuous-time Markov chains and semi-Markov chains. In recent years, much work has been done to extend the application of asynchronous iterative solution methods to different contexts. This work has been motivated by the potential for faster solution, more efficient use of the communication channel and access to memory, and simplification of task management and programming. In this paper, we show how the key performance metrics mentioned above can be transformed into problems which can be solved using asynchronous iterative methods with guaranteed convergence—using the full breadth of Chazan and Miranker’s classes of asynchronous iterations. We introduce the application of asynchronous iterative solution methods within this context by applying several algorithm variants to the steady-state analysis of a GSPN model of a flexible manufacturing system. We show that for varying numbers of processors and different problem sizes one of these algorithm variants offers consistently better wall time until convergence, a consistently better communication profile, and often requires fewer update iterations than a standard parallel Jacobi algorithm, seemingly benefiting from a form of Gauss–Seidel effect.

Asynchronous multisplitting iteration with different weighting schemes

The multisplitting iteration method was presented by O’Leary and White [5] for solving large sparse linear systems on parallel multiprocessor system. In this paper, we further set up an asynchronous variant for the multisplitting iteration method with different weighting schemes studied by White [8]. Moreover, we establish a general convergence criterion for asynchronous iteration framework, and then prove the convergence of the new asynchronous multisplitting iteration method with different weighting schemes by making use of this general criterion.

A decentralized and fault tolerant convergence detection algorithm for asynchronous iterative algorithms

This article presents an algorithm that performs a decentralized detection of the global convergence of parallel asynchronous iterative applications. This algorithm is fault tolerant. It runs a decentralized saving procedure which enables this algorithm, after a node’s crash, to replace the dead node by a new one which will continue the computing task from the last check point. Combined with the advantages of the asynchronous iteration model, this method allows us to compute very large scale problems using highly volatile parallel architectures like Peer-to-Peer and distributed clusters architectures. We also present the implementation of this algorithm in the JaceP2P platform which is dedicated to designing and executing parallel asynchronous iterative applications in volatile environments. Numerous experiments show the robustness and the efficiency of our algorithm.

Asynchronous Consensus Protocols Using Nonlinear Paracontractions Theory

Several consensus protocols have been proposed in the literature and their convergence properties studied via a variety of methods. In all these methods, the communication topologies play a key role in the convergence of consensus processes. In this note, based on asynchronous iteration methods for nonlinear paracontractions, we establish a new result which shows that consensus is reachable under directional and time-varying topologies by using asynchronous nonlinear protocols. Our result makes use of the confluent iteration graph which unifies various communication assumptions and contributes to a fundamental understanding of convergent consensus processes. This result extends existing ones in the literature and has many potential applications. As an illustration, we consider a special case of our model and discuss the robot rendezvous problem via a center-of-gravity algorithm.

Parallel asynchronous iterations for the solution of a 3D continuous flow electrophoresis problem

This paper deals with the parallel numerical solution of a 3D continuous flow electrophoresis problem governed by Navier–Stokes equations coupled with transport and potential equations. For this problem, using the properties of the discrete operators, the convergence of synchronous and asynchronous parallel Schwarz alternating methods is analysed. Finally, parallel solvers are implemented and the results of simulations are given.

Asynchronous iterative solution for state-based performance metrics

Solution of large sparse fixed-point problems, Mline over x = over x and Mline over x + line over b = over x, may be seen as underpinning many important performance-analysis calculations. These calculations include steady-state, passage-time and transient-time calculations in discrete-time Markov chains, continuous-time Markov chains and semi-Markov chains. In recent years, much work has been done to extend the application of asynchronous iterative fixed-point solution methods to many different contexts. This work has been motivated by the potential for faster solution, more efficient use of the communication channel and/or access to memory, and simplification of task management and programming. In this paper, we present theoretical developments which allow us to extend theapplication of asynchronous iterative solution methods to solve for the key performance metrics mentioned above-such that we may employ the full breadth of Chazan and Miranker's classes of asynchronous iterations.

Multisplitting Iteration Schemes for Solving a Class of Nonlinear Complementarity Problems

We consider several synchronous and asynchronous multisplitting iteration schemes for solving a class of nonlinear complementarity problems with the system matrix being an H−matrix. We establish the convergence theorems for the schemes. The numerical experiments show that the schemes are efficient for solving the class of nonlinear complementarity problems.

Integral equation‐based overlapped domain decomposition method for the analysis of electromagnetic scattering of 3D conducting objects

AbstractAn integral equation‐based overlapped domain decomposition method (IE‐ODDM) is presented for the analysis of three‐dimensional (3D) electromagnetic scattering problems. Compared with the forward and backward buffer region iterative method proposed by Brennan et al., the convergence of the IE‐ODDM is faster due to the edge‐effect of the current in each subdomain being effectively depressed. Moreover, the iterative sequence among the subdomains of IE‐ODDM is more flexible, which means that asynchronous iteration and even parallel computing are doable. Simulated RCS results of several convex and concave 3D conducting objects verify the speedup and validity of this method. © 2006 Wiley Periodicals, Inc. Microwave Opt Technol Lett 49: 265–274, 2007; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/mop.22110

Robust Distributed Estimation Using the Embedded Subgraphs Algorithm

We propose a new iterative, distributed approach for linear minimum mean-square-error (LMMSE) estimation in graphical models with cycles. The embedded subgraphs algorithm (ESA) decomposes a loopy graphical model into a number of linked embedded subgraphs and applies the classical parallel block Jacobi iteration comprising local LMMSE estimation in each subgraph (involving inversion of a small matrix) followed by an information exchange between neighboring nodes and subgraphs. Our primary application is sensor networks, where the model encodes the correlation structure of the sensor measurements, which are assumed to be Gaussian. The resulting LMMSE estimation problem involves a large matrix inverse, which must be solved in-network with distributed computation and minimal intersensor communication. By invoking the theory of asynchronous iterations, we prove that ESA is robust to temporary communication faults such as failing links and sleeping nodes, and enjoys guaranteed convergence under relatively mild conditions. Simulation studies demonstrate that ESA compares favorably with other recently proposed algorithms for distributed estimation. Simulations also indicate that energy consumption for iterative estimation increases substantially as more links fail or nodes sleep. Thus, somewhat surprisingly, sensor network energy conservation strategies such as low-powered transmission and aggressive sleep schedules could actually prove counterproductive. Our results can be replicated using MATLAB code from www.dsp.rice.edu/software

A Decentralized Computational Infrastructure for Grid-Based Parallel Asynchronous Iterative Applications

Parallel asynchronous iterative algo- rithms relax synchronization and communication requirements, and can potentially extend Desktop Grids beyond embarrassingly parallel applica- tions to support a broader class of parallel iterative applications. This paper presents the design and implementation of CometG, a decentralized (peer- to-peer) computational infrastructure that extends Desktop Grid environments to support these ap- plications. CometG provides a decentralized and scalable tuple space, efficient communication and coordination support, and application-level ab- stractions that can be used to implement Desktop Grid applications based on parallel asynchronous iterative algorithms using the master-worker/BOT paradigm. The deployment and evaluations of CometG and a CometG-based application in a wide-area environment using the PlanetLab (7) test bed, as well as a campus network are presented.

Performance Comparison of Parallel Programming Environments for Implementing AIAC Algorithms

AIAC algorithms (Asynchronous Iterations Asynchronous Communications) are a particular class of parallel iterative algorithms. Their asynchronous nature makes them more efficient than their synchronous counterparts in numerous cases as has already been shown in previous works. The first goal of this article is to compare several parallel programming environments in order to see if there is one of them which is best suited to efficiently implement AIAC algorithms. The main criterion for this comparison consists in the performances achieved in a global context of grid computing for two classical scientific problems. Nevertheless, we also take into account two secondary criteria which are the ease of programming and the ease of deployment. The second goal of this study is to extract from this comparison the important features that a parallel programming environment must have in order to be suited for the implementation of AIAC algorithms.

The virtual reality applied to biology understanding: The in virtuo experimentation

The advent of the computer and computer science, and in particular virtual reality, offers new experiment possibilities with numerical simulations and introduces a new type of investigation for the complex systems study: the in virtuo experiment. This work lies on the framework of multi-agent systems. We propose a generic model for systems biology based on reification of the interactions, on a concept of organization and on a multi-model approach. By 'reification' we understand that interactions are considered as autonomous agents. The aim has been to combine the systemic paradigm and the virtual reality to provide an application able to collect, simulate, experiment and understand the knowledge owned by different biologists working around an interdisciplinary subject. Here, we have been focused on the urticaria disease understanding. Autonomy is taken as a principle. The method permits to integrate different natures of model in the same application using chaotic asynchronous iterations and C++ library: AReVi. We have modeled biochemical reactions, molecular 3D diffusion, cell organizations and mechanical 3D interactions. It also permits to embed different expert system modeling methods like fuzzy cognitive maps. This work provides a toolbox easily adaptable to new biological studies.

MPMD 방식의 비동기 연산을 이용한 응용 수준의 무정지 선형 시스템의 해법

대규모 병렬 연산에 있어서, 계산 노드 혹은 이들을 연결한 통신 네트워크의 장애는 연산 실패로 끝나며, 소중한 계산 시간이 낭비된다. 그러나 현재의 MPI 표준은 이에 대한 대안을 제시하지 않고 있다. 본 논문에서는, 비표준의 무정지형 MPI 라이브러리가 아닌 MPI 표준 함수들만을 사용하여, MPMD 방식의 비동기 연산을 도입한 응용 수준의 무정지형 선형 시스템의 해법을 제안한다. In a large scale parallel computation, some processor or communication link failure results in a waste of huge amount of CPU hours. However, MPI in its current specification gives the user no possibility to handle such a problem. In this paper, we propose an application-level fault tolerant linear system solver by using an MPMD-type asynchronous iteration, purely on the basis of the MPI standard without using any non-standard fault-tolerant MPI library.

Stopping criteria, forward and backward errors for perturbed asynchronous linear fixed point methods in finite precision

This paper deals with perturbed linear fixed point methods in the presence of round off errors. Successive approximations as well as the more general, asynchronous iterations are treated. Forward and backward error estimates are presented and are used in order to propose theoretical stoppig criteria for these methods. In the case of asynchronous iterations, macro iterations are used as a tool in order to obtain estimates.

Asynchronous iterations with flexible communication: contracting operators

The concept of flexible communication permits one to model efficient asynchronous iterations on parallel computers. This concept is particularly useful in two practical situations. Firstly, when communications are requested while a processor has completed the current update only partly, and secondly, in the context of inner/outer iterations, when processors are also allowed to make use of intermediate results obtained during the inner iteration in other processors. In the general case of nonlinear or linear fixed point problems, we give a global convergence results for asynchronous iterations with flexible communication whereby the iteration operators satisfy certain contraction hypotheses. In this manner we extend to a contraction context previous results obtained for monotone operators with respect to a partial ordering.

On generalized max-min rate allocation and distributed convergence algorithm for packet networks

We consider the fundamental problem of bandwidth allocation among flows in a packet-switched network. The classical max-min rate allocation has been widely regarded as a fair rate allocation policy. But, for a flow with a minimum rate requirement and a peak rate constraint, the classical max-min policy no longer suffices to determine rate allocation since it is not capable of supporting either the minimum rate or the peak rate constraint from a flow. We generalize the theory of the classical max-min rate allocation with the support of both the minimum rate and peak rate constraints for each flow. Additionally, to achieve generalized max-min rate allocation in a fully distributed packet network, we present a distributed algorithm that uses a feedback-based flow control mechanism. Our design not only offers a fresh perspective on flow marking technique, but also advances the state-of-the-art flow marking technique favored by other researchers. We provide proof that such a distributed algorithm, through asynchronous iterations, will always converge to the generalized max-min rate allocation under any network configuration and any set of link distances. We use simulation results to demonstrate the fast convergence property of the distributed algorithm.

Pagerank Computation and Keyword Search on Distributed Systems and P2P Networks

This paper presents a fully distributed computation for Google's pagerank algorithm. The computation is based on solution of the matrix equation defining pageranks by a distributed implementation of asynchronous iteration. Pageranks for the documents stored on a web server or on a host in a peer-to-peer network are computed in place and stored with the documents. The matrix is never assembled and no crawls of the web are required. Continuously accurate pageranks are enabled by incremental computation of pageranks for documents as they are inserted onto a network storage host and incremental recomputation of pageranks when documents are deleted. Intrahost and intradomain dominance of document link structure is naturally exploited by the distributed asynchronous iteration algorithm.

Self-stabilization with path algebra

Self-stabilizing protocols can resist transient failures and guarantee system recovery in a finite time. We highlight the connexion between the formalism of self-stabilizing distributed systems and the formalism of generalized path algebra and asynchronous iterations with delay. We use the later to prove that a local condition on locally executed algorithm (being a strictly idempotent r-operator) ensures self-stabilization of the global system. As a result, a parametrized distributed algorithm applicable to any directed graph topology is proposed, and the function parameter of our algorithm is instantiated to produce distributed algorithms for both fundamental and high-level applications. Due to fault resilience properties of our algorithm, the resulting protocols are self-stabilizing at no additional cost.