Message-passing Model Research Articles

Asynchronous message-passing distributed algorithm for the global critical section problem

This paper considers the global $(l,k)$-CS problem which is the problem of controlling the system in such a way that, at least $l$ and at most $k$ processes must be in the CS at a time in the network. In this paper, a distributed solution is proposed in the asynchronous message-passing model. Our solution is a versatile composition method of algorithms for $l$-mutual inclusion and $k$-mutual exclusion. Its message complexity is $O(|Q|)$, where $|Q|$ is the maximum size for the quorum of a coterie used by the algorithm, which is typically $|Q| = \sqrt{n}$.

Komputasi Paralel Menggunakan Model Message Passing Pada SIM RS (Sistem Informasi Manajemen Rumah Sakit)

The growth of technology will impact to the growth of data beyond limits of database management tools. One of the system is Information Management System for Hospital it’s a high complexity problem solving method related to the load of data. Parallel computing is one of the technique that been used in HPC. The focus of this research will be emphasized to the design of parallel computing using message – passing model as a search system process in Information Management System for Hospital to find patient data. The design of parallel computing will be done in a way to shared computing data to a number of CPU (Master and Slave), the parallel computing configuration used on CPU (Master and Slave) will be using some stage FOSTER method that are partition, communication, aglomerasi and mapping. The test will be done computing data processing time between sequential and parallel time. Parallel computing design will be tested using speedup and efficiency calculation. The result of designing and testing parallel computing using message-passing model proved the patient data processing speed using parallel program is capable to overcome 1 CPU using sequential network topologi. On speed up method , the test indicate an increase on data transfer speed up using 3 parallel computing CPU. While using efficiency testing method, the efficiency point reached its peak when using 2 and 3 CPU. The Occurrence of decrease on speedup and efficiency point were caused by the minimal amount of data if handled by 7 parallel computing CPU. The Conclusion for this method is, the increase amount of CPU involved in the data processing using parallel computing is not proportional to the amount of time to processing data itself. It happened because every data processing task in terms of the amount of data that’s handled have an ideal amount of CPU limits to do the task itself.

Message lower bounds via efficient network synchronization

We present a uniform approach to derive message-time tradeoffs and message lower bounds for synchronous distributed computations using results from communication complexity theory.Since the models used in the classical theory of communication complexity are inherently asynchronous, lower bounds do not directly apply in a synchronous setting. To address this issue, we show a general result called Synchronous Simulation Theorem (SST) which allows to obtain message lower bounds for synchronous distributed computations by leveraging lower bounds on communication complexity. The SST is a by-product of a new efficient synchronizer for complete networks, called σ, which has simulation overheads that are only logarithmic in the number of synchronous rounds with respect to both time and message complexity, even in networks with limited bandwidth. Synchronizer σ is particularly efficient in simulating synchronous algorithms which employ silence, a situation that occurs when in some round no processor sends any message. In particular, a curious property of this synchronizer, which sets it apart from its predecessors, is that it is time-compressing, and hence in some cases it may result in a simulation that is faster than the original execution.While the SST gives near-optimal message lower bounds up to large values of the number of allowed synchronous rounds r (usually polynomial in the size of the input), it fails to provide meaningful bounds when the synchronous algorithm to be simulated may comprise a very large number of rounds. To complement the bounds provided by the SST, we then derive message lower bounds for the synchronous message-passing model that are unconditional, that is, independent of r, by establishing novel lower bounds for multi-party synchronous communication complexity.We apply our approach to show (almost) tight message-time tradeoffs and message lower bounds for several fundamental problems in the synchronous message-passing model of distributed computation. These include sorting, matrix multiplication, and several graph problems. All these lower bounds hold for any distributed algorithms, including randomized Monte Carlo algorithms.

Fast Distributed Algorithms for Connectivity and MST in Large Graphs

Motivated by the increasing need to understand the algorithmic foundations of distributed large-scale graph computations, we study a number of fundamental graph problems in a message-passing model for distributed computing where k ≥2 machines jointly perform computations on graphs with n nodes (typically, n > k ). The input graph is assumed to be initially randomly partitioned among the k machines, a common implementation in many real-world systems. Communication is point-to-point, and the goal is to minimize the number of communication rounds of the computation. Our main result is an (almost) optimal distributed randomized algorithm for graph connectivity. Our algorithm runs in Õ( n / k 2 ) rounds (Õ notation hides a polylog( n ) factor and an additive polylog( n ) term). This improves over the best previously known bound of Õ( n / k ) [Klauck et al., SODA 2015] and is optimal (up to a polylogarithmic factor) in light of an existing lower bound of Ω ˜ ( n / k 2 ). Our improved algorithm uses a bunch of techniques, including linear graph sketching, that prove useful in the design of efficient distributed graph algorithms. Using the connectivity algorithm as a building block, we then present fast randomized algorithms for computing minimum spanning trees, (approximate) min-cuts, and for many graph verification problems. All these algorithms take Õ( n / k 2 ) rounds and are optimal up to polylogarithmic factors. We also show an almost matching lower bound of Ω ˜ ( n / k 2 ) rounds for many graph verification problems by leveraging lower bounds in random-partition communication complexity.

Communication-efficient randomized consensus

We consider the problem of consensus in the challenging classic model. In this model, the adversary is adaptive; it can choose which processors crash at any point during the course of the algorithm. Further, communication is via asynchronous message passing: there is no known upper bound on the time to send a message from one processor to another, and all messages and coin flips are seen by the adversary. We describe a new randomized consensus protocol with expected message complexity O( n^2 log ^2 n ) when fewer than n / 2 processes may fail by crashing. This is an almost-linear improvement over the best previously known protocol, and within logarithmic factors of a known Omega ( n^2 ) message lower bound. The protocol further ensures that no process sends more than O( n log ^3 n ) messages in expectation, which is again within logarithmic factors of optimal. We also present a generalization of the algorithm to an arbitrary number of failures t, which uses expected O( n t + t^2 log ^{2} t ) total messages. Our approach is to build a message-efficient, resilient mechanism for aggregating individual processor votes, implementing the message-passing equivalent of a weak shared coin. Roughly, in our protocol, a processor first announces its votes to small groups, then propagates them to increasingly larger groups as it generates more and more votes. To bound the number of messages that an individual process might have to send or receive, the protocol progressively increases the weight of generated votes. The main technical challenge is bounding the impact of votes that are still “in flight” (generated, but not fully propagated) on the final outcome of the shared coin, especially since such votes might have different weights. We achieve this by leveraging the structure of the algorithm, and a technical argument based on martingale concentration bounds. Overall, we show that it is possible to build an efficient message-passing implementation of a shared coin, and in the process (almost-optimally) solve the classic consensus problem in the asynchronous message-passing model.

Scalable Failure Masking for Stencil Computations using Ghost Region Expansion and Cell to Rank Remapping

In order to achieve exascale systems, application resilience needs to be addressed. Some programming models, such as task-DAG (directed acyclic graphs) architectures, currently embed resilience features whereas traditional SPMD (single program, multiple data) and message-passing models do not. Since a large part of the community's code base follows the latter models, it is still required to take advantage of application characteristics to minimize the overheads of fault tolerance. To that end, this paper explores how recovering from hard process/node failures in a local manner is a natural approach for certain applications to obtain resilience at lower costs in faulty environments. In particular, this paper targets enabling online, semitransparent local recovery for stencil computations on current leadership-class systems as well as presents programming support and scalable runtime mechanisms. Also described and demonstrated in this paper is the effect of failure masking, which allows the effective reduct...

Partition Expanders

We introduce a new concept, which we call partition expanders. The basic idea is to study quantitative properties of graphs in a slightly different way than it is in the standard definition of expanders. While in the definition of expanders it is required that the number of edges between any pair of sufficiently large sets is close to the expected number, we consider partitions and require this condition only for most of the pairs of blocks. As a result, the blocks can be substantially smaller. We show that for some range of parameters, to be a partition expander a random graph needs exponentially smaller degree than any expander would require in order to achieve similar expanding properties. We apply the concept of partition expanders in communication complexity. First, we construct an optimal pseudo-random generator (PRG) for the Simultaneous Message Passing (SMP) model: it needs n + log k random bits against protocols of cost Ω(k). Second, we compare the SMP model to that of Simultaneous Two-Way Communication, and give a new separation that is stronger both qualitatively and quantitatively than the previously known ones.

A fast network-decomposition algorithm and its applications to constant-time distributed computation

A partition (C1,C2,…,Cq) of G=(V,E) into clusters of strong (respectively, weak) diameter d, such that the supergraph obtained by contracting each Ci is ℓ-colorable is called a strong (resp., weak) (d,ℓ)-network-decomposition. Network-decompositions were introduced in a seminal paper by Awerbuch, Goldberg, Luby and Plotkin in 1989. Awerbuch et al. showed that strong (d,ℓ)-network-decompositions with d=ℓ=exp⁡{O(log⁡nlog⁡log⁡n)} can be computed in distributed deterministic time O(d). Even more importantly, they demonstrated that network-decompositions can be used for a great variety of applications in the message-passing model of distributed computing.The result of Awerbuch et al. was improved by Panconesi and Srinivasan in 1992: in the latter result d=ℓ=exp⁡{O(log⁡n)}, and the running time is O(d) as well. In another remarkable breakthrough Linial and Saks (in 1992) showed that weak (O(log⁡n),O(log⁡n))-network-decompositions can be computed in distributed randomized time O(log2⁡n). Much more recently Barenboim (2012) devised a distributed randomized constant-time algorithm for computing strong network decompositions with d=O(1). However, the parameter ℓ in his result is O(n1/2+ϵ).In this paper we drastically improve the result of Barenboim and devise a distributed randomized constant-time algorithm for computing strong (O(1),O(nϵ))-network-decompositions. As a corollary we derive a constant-time randomized O(nϵ)-approximation algorithm for the distributed minimum coloring problem, improving the previously best-known O(n1/2+ϵ) approximation guarantee. We also derive other improved distributed algorithms for a variety of problems.Most notably, for the extremely well-studied distributed minimum dominating set problem currently there is no known deterministic polylogarithmic-time algorithm. We devise a deterministic polylogarithmic-time approximation algorithm for this problem, addressing an open problem of Lenzen and Wattenhofer (2010).

Lower Bounds for Number-in-Hand Multiparty Communication Complexity, Made Easy

In this paper we prove lower bounds on randomized multiparty communication complexity, mainly in the message-passing model, where messages are sent player-to-player. Some of our results apply to the blackboard model, where each message is written on a blackboard for all players to see. We introduce a new technique for proving such bounds, called symmetrization, which is natural, intuitive, and often easy to use. For example, for the problem where each of $k$ players gets a bit-vector of length $n$, and the goal is to compute the coordinatewise XOR of these vectors, we prove a tight lower bound of $\Omega(nk)$ in the blackboard model. For the same problem with AND instead of XOR, we prove a lower bound of roughly $\Omega(nk)$ in the message-passing model (assuming $k \le n/3200$) and $\Omega(n \log k)$ in the blackboard model. We also prove lower bounds for bitwise majority, for a graph-connectivity problem, and for other problems; the technique seems applicable to a wide range of other problems as well. All of our lower bounds allow randomized communication protocols with two-sided error. We also use the symmetrization technique to prove several direct-sum-like results for multiparty communication.

RDA: A Coq Library to Reason about Randomised Distributed Algorithms in the Message Passing Model

RDA: A Coq Library to Reason about Randomised Distributed Algorithms in the Message Passing Model

Mutual inclusion in asynchronous message-passing distributed systems

In the mutual inclusion problem, at least one process is in the critical section. However, only a solution for two processes with semaphores has been reported previously. In this study, a generalized problem setting is formalized and two distributed solutions are proposed based on an asynchronous message-passing model. In the local problem setting (the local mutual inclusion problem), for each process P, at least one of P and its neighbors must be in the critical section. For the local problem setting, a solution is proposed with O(Δ) message complexity, where Δ is the maximum degree (number of neighboring processes) of a network. In a global setting (the global mutual inclusion problem), at least one of the processes must be in the critical section. For the global problem setting, a solution is proposed with O(|Q|) message complexity, where |Q| is the maximum size for the quorum of a coterie used by the algorithm, which is typically |Q|=n, where n is the number of processes in a network.

Distributed Learning of Distributions via Social Sampling

A protocol for distributed estimation of discrete distributions is proposed. Each agent begins with a single sample from the distribution, and the goal is to learn the empirical distribution of the samples. The protocol is based on a simple message-passing model motivated by communication in social networks. Agents sample a message randomly from their current estimates of the distribution, resulting in a protocol with quantized messages. Using tools from stochastic approximation, the algorithm is shown to converge almost surely. Examples illustrate three regimes with different consensus phenomena. Simulations demonstrate this convergence and give some insight into the effect of network topology.

Steiner Shallow-Light Trees Are Exponentially Lighter than Spanning Ones

For a pair of parameters $\alpha,\beta \ge 1$, a spanning tree $T$ of a weighted undirected $n$-vertex graph $G = (V,E,w)$ is called an $(\alpha,\beta)$-shallow-light tree (shortly, $(\alpha,\beta)$-SLT) of $G$ with respect to a designated vertex $rt \in V$ if (1) it approximates all distances from $rt$ to the other vertices up to a factor of $\alpha$, and (2) its weight is at most $\beta$ times the weight of the minimum spanning tree $MST(G)$ of $G$. The parameter $\alpha$ (resp., $\beta$) is called the root-distortion (resp., lightness) of the tree $T$. Shallow-light trees (SLTs) constitute a fundamental graph structure, with numerous theoretical and practical applications. In particular, they were used for constructing spanners in network design, for VLSI-circuit design, for various data gathering and dissemination tasks in wireless and sensor networks, in overlay networks, and in the message-passing model of distributed computing. Tight tradeoffs between the parameters of SLTs were established by Awerbuch, Baratz, and Peleg [Proceedings of the 9th Annual ACM Symposium on Principles of Distributed Computing (PODC), 1990, pp. 177--187, Efficient Broadcast and Light-Weight Spanners, manuscript, 1991] and Khuller, Raghavachari, and Young [Proceedings of the Fourth Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), 1993, pp. 243--250]. They showed that for any $\epsilon > 0$ there always exist $(1+\epsilon,O(\frac{1}{\epsilon}))$-SLTs and that the upper bound $\beta = O(\frac{1}{\epsilon})$ on the lightness of SLTs cannot be improved. In this paper we show that using Steiner points one can build SLTs with logarithmic lightness, i.e., $\beta = O(\log \frac{1}{\epsilon})$. This establishes an exponential separation between spanning SLTs and Steiner ones. In the regime $\epsilon = 0$ our construction provides a shortest-path tree with weight at most $O(\log n) \cdot w(MST(G))$. Moreover, we prove matching lower bounds that show that all our results are tight up to constant factors.

A simple characterization of asynchronous computations

In this paper we investigate synchronous message-passing systems with reliable processors but different scenarios of message loss. In particular, we investigate and present the minimum set of messages whose delivery must be guaranteed to ensure the equivalence of this model to asynchronous wait–free shared-memory. Delivery guarantees were investigated in the past at the other extreme – delivery guarantees that ensure consensus.Because message failure is a more refined type of fault than processor crash failure, we were able to use in this paper an extremely simple message-passing model to characterize exactly what is computable in wait–free read–write shared memory. We use a synchronous complete network where in each round a subset from a defined family of subsets of messages, must be successfully delivered. With this model we obtain an extremely simple derivation of the Herlihy–Shavit condition that equates the wait–free read–write model with a subdivided-simplex. We show how each step in the computation inductively takes a subdivided-simplex and further subdivides it in the simplest way possible, making the characterization of read–write wait–free widely accessible.

Partition consistency

This paper provides a case study of specifying an abstract memory consistency model, providing possible implementations for the model, and proving the correctness of implementations. Specifically, we introduce a class of memory consistency models called partition consistency. Existing abstract consistency models such as sequential consistency, piplined-RAM, Goodman's processor consistency, and coherence are all members of the partition consistency class. A concrete message-passing network model is also specified. Implementations of partition consistency on this network model are then presented and proved correct. A middle level of abstraction is utilized to facilitate the proofs. All three levels of abstraction are specified using the same framework. The paper aims to illustrate a general methodology and techniques for specifying memory consistency models and proving the correctness of their implementations.

When distributed computation is communication expensive

We consider a number of fundamental statistical and graph problems in the message-passing model, where we have \(k\) machines (sites), each holding a piece of data, and the machines want to jointly solve a problem defined on the union of the \(k\) data sets. The communication is point-to-point, and the goal is to minimize the total communication among the \(k\) machines. This model captures all point-to-point distributed computational models with respect to minimizing communication costs. Our analysis shows that exact computation of many statistical and graph problems in this distributed setting requires a prohibitively large amount of communication, and often one cannot improve upon the communication of the simple protocol in which all machines send their data to a centralized server. Thus, in order to obtain protocols that are communication-efficient, one has to allow approximation, or investigate the distribution or layout of the data sets.

A GRASS GIS parallel module for radio-propagation predictions

Geographical information systems are ideal candidates for the application of parallel programming techniques, mainly because they usually handle large data sets. To help us deal with complex calculations over such data sets, we investigated the performance constraints of a classic master–worker parallel paradigm over a message-passing communication model. To this end, we present a new approach that employs an external database in order to improve the calculation–communication overlap, thus reducing the idle times for the worker processes. The presented approach is implemented as part of a parallel radio-coverage prediction tool for the Geographic Resources Analysis Support System (GRASS) environment. The prediction calculation employs digital elevation models and land-usage data in order to analyze the radio coverage of a geographical area. We provide an extended analysis of the experimental results, which are based on real data from an Long Term Evolution (LTE) network currently deployed in Slovenia. Based on the results of the experiments, which were performed on a computer cluster, the new approach exhibits better scalability than the traditional master–worker approach. We successfully tackled real-world-sized data sets, while greatly reducing the processing time and saturating the hardware utilization.

A Distributed Rule Engine Based on Message-Passing Model to Deal with Big Data

 Abstract—Rule engine is a good way of knowledge representation and inference. It has been extensively studied and used in artificial intelligence. However, because of its low computational efficiency and the limitation of single machine's capacity, it cannot deal well with big data. In order to address this problem, we propose a distributed implementation of the rule engine based on message-passing model. It is designed to make use of multiple machines in a computing cluster to deal with a large amount of data in a parallel and distributed way. This paper not only describes details of the design and implementation of it, but also shows its high performance through several experiments.

Energy-Aware Self-Stabilizing Distributed Clustering Protocol for Ad Hoc Networks: the case of WSNs

In this paper, we present an Energy-Aware Self-Stabilizing Distributed Clustering protocol based on message-passing model for Ad Hoc networks. The latter does not require any initialization. Starting from an arbitrary configuration, the network converges to a stable state in a finite time. Our contribution is twofold. We firstly give the formal proof that the stabilization is reached after at most transitions and requires at most n*log(2n+k+3) memory space, where n is the number of network nodes and k represents the maximum hops number in the clusters. Furthermore, using the OMNeT++ simulator, we perform an evaluation of our approach. Secondly, we propose an adaptation of our solution in the context of Wireless Sensor Networks (WSNs) with energy constraint. We notably show that our protocol can be easily used for constructing clusters according to multiple criteria in the election of cluster-heads, such as nodes` identity, residual energy or degree. We give a comparison under the different election metrics by evaluating their communication cost and energy consumption. Simulation results show that in terms of number of exchanged messages and energy consumption, it is better to use the Highest-ID metric for electing CHs.

Distributed Graph Coloring: Fundamentals and Recent Developments

The focus of this monograph is on symmetry breaking problems in the message-passing model of distributed computing. In this model a communication network is represented by a n-vertex graph G = (V,E), whose vertices host autonomous processors. The processors communicate over the edges of G in discrete rounds. The goal is to devise algorithms that use as few rounds as possible. A typical symmetry-breaking problem is the problem of graph coloring. Denote by [delta] the maximum degree of G. While coloring G with [delta]+ 1 colors is trivial in the centralized setting, the problem becomes much more challenging in the distributed one. One can also compromise on the number of colors, if this allows for more efficient algorithms. Other typical symmetry-breaking problems are the problems of computing a maximal independent set (MIS) and a maximal matching (MM). The study of these problems dates back to the very early days of distributed computing. The founding fathers of distributed computing laid firm foundations for the area of distributed symmetry breaking already in the eighties. In particular, they showed that all these problems can be solved in randomized logarithmic time. Also, Linial showed that an O([delta]2)-coloring can be solved very efficiently deterministically. However, fundamental questions were left open for decades. In particular, it is not known if the MIS or the ([delta] + 1)-coloring can be solved in deterministic polylogarithmic time. Moreover, until recently it was not known if in deterministic polylogarithmic time one can color a graph with significantly fewer than [delta]2 colors. Additionally, it was open (and still open to some extent) if one can have sublogarithmic randomized algorithms for the symmetry breaking problems. Recently, significant progress was achieved in the study of these questions. More efficient deterministic and randomized ([delta] + 1)-coloring algorithms were achieved. Deterministic [delta]1 + o(1)-coloring algorithms with polylogarithmic running time were devised. Improved (and often sublogarithmic-time) randomized algorithms were devised. Drastically improved lower bounds were given. Wide families of graphs in which these problems are solvable much faster than on general graphs were identified. The objective of our monograph is to cover most of these developments, and as a result to provide a treatise on theoretical foundations of distributed symmetry breaking in the message-passing model. We hope that our monograph will stimulate further progress in this exciting area. Table of Contents: Acknowledgments / Introduction / Basics of Graph Theory / Basic Distributed Graph Coloring Algorithns / Lower Bounds / Forest-Decomposition Algorithms and Applications / Defective Coloring / Arbdefective Coloring / Edge-Coloring and Maximal Matching / Network Decompositions / Introduction to Distributed Randomized Algorithms / Conclusion and Open Questions / Bibliography / Authors' Biographies