Abstract
Abstract Cycle sharing over the Internet has increased in popularity during the last decade, with increasingly powerful machines being made available to existing projects. In this paper, we present GiGi-MR, a framework that allows non-expert users to run CPU-intensive jobs on top of volunteer resources over the Internet. GiGi-MR has several distinctive features: it allows non-expert users to easily partition their jobs in several parallel tasks; such Bag-of-Tasks (BoT) are executed in parallel as a set of MapReduce applications; the volunteer resources that provide the best match for the tasks being executed are chosen (using attenuated bloom filters); it provides a portable checkpointing fault-tolerance mechanism based on virtualization; it does not rely exclusively on a central server (or servers) at all times (thus minimizing the bottleneck effect); it deals with malicious participants (possibly byzantine) using an efficient partial replication mechanism to validate the results obtained; and it is compatible with BOINC (one of the most popular open-source software platforms for computing using volunteered resources). We describe GiGi-MR’s architecture and evaluate its performance by executing several MapReduce applications on a wide area testbed. Furthermore, we use micro-benchmarks to assess each one of GiGi-MR’s components independently. The system’s overhead is minimal. When compared to an unmodified volunteer computing system, GiGi-MR obtains a performance increase of over 60 % in application turnaround time, while reducing the bandwidth used by an order of magnitude.
Highlights
The use of volunteer PCs across the Internet to execute distributed applications has been increasing in popularity since its inception in the early 1990s, with the creation of projects such as Distributed.net,1 Seti@home [3] or Folding@home [19]
We evaluate the performance of GiGi-MR in terms of application turnaround and network use by running several tests over the Internet, in a scenario that resembles a typical Volunteer Computing (VC) environment
We have presented GiGi-MR, a Volunteer Computing platform that allows ordinary users to create and submit jobs for execution on volunteer machines over the Internet
Summary
The use of volunteer PCs across the Internet to execute distributed applications has been increasing in popularity since its inception in the early 1990s, with the creation of projects such as Distributed.net, Seti@home [3] or Folding@home [19]. These Volunteer Computing (VC) systems harness computing resources from machines running commodity hardware and software, and perform highly parallel computations, called Bag-of-Tasks (BoT), that do not require any interaction between network participants. To achieve fault tolerance during task execution, developers must modify their application code and insert explicit checkpoints Users not satisfying these requirements cannot take advantage of available remote cycles.
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
