Desktop Grid Research Articles

Towards a Powerful European DCI Based on Desktop Grids

Service Grids like the EGEE Grid can not provide the required number of resources for many VOs. Therefore extending the capacity of these VOs with volunteer or institutional desktop Grids would significantly increase the number of accessible computing resources that can particularly advantageously be exploited in case of parameter sweep applications. This objective has been achieved by the EDGeS project that built a production infrastructure enabling the extension of gLite VOs with several volunteer and institutional desktop Grids. The paper describes the technical solution of integrating service Grids and desktop Grids and, the actual EDGeS production infrastructure. The main objectives and current achievements of the follow-up EDGI project have also been described showing how the existing EDGeS infrastructure can be further extended with clouds.

An architecture for P2P bag-of-tasks execution with multiple task allocation policies in desktop grids

In this paper, we propose and evaluate a flexible architecture for desktop grids that supports multiple task allocation policies on top of a structured P2P overlay. In our proposal, a Bag-of-Tasks application is submitted to random nodes and placed in their local queue, that is processed in a FIFO way. When a node becomes idle, a task allocation policy is executed that fetches tasks from remote nodes. The proposed architecture is flexible since it is decoupled from both the P2P middleware and the P2P overlay. A prototype of the proposed architecture was implemented on top of the JXTA middleware, using the Chord P2P search overlay. The results obtained in a 16-machine heterogeneous desktop grid show that very good performance gains are obtained with multiple task allocation policies. Also, a speedup of 9.85 was achieved for an application composed of 270 network flow balancing tasks, reducing its wallclock execution time from 32.51 min to 3.3 min.

Zorilla: a peer‐to‐peer middleware for real‐world distributed systems

AbstractThe inherent complex nature of current distributed computing architectures hinders the widespread adoption of these systems for mainstream use. In general, users have access to a highly heterogeneous set of compute resources, which may include clusters, grids, desktop grids, clouds, and other compute platforms. This heterogeneity is especially problematic when running parallel and distributed applications. Software is needed which easily combines as many resources as possible into one coherent computing platform.In this paper, we introduce Zorilla: peer‐to‐peer (P2P) middleware that creates a single distributed environment from any available set of compute resources. Zorilla imposes minimal requirements on the resource used, is platform independent, and does not rely on central components. In addition to providing functionality on bare resources, Zorilla can exploit locally available middleware. Zorilla explicitly supports distributed and parallel applications, and allows resources from multiple sites to cooperate in a single computation.Zorilla makes extensive use of both virtualization and P2P techniques. We will demonstrate how virtualization and P2P combine into a simple design, while enhancing functionality and ease of use. Together, these techniques bring our goal a step closer: transparent, easy use of resources, even on very heterogeneous distributed systems. Copyright © 2011 John Wiley & Sons, Ltd.

Statistical Properties of Deformed Single-Crystal Surface under Real-Time Video Monitoring and Processing in the Desktop Grid Distributed Computing Environment

Many crucial processes of damage initiation and propagation take place dynamically in the wide observable time domain from 10-3 seconds to 103 seconds and on the many scales from 10 6 m (solitary defects places) to 10-2 m (correlated linked network of defects). The new multiscale image and video processing method is used to observe and characterize the dynamic behavior of metals under mechanical deformation, record its surface evolution, and process the observed surface statistical parameters. The calculations include many parameters of physical process and video processing parameters (size distribution, anisotropy, localization, scaling parameters, etc.). The typical multiscale analysis of the dynamic video sequence (with 103-106 frames) of deformed metal surface (depicting it time evolution) takes approximately several weeks on a single modern processor. That is why the more powerful computational resources are in extreme need. The free high-performance grid-enabled version of the video processing software is designed and tested in our Desktop Grid (DG) distributed computing infrastructure (DCI).

Green Desktop-Grids: Scientific Impact, Carbon Footprint, Power Usage Efficiency

Desktop Grids take their place in the e-Science distributed computing infrastructure - scaling into the millions of PCs. Desktop-Grids collect CPU cycles from PCs contributed by donors, by volunteers who are willing to support science and research. The Green key advantage of Desktop-Grids over service Grids and data centers based on clusters of servers is the minimal heat density. Compute Clusters without energy intensive aircondition would run into thermal disaster within minutes. PCs participating in Desktop-Grids usually do not make use of any aircondition. In return and with raising energy prices, data centers have implemented lower costs air-conditioning means like e.g. free cooling, improving their thermodynamic-efficiency and PUE rating. This paper, based on [24], investigates whether Desktop-Grids still have a Green advantage over Service-Grids and describe several distinct Green Methodologies to optimize compute unit specific energy consumption. Green-IT metrics as “Carbon Footprint” and PUE are analyzed for their relevance and applied to Desktop-Grids. Pragmatic implementation steps to Green-Desktop-Grids are described.

Using a private desktop grid system for accelerating drug discovery

Drug design is a challenging and computationally intensive task. Chemists are often faced with handling huge amounts of data as well as with working with huge amounts of computational resources. This paper shows that using the BOINC system as a private, dedicated desktop grid a community, like the EU CancerGrid chemist community, can easily set up its own high-end infrastructure based on available and inexpensive desktop computers. The CancerGrid Computing System described in the paper can easily be adapted for the needs of other user communities having similar infrastructure requirements.

Evaluation of hierarchical desktop grid scheduling algorithms

Desktop grids as opposed to service grids are the cost-effective way to gather large amount of volunteer computing resources for solving scientific problems. It is possible to create desktop grids with the help of only one single machine, where volunteer desktops will connect to process work. MTA SZTAKI has created the hierarchical desktop grid concept, where not only single computers, but also desktop grids may join other systems increasing their performance significantly. In the paper we investigate scheduling issues of hierarchical desktop grid systems, present scheduling algorithms focusing on different properties, and compare them using HierDGSim, the Hierarchical Desktop Grid Simulator.

Parallel processing on block-based Gauss-Jordan algorithm for desktop grid

Two kinds of parallel possibilities exist in the block-based Gauss-Jordan (BbGJ) algorithm, which are intra-step and inter-steps based parallelism. But the existing parallel paradigm of BbGJ algorithm just aiming at the intra-step based parallelism, can?t meet the requirement of dispatching simultaneously as many tasks as possible to computing nodes of desktop grid platform exploiting thousands of volunteer computing resources. To overcome the problem described above, this paper presents a hybrid parallel paradigm for desktop grid platform, exploiting all the possible parallelizable parts of the BbGJ algorithm. As well known to us all, volatility is the key issue of desktop grid platform and faults are unavoidable during the process of program execution. So the adapted version of block BbGJ algorithm for desktop grid platform should take the volatility into consideration. To solve the problem presented above, the paper adopts multi-copy distribution strategy and multi-queue based task preemption method to ensure the key tasks can be executed on time, thus ensure the whole tasks can be finished in shorter period of time.

P-GRADE Portal: A generic workflow system to support user communities

P-GRADE portal is an open-source multi-grid portal that supports creation, execution and management of traditional and parameter study workflows on gLite and GT-2 infrastructures. However, some user communities need support for other grid types, like desktop grids or the Amazon EC2 cloud, as well. In this paper we will present the internal architecture of P-GRADE Portal as well as the implementation details of grid support methods. This paper intends to give support for those communities who would like to further develop the portal or create a similar one for other workflow systems.

Decentralized approach to resource availability prediction using group availability in a P2P desktop grid

In a desktop grid model, the job (computational task) is submitted for execution in the resource only when the resource is idle. There is no guarantee that the job which has started to execute in a resource will complete its execution without any disruption from user activity (such as a keyboard stroke or mouse move) if the desktop machines are used for other purposes. This problem becomes more challenging in a Peer-to-Peer (P2P) model for a desktop grid where there is no central server that decides to allocate a job to a particular resource. This paper describes a P2P desktop grid framework that utilizes resource availability prediction, using group availability data. We improve the functionality of the system by submitting the jobs on machines that have a higher probability of being available at a given time. We benchmark our framework and provide an analysis of our results.

Exploiting grid computing, desktop grids and cloud computing for e‐science

PurposeThe purpose of this paper is to examine three different, but related, distributed computing technologies in the context of public‐funded e‐science research, and to present the author's viewpoint on future directions.Design/methodology/approachThe paper takes a critical look at the state‐of‐the‐art with regard to three enabling technologies for e‐science. It forms a set of arguments to support views on the evolution of these technologies in support of the e‐science applications of the future.FindingsAlthough grid computing has been embraced in public‐funded higher education institutions and research centres as an enabler for projects pertaining to e‐science, the adoption of desktop grids is low. With the advent of cloud computing and its promise of on‐demand provisioning of computing resources, it is expected that the conventional form of grid computing will gradually move towards cloud‐based computing. However, cloud computing also brings with it the “pay‐per‐use” economic model, and this may act as stimulus for organisations engaged in e‐science to harvest existing underutilised computation capacity through the deployment of organisation‐wide desktop grid infrastructures. Conventional grid computing will continue to support future e‐science applications, although its growth may remain stagnant.Originality/valueThe paper argues that there will be a gradual shift in the underlying distributed computing technologies that support e‐science applications of the future. While cloud computing and desktop grid computing will gain in prominence, the growth of traditional cluster‐based grid computing may remain dormant.

A taxonomy of peer-to-peer desktop grid paradigms

Desktop grid systems and applications have generated significant impacts on science and engineering. The emerging convergence of grid and peer-to-peer (P2P) computing technologies further opens new opportunities for enabling P2P Desktop Grid systems. This paper presents a taxonomy for classifying P2P desktop grid implementation paradigms, aiming to summarize the state-of-the-art technologies and explore the current and potential solution space. To have a comprehensive taxonomy for P2P desktop grid paradigms, we investigate both computational and data grid systems. Moreover, to ease the understanding, the taxonomy is applied to selected case studies of P2P desktop grid systems. The taxonomy is expected to be used as a survey of the state-of-the-art, a design map, a guideline for novice researchers, a common vocabulary, or a design space for simulation and benchmark, and to be extended as the technologies rapidly evolve.

Parallel pricing algorithms for multi-dimensional Bermudan/American options using Monte Carlo methods

Abstract In this paper we present two parallel Monte Carlo based algorithms for pricing multi-dimensional Bermudan/American options. First approach relies on computation of the optimal exercise boundary while the second relies on classification of continuation and exercise values. We also evaluate the performance of both the algorithms in a desktop grid environment. We show the effectiveness of the proposed approaches in a heterogeneous computing environment, and identify scalability constraints due to the algorithmic structure.

Dynamic scheduling for heterogeneous Desktop Grids

Desktop Grids have emerged as an important methodology to harness the idle cycles of millions of participant desktop PCs over the Internet. However, to effectively utilize the resources of a Desktop Grid, it is necessary to use scheduling policies suitable for such systems. In this paper, we analyze the performance of a policy which is shown to perform well in highly heterogeneous Desktop Grids. The policy utilizes the solution to a linear programming (LP) problem which maximizes system capacity. We suggest robust modifications to address several limitations of the policy.

Parallel SAT Solving on Peer-to-Peer Desktop Grids

Satciety is a distributed parallel satisfiability (SAT) solver which focuses on tackling the domain-specific problems inherent to one of the most challenging environments for parallel computing—Peer-to-Peer Desktop Grids. Satciety efficiently addresses issues related to resource volatility and heterogeneity, limited node and network capabilities, as well as non-uniform communication costs. This is achieved through a sophisticated distributed task pool execution model, problem size reduction through multi-stage SAT formula preprocessing, context-aware memory management, and adaptive topology-aware distributed dynamic learning. Despite the demanding conditions prevailing in Desktop Grids, Satciety achieves considerable speedups compared to state-of-the-art sequential SAT solvers.

The ShareGrid Peer-to-Peer Desktop Grid: Infrastructure, Applications, and Performance Evaluation

Peer-to-Peer (P2P) Desktop Grids are computing infrastructures that aggregate a set of desktop-class machines in which all the participating entities have the same roles, responsibilities, and rights. In this paper, we present ShareGrid, a P2P Desktop Grid infrastructure based on the OurGrid middleware, that federates the resources provided by a set of small research laboratories to easily share and use their computing resources. We discuss the techniques and tools we employed to ensure scalability, efficiency, and usability, and describe the various applications used on it. We also demonstrate the ability of ShareGrid of providing good performance and scalability by reporting the results of experimental evaluations carried out by running various applications with different resource requirements. Our experience with ShareGrid indicates that P2P Desktop Grids can represent an effective answer to the computing needs of small research laboratories, as long as they provide both ease of management and use, and good scalability and performance.

Interoperability of BOINC and EGEE

Today basically two types of grid systems are in use: service grids and desktop grids. Service grids offer an infrastructure for grid users, thus require notable management to keep the service running. On the other hand, desktop grids aim to utilize free CPU cycles of cheap desktop PCs, are easy to set up, but the availability towards users is limited compared to the service grid. The aim of the EDGeS project is to create an integrated infrastructure that combines the advantages of the two grid concepts. A building block of this infrastructure is bridging between the different grid types. In this paper, we first focus on bridging from BOINC-based desktop grids towards EGEE-like service grids, i.e., making desktop grids able to utilize free service grid resources. The solution is based on a generic grid to grid bridge, called as, 3G Bridge. In the second part of the paper we show how the 3G Bridge and EDGeS Bridge services can be used to realize the reverse direction interconnection of BOINC and EGEE grids, i.e., sending EGEE jobs in a user transparent way to BOINC systems that are connected to EGEE VOs. This is the first paper in which we publish the full two-directional bridging between BOINC and EGEE grids.

AVAILABILITY OF UNUSED COMPUTATIONAL RESOURCES IN AN ORDINARY OFFICE ENVIRONMENT

The study presented in this paper highlights an important issue that was subject for discussions and research about a decade ago and now have gained new interest with the current advances of grid computing and desktop grids. New techniques are being invented on how to utilize desktop computers for computational tasks but no other study, to our knowledge, has explored the availability of the said resources. The general assumption has been that there are resources and that they are available. The study is based on a survey on the availability of resources in an ordinary office environment. The aim of the study was to determine if there are truly usable under-utilized networked desktop computers available for non-desktop tasks during the off-hours. We found that in more than 96% of the cases the computers in the current investigation was available for the formation of part-time (night and weekend) computer clusters. Finally we compare the performance of a full time and a metamorphosic cluster, based on one hypothetical linear scalable application and a real world welding simulation.

Characterizing fault tolerance in genetic programming

Evolutionary algorithms, including genetic programming (GP), are frequently employed to solve difficult real-life problems, which can require up to days or months of computation. An approach for reducing the time-to-solution is to use parallel computing on distributed platforms. Large platforms such as these are prone to failures, which can even be commonplace events rather than rare occurrences. Thus, fault tolerance and recovery techniques are typically necessary. The aim of this article is to show the inherent ability of parallel GP to tolerate failures in distributed platforms without using any fault-tolerant technique. This ability is quantified via simulation experiments performed using failure traces from real-world distributed platforms, namely, desktop grids, for two well-known problems.

A decentralized and fault‐tolerant Desktop Grid system for distributed applications

AbstractThis paper proposes a decentralized and fault‐tolerant software system for the purpose of managing Desktop Grid resources. Its main design principle is to eliminate the need for a centralized server, therefore to remove the single point of failure and bottleneck of existing Desktop Grids. Instead, each node can play alternatively the role of client or server. Our main contribution is to design the PastryGrid protocol (based on Pastry) for Desktop Grid in order to support a wider class of applications, especially the distributed application with precedence between tasks. Compared with a centralized system, we evaluate our approach over 205 machines executing 2500 tasks. The results we obtain show that our decentralized system outperforms XtremWeb‐CH which is configured as a master/slave, with respect to the turnaround time. Copyright © 2009 John Wiley & Sons, Ltd.