XtremLab: A System for Characterizing Internet Desktop Grids

Abstract

Desktop grid (DG) systems use the idle computing power of many volunteered desktop PC's on the Internet to support large-scale computation and storage. For over a decade, DG systems have been the largest and the most powerful distributed computing systems, offering a plethora of computing power at a fraction of the cost of supercomputers. The volunteer desktops participating in DG projects are volatile and heterogeneous, but there is little detailed information about their volatility and heterogeneity. Yet this characterization is essential for the simulation and modelling of such systems. We are conducting a project whose goal is to obtain a detailed picture of the DG landscape. To this end, we are deploying resource sensors on the volunteer desktops to measure the availability of CPU, disk space, memory, and network bandwidth. An initial version of the project has been deployed and currently has about 1,800 participating hosts. The resulting resource measurement data and characterization will be useful for a broad range of research areas, including distributed and peer-to-peer computing, and fault tolerance