The DO Experiment Data Grid—SAM

Abstract

SAM(Sequential Access through Meta-data) is a data grid and data cataloging system developed for the DO high energy physics (HEP) experiment at Fermilab. Since March 2001, DO has been acquiring data in real time from the detector and will archive up to 1/2 Petabyte a year of simulated, raw detector and processed physics data. SAM catalogs the event and calibration data, provides distributed file delivery and caching services, and manages the processing and analysis jobs for the hundreds of DO collaborators around the world. The DO applications are data-intensive and the physics analysis programs execute on the order of 1-1000 cpuseconds per 250KByte of data. SAM manages the transfer of data between the archival storage systems through the globally distributed disk caches and delivers the data files to the users batch and interactive jobs. Additionally, SAM handles the user job requests and execution scheduling, and manages the use of the available compute, storage and network resources to implement experiment resource allocation policies. DO has been using early versions of the SAM system for two years for the management of the simulation and test data. The system is in production use with round the clock support. DO is a participant in the Particle Physics Data Grid (PPDG) project. Aspects of the ongoing SAM developments are in collaboration with the computer science groups and other experiments on PPDG. The DO emphasis is to develop the more sophisticated global grid job, resource management, authentication and information services needed to fully meet the needs of the experiment during the next 6 years of data taking and analysis.KeywordsCache ServiceCache Replacement PolicyDisk CacheMass Storage SystemMetadata CatalogThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.