Scalability tests of R-GMA based grid job monitoring system for CMS Monte Carlo data production

Abstract

High Energy Physics experiments such as CMS (Compact Muon Solenoid) at the Large Hadron Collider have unprecedented, large-scale data processing computing requirements, with data accumulating at around 1 Gbyte/s. The Grid distributed computing paradigm has been chosen as the solution to provide the requisite computing power. The demanding nature of CMS software and computing requirements, such as the production of large quantities of Monte Carlo simulated data, makes them an ideal test case for the Grid and a major driver for the development of Grid technologies. One important challenge when using the Grid for large-scale data analysis is the ability to monitor the large numbers of jobs that are being executed simultaneously at multiple remote sites. R-GMA is a monitoring and information management service for distributed resources based on the Grid Monitoring Architecture of the Global Grid Forum. In this paper we report on the first measurements of R-GMA as part of a monitoring architecture to be used for batch submission of multiple CMS Monte Carlo simulation jobs running on the CMS-LCGO Grid test-bed. Monitoring information was transferred in real time from remote executing nodes back to the submitting host and stored in a database. Scalability tests were undertaken whereby the job submission rate was ramped up to rates comparable with those expected in a full-scale production.