Abstract
This paper provides a detailed overview of the Unified Monitoring Architecture (UMA) that aims at merging the monitoring of the CERN IT data centres and the WLCG monitoring using common and widely-adopted open source technologies such as Flume, Elasticsearch, Hadoop, Spark, Kibana, Grafana and Zeppelin. It provides insights and details on the lessons learned, explaining the work performed in order to monitor the CERN IT data centres and the WLCG computing activities such as the job processing, data access and transfers, and the status of sites and services.
Highlights
For over a decade, the Large Hadron Collider (LHC) [1] experiments have been relying on advanced and specialised Worldwide LHC Computing Grid (WLCG) [2] dashboards [3] for monitoring, visualising and reporting the computing activities of the LHC across its distributed grid and cloud resources.In the recent years, in order to cope with the increase of volume and variety of the resources, the WLCG monitoring started to evolve [4][5] towards data analytics technologies such as Elasticsearch [6], Apache Hadoop [7] and Apache Spark [8]
In order to cope with the increase of volume and variety of the resources, the WLCG monitoring started to evolve [4][5] towards data analytics technologies such as Elasticsearch [6], Apache Hadoop [7] and Apache Spark [8]
This paper covers the work performed, starting from early 2016, that resulted in the definition, the development and the deployment of a Unified Monitoring Architecture (UMA) aiming at satisfying the requirements to collect, transport, store, search, process and visualise both IT and WLCG monitoring data
Summary
- Monitoring of IaaS and scientific applications on the Cloud using the Elasticsearch ecosystem S Bagnasco, D Berzano, A Guarise et al. - WLCG Monitoring Consolidation and further evolution P Saiz, A Aimar, J Andreeva et al. - Processing of the WLCG monitoring data using NoSQL J Andreeva, A Beche, S Belov et al. CHEP IOP Conf. A Aimar, A Aguado Corman, P Andrade, S Belov, J Delgado Fernandez, B Garrido Bear, M Georgiou, E Karavakis, L Magnoni, R Rama Ballesteros, H Riahi, J Rodriguez Martinez, P Saiz and D Zolnai
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
