A Data Space environment for big data and ML-based climate applications in the European Open Science Cloud

Abstract

Several scientific fields, including climate science, have undergone radical changes in the last years due to the increase in the data volumes and the emergence of data science and Machine Learning (ML) approaches. In this context, providing fast data access and analytics has become of paramount importance. The data space concept has emerged to address some of the key challenges and support scientific communities  towards a more sustainable and FAIR use of data. The ENES Data Space (EDS) represents a domain-specific example of this concept for the climate community developed under the umbrella of the European Open Science Cloud (EOSC) initiative by the European Commission. EDS provides an open, scalable, cloud-enabled data science environment for climate data analysis on top of the EOSC Compute Platform made available through a user-friendly JupyterLab GUI. The service integrates into a single environment climate datasets from well-known initiatives (e.g., CMIP6), compute resources, data science and ML tools. It was launched in the context of the EGI-ACE project, it is accessible through the EOSC Catalogue and Marketplace (https://marketplace.eosc-portal.eu/services/enes-data-space) and it also provides a web portal (https://enesdataspace.vm.fedcloud.eu) including information, tutorials and training materials. It has recently been added to the Data Spaces radar, an initiative launched by the IDSA (International Data Space Association) with the main goal of mapping data spaces from several domains into one easy-to-use tool. The EDS is being employed in climate applications targeting big data processing, interactive analytics and visualization, and recently it has been extended to support more advanced scientific applications based on ML. In particular, it has been enhanced to provide a cloud-based development and testing environment for the implementation of some data-driven Digital Twin applications for extreme climate events in the context of the interTwin project. Finally, the ENES Data Space will be also one of the pilots in EOSC Beyond, a Horizon Europe project where the EDS will integrate and validate the new EOSC Core capabilities developed by the project. This work has been supported in part by interTwin; interTwin is funded by the European Union (Horizon Europe) under grant agreement No 101058386.