Earth System Grid Federation Research Articles

FIO-ESM v2.0 Outputs for the CMIP6 Global Monsoons Model Intercomparison Project Experiments

Three tiers of experiments in the Global Monsoons Model Intercomparison Project (GMMIP), one of the endorsed model intercomparison projects of phase 6 of the Coupled Model Intercomparison Project (CMIP6), are implemented by the First Institute of Oceanography Earth System Model version 2 (FIO-ESM v2.0), following the GMMIP protocols. Evaluation of global mean surface air temperature from 1870 to 2014 and climatological precipitation (1979–2014) in tier-1 shows that the atmosphere model of FIO-ESM v2.0 can reproduce the basic observed atmospheric features. In tier-2, the internal variability is captured by the coupled model, with the SST restoring to the model climatology plus the observed anomalies in the tropical Pacific and North Atlantic. Simulation of the Northern Hemisphere summer monsoon circulation is significantly improved by the SST restoration in the North Atlantic. In tier-3, five orographic perturbation experiments are conducted covering the period 1979–2014 by modifying the surface elevation or vertical heating in the prescribed region. In particular, the strength of the South Asian summer monsoon is reduced by removing the topography or thermal forcing above 500 m over the Asian continent. Monthly and daily simulated outputs of FIO-ESM v2.0 are provided through the Earth System Grid Federation (ESGF) node to contribute to a better understanding of the global monsoon system.

CAS-FGOALS Datasets for the Two Interglacial Epochs of the Holocene and the Last Interglacial in PMIP4

Two versions of the Chinese Academy of Sciences Flexible Global Ocean-Atmosphere-Land System model (CAS-FGOALS), version f3-L and g3, are used to simulate the two interglacial epochs of the mid-Holocene and the Last Interglacial in phase 4 of the Paleoclimate Modelling Intercomparison Project (PMIP4), which aims to study the impact of changes in orbital parameters on the Earth’s climate. Following the PMIP4 experimental protocols, four simulations for the mid-Holocene and two simulations for the Last Interglacial have been completed, and all the data, including monthly and daily outputs for the atmospheric, oceanic, land and sea-ice components, have been released on the Earth System Grid Federation (ESGF) node. These datasets contribute to PMIP4 and CMIP6 (phase 6 of the Coupled Model Intercomparison Project) by providing the variables necessary for the two interglacial periods. In this paper, the basic information of the CAS-FGOALS models and the protocols for the two interglacials are briefly described, and the datasets are validated using proxy records. Results suggest that the CAS-FGOALS models capture the large-scale changes in the climate system in response to changes in solar insolation during the interglacial epochs, including warming in mid-to-high latitudes, changes in the hydrological cycle, the seasonal variation in the extent of sea ice, and the damping of interannual variabilities in the tropical Pacific. Meanwhile, disagreements within and between the models and the proxy data are also presented. These datasets will help the modeling and the proxy data communities with a better understanding of model performance and biases in paleoclimate simulations.

Observations for Model Intercomparison Project (Obs4MIPs): status for CMIP6

Abstract. The Observations for Model Intercomparison Project (Obs4MIPs) was initiated in 2010 to facilitate the use of observations in climate model evaluation and research, with a particular target being the Coupled Model Intercomparison Project (CMIP), a major initiative of the World Climate Research Programme (WCRP). To this end, Obs4MIPs (1) targets observed variables that can be compared to CMIP model variables; (2) utilizes dataset formatting specifications and metadata requirements closely aligned with CMIP model output; (3) provides brief technical documentation for each dataset, designed for nonexperts and tailored towards relevance for model evaluation, including information on uncertainty, dataset merits, and limitations; and (4) disseminates the data through the Earth System Grid Federation (ESGF) platforms, making the observations searchable and accessible via the same portals as the model output. Taken together, these characteristics of the organization and structure of obs4MIPs should entice a more diverse community of researchers to engage in the comparison of model output with observations and to contribute to a more comprehensive evaluation of the climate models. At present, the number of obs4MIPs datasets has grown to about 80; many are undergoing updates, with another 20 or so in preparation, and more than 100 are proposed and under consideration. A partial list of current global satellite-based datasets includes humidity and temperature profiles; a wide range of cloud and aerosol observations; ocean surface wind, temperature, height, and sea ice fraction; surface and top-of-atmosphere longwave and shortwave radiation; and ozone (O3), methane (CH4), and carbon dioxide (CO2) products. A partial list of proposed products expected to be useful in analyzing CMIP6 results includes the following: alternative products for the above quantities, additional products for ocean surface flux and chlorophyll products, a number of vegetation products (e.g., FAPAR, LAI, burned area fraction), ice sheet mass and height, carbon monoxide (CO), and nitrogen dioxide (NO2). While most existing obs4MIPs datasets consist of monthly-mean gridded data over the global domain, products with higher time resolution (e.g., daily) and/or regional products are now receiving more attention. Along with an increasing number of datasets, obs4MIPs has implemented a number of capability upgrades including (1) an updated obs4MIPs data specifications document that provides additional search facets and generally improves congruence with CMIP6 specifications for model datasets, (2) a set of six easily understood indicators that help guide users as to a dataset's maturity and suitability for application, and (3) an option to supply supplemental information about a dataset beyond what can be found in the standard metadata. With the maturation of the obs4MIPs framework, the dataset inclusion process, and the dataset formatting guidelines and resources, the scope of the observations being considered is expected to grow to include gridded in situ datasets as well as datasets with a regional focus, and the ultimate intent is to judiciously expand this scope to any observation dataset that has applicability for evaluation of the types of Earth system models used in CMIP.

Overview of the CMIP6 Historical Experiment Datasets with the Climate System Model CAS FGOALS-f3-L

The three-member historical simulations by the Chinese Academy of Sciences Flexible Global Ocean-Atmosphere-Land System model, version f3-L (CAS FGOALS-f3-L), which is contributing to phase 6 of the Coupled Model Intercomparison Project (CMIP6), are described in this study. The details of the CAS FGOALS-f3-L model, experiment settings and output datasets are briefly introduced. The datasets include monthly and daily outputs from the atmospheric, oceanic, land and sea-ice component models of CAS FGOALS-f3-L, and all these data have been published online in the Earth System Grid Federation (ESGF, https://esgf-node.llnl.gov/projects/cmip6/). The three ensembles are initialized from the 600th, 650th and 700th model year of the preindustrial experiment (piControl) and forced by the same historical forcing provided by CMIP6 from 1850 to 2014. The performance of the coupled model is validated in comparison with some recent observed atmospheric and oceanic datasets. It is shown that CAS FGOALS-f3-L is able to reproduce the main features of the modern climate, including the climatology of air surface temperature and precipitation, the long-term changes in global mean surface air temperature, ocean heat content and sea surface steric height, and the horizontal and vertical distribution of temperature in the ocean and atmosphere. Meanwhile, like other state-of-the-art coupled GCMs, there are still some obvious biases in the historical simulations, which are also illustrated. This paper can help users to better understand the advantages and biases of the model and the datasets.

Developing an Open Data Portal for the ESA Climate Change Initiative

We introduce the rationale for, and architecture of, the European Space Agency Climate Change Initiative (CCI) Open Data Portal ( http://cci.esa.int/data/ ). The Open Data Portal hosts a set of richly diverse datasets – 13 “Essential Climate Variables” – from the CCI programme in a consistent and harmonised form and to provides a single point of access for the (>100 TB) data for broad dissemination to an international user community. These data have been produced by a range of different institutions and vary across both scientific and spatio-temporal characteristics. This heterogeneity of the data together with the range of services to be supported presented significant technical challenges. An iterative development methodology was key to tackling these challenges: the system developed exploits a workflow which takes data that conforms to the CCI data specification, ingests it into a managed archive and uses both manual and automatically generated metadata to support data discovery, browse, and delivery services. It utilises both Earth System Grid Federation (ESGF) data nodes and the Open Geospatial Consortium Catalogue Service for the Web (OGC-CSW) interface, serving data into both the ESGF and the Global Earth Observation System of Systems (GEOSS). A key part of the system is a new vocabulary server, populated with CCI specific terms and relationships which integrates OGC-CSW and ESGF search services together, developed as part of a dialogue between domain scientists and linked data specialists. These services have enabled the development of a unified user interface for graphical search and visualisation – the CCI Open Data Portal Web Presence.

A climate projection dataset tailored for the European energy sector

Climate information is necessary for the energy sector. However, the use of climate projections has remained limited so far for a number of reasons such us the lack of consistency among climate projections, the inadequate temporal and spatial resolution, the climate model biases, the lack of guidance for users, and the size of data sets. In this work, we develop and assess a consistent ensemble of high time and space resolution climate projections that address these problems. First, a methodology for sub-ensemble selection is developed and proposed. Our ensemble dataset includes eleven 12 km-resolution EURO-CORDEX simulations of temperature, precipitation, wind speed and surface solar radiation on 3-hourly and daily time scales. These variables are bias-corrected for a more effective use into impact studies. The assessment of bias-corrected model simulations against observational data indicates reduced biases and increased coherence in projected changes among models compared to the raw climate projections. We provide a well-documented dataset for energy practitioners and decision-makers to facilitate the access and use of energy-relevant high-quality climate information in operation and planning. The new dataset is freely available via the Earth System Grid Federation (ESGF) platform.

The Zero Emissions Commitment Model Intercomparison Project (ZECMIP) contribution to C4MIP: quantifying committed climate changes following zero carbon emissions

Abstract. The amount of additional future temperature change following a complete cessation of CO2 emissions is a measure of the unrealized warming to which we are committed due to CO2 already emitted to the atmosphere. This “zero emissions commitment” (ZEC) is also an important quantity when estimating the remaining carbon budget – a limit on the total amount of CO2 emissions consistent with limiting global mean temperature at a particular level. In the recent IPCC Special Report on Global Warming of 1.5 ∘C, the carbon budget framework used to calculate the remaining carbon budget for 1.5 ∘C included the assumption that the ZEC due to CO2 emissions is negligible and close to zero. Previous research has shown significant uncertainty even in the sign of the ZEC. To close this knowledge gap, we propose the Zero Emissions Commitment Model Intercomparison Project (ZECMIP), which will quantify the amount of unrealized temperature change that occurs after CO2 emissions cease and investigate the geophysical drivers behind this climate response. Quantitative information on ZEC is a key gap in our knowledge, and one that will not be addressed by currently planned CMIP6 simulations, yet it is crucial for verifying whether carbon budgets need to be adjusted to account for any unrealized temperature change resulting from past CO2 emissions. We request only one top-priority simulation from comprehensive general circulation Earth system models (ESMs) and Earth system models of intermediate complexity (EMICs) – a branch from the 1 % CO2 run with CO2 emissions set to zero at the point of 1000 PgC of total CO2 emissions in the simulation – with the possibility for additional simulations, if resources allow. ZECMIP is part of CMIP6, under joint sponsorship by C4MIP and CDRMIP, with associated experiment names to enable data submissions to the Earth System Grid Federation. All data will be published and made freely available.

The Atmospheric Infrared Sounder Obs4MIPs Version 2 Data Set

AbstractThe Atmospheric Infrared Sounder (AIRS) Obs4MIPs (Observations for Model Intercomparison Projects) Version 2 data set includes monthly mean tropospheric air temperature, specific humidity, and relative humidity for each calendar month from September 2002 to September 2016, on a global 1° × 1° latitude‐longitude spatial grid and on the Coupled Model Intercomparison Project eight mandatory vertical pressure levels from 1,000 to 300 hPa. It also includes standard error and number of observations, for an estimate of AIRS data retrieval error and sampling uncertainty. Three new technical notes describe the data set. This data set is designed for Coupled Model Intercomparison Project climate model evaluation and has been published at Earth System Grid Federation data centers since April 2018. It adds new monthly mean tropospheric relative humidity data to Obs4MIPs and updates and extends the monthly mean tropospheric air temperature and specific humidity data in AIRS Obs4MIPs Version 1 data set. The data source for this data set is the AIRS Version 6 Level 3 standard monthly mean air temperature, specific humidity, and relative humidity data products in the “TqJoint” grids from the AIRS and Advanced Microwave Sounding Unit A combined physical retrievals. This paper documents this data set in terms of motivation for this data set, data description, data origin and processing procedures, major improvements from the previous version, and caveats for its use in climate model evaluation.

Regional Climate Model Evaluation System powered by Apache Open Climate Workbench v1.3.0: an enabling tool for facilitating regional climate studies

Abstract. The Regional Climate Model Evaluation System (RCMES) is an enabling tool of the National Aeronautics and Space Administration to support the United States National Climate Assessment. As a comprehensive system for evaluating climate models on regional and continental scales using observational datasets from a variety of sources, RCMES is designed to yield information on the performance of climate models and guide their improvement. Here, we present a user-oriented document describing the latest version of RCMES, its development process, and future plans for improvements. The main objective of RCMES is to facilitate the climate model evaluation process at regional scales. RCMES provides a framework for performing systematic evaluations of climate simulations, such as those from the Coordinated Regional Climate Downscaling Experiment (CORDEX), using in situ observations, as well as satellite and reanalysis data products. The main components of RCMES are (1) a database of observations widely used for climate model evaluation, (2) various data loaders to import climate models and observations on local file systems and Earth System Grid Federation (ESGF) nodes, (3) a versatile processor to subset and regrid the loaded datasets, (4) performance metrics designed to assess and quantify model skill, (5) plotting routines to visualize the performance metrics, (6) a toolkit for statistically downscaling climate model simulations, and (7) two installation packages to maximize convenience of users without Python skills. RCMES website is maintained up to date with a brief explanation of these components. Although there are other open-source software (OSS) toolkits that facilitate analysis and evaluation of climate models, there is a need for climate scientists to participate in the development and customization of OSS to study regional climate change. To establish infrastructure and to ensure software sustainability, development of RCMES is an open, publicly accessible process enabled by leveraging the Apache Software Foundation's OSS library, Apache Open Climate Workbench (OCW). The OCW software that powers RCMES includes a Python OSS library for common climate model evaluation tasks as well as a set of user-friendly interfaces for quickly configuring a model evaluation task. OCW also allows users to build their own climate data analysis tools, such as the statistical downscaling toolkit provided as a part of RCMES.

Requirements for a global data infrastructure in support of CMIP6

Abstract. The World Climate Research Programme (WCRP)'s Working Group on Climate Modelling (WGCM) Infrastructure Panel (WIP) was formed in 2014 in response to the explosive growth in size and complexity of Coupled Model Intercomparison Projects (CMIPs) between CMIP3 (2005–2006) and CMIP5 (2011–2012). This article presents the WIP recommendations for the global data infrastructure needed to support CMIP design, future growth, and evolution. Developed in close coordination with those who build and run the existing infrastructure (the Earth System Grid Federation; ESGF), the recommendations are based on several principles beginning with the need to separate requirements, implementation, and operations. Other important principles include the consideration of the diversity of community needs around data – a data ecosystem – the importance of provenance, the need for automation, and the obligation to measure costs and benefits.This paper concentrates on requirements, recognizing the diversity of communities involved (modelers, analysts, software developers, and downstream users). Such requirements include the need for scientific reproducibility and accountability alongside the need to record and track data usage. One key element is to generate a dataset-centric rather than system-centric focus, with an aim to making the infrastructure less prone to systemic failure.With these overarching principles and requirements, the WIP has produced a set of position papers, which are summarized in the latter pages of this document. They provide specifications for managing and delivering model output, including strategies for replication and versioning, licensing, data quality assurance, citation, long-term archiving, and dataset tracking. They also describe a new and more formal approach for specifying what data, and associated metadata, should be saved, which enables future data volumes to be estimated, particularly for well-defined projects such as CMIP6.The paper concludes with a future facing consideration of the global data infrastructure evolution that follows from the blurring of boundaries between climate and weather, and the changing nature of published scientific results in the digital age.

CMIP6 Data Citation of Evolving Data

Data citations have become widely accepted. Technical infrastructures as well as principles and recommendations for data citation are in place but best practices or guidelines for their implementation are not yet available. On the other hand, the scientific climate community requests early citations on evolving data for credit, e.g. for CMIP6 (Coupled Model Intercomparison Project Phase 6). The data citation concept for CMIP6 is presented. The main challenges lie in limited resources, a strict project timeline and the dependency on changes of the data dissemination infrastructure ESGF (Earth System Grid Federation) to meet the data citation requirements. Therefore a pragmatic, flexible and extendible approach for the CMIP6 data citation service was developed, consisting of a citation for the full evolving data superset and a data cart approach for citing the concrete used data subset. This two citation approach can be implemented according to the RDA recommendations for evolving data. Because of resource constraints and missing project policies, the implementation of the second part of the citation concept is postponed to CMIP7.

Benchmarking CMIP5 models with a subset of ESA CCI Phase 2 data using the ESMValTool

The Coupled Model Intercomparison Project (CMIP) is now moving into its sixth phase and aims at a more routine evaluation of the models as soon as the model output is published to the Earth System Grid Federation (ESGF). To meet this goal the Earth System Model Evaluation Tool (ESMValTool), a community diagnostics and performance metrics tool for the systematic evaluation of Earth system models (ESMs) in CMIP, has been developed and a first version (1.0) released as open source software in 2015. Here, an enhanced version of the ESMValTool is presented that exploits a subset of Essential Climate Variables (ECVs) from the European Space Agency's Climate Change Initiative (ESA CCI) Phase 2 and this version is used to demonstrate the value of the data for model evaluation. This subset includes consistent, long-term time series of ECVs obtained from harmonized, reprocessed products from different satellite instruments for sea surface temperature, sea ice, cloud, soil moisture, land cover, aerosol, ozone, and greenhouse gases. The ESA CCI data allow extending the calculation of performance metrics as summary statistics for some variables and add an important alternative data set in other cases where observations are already available. The provision of uncertainty estimates on a per grid basis for the ESA CCI data sets is used in a new extended version of the Taylor diagram and provides important additional information for a more objective evaluation of the models. In our analysis we place a specific focus on the comparability of model and satellite data both in time and space. The ESA CCI data are well suited for an evaluation of results from global climate models across ESM compartments as well as an analysis of long-term trends, variability and change in the context of a changing climate. The enhanced version of the ESMValTool is released as open source software and ready to support routine model evaluation in CMIP6 and at individual modeling centers.

Towards improved and more routine Earth system model evaluation in CMIP

Abstract. The Coupled Model Intercomparison Project (CMIP) has successfully provided the climate community with a rich collection of simulation output from Earth system models (ESMs) that can be used to understand past climate changes and make projections and uncertainty estimates of the future. Confidence in ESMs can be gained because the models are based on physical principles and reproduce many important aspects of observed climate. More research is required to identify the processes that are most responsible for systematic biases and the magnitude and uncertainty of future projections so that more relevant performance tests can be developed. At the same time, there are many aspects of ESM evaluation that are well established and considered an essential part of systematic evaluation but have been implemented ad hoc with little community coordination. Given the diversity and complexity of ESM analysis, we argue that the CMIP community has reached a critical juncture at which many baseline aspects of model evaluation need to be performed much more efficiently and consistently. Here, we provide a perspective and viewpoint on how a more systematic, open, and rapid performance assessment of the large and diverse number of models that will participate in current and future phases of CMIP can be achieved, and announce our intention to implement such a system for CMIP6. Accomplishing this could also free up valuable resources as many scientists are frequently "re-inventing the wheel" by re-writing analysis routines for well-established analysis methods. A more systematic approach for the community would be to develop and apply evaluation tools that are based on the latest scientific knowledge and observational reference, are well suited for routine use, and provide a wide range of diagnostics and performance metrics that comprehensively characterize model behaviour as soon as the output is published to the Earth System Grid Federation (ESGF). The CMIP infrastructure enforces data standards and conventions for model output and documentation accessible via the ESGF, additionally publishing observations (obs4MIPs) and reanalyses (ana4MIPs) for model intercomparison projects using the same data structure and organization as the ESM output. This largely facilitates routine evaluation of the ESMs, but to be able to process the data automatically alongside the ESGF, the infrastructure needs to be extended with processing capabilities at the ESGF data nodes where the evaluation tools can be executed on a routine basis. Efforts are already underway to develop community-based evaluation tools, and we encourage experts to provide additional diagnostic codes that would enhance this capability for CMIP. At the same time, we encourage the community to contribute observations and reanalyses for model evaluation to the obs4MIPs and ana4MIPs archives. The intention is to produce through the ESGF a widely accepted quasi-operational evaluation framework for CMIP6 that would routinely execute a series of standardized evaluation tasks. Over time, as this capability matures, we expect to produce an increasingly systematic characterization of models which, compared with early phases of CMIP, will more quickly and openly identify the strengths and weaknesses of the simulations. This will also reveal whether long-standing model errors remain evident in newer models and will assist modelling groups in improving their models. This framework will be designed to readily incorporate updates, including new observations and additional diagnostics and metrics as they become available from the research community.

Big Data Challenges in Climate Science.

The knowledge we gain from research in climate science depends on the generation, dissemination, and analysis of high-quality data. This work comprises technical practice as well as social practice, both of which are distinguished by their massive scale and global reach. As a result, the amount of data involved in climate research is growing at an unprecedented rate. Climate model intercomparison (CMIP) experiments, the integration of observational data and climate reanalysis data with climate model outputs, as seen in the Obs4MIPs, Ana4MIPs, and CREATE-IP activities, and the collaborative work of the Intergovernmental Panel on Climate Change (IPCC) provide examples of the types of activities that increasingly require an improved cyberinfrastructure for dealing with large amounts of critical scientific data. This paper provides an overview of some of climate science's big data problems and the technical solutions being developed to advance data publication, climate analytics as a service, and interoperability within the Earth System Grid Federation (ESGF), the primary cyberinfrastructure currently supporting global climate research activities.

ESMValTool (v1.0) – a community diagnostic and performance metrics tool for routine evaluation of Earth system models in CMIP

Abstract. A community diagnostics and performance metrics tool for the evaluation of Earth system models (ESMs) has been developed that allows for routine comparison of single or multiple models, either against predecessor versions or against observations. The priority of the effort so far has been to target specific scientific themes focusing on selected essential climate variables (ECVs), a range of known systematic biases common to ESMs, such as coupled tropical climate variability, monsoons, Southern Ocean processes, continental dry biases, and soil hydrology–climate interactions, as well as atmospheric CO2 budgets, tropospheric and stratospheric ozone, and tropospheric aerosols. The tool is being developed in such a way that additional analyses can easily be added. A set of standard namelists for each scientific topic reproduces specific sets of diagnostics or performance metrics that have demonstrated their importance in ESM evaluation in the peer-reviewed literature. The Earth System Model Evaluation Tool (ESMValTool) is a community effort open to both users and developers encouraging open exchange of diagnostic source code and evaluation results from the Coupled Model Intercomparison Project (CMIP) ensemble. This will facilitate and improve ESM evaluation beyond the state-of-the-art and aims at supporting such activities within CMIP and at individual modelling centres. Ultimately, we envisage running the ESMValTool alongside the Earth System Grid Federation (ESGF) as part of a more routine evaluation of CMIP model simulations while utilizing observations available in standard formats (obs4MIPs) or provided by the user.

A Global Repository for Planet-Sized Experiments and Observations

Abstract Working across U.S. federal agencies, international agencies, and multiple worldwide data centers, and spanning seven international network organizations, the Earth System Grid Federation (ESGF) allows users to access, analyze, and visualize data using a globally federated collection of networks, computers, and software. Its architecture employs a system of geographically distributed peer nodes that are independently administered yet united by common federation protocols and application programming interfaces (APIs). The full ESGF infrastructure has now been adopted by multiple Earth science projects and allows access to petabytes of geophysical data, including the Coupled Model Intercomparison Project (CMIP)—output used by the Intergovernmental Panel on Climate Change assessment reports. Data served by ESGF not only include model output (i.e., CMIP simulation runs) but also include observational data from satellites and instruments, reanalyses, and generated images. Metadata summarize basic information about the data for fast and easy data discovery.

The Need for a Tool to Support Users of e-Science Infrastructures in a Virtual Laboratory Environment

Support processes play an important role to facilitate researchers (users) to accomplish their research activities with the help of cyber-infrastructure(s). However, the current user-support process in cyber-infrastructure needs a feasible tool to support users. This tool must enable the users of a cyber-infrastructure to communicate efficiently with the staffs of a cyber-infrastructure in order to get technical and scientific assistance, whilst saving resources at the same time. This research paper narrates the real story of employing various forms of tools to support the user and staff communication. In addition this paper projects the lessons learned from an exploration of the help-desk tools in the current state of user support process in Earth System Grid Federation (ESGF) from support staffs’ perspective. ESGF is a climate cyber-infrastructure that facilitates Earth System Modeling (ESM) and is taken as a case study in this paper. Finally, this study proposes a need for a tool, a framework or a platform that not only improves the user support process to address support servicing needs of end-users of e-Science infrastructures but also eases the life of staffs in providing assistance to the users. With the help of such a tool; the collaboration between users and the staffs of cyber-infrastructures is made easier. Consequently, the research activities of the users of e-Science infrastructure will thrive as the scientific and technical support will be available to users. Finally, this results into painless and productive e-Research.

The Earth System Grid Federation: An open infrastructure for access to distributed geospatial data

The Earth System Grid Federation (ESGF) is a multi-agency, international collaboration that aims at developing the software infrastructure needed to facilitate and empower the study of climate change on a global scale. The ESGF’s architecture employs a system of geographically distributed peer nodes, which are independently administered yet united by the adoption of common federation protocols and application programming interfaces (APIs). The cornerstones of its interoperability are the peer-to-peer messaging that is continuously exchanged among all nodes in the federation; a shared architecture and API for search and discovery; and a security infrastructure based on industry standards (OpenID, SSL, GSI and SAML). The ESGF software stack integrates custom components (for data publishing, searching, user interface, security and messaging), developed collaboratively by the team, with popular application engines (Tomcat, Solr) available from the open source community. The full ESGF infrastructure has now been adopted by multiple Earth science projects and allows access to petabytes of geophysical data, including the entire Fifth Coupled Model Intercomparison Project (CMIP5) output used by the Intergovernmental Panel on Climate Change (IPCC) Fifth Assessment Report (AR5) and a suite of satellite observations (obs4MIPs) and reanalysis data sets (ANA4MIPs). This paper presents ESGF as a successful example of integration of disparate open source technologies into a cohesive, wide functional system, and describes our experience in building and operating a distributed and federated infrastructure to serve the needs of the global climate science community.

The Earth System Grid Federation: Delivering globally accessible petascale data for CMIP5

The fifth Coupled Model Intercomparison Project (CMIP5) will involve the global production and analysis of petabytes of data. The Program for Climate Model Diagnosis and Intercomparison (PCMDI), with responsibility for archival for CMIP5, has established the global “Earth System Grid Federation” (ESGF) of data producers and data archives to support CMIP5. ESGF will provide a set of globally synchronised views of globally distributed data – including some large cache replicants which will be persisted for (at least) decades. Here we describe the archive requirements and key aspects of the resulting architecture. ESGF will stress international networks, as well as the data archives themselves – but significantly less than would have been the case of a centralised archive. Developing and deploying the ESGF has exploited good will and best efforts, but future developments are likely to require more formalised architecture and management.